sections_fts
714 rows
This data as json, CSV (advanced)
| Link | rowid ▼ | title | content | sections_fts | rank |
|---|---|---|---|---|---|
| 1 | 1 | Events | Datasette includes a mechanism for tracking events that occur while the software is running. This is primarily intended to be used by plugins, which can both trigger events and listen for events. The core Datasette application triggers events when certain things happen. This page describes those events. Note that these events will not fire for changes made to a SQLite database by a process other than Datasette itself. Plugins can listen for events using the track_event(datasette, event) plugin hook, which will be called with instances of the following classes - or additional classes registered by other plugins . class datasette.events. LoginEvent actor : dict | None Event name: login A user (represented by event.actor ) has logged in. class datasette.events. LogoutEvent actor : dict | None Event name: logout A user (represented by event.actor ) has logged out. class datasette.events. CreateTokenEvent actor : dict | None expires_after : int | None restrict_all : list restrict_database : dict restrict_resource : dict Event name: create-token A user created an API token. Variables expires_after -- Number of seconds after which this token will expire. restrict_all -- Restricted permissions for this token. restrict_database -- R… | 76 | |
| 2 | 2 | CLI reference | The datasette CLI tool provides a number of commands. Running datasette without specifying a command runs the default command, datasette serve . See datasette serve for the full list of options for that command. [[[cog from datasette import cli from click.testing import CliRunner import textwrap def help(args): title = "datasette " + " ".join(args) cog.out("\n::\n\n") result = CliRunner().invoke(cli.cli, args) output = result.output.replace("Usage: cli ", "Usage: datasette ") cog.out(textwrap.indent(output, ' ')) cog.out("\n\n") ]]] [[[end]]] | 76 | |
| 3 | 3 | datasette --help | Running datasette --help shows a list of all of the available commands. [[[cog help(["--help"]) ]]] Usage: datasette [OPTIONS] COMMAND [ARGS]... Datasette is an open source multi-tool for exploring and publishing data About Datasette: https://datasette.io/ Full documentation: https://docs.datasette.io/ Options: --version Show the version and exit. --help Show this message and exit. Commands: serve* Serve up specified SQLite database files with a web UI create-token Create a signed API token for the specified actor ID inspect Generate JSON summary of provided database files install Install plugins and packages from PyPI into the same... package Package SQLite files into a Datasette Docker container plugins List currently installed plugins publish Publish specified SQLite database files to the internet... uninstall Uninstall plugins and Python packages from the Datasette... [[[end]]] Additional commands added by plugins that use the register_commands(cli) hook will be listed here as well. | 76 | |
| 4 | 4 | datasette serve | This command starts the Datasette web application running on your machine: datasette serve mydatabase.db Or since this is the default command you can run this instead: datasette mydatabase.db Once started you can access it at http://localhost:8001 [[[cog help(["serve", "--help"]) ]]] Usage: datasette serve [OPTIONS] [FILES]... Serve up specified SQLite database files with a web UI Options: -i, --immutable PATH Database files to open in immutable mode -h, --host TEXT Host for server. Defaults to 127.0.0.1 which means only connections from the local machine will be allowed. Use 0.0.0.0 to listen to all IPs and allow access from other machines. -p, --port INTEGER RANGE Port for server, defaults to 8001. Use -p 0 to automatically assign an available port. [0<=x<=65535] --uds TEXT Bind to a Unix domain socket --reload Automatically reload if code or metadata change detected - useful for development --cors Enable CORS by serving Access-Control-Allow- Origin: * --load-extension PATH:ENTRYPOINT? Path to a SQLite extension to load, and optional entrypoint --inspect-file TEXT Path to JSON file created using "datasette inspect" -m, --metadata FILENAME Path to JSON/YAML file containing license/source metadata --template-dir DIRECTORY Path to directory containing custom templates --plugins-dir DIRECTORY Path to directory containing custom plugins --static MOUNT:DIRECTORY Serve static files fr… | 76 | |
| 5 | 5 | Environment variables | Some of the datasette serve options can be provided by environment variables: DATASETTE_SECRET : Equivalent to the --secret option. DATASETTE_SSL_KEYFILE : Equivalent to the --ssl-keyfile option. DATASETTE_SSL_CERTFILE : Equivalent to the --ssl-certfile option. DATASETTE_LOAD_EXTENSION : Equivalent to the --load-extension option. | 76 | |
| 6 | 6 | datasette --get | The --get option to datasette serve (or just datasette ) specifies the path to a page within Datasette and causes Datasette to output the content from that path without starting the web server. This means that all of Datasette's functionality can be accessed directly from the command-line. For example: datasette --get '/-/versions.json' | jq . { "python": { "version": "3.8.5", "full": "3.8.5 (default, Jul 21 2020, 10:48:26) \n[Clang 11.0.3 (clang-1103.0.32.62)]" }, "datasette": { "version": "0.46+15.g222a84a.dirty" }, "asgi": "3.0", "uvicorn": "0.11.8", "sqlite": { "version": "3.32.3", "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "extensions": { "json1": null }, "compile_options": [ "COMPILER=clang-11.0.3", "ENABLE_COLUMN_METADATA", "ENABLE_FTS3", "ENABLE_FTS3_PARENTHESIS", "ENABLE_FTS4", "ENABLE_FTS5", "ENABLE_GEOPOLY", "ENABLE_JSON1", "ENABLE_PREUPDATE_HOOK", "ENABLE_RTREE", "ENABLE_SESSION", "MAX_VARIABLE_NUMBER=250000", "THREADSAFE=1" ] } } You can use the --token TOKEN option to send an API token with the simulated request. Or you can make a request as a specific actor by passing a JSON representation of that actor to --actor : datasette --memory --actor '{"id": "root"}' --get '/-/actor.json' The exit code of datasette --get will be 0 if the request succeeds and 1 if the request produced an HTTP status code other than 200 - e.g. a 404 or 500 error. This lets you use datasette --get / to run tests against a Datasette application in a continuous integration environment such as GitHub Actions. | 76 | |
| 7 | 7 | datasette serve --help-settings | This command outputs all of the available Datasette settings . These can be passed to datasette serve using datasette serve --setting name value . [[[cog help(["--help-settings"]) ]]] Settings: default_page_size Default page size for the table view (default=100) max_returned_rows Maximum rows that can be returned from a table or custom query (default=1000) max_insert_rows Maximum rows that can be inserted at a time using the bulk insert API (default=100) num_sql_threads Number of threads in the thread pool for executing SQLite queries (default=3) sql_time_limit_ms Time limit for a SQL query in milliseconds (default=1000) default_facet_size Number of values to return for requested facets (default=30) facet_time_limit_ms Time limit for calculating a requested facet (default=200) facet_suggest_time_limit_ms Time limit for calculating a suggested facet (default=50) allow_facet Allow users to specify columns to facet using ?_facet= parameter (default=True) allow_download Allow users to download the original SQLite database files (default=True) allow_signed_tokens Allow users to create and use signed API tokens (default=True) default_allow_sql Allow anyone to run arbitrary SQL queries (default=True) max_signed_tokens_ttl Maximum allowed expiry time for signed API tokens (default=0) suggest_facets Calculate and display suggested facets (default… | 76 | |
| 8 | 8 | datasette plugins | Output JSON showing all currently installed plugins, their versions, whether they include static files or templates and which Plugin hooks they use. [[[cog help(["plugins", "--help"]) ]]] Usage: datasette plugins [OPTIONS] List currently installed plugins Options: --all Include built-in default plugins --requirements Output requirements.txt of installed plugins --plugins-dir DIRECTORY Path to directory containing custom plugins --help Show this message and exit. [[[end]]] Example output: [ { "name": "datasette-geojson", "static": false, "templates": false, "version": "0.3.1", "hooks": [ "register_output_renderer" ] }, { "name": "datasette-geojson-map", "static": true, "templates": false, "version": "0.4.0", "hooks": [ "extra_body_script", "extra_css_urls", "extra_js_urls" ] }, { "name": "datasette-leaflet", "static": true, "templates": false, "version": "0.2.2", "hooks": [ "extra_body_script", "extra_template_vars" ] } ] | 76 | |
| 9 | 9 | datasette install | Install new Datasette plugins. This command works like pip install but ensures that your plugins will be installed into the same environment as Datasette. This command: datasette install datasette-cluster-map Would install the datasette-cluster-map plugin. [[[cog help(["install", "--help"]) ]]] Usage: datasette install [OPTIONS] [PACKAGES]... Install plugins and packages from PyPI into the same environment as Datasette Options: -U, --upgrade Upgrade packages to latest version -r, --requirement PATH Install from requirements file -e, --editable TEXT Install a project in editable mode from this path --help Show this message and exit. [[[end]]] | 76 | |
| 10 | 10 | datasette uninstall | Uninstall one or more plugins. [[[cog help(["uninstall", "--help"]) ]]] Usage: datasette uninstall [OPTIONS] PACKAGES... Uninstall plugins and Python packages from the Datasette environment Options: -y, --yes Don't ask for confirmation --help Show this message and exit. [[[end]]] | 76 | |
| 11 | 11 | datasette publish | Shows a list of available deployment targets for publishing data with Datasette. Additional deployment targets can be added by plugins that use the publish_subcommand(publish) hook. [[[cog help(["publish", "--help"]) ]]] Usage: datasette publish [OPTIONS] COMMAND [ARGS]... Publish specified SQLite database files to the internet along with a Datasette-powered interface and API Options: --help Show this message and exit. Commands: cloudrun Publish databases to Datasette running on Cloud Run heroku Publish databases to Datasette running on Heroku [[[end]]] | 76 | |
| 12 | 12 | datasette publish cloudrun | See Publishing to Google Cloud Run . [[[cog help(["publish", "cloudrun", "--help"]) ]]] Usage: datasette publish cloudrun [OPTIONS] [FILES]... Publish databases to Datasette running on Cloud Run Options: -m, --metadata FILENAME Path to JSON/YAML file containing metadata to publish --extra-options TEXT Extra options to pass to datasette serve --branch TEXT Install datasette from a GitHub branch e.g. main --template-dir DIRECTORY Path to directory containing custom templates --plugins-dir DIRECTORY Path to directory containing custom plugins --static MOUNT:DIRECTORY Serve static files from this directory at /MOUNT/... --install TEXT Additional packages (e.g. plugins) to install --plugin-secret <TEXT TEXT TEXT>... Secrets to pass to plugins, e.g. --plugin- secret datasette-auth-github client_id xxx --version-note TEXT Additional note to show on /-/versions --secret TEXT Secret used for signing secure values, such as signed cookies --title TEXT Title for metadata --license TEXT License label for metadata --license_url TEXT License URL for metadata --source TEXT Source label for metadata --source_url TEXT Source URL for metadata --about TEXT About label for metadata --about_url TEXT About URL for metadata -n, --name TEXT Application name to use when building --service TEXT Cloud Run service to deploy (or over-write) --spatialite Enable SpatialLite extension --show-files Output the generated Dockerfile and metad… | 76 | |
| 13 | 13 | datasette publish heroku | See Publishing to Heroku . [[[cog help(["publish", "heroku", "--help"]) ]]] Usage: datasette publish heroku [OPTIONS] [FILES]... Publish databases to Datasette running on Heroku Options: -m, --metadata FILENAME Path to JSON/YAML file containing metadata to publish --extra-options TEXT Extra options to pass to datasette serve --branch TEXT Install datasette from a GitHub branch e.g. main --template-dir DIRECTORY Path to directory containing custom templates --plugins-dir DIRECTORY Path to directory containing custom plugins --static MOUNT:DIRECTORY Serve static files from this directory at /MOUNT/... --install TEXT Additional packages (e.g. plugins) to install --plugin-secret <TEXT TEXT TEXT>... Secrets to pass to plugins, e.g. --plugin- secret datasette-auth-github client_id xxx --version-note TEXT Additional note to show on /-/versions --secret TEXT Secret used for signing secure values, such as signed cookies --title TEXT Title for metadata --license TEXT License label for metadata --license_url TEXT License URL for metadata --source TEXT Source label for metadata --source_url TEXT Source URL for metadata --about TEXT About label for metadata --about_url TEXT About URL for metadata -n, --name TEXT Application name to use when deploying --tar TEXT --tar option to pass to Heroku, e.g. --tar=/usr/local/bin/gtar --generate-dir DIRECTORY Output generated application files and stop without deploying --h… | 76 | |
| 14 | 14 | datasette package | Package SQLite files into a Datasette Docker container, see datasette package . [[[cog help(["package", "--help"]) ]]] Usage: datasette package [OPTIONS] FILES... Package SQLite files into a Datasette Docker container Options: -t, --tag TEXT Name for the resulting Docker container, can optionally use name:tag format -m, --metadata FILENAME Path to JSON/YAML file containing metadata to publish --extra-options TEXT Extra options to pass to datasette serve --branch TEXT Install datasette from a GitHub branch e.g. main --template-dir DIRECTORY Path to directory containing custom templates --plugins-dir DIRECTORY Path to directory containing custom plugins --static MOUNT:DIRECTORY Serve static files from this directory at /MOUNT/... --install TEXT Additional packages (e.g. plugins) to install --spatialite Enable SpatialLite extension --version-note TEXT Additional note to show on /-/versions --secret TEXT Secret used for signing secure values, such as signed cookies -p, --port INTEGER RANGE Port to run the server on, defaults to 8001 [1<=x<=65535] --title TEXT Title for metadata --license TEXT License label for metadata --license_url TEXT License URL for metadata --source TEXT Source label for metadata --source_url TEXT Source URL for metadata --about TEXT About label for metadata --about_url TEXT About URL for metadata --help Show this message and exit. [[[end]]] | 76 | |
| 15 | 15 | datasette inspect | Outputs JSON representing introspected data about one or more SQLite database files. If you are opening an immutable database, you can pass this file to the --inspect-data option to improve Datasette's performance by allowing it to skip running row counts against the database when it first starts running: datasette inspect mydatabase.db > inspect-data.json datasette serve -i mydatabase.db --inspect-file inspect-data.json This performance optimization is used automatically by some of the datasette publish commands. You are unlikely to need to apply this optimization manually. [[[cog help(["inspect", "--help"]) ]]] Usage: datasette inspect [OPTIONS] [FILES]... Generate JSON summary of provided database files This can then be passed to "datasette --inspect-file" to speed up count operations against immutable database files. Options: --inspect-file TEXT --load-extension PATH:ENTRYPOINT? Path to a SQLite extension to load, and optional entrypoint --help Show this message and exit. [[[end]]] | 76 | |
| 16 | 16 | datasette create-token | Create a signed API token, see datasette create-token . [[[cog help(["create-token", "--help"]) ]]] Usage: datasette create-token [OPTIONS] ID Create a signed API token for the specified actor ID Example: datasette create-token root --secret mysecret To allow only "view-database-download" for all databases: datasette create-token root --secret mysecret \ --all view-database-download To allow "create-table" against a specific database: datasette create-token root --secret mysecret \ --database mydb create-table To allow "insert-row" against a specific table: datasette create-token root --secret myscret \ --resource mydb mytable insert-row Restricted actions can be specified multiple times using multiple --all, --database, and --resource options. Add --debug to see a decoded version of the token. Options: --secret TEXT Secret used for signing the API tokens [required] -e, --expires-after INTEGER Token should expire after this many seconds -a, --all ACTION Restrict token to this action -d, --database DB ACTION Restrict token to this action on this database -r, --resource DB RESOURCE ACTION Restrict token to this action on this database resource (a table, SQL view or named query) --debug Show decoded token --plugins-dir DIRECTORY Path to directory containing custom plugins --help Show this message and exit. [[[end]]] | 76 | |
| 17 | 17 | Authentication and permissions | Datasette doesn't require authentication by default. Any visitor to a Datasette instance can explore the full data and execute read-only SQL queries. Datasette can be configured to only allow authenticated users, or to control which databases, tables, and queries can be accessed by the public or by specific users. Datasette's plugin system can be used to add many different styles of authentication, such as user accounts, single sign-on or API keys. | 76 | |
| 18 | 18 | Actors | Through plugins, Datasette can support both authenticated users (with cookies) and authenticated API clients (via authentication tokens). The word "actor" is used to cover both of these cases. Every request to Datasette has an associated actor value, available in the code as request.actor . This can be None for unauthenticated requests, or a JSON compatible Python dictionary for authenticated users or API clients. The actor dictionary can be any shape - the design of that data structure is left up to the plugins. Actors should always include a unique "id" string, as demonstrated by the "root" actor below. Plugins can use the actor_from_request(datasette, request) hook to implement custom logic for authenticating an actor based on the incoming HTTP request. | 76 | |
| 19 | 19 | Using the "root" actor | Datasette currently leaves almost all forms of authentication to plugins - datasette-auth-github for example. The one exception is the "root" account, which you can sign into while using Datasette on your local machine. The root user has all permissions - they can perform any action regardless of other permission rules. The --root flag is designed for local development and testing. When you start Datasette with --root , the root user automatically receives every permission, including: All view permissions ( view-instance , view-database , view-table , etc.) All write permissions ( insert-row , update-row , delete-row , create-table , alter-table , set-column-type , drop-table ) Debug permissions ( permissions-debug , debug-menu ) Any custom permissions defined by plugins If you add explicit deny rules in datasette.yaml those can still block the root actor from specific databases or tables. The --root flag sets an internal root_enabled switch—without it, a signed-in user with {"id": "root"} is treated like any other actor. To sign in as root, start Datasette using the --root command-line option, like this: datasette --root Datasette will output a single-use-only login URL on startup: http://127.0.0.1:8001/-/auth-token?token=786fc524e0199d70dc9a581d851f466244e114ca92f33aa3b42a139e9388daa7 INFO: Started server process [25801] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit) Click on that link… | 76 | |
| 20 | 20 | Permissions | Datasette's permissions system is built around SQL queries. Datasette and its plugins construct SQL queries to resolve the list of resources that an actor cas access. The key question the permissions system answers is this: Is this actor allowed to perform this action , optionally against this particular resource ? Actors are described above . An action is a string describing the action the actor would like to perform. A full list is provided below - examples include view-table and execute-sql . A resource is the item the actor wishes to interact with - for example a specific database or table. Some actions, such as permissions-debug , are not associated with a particular resource. Datasette's built-in view actions ( view-database , view-table etc) are allowed by Datasette's default configuration: unless you configure additional permission rules unauthenticated users will be allowed to access content. Other actions, including those introduced by plugins, will default to deny . | 76 | |
| 21 | 21 | Denying all permissions by default | By default, Datasette allows unauthenticated access to view databases, tables, and execute SQL queries. You may want to run Datasette in a mode where all access is denied by default, and you explicitly grant permissions only to authenticated users, either using the --root mechanism or through configuration file rules or plugins. Use the --default-deny command-line option to run Datasette in this mode: datasette --default-deny data.db --root With --default-deny enabled: Anonymous users are denied access to view the instance, databases, tables, and queries Authenticated users are also denied access unless they're explicitly granted permissions The root user (when using --root ) still has access to everything You can grant permissions using configuration file rules or plugins For example, to allow only a specific user to access your instance: datasette --default-deny data.db --config datasette.yaml Where datasette.yaml contains: allow: id: alice This configuration will deny access to everyone except the user with id of alice . | 76 | |
| 22 | 22 | How permissions are resolved | Datasette performs permission checks using the internal await .allowed(*, action, resource, actor=None) , method which accepts keyword arguments for action , resource and an optional actor . resource should be an instance of the appropriate Resource subclass from datasette.resources —for example InstanceResource() , DatabaseResource(database="... )`` or TableResource(database="...", table="...") . This defaults to InstanceResource() if not specified. When a check runs Datasette gathers allow/deny rules from multiple sources and compiles them into a SQL query. The resulting query describes all of the resources an actor may access for that action, together with the reasons those resources were allowed or denied. The combined sources are: allow blocks configured in datasette.yaml . Actor restrictions encoded into the actor dictionary or API token. The "root" user shortcut when --root (or Datasette.root_enabled ) is active, replying True to all permission chucks unless configuration rules deny them at a more specific level. Any additional SQL provided by plugins implementing permission_resources_sql(datasette, actor, action) . Datasette evaluates the SQL to determine if the requested resource is included. Explicit deny rules returned by configuration or plugins will block access even if other rules allowed it. | 76 | |
| 23 | 23 | Defining permissions with "allow" blocks | One way to define permissions in Datasette is to use an "allow" block in the datasette.yaml file . This is a JSON document describing which actors are allowed to perform an action against a specific resource. Each allow block is compiled into SQL and combined with any plugin-provided rules to produce the cascading allow/deny decisions that power await .allowed(*, action, resource, actor=None) . The most basic form of allow block is this ( allow demo , deny demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: id: root """).strip(), "YAML", "JSON" ) ]]] [[[end]]] This will match any actors with an "id" property of "root" - for example, an actor that looks like this: { "id": "root", "name": "Root User" } An allow block can specify "deny all" using false ( demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: false """).strip(), "YAML", "JSON" ) ]]] [[[end]]] An "allow" of true allows all access ( demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: true """).strip(), "YAML", "JSON" ) ]]] [[[end]]] Allow keys can provide a list of values. These will match any actor that has any of those values ( allow demo , deny demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: id: - simon - cleopaws """).strip(), "YAML", "JSON" ) ]]] [[[end]]] This will match any a… | 76 | |
| 24 | 24 | The /-/allow-debug tool | The /-/allow-debug tool lets you try out different "action" blocks against different "actor" JSON objects. You can try that out here: https://latest.datasette.io/-/allow-debug | 76 | |
| 25 | 25 | Access permissions in | There are two ways to configure permissions using datasette.yaml (or datasette.json ). For simple visibility permissions you can use "allow" blocks in the root, database, table and query sections. For other permissions you can use a "permissions" block, described in the next section . You can limit who is allowed to view different parts of your Datasette instance using "allow" keys in your Configuration . You can control the following: Access to the entire Datasette instance Access to specific databases Access to specific tables and views Access to specific Canned queries If a user has permission to view a table they will be able to view that table, independent of if they have permission to view the database or instance that the table exists within. | 76 | |
| 26 | 26 | Access to an instance | Here's how to restrict access to your entire Datasette instance to just the "id": "root" user: [[[cog from metadata_doc import config_example config_example(cog, """ title: My private Datasette instance allow: id: root """) ]]] [[[end]]] To deny access to all users, you can use "allow": false : [[[cog config_example(cog, """ title: My entirely inaccessible instance allow: false """) ]]] [[[end]]] One reason to do this is if you are using a Datasette plugin - such as datasette-permissions-sql - to control permissions instead. | 76 | |
| 27 | 27 | Access to specific databases | To limit access to a specific private.db database to just authenticated users, use the "allow" block like this: [[[cog config_example(cog, """ databases: private: allow: id: "*" """) ]]] [[[end]]] | 76 | |
| 28 | 28 | Access to specific tables and views | To limit access to the users table in your bakery.db database: [[[cog config_example(cog, """ databases: bakery: tables: users: allow: id: '*' """) ]]] [[[end]]] This works for SQL views as well - you can list their names in the "tables" block above in the same way as regular tables. Restricting access to tables and views in this way will NOT prevent users from querying them using arbitrary SQL queries, like this for example. If you are restricting access to specific tables you should also use the "allow_sql" block to prevent users from bypassing the limit with their own SQL queries - see Controlling the ability to execute arbitrary SQL . | 76 | |
| 29 | 29 | Access to specific canned queries | Canned queries allow you to configure named SQL queries in your datasette.yaml that can be executed by users. These queries can be set up to both read and write to the database, so controlling who can execute them can be important. To limit access to the add_name canned query in your dogs.db database to just the root user : [[[cog config_example(cog, """ databases: dogs: queries: add_name: sql: INSERT INTO names (name) VALUES (:name) write: true allow: id: - root """) ]]] [[[end]]] | 76 | |
| 30 | 30 | Controlling the ability to execute arbitrary SQL | Datasette defaults to allowing any site visitor to execute their own custom SQL queries, for example using the form on the database page or by appending a ?_where= parameter to the table page like this . Access to this ability is controlled by the execute-sql permission. The easiest way to disable arbitrary SQL queries is using the default_allow_sql setting when you first start Datasette running. You can alternatively use an "allow_sql" block to control who is allowed to execute arbitrary SQL queries. To prevent any user from executing arbitrary SQL queries, use this: [[[cog config_example(cog, """ allow_sql: false """) ]]] [[[end]]] To enable just the root user to execute SQL for all databases in your instance, use the following: [[[cog config_example(cog, """ allow_sql: id: root """) ]]] [[[end]]] To limit this ability for just one specific database, use this: [[[cog config_example(cog, """ databases: mydatabase: allow_sql: id: root """) ]]] [[[end]]] | 76 | |
| 31 | 31 | Other permissions in | For all other permissions, you can use one or more "permissions" blocks in your datasette.yaml configuration file. To grant access to the permissions debug tool to all signed in users, you can grant permissions-debug to any actor with an id matching the wildcard * by adding this a the root of your configuration: [[[cog config_example(cog, """ permissions: debug-menu: id: '*' """) ]]] [[[end]]] To grant create-table to the user with id of editor for the docs database: [[[cog config_example(cog, """ databases: docs: permissions: create-table: id: editor """) ]]] [[[end]]] Other table-scoped write permissions, including set-column-type , can be configured in the same place. And for insert-row against the reports table in that docs database: [[[cog config_example(cog, """ databases: docs: tables: reports: permissions: insert-row: id: editor """) ]]] [[[end]]] The permissions debug tool can be useful for helping test permissions that you have configured in this way. | 76 | |
| 32 | 32 | API Tokens | Datasette includes a default mechanism for generating API tokens that can be used to authenticate requests. Authenticated users can create new API tokens using a form on the /-/create-token page. Tokens created in this way can be further restricted to only allow access to specific actions, or to limit those actions to specific databases, tables or queries. Created tokens can then be passed in the Authorization: Bearer $token header of HTTP requests to Datasette. A token created by a user will include that user's "id" in the token payload, so any permissions granted to that user based on their ID can be made available to the token as well. When one of these a token accompanies a request, the actor for that request will have the following shape: { "id": "user_id", "token": "dstok", "token_expires": 1667717426 } The "id" field duplicates the ID of the actor who first created the token. The "token" field identifies that this actor was authenticated using a Datasette signed token ( dstok ). The "token_expires" field, if present, indicates that the token will expire after that integer timestamp. The /-/create-token page cannot be accessed by actors that are authenticated with a "token": "some-value" property. This is to prevent API tokens from being used to create more tokens. Datasette plugins that implement their own form of API token authentication should follow this convention. You can disable the signed token feature entirely using the allow_signed_tokens setting. | 76 | |
| 33 | 33 | datasette create-token | You can also create tokens on the command line using the datasette create-token command. This command takes one required argument - the ID of the actor to be associated with the created token. You can specify a -e/--expires-after option in seconds. If omitted, the token will never expire. The command will sign the token using the DATASETTE_SECRET environment variable, if available. You can also pass the secret using the --secret option. This means you can run the command locally to create tokens for use with a deployed Datasette instance, provided you know that instance's secret. To create a token for the root actor that will expire in one hour: datasette create-token root --expires-after 3600 To create a token that never expires using a specific secret: datasette create-token root --secret my-secret-goes-here | 76 | |
| 34 | 34 | Restricting the actions that a token can perform | Tokens created using datasette create-token ACTOR_ID will inherit all of the permissions of the actor that they are associated with. You can pass additional options to create tokens that are restricted to a subset of that actor's permissions. To restrict the token to just specific permissions against all available databases, use the --all option: datasette create-token root --all insert-row --all update-row This option can be passed as many times as you like. In the above example the token will only be allowed to insert and update rows. You can also restrict permissions such that they can only be used within specific databases: datasette create-token root --database mydatabase insert-row The resulting token will only be able to insert rows, and only to tables in the mydatabase database. Finally, you can restrict permissions to individual resources - tables, SQL views and named queries - within a specific database: datasette create-token root --resource mydatabase mytable insert-row These options have short versions: -a for --all , -d for --database and -r for --resource . You can add --debug to see a JSON representation of the token that has been created. Here's a full example: datasette create-token root \ --secret mysecret \ --all view-instance \ --all view-table \ --database docs view-query \ --resource docs documents insert-row \ --resource docs documents update-row \ --debug This example outputs the following: dstok_.eJxFizEKgDAMRe_y5w4qYrFXERGxDkVsMI0uxbubdjFL8l_ez1jhwEQCA6Fjjxp90qtkuHawzdjYrh8MFobLxZ_wBH0_gtnAF-hpS5VfmF8D_lnd97lHqUJgLd6sls4H1qwlhA.nH_7RecYHj5qSzvjhMU95iy0Xlc Decoded: { "a": "root", "token": "dstok", "t": 1670907246, "_r": { "a… | 76 | |
| 35 | 35 | Checking permissions in plugins | Datasette plugins can check if an actor has permission to perform an action using await .allowed(*, action, resource, actor=None) —for example: from datasette.resources import TableResource can_edit = await datasette.allowed( action="update-row", resource=TableResource(database="fixtures", table="facetable"), actor=request.actor, ) Use await .ensure_permission(action, resource=None, actor=None) when you need to enforce a permission and raise a Forbidden error automatically. Plugins that define new operations should return Action objects from register_actions(datasette) and can supply additional allow/deny rules by returning PermissionSQL objects from the permission_resources_sql(datasette, actor, action) hook. Those rules are merged with configuration allow blocks and actor restrictions to determine the final result for each check. | 76 | |
| 36 | 36 | actor_matches_allow() | Plugins that wish to implement this same "allow" block permissions scheme can take advantage of the datasette.utils.actor_matches_allow(actor, allow) function: from datasette.utils import actor_matches_allow actor_matches_allow({"id": "root"}, {"id": "*"}) # returns True The currently authenticated actor is made available to plugins as request.actor . | 76 | |
| 37 | 37 | Permissions debug tools | The debug tool at /-/permissions is available to any actor with the permissions-debug permission. By default this is just the authenticated root user but you can open it up to all users by starting Datasette like this: datasette -s permissions.permissions-debug true data.db The page shows the permission checks that have been carried out by the Datasette instance. It also provides an interface for running hypothetical permission checks against a hypothetical actor. This is a useful way of confirming that your configured permissions work in the way you expect. This is designed to help administrators and plugin authors understand exactly how permission checks are being carried out, in order to effectively configure Datasette's permission system. | 76 | |
| 38 | 38 | Allowed resources view | The /-/allowed endpoint displays resources that the current actor can access for a specified action . This endpoint provides an interactive HTML form interface. Add .json to the URL path (e.g. /-/allowed.json ) to get the raw JSON response instead. Pass ?action=view-table (or another action) to select the action. Optional parent= and child= query parameters can narrow the results to a specific database/table pair. This endpoint is publicly accessible to help users understand their own permissions. The potentially sensitive reason field is only shown to users with the permissions-debug permission - it shows the plugins and explanatory reasons that were responsible for each decision. | 76 | |
| 39 | 39 | Permission rules view | The /-/rules endpoint displays all permission rules (both allow and deny) for each candidate resource for the requested action. This endpoint provides an interactive HTML form interface. Add .json to the URL path (e.g. /-/rules.json?action=view-table ) to get the raw JSON response instead. Pass ?action= as a query parameter to specify which action to check. This endpoint requires the permissions-debug permission. | 76 | |
| 40 | 40 | Permission check view | The /-/check endpoint evaluates a single action/resource pair and returns information indicating whether the access was allowed along with diagnostic information. This endpoint provides an interactive HTML form interface. Add .json to the URL path (e.g. /-/check.json?action=view-instance ) to get the raw JSON response instead. Pass ?action= to specify the action to check, and optional ?parent= and ?child= parameters to specify the resource. | 76 | |
| 41 | 41 | The ds_actor cookie | Datasette includes a default authentication plugin which looks for a signed ds_actor cookie containing a JSON actor dictionary. This is how the root actor mechanism works. Authentication plugins can set signed ds_actor cookies themselves like so: response = Response.redirect("/") datasette.set_actor_cookie(response, {"id": "cleopaws"}) The shape of data encoded in the cookie is as follows: { "a": { "id": "cleopaws" } } To implement logout in a plugin, use the delete_actor_cookie() method: response = Response.redirect("/") datasette.delete_actor_cookie(response) | 76 | |
| 42 | 42 | Including an expiry time | ds_actor cookies can optionally include a signed expiry timestamp, after which the cookies will no longer be valid. Authentication plugins may chose to use this mechanism to limit the lifetime of the cookie. For example, if a plugin implements single-sign-on against another source it may decide to set short-lived cookies so that if the user is removed from the SSO system their existing Datasette cookies will stop working shortly afterwards. To include an expiry pass expire_after= to datasette.set_actor_cookie() with a number of seconds. For example, to expire in 24 hours: response = Response.redirect("/") datasette.set_actor_cookie( response, {"id": "cleopaws"}, expire_after=60 * 60 * 24 ) The resulting cookie will encode data that looks something like this: { "a": { "id": "cleopaws" }, "e": "1jjSji" } | 76 | |
| 43 | 43 | The /-/logout page | The page at /-/logout provides the ability to log out of a ds_actor cookie authentication session. | 76 | |
| 44 | 44 | Built-in actions | This section lists all of the permission checks that are carried out by Datasette core, along with the resource if it was passed. | 76 | |
| 45 | 45 | view-instance | Top level permission - Actor is allowed to view any pages within this instance, starting at https://latest.datasette.io/ | 76 | |
| 46 | 46 | view-database | Actor is allowed to view a database page, e.g. https://latest.datasette.io/fixtures resource - datasette.permissions.DatabaseResource(database) database is the name of the database (string) | 76 | |
| 47 | 47 | view-database-download | Actor is allowed to download a database, e.g. https://latest.datasette.io/fixtures.db resource - datasette.resources.DatabaseResource(database) database is the name of the database (string) | 76 | |
| 48 | 48 | view-table | Actor is allowed to view a table (or view) page, e.g. https://latest.datasette.io/fixtures/complex_foreign_keys resource - datasette.resources.TableResource(database, table) database is the name of the database (string) table is the name of the table (string) | 76 | |
| 49 | 49 | view-query | Actor is allowed to view (and execute) a canned query page, e.g. https://latest.datasette.io/fixtures/pragma_cache_size - this includes executing Writable canned queries . resource - datasette.resources.QueryResource(database, query) database is the name of the database (string) query is the name of the canned query (string) | 76 | |
| 50 | 50 | insert-row | Actor is allowed to insert rows into a table. resource - datasette.resources.TableResource(database, table) database is the name of the database (string) table is the name of the table (string) | 76 | |
| 51 | 51 | delete-row | Actor is allowed to delete rows from a table. resource - datasette.resources.TableResource(database, table) database is the name of the database (string) table is the name of the table (string) | 76 | |
| 52 | 52 | update-row | Actor is allowed to update rows in a table. resource - datasette.resources.TableResource(database, table) database is the name of the database (string) table is the name of the table (string) | 76 | |
| 53 | 53 | create-table | Actor is allowed to create a database table. resource - datasette.resources.DatabaseResource(database) database is the name of the database (string) | 76 | |
| 54 | 54 | alter-table | Actor is allowed to alter a database table. resource - datasette.resources.TableResource(database, table) database is the name of the database (string) table is the name of the table (string) | 76 | |
| 55 | 55 | set-column-type | Actor is allowed to set assigned column types for columns in a table. resource - datasette.resources.TableResource(database, table) database is the name of the database (string) table is the name of the table (string) | 76 | |
| 56 | 56 | drop-table | Actor is allowed to drop a database table. resource - datasette.resources.TableResource(database, table) database is the name of the database (string) table is the name of the table (string) | 76 | |
| 57 | 57 | execute-sql | Actor is allowed to run arbitrary SQL queries against a specific database, e.g. https://latest.datasette.io/fixtures/-/query?sql=select+100 resource - datasette.resources.DatabaseResource(database) database is the name of the database (string) See also the default_allow_sql setting . | 76 | |
| 58 | 58 | permissions-debug | Actor is allowed to view the /-/permissions debug tools. | 76 | |
| 59 | 59 | debug-menu | Controls if the various debug pages are displayed in the navigation menu. | 76 | |
| 60 | 60 | JavaScript plugins | Datasette can run custom JavaScript in several different ways: Datasette plugins written in Python can use the extra_js_urls() or extra_body_script() plugin hooks to inject JavaScript into a page Datasette instances with custom templates can include additional JavaScript in those templates The extra_js_urls key in datasette.yaml can be used to include extra JavaScript There are no limitations on what this JavaScript can do. It is executed directly by the browser, so it can manipulate the DOM, fetch additional data and do anything else that JavaScript is capable of. Custom JavaScript has security implications, especially for authenticated Datasette instances where the JavaScript might run in the context of the authenticated user. It's important to carefully review any JavaScript you run in your Datasette instance. | 76 | |
| 61 | 61 | The datasette_init event | Datasette emits a custom event called datasette_init when the page is loaded. This event is dispatched on the document object, and includes a detail object with a reference to the datasetteManager object. Your JavaScript code can listen out for this event using document.addEventListener() like this: document.addEventListener("datasette_init", function (evt) { const manager = evt.detail; console.log("Datasette version:", manager.VERSION); }); | 76 | |
| 62 | 62 | datasetteManager | The datasetteManager object VERSION - string The version of Datasette plugins - Map() A Map of currently loaded plugin names to plugin implementations registerPlugin(name, implementation) Call this to register a plugin, passing its name and implementation selectors - object An object providing named aliases to useful CSS selectors, listed below | 76 | |
| 63 | 63 | JavaScript plugin objects | JavaScript plugins are blocks of code that can be registered with Datasette using the registerPlugin() method on the datasetteManager object. The implementation object passed to this method should include a version key defining the plugin version, and one or more of the following named functions providing the implementation of the plugin: | 76 | |
| 64 | 64 | makeAboveTablePanelConfigs() | This method should return a JavaScript array of objects defining additional panels to be added to the top of the table page. Each object should have the following: id - string A unique string ID for the panel, for example map-panel label - string A human-readable label for the panel render(node) - function A function that will be called with a DOM node to render the panel into This example shows how a plugin might define a single panel: document.addEventListener('datasette_init', function(ev) { ev.detail.registerPlugin('panel-plugin', { version: 0.1, makeAboveTablePanelConfigs: () => { return [ { id: 'first-panel', label: 'First panel', render: node => { node.innerHTML = '<h2>My custom panel</h2><p>This is a custom panel that I added using a JavaScript plugin</p>'; } } ] } }); }); When a page with a table loads, all registered plugins that implement makeAboveTablePanelConfigs() will be called and panels they return will be added to the top of the table page. | 76 | |
| 65 | 65 | makeColumnActions(columnDetails) | This method, if present, will be called when Datasette is rendering the cog action menu icons that appear at the top of the table view. By default these include options like "Sort ascending/descending" and "Facet by this", but plugins can return additional actions to be included in this menu. The method will be called with a columnDetails object with the following keys: columnName - string The name of the column columnNotNull - boolean True if the column is defined as NOT NULL columnType - string The SQLite data type of the column isPk - boolean True if the column is part of the primary key It should return a JavaScript array of objects each with a label and onClick property: label - string The human-readable label for the action onClick(evt) - function A function that will be called when the action is clicked The evt object passed to the onClick is the standard browser event object that triggered the click. This example plugin adds two menu items - one to copy … | 76 | |
| 66 | 66 | Selectors | These are available on the selectors property of the datasetteManager object. const DOM_SELECTORS = { /** Should have one match */ jsonExportLink: ".export-links a[href*=json]", /** Event listeners that go outside of the main table, e.g. existing scroll listener */ tableWrapper: ".table-wrapper", table: "table.rows-and-columns", aboveTablePanel: ".above-table-panel", // These could have multiple matches /** Used for selecting table headers. Use makeColumnActions if you want to add menu items. */ tableHeaders: `table.rows-and-columns th`, /** Used to add "where" clauses to query using direct manipulation */ filterRows: ".filter-row", /** Used to show top available enum values for a column ("facets") */ facetResults: ".facet-results [data-column]", }; | 76 | |
| 67 | 67 | Deploying Datasette | The quickest way to deploy a Datasette instance on the internet is to use the datasette publish command, described in Publishing data . This can be used to quickly deploy Datasette to a number of hosting providers including Heroku, Google Cloud Run and Vercel. You can deploy Datasette to other hosting providers using the instructions on this page. | 76 | |
| 68 | 68 | Deployment fundamentals | Datasette can be deployed as a single datasette process that listens on a port. Datasette is not designed to be run as root, so that process should listen on a higher port such as port 8000. If you want to serve Datasette on port 80 (the HTTP default port) or port 443 (for HTTPS) you should run it behind a proxy server, such as nginx, Apache or HAProxy. The proxy server can listen on port 80/443 and forward traffic on to Datasette. | 76 | |
| 69 | 69 | Running Datasette using systemd | You can run Datasette on Ubuntu or Debian systems using systemd . First, ensure you have Python 3 and pip installed. On Ubuntu you can use sudo apt-get install python3 python3-pip . You can install Datasette into a virtual environment, or you can install it system-wide. To install system-wide, use sudo pip3 install datasette . Now create a folder for your Datasette databases, for example using mkdir /home/ubuntu/datasette-root . You can copy a test database into that folder like so: cd /home/ubuntu/datasette-root curl -O https://latest.datasette.io/fixtures.db Create a file at /etc/systemd/system/datasette.service with the following contents: [Unit] Description=Datasette After=network.target [Service] Type=simple User=ubuntu Environment=DATASETTE_SECRET= WorkingDirectory=/home/ubuntu/datasette-root ExecStart=datasette serve . -h 127.0.0.1 -p 8000 Restart=on-failure [Install] WantedBy=multi-user.target Add a random value for the DATASETTE_SECRET - this will be used to sign Datasette cookies such as the CSRF token cookie. You can generate a suitable value like so: python3 -c 'import secrets; print(secrets.token_hex(32))' This configuration will run Datasette against all database files contained in the /home/ubuntu/datasette-root directory. If that directory contains a metadata.yml (or .json ) file or a templates/ or plugins/ sub-directory those will automatically be loaded by Datasette - see Configuration directory mode for details. You can start the Datasette process running using the following: sudo systemctl daemon-reload sudo systemctl start datasette.service You will need to restart the Datasette service after making changes to its metadata.json configuration or adding a new database file to that directory. You can do that using: sudo systemctl restart datasette.service Once the … | 76 | |
| 70 | 70 | Running Datasette using OpenRC | OpenRC is the service manager on non-systemd Linux distributions like Alpine Linux and Gentoo . Create an init script at /etc/init.d/datasette with the following contents: #!/sbin/openrc-run name="datasette" command="datasette" command_args="serve -h 0.0.0.0 /path/to/db.db" command_background=true pidfile="/run/${RC_SVCNAME}.pid" You then need to configure the service to run at boot and start it: rc-update add datasette rc-service datasette start | 76 | |
| 71 | 71 | Deploying using buildpacks | Some hosting providers such as Heroku , DigitalOcean App Platform and Scalingo support the Buildpacks standard for deploying Python web applications. Deploying Datasette on these platforms requires two files: requirements.txt and Procfile . The requirements.txt file lets the platform know which Python packages should be installed. It should contain datasette at a minimum, but can also list any Datasette plugins you wish to install - for example: datasette datasette-vega The Procfile lets the hosting platform know how to run the command that serves web traffic. It should look like this: web: datasette . -h 0.0.0.0 -p $PORT --cors The $PORT environment variable is provided by the hosting platform. --cors enables CORS requests from JavaScript running on other websites to your domain - omit this if you don't want to allow CORS. You can add additional Datasette Settings options here too. These two files should be enough to deploy Datasette on any host that supports buildpacks. Datasette will serve any SQLite files that are included in the root directory of the application. If you want to build SQLite files or download them as part of the deployment process you can do so using a bin/post_compile file. For example, the following bin/post_compile will download an example database that will then be served by Datasette: wget https://fivethirtyeight.datasettes.com/fivethirtyeight.db simonw/buildpack-datasette-demo is an example GitHub repository showing a Datasette configuration that can be deployed to a buildpack-supporting host. | 76 | |
| 72 | 72 | Running Datasette behind a proxy | You may wish to run Datasette behind an Apache or nginx proxy, using a path within your existing site. You can use the base_url configuration setting to tell Datasette to serve traffic with a specific URL prefix. For example, you could run Datasette like this: datasette my-database.db --setting base_url /my-datasette/ -p 8009 This will run Datasette with the following URLs: http://127.0.0.1:8009/my-datasette/ - the Datasette homepage http://127.0.0.1:8009/my-datasette/my-database - the page for the my-database.db database http://127.0.0.1:8009/my-datasette/my-database/some_table - the page for the some_table table You can now set your nginx or Apache server to proxy the /my-datasette/ path to this Datasette instance. | 76 | |
| 73 | 73 | Nginx proxy configuration | Here is an example of an nginx configuration file that will proxy traffic to Datasette: daemon off; events { worker_connections 1024; } http { server { listen 80; location /my-datasette { proxy_pass http://127.0.0.1:8009/my-datasette; proxy_set_header Host $host; } } } You can also use the --uds option to Datasette to listen on a Unix domain socket instead of a port, configuring the nginx upstream proxy like this: daemon off; events { worker_connections 1024; } http { server { listen 80; location /my-datasette { proxy_pass http://datasette/my-datasette; proxy_set_header Host $host; } } upstream datasette { server unix:/tmp/datasette.sock; } } Then run Datasette with datasette --uds /tmp/datasette.sock path/to/database.db --setting base_url /my-datasette/ . | 76 | |
| 74 | 74 | Apache proxy configuration | For Apache , you can use the ProxyPass directive. First make sure the following lines are uncommented: LoadModule proxy_module lib/httpd/modules/mod_proxy.so LoadModule proxy_http_module lib/httpd/modules/mod_proxy_http.so Then add these directives to proxy traffic: ProxyPass /my-datasette/ http://127.0.0.1:8009/my-datasette/ ProxyPreserveHost On A live demo of Datasette running behind Apache using this proxy setup can be seen at datasette-apache-proxy-demo.datasette.io/prefix/ . The code for that demo can be found in the demos/apache-proxy directory. Using --uds you can use Unix domain sockets similar to the nginx example: ProxyPass /my-datasette/ unix:/tmp/datasette.sock|http://localhost/my-datasette/ The ProxyPreserveHost On directive ensures that the original Host: header from the incoming request is passed through to Datasette. Datasette needs this to correctly assemble links to other pages using the .absolute_url(request, path) method. | 76 | |
| 75 | 75 | Publishing data | Datasette includes tools for publishing and deploying your data to the internet. The datasette publish command will deploy a new Datasette instance containing your databases directly to a Heroku or Google Cloud hosting account. You can also use datasette package to create a Docker image that bundles your databases together with the datasette application that is used to serve them. | 76 | |
| 76 | 76 | datasette publish | Once you have created a SQLite database (e.g. using csvs-to-sqlite ) you can deploy it to a hosting account using a single command. You will need a hosting account with Heroku or Google Cloud . Once you have created your account you will need to install and configure the heroku or gcloud command-line tools. | 76 | |
| 77 | 77 | Publishing to Google Cloud Run | Google Cloud Run allows you to publish data in a scale-to-zero environment, so your application will start running when the first request is received and will shut down again when traffic ceases. This means you only pay for time spent serving traffic. Cloud Run is a great option for inexpensively hosting small, low traffic projects - but costs can add up for projects that serve a lot of requests. Be particularly careful if your project has tables with large numbers of rows. Search engine crawlers that index a page for every row could result in a high bill. The datasette-block-robots plugin can be used to request search engine crawlers omit crawling your site, which can help avoid this issue. You will first need to install and configure the Google Cloud CLI tools by following these instructions . You can then publish one or more SQLite database files to Google Cloud Run using the following command: datasette publish cloudrun mydatabase.db --service=my-database A Cloud Run service is a single hosted application. The service name you specify will be used as part of the Cloud Run URL. If you deploy to a service name that you have used in the past your new deployment will replace the previous one. If you omit the --service option you will be asked to pick a service name interactively during the deploy. You may need to interact with prompts from the tool. Many of the prompts ask for values that can be set as properties for the Google Cloud SDK if you want to avoid the prompts. For example, the default region for the deployed instance can be set using the command: gcloud config set run/region us-central1 You should replace us-central1 with your desired region . Alternately, you can specify the region by setting the CLOUDSDK_RUN_REGION environment… | 76 | |
| 78 | 78 | Publishing to Heroku | To publish your data using Heroku , first create an account there and install and configure the Heroku CLI tool . You can publish one or more databases to Heroku using the following command: datasette publish heroku mydatabase.db This will output some details about the new deployment, including a URL like this one: https://limitless-reef-88278.herokuapp.com/ deployed to Heroku You can specify a custom app name by passing -n my-app-name to the publish command. This will also allow you to overwrite an existing app. Rather than deploying directly you can use the --generate-dir option to output the files that would be deployed to a directory: datasette publish heroku mydatabase.db --generate-dir=/tmp/deploy-this-to-heroku See datasette publish heroku for the full list of options for this command. | 76 | |
| 79 | 79 | Publishing to Vercel | Vercel - previously known as Zeit Now - provides a layer over AWS Lambda to allow for quick, scale-to-zero deployment. You can deploy Datasette instances to Vercel using the datasette-publish-vercel plugin. pip install datasette-publish-vercel datasette publish vercel mydatabase.db --project my-database-project Not every feature is supported: consult the datasette-publish-vercel README for more details. | 76 | |
| 80 | 80 | Publishing to Fly | Fly is a competitively priced Docker-compatible hosting platform that supports running applications in globally distributed data centers close to your end users. You can deploy Datasette instances to Fly using the datasette-publish-fly plugin. pip install datasette-publish-fly datasette publish fly mydatabase.db --app="my-app" Consult the datasette-publish-fly README for more details. | 76 | |
| 81 | 81 | Custom metadata and plugins | datasette publish accepts a number of additional options which can be used to further customize your Datasette instance. You can define your own Metadata and deploy that with your instance like so: datasette publish cloudrun --service=my-service mydatabase.db -m metadata.json If you just want to set the title, license or source information you can do that directly using extra options to datasette publish : datasette publish cloudrun mydatabase.db --service=my-service \ --title="Title of my database" \ --source="Where the data originated" \ --source_url="http://www.example.com/" You can also specify plugins you would like to install. For example, if you want to include the datasette-vega visualization plugin you can use the following: datasette publish cloudrun mydatabase.db --service=my-service --install=datasette-vega If a plugin has any Secret configuration values you can use the --plugin-secret option to set those secrets at publish time. For example, using Heroku with datasette-auth-github you might run the following command: datasette publish heroku my_database.db \ --name my-heroku-app-demo \ --install=datasette-auth-github \ --plugin-secret datasette-auth-github client_id your_client_id \ --plugin-secret datasette-auth-github client_secret your_client_secret | 76 | |
| 82 | 82 | datasette package | If you have docker installed (e.g. using Docker for Mac ) you can use the datasette package command to create a new Docker image in your local repository containing the datasette app bundled together with one or more SQLite databases: datasette package mydatabase.db Here's example output for the package command: datasette package parlgov.db --extra-options="--setting sql_time_limit_ms 2500" Sending build context to Docker daemon 4.459MB Step 1/7 : FROM python:3.11.0-slim-bullseye ---> 79e1dc9af1c1 Step 2/7 : COPY . /app ---> Using cache ---> cd4ec67de656 Step 3/7 : WORKDIR /app ---> Using cache ---> 139699e91621 Step 4/7 : RUN pip install datasette ---> Using cache ---> 340efa82bfd7 Step 5/7 : RUN datasette inspect parlgov.db --inspect-file inspect-data.json ---> Using cache ---> 5fddbe990314 Step 6/7 : EXPOSE 8001 ---> Using cache ---> 8e83844b0fed Step 7/7 : CMD datasette serve parlgov.db --port 8001 --inspect-file inspect-data.json --setting sql_time_limit_ms 2500 ---> Using cache ---> 1bd380ea8af3 Successfully built 1bd380ea8af3 You can now run the resulting container like so: docker run -p 8081:8001 1bd380ea8af3 This exposes port 8001 inside the container as port 8081 on your host machine, so you can access the application at http://localhost:8081/ You can customize the port that is exposed by the container using the --port option: datasette package mydatabase.db --port 8080 A full list of options can be seen by running datasette package --help : See datasette package for the full list of options for this command. | 76 | |
| 83 | 83 | Facets | Datasette facets can be used to add a faceted browse interface to any database table. With facets, tables are displayed along with a summary showing the most common values in specified columns. These values can be selected to further filter the table. Here's an example : Facets can be specified in two ways: using query string parameters, or in metadata.json configuration for the table. | 76 | |
| 84 | 84 | Facets in query strings | To turn on faceting for specific columns on a Datasette table view, add one or more _facet=COLUMN parameters to the URL. For example, if you want to turn on facets for the city_id and state columns, construct a URL that looks like this: /dbname/tablename?_facet=state&_facet=city_id This works for both the HTML interface and the .json view. When enabled, facets will cause a facet_results block to be added to the JSON output, looking something like this: { "state": { "name": "state", "results": [ { "value": "CA", "label": "CA", "count": 10, "toggle_url": "http://...?_facet=city_id&_facet=state&state=CA", "selected": false }, { "value": "MI", "label": "MI", "count": 4, "toggle_url": "http://...?_facet=city_id&_facet=state&state=MI", "selected": false }, { "value": "MC", "label": "MC", "count": 1, "toggle_url": "http://...?_facet=city_id&_facet=state&state=MC", "selected": false } ], "truncated": false } "city_id": { "name": "city_id", "results": [ { "value": 1, "label": "San Francisco", "count": 6, "toggle_url": "http://...?_facet=city_id&_facet=state&city_id=1", "selected": false }, { "value": 2, "label": "Los Angeles", "count": 4, "toggle_url": "http://...?_facet=city_id&_facet=state&city_id=2", "selected": false }, { "value": 3, "label": "Detroit", "count": 4, "toggle_url": "http://...?_facet=city_id&_facet=state&city_id=3", "selected": false }, { "value": 4, "label": "Memnonia", "count": 1, "toggle_url": "http://...?_facet=city_id&_facet=state&city_id=4", "selected": false } ], "truncated": false } } If Datasette detect… | 76 | |
| 85 | 85 | Facets in configuration | You can turn facets on by default for specific tables by adding a "facets" key to the table configuration in datasette.yaml . See also the table configuration reference for a quick overview. Here's an example that turns on faceting by default for the qLegalStatus column in the Street_Tree_List table in the sf-trees database: [[[cog from metadata_doc import config_example config_example(cog, { "databases": { "sf-trees": { "tables": { "Street_Tree_List": { "facets": ["qLegalStatus"] } } } } }) ]]] [[[end]]] Facets defined in this way will always be shown in the interface and returned in the API, regardless of the _facet arguments passed to the view. Facets defined in configuration will be displayed in the order they are listed. Any additional facets added via query string parameters (e.g. ?_facet=column_name ) will appear after the configured facets, sorted by the number of unique values. You can specify array or date facets using JSON objects with a single key of array or date and a value specifying the column, like this: [[[cog config_example(cog, { "facets": [ {"array": "tags"}, {"date": "created"} ] }) ]]] [[[end]]] You can change the default facet size (the number of results shown for each facet) for a table using facet_size : [[[cog config_example(cog, { "databases": { "sf-trees": { "tables": { "Street_Tree_List": { "facets": ["qLegalStatus"], "facet_size": 10 } } } } }) ]]] [[[end]]] | 76 | |
| 86 | 86 | Suggested facets | Datasette's table UI will suggest facets for the user to apply, based on the following criteria: For the currently filtered data are there any columns which, if applied as a facet... Will return 30 or less unique options Will return more than one unique option Will return less unique options than the total number of filtered rows And the query used to evaluate this criteria can be completed in under 50ms That last point is particularly important: Datasette runs a query for every column that is displayed on a page, which could get expensive - so to avoid slow load times it sets a time limit of just 50ms for each of those queries. This means suggested facets are unlikely to appear for tables with millions of records in them. | 76 | |
| 87 | 87 | Speeding up facets with indexes | The performance of facets can be greatly improved by adding indexes on the columns you wish to facet by. Adding indexes can be performed using the sqlite3 command-line utility. Here's how to add an index on the state column in a table called Food_Trucks : sqlite3 mydatabase.db SQLite version 3.19.3 2017-06-27 16:48:08 Enter ".help" for usage hints. sqlite> CREATE INDEX Food_Trucks_state ON Food_Trucks("state"); Or using the sqlite-utils command-line utility: sqlite-utils create-index mydatabase.db Food_Trucks state | 76 | |
| 88 | 88 | Facet by JSON array | If your SQLite installation provides the json1 extension (you can check using /-/versions ) Datasette will automatically detect columns that contain JSON arrays of values and offer a faceting interface against those columns. This is useful for modelling things like tags without needing to break them out into a new table. Example here: latest.datasette.io/fixtures/facetable?_facet_array=tags | 76 | |
| 89 | 89 | Facet by date | If Datasette finds any columns that contain dates in the first 100 values, it will offer a faceting interface against the dates of those values. This works especially well against timestamp values such as 2019-03-01 12:44:00 . Example here: latest.datasette.io/fixtures/facetable?_facet_date=created | 76 | |
| 90 | 90 | Upgrade guide | 76 | ||
| 91 | 91 | Datasette 0.X -> 1.0 | This section reviews breaking changes Datasette 1.0 has when upgrading from a 0.XX version. For new features that 1.0 offers, see the Changelog . | 76 | |
| 92 | 92 | New URL for SQL queries | Prior to 1.0a14 the URL for executing a SQL query looked like this: /databasename?sql=select+1 # Or for JSON: /databasename.json?sql=select+1 This endpoint served two purposes: without a ?sql= it would list the tables in the database, but with that option it would return results of a query instead. The URL for executing a SQL query now looks like this: /databasename/-/query?sql=select+1 # Or for JSON: /databasename/-/query.json?sql=select+1 This isn't a breaking change. API calls to the older /databasename?sql=... endpoint will redirect to the new databasename/-/query?sql=... endpoint. Upgrading to the new URL is recommended to avoid the overhead of the additional redirect. | 76 | |
| 93 | 93 | Metadata changes | Metadata was completely revamped for Datasette 1.0. There are a number of related breaking changes, from the metadata.yaml file to Python APIs, that you'll need to consider when upgrading. | 76 | |
| 94 | 94 | Before Datasette 1.0, the metadata.yaml file became a kitchen sink if a mix of metadata, configuration, and settings. Now metadata.yaml is strictly for metadata (ex title and descriptions of database and tables, licensing info, etc). Other settings have been moved to a datasette.yml configuration file, described in Configuration . To start Datasette with both metadata and configuration files, run it like this: datasette --metadata metadata.yaml --config datasette.yaml # Or the shortened version: datasette -m metadata.yml -c datasette.yml | 76 | ||
| 95 | 95 | Upgrading an existing | The datasette-upgrade plugin can be used to split a Datasette 0.x.x metadata.yaml (or .json ) file into separate metadata.yaml and datasette.yaml files. First, install the plugin: datasette install datasette-upgrade Then run it like this to produce the two new files: datasette upgrade metadata-to-config metadata.json -m metadata.yml -c datasette.yml | 76 | |
| 96 | 96 | Metadata "fallback" has been removed | Certain keys in metadata like license used to "fallback" up the chain of ownership. For example, if you set an MIT to a database and a table within that database did not have a specified license, then that table would inherit an MIT license. This behavior has been removed in Datasette 1.0. Now license fields must be placed on all items, including individual databases and tables. | 76 | |
| 97 | 97 | The | In Datasette 0.x plugins could implement a get_metadata() plugin hook to customize how metadata was retrieved for different instances, databases and tables. This hook could be inefficient, since some pages might load metadata for many different items (to list a large number of tables, for example) which could result in a large number of calls to potentially expensive plugin hook implementations. As of Datasette 1.0a14 (2024-08-05), the get_metadata() hook has been deprecated: # ❌ DEPRECATED in Datasette 1.0 @hookimpl def get_metadata(datasette, key, database, table): pass Instead, plugins are encouraged to interact directly with Datasette's in-memory metadata tables in SQLite using the following methods on the Datasette class : get_instance_metadata() and set_instance_metadata() get_database_metadata() and set_database_metadata() get_resource_metadata() and set_resource_metadata() get_column_metadata() and set_column_metadata() A plugin that stores or calculates its own metadata can implement the startup(datasette) hook to populate those items on startup, and then call those methods while it is running to persist any new metadata changes. | 76 | |
| 98 | 98 | The | As of Datasette 1.0a14 , the root level /metadata.json endpoint has been removed. Metadata for tables will become available through currently in-development extras in a future alpha. | 76 | |
| 99 | 99 | The | As of Datasette 1.0a14 , the .metadata() method on the Datasette Python API has been removed. Instead, one should use the following methods on a Datasette class: get_instance_metadata() get_database_metadata() get_resource_metadata() get_column_metadata() | 76 | |
| 100 | 100 | Datasette 1.0a20 plugin upgrade guide | Datasette 1.0a20 makes some breaking changes to Datasette's permission system. Plugins need to be updated if they use any of the following : The register_permissions() plugin hook - this should be replaced with register_actions The permission_allowed() plugin hook - this should be upgraded to use permission_resources_sql() . The datasette.permission_allowed() internal method - this should be replaced with datasette.allowed() Logic that grants access to the "root" actor can be removed. | 76 |
Advanced export
JSON shape: default, array, newline-delimited
CREATE VIRTUAL TABLE [sections_fts] USING FTS5 (
[title], [content],
tokenize='porter',
content=[sections]
);