sections
582 rows sorted by content
This data as json, CSV (advanced)
id | page | ref | title | content ▼ | breadcrumbs | references |
---|---|---|---|---|---|---|
binary_data:binary-plugins | binary_data | binary-plugins | Binary plugins | Several Datasette plugins are available that change the way Datasette treats binary data. datasette-render-binary modifies Datasette's default interface to show an automatic guess at what type of binary data is being stored, along with a visual representation of the binary value that displays ASCII strings directly in the interface. datasette-render-images detects common image formats and renders them as images directly in the Datasette interface. datasette-media allows Datasette interfaces to be configured to serve binary files from configured SQL queries, and includes the ability to resize images directly before serving them. | ["Binary data"] | [{"href": "https://github.com/simonw/datasette-render-binary", "label": "datasette-render-binary"}, {"href": "https://github.com/simonw/datasette-render-images", "label": "datasette-render-images"}, {"href": "https://github.com/simonw/datasette-media", "label": "datasette-media"}] |
settings:setting-suggest-facets | settings | setting-suggest-facets | suggest_facets | Should Datasette calculate suggested facets? On by default, turn this off like so: datasette mydatabase.db --setting suggest_facets off | ["Settings", "Settings"] | [] |
settings:setting-allow-signed-tokens | settings | setting-allow-signed-tokens | allow_signed_tokens | Should users be able to create signed API tokens to access Datasette? This is turned on by default. Use the following to turn it off: datasette mydatabase.db --setting allow_signed_tokens off Turning this setting off will disable the /-/create-token page, described here . It will also cause any incoming Authorization: Bearer dstok_... API tokens to be ignored. | ["Settings", "Settings"] | [] |
settings:setting-allow-download | settings | setting-allow-download | allow_download | Should users be able to download the original SQLite database using a link on the database index page? This is turned on by default. However, databases can only be downloaded if they are served in immutable mode and not in-memory. If downloading is unavailable for either of these reasons, the download link is hidden even if allow_download is on. To disable database downloads, use the following: datasette mydatabase.db --setting allow_download off | ["Settings", "Settings"] | [] |
settings:setting-default-allow-sql | settings | setting-default-allow-sql | default_allow_sql | Should users be able to execute arbitrary SQL queries by default? Setting this to off causes permission checks for execute-sql to fail by default. datasette mydatabase.db --setting default_allow_sql off Another way to achieve this is to add "allow_sql": false to your datasette.yaml file, as described in Controlling the ability to execute arbitrary SQL . This setting offers a more convenient way to do this. | ["Settings", "Settings"] | [] |
changelog:id39 | changelog | id39 | 0.52.4 (2020-12-05) | Show pysqlite3 version on /-/versions , if installed. ( #1125 ) Errors output by Datasette (e.g. for invalid SQL queries) now go to stderr , not stdout . ( #1131 ) Fix for a startup error on windows caused by unnecessary from os import EX_CANTCREAT - thanks, Abdussamet Koçak. ( #1094 ) | ["Changelog"] | [{"href": "https://github.com/coleifer/pysqlite3", "label": "pysqlite3"}, {"href": "https://github.com/simonw/datasette/issues/1125", "label": "#1125"}, {"href": "https://github.com/simonw/datasette/issues/1131", "label": "#1131"}, {"href": "https://github.com/simonw/datasette/issues/1094", "label": "#1094"}] |
cli-reference:cli-help-publish-help | cli-reference | cli-help-publish-help | datasette publish | Shows a list of available deployment targets for publishing data with Datasette. Additional deployment targets can be added by plugins that use the publish_subcommand(publish) hook. [[[cog help(["publish", "--help"]) ]]] Usage: datasette publish [OPTIONS] COMMAND [ARGS]... Publish specified SQLite database files to the internet along with a Datasette-powered interface and API Options: --help Show this message and exit. Commands: cloudrun Publish databases to Datasette running on Cloud Run heroku Publish databases to Datasette running on Heroku [[[end]]] | ["CLI reference"] | [] |
introspection:jsondataview-plugins | introspection | jsondataview-plugins | /-/plugins | Shows a list of currently installed plugins and their versions. Plugins example : [ { "name": "datasette_cluster_map", "static": true, "templates": false, "version": "0.10", "hooks": ["extra_css_urls", "extra_js_urls", "extra_body_script"] } ] Add ?all=1 to include details of the default plugins baked into Datasette. | ["Introspection"] | [{"href": "https://san-francisco.datasettes.com/-/plugins", "label": "Plugins example"}] |
introspection:jsondataview-databases | introspection | jsondataview-databases | /-/databases | Shows currently attached databases. Databases example : [ { "hash": null, "is_memory": false, "is_mutable": true, "name": "fixtures", "path": "fixtures.db", "size": 225280 } ] | ["Introspection"] | [{"href": "https://latest.datasette.io/-/databases", "label": "Databases example"}] |
introspection:jsondataview-threads | introspection | jsondataview-threads | /-/threads | Shows details of threads and asyncio tasks. Threads example : { "num_threads": 2, "threads": [ { "daemon": false, "ident": 4759197120, "name": "MainThread" }, { "daemon": true, "ident": 123145319682048, "name": "Thread-1" }, ], "num_tasks": 3, "tasks": [ "<Task pending coro=<RequestResponseCycle.run_asgi() running at uvicorn/protocols/http/httptools_impl.py:385> cb=[set.discard()]>", "<Task pending coro=<Server.serve() running at uvicorn/main.py:361> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10365c3d0>()]> cb=[run_until_complete.<locals>.<lambda>()]>", "<Task pending coro=<LifespanOn.main() running at uvicorn/lifespan/on.py:48> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10364f050>()]>>" ] } | ["Introspection"] | [{"href": "https://latest.datasette.io/-/threads", "label": "Threads example"}] |
introspection:jsondataview-settings | introspection | jsondataview-settings | /-/settings | Shows the Settings for this instance of Datasette. Settings example : { "default_facet_size": 30, "default_page_size": 100, "facet_suggest_time_limit_ms": 50, "facet_time_limit_ms": 1000, "max_returned_rows": 1000, "sql_time_limit_ms": 1000 } | ["Introspection"] | [{"href": "https://fivethirtyeight.datasettes.com/-/settings", "label": "Settings example"}] |
introspection:jsondataview-config | introspection | jsondataview-config | /-/config | Shows the configuration for this instance of Datasette. This is generally the contents of the datasette.yaml or datasette.json file, which can include plugin configuration as well. Config example : { "settings": { "template_debug": true, "trace_debug": true, "force_https_urls": true } } Any keys that include the one of the following substrings in their names will be returned as redacted *** output, to help avoid accidentally leaking private configuration information: secret , key , password , token , hash , dsn . | ["Introspection"] | [{"href": "https://latest.datasette.io/-/config", "label": "Config example"}] |
introspection:jsondataview-metadata | introspection | jsondataview-metadata | /-/metadata | Shows the contents of the metadata.json file that was passed to datasette serve , if any. Metadata example : { "license": "CC Attribution 4.0 License", "license_url": "http://creativecommons.org/licenses/by/4.0/", "source": "fivethirtyeight/data on GitHub", "source_url": "https://github.com/fivethirtyeight/data", "title": "Five Thirty Eight", "databases": { } } | ["Introspection"] | [{"href": "https://fivethirtyeight.datasettes.com/-/metadata", "label": "Metadata example"}] |
introspection:jsondataview-actor | introspection | jsondataview-actor | /-/actor | Shows the currently authenticated actor. Useful for debugging Datasette authentication plugins. { "actor": { "id": 1, "username": "some-user" } } | ["Introspection"] | [] |
introspection:jsondataview-versions | introspection | jsondataview-versions | /-/versions | Shows the version of Datasette, Python and SQLite. Versions example : { "datasette": { "version": "0.60" }, "python": { "full": "3.8.12 (default, Dec 21 2021, 10:45:09) \n[GCC 10.2.1 20210110]", "version": "3.8.12" }, "sqlite": { "extensions": { "json1": null }, "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "compile_options": [ "COMPILER=gcc-6.3.0 20170516", "ENABLE_FTS3", "ENABLE_FTS4", "ENABLE_FTS5", "ENABLE_JSON1", "ENABLE_RTREE", "THREADSAFE=1" ], "version": "3.37.0" } } | ["Introspection"] | [{"href": "https://latest.datasette.io/-/versions", "label": "Versions example"}] |
changelog:csrf-protection | changelog | csrf-protection | CSRF protection | Since writable canned queries are built using POST forms, Datasette now ships with CSRF protection ( #798 ). This applies automatically to any POST request, which means plugins need to include a csrftoken in any POST forms that they render. They can do that like so: <input type="hidden" name="csrftoken" value="{{ csrftoken() }}"> | ["Changelog", "0.44 (2020-06-11)"] | [{"href": "https://github.com/simonw/datasette/issues/798", "label": "#798"}] |
deploying:deploying-buildpacks | deploying | deploying-buildpacks | Deploying using buildpacks | Some hosting providers such as Heroku , DigitalOcean App Platform and Scalingo support the Buildpacks standard for deploying Python web applications. Deploying Datasette on these platforms requires two files: requirements.txt and Procfile . The requirements.txt file lets the platform know which Python packages should be installed. It should contain datasette at a minimum, but can also list any Datasette plugins you wish to install - for example: datasette datasette-vega The Procfile lets the hosting platform know how to run the command that serves web traffic. It should look like this: web: datasette . -h 0.0.0.0 -p $PORT --cors The $PORT environment variable is provided by the hosting platform. --cors enables CORS requests from JavaScript running on other websites to your domain - omit this if you don't want to allow CORS. You can add additional Datasette Settings options here too. These two files should be enough to deploy Datasette on any host that supports buildpacks. Datasette will serve any SQLite files that are included in the root directory of the application. If you want to build SQLite files or download them as part of the deployment process you can do so using a bin/post_compile file. For example, the following bin/post_compile will download an example database that will then be served by Datasette: wget https://fivethirtyeight.datasettes.com/fivethirtyeight.db simonw/buildpack-datasette-demo is an example GitHub repository showing a Datasette configuration that can be deployed to a buildpack-supporting host. | ["Deploying Datasette"] | [{"href": "https://www.heroku.com/", "label": "Heroku"}, {"href": "https://www.digitalocean.com/docs/app-platform/", "label": "DigitalOcean App Platform"}, {"href": "https://scalingo.com/", "label": "Scalingo"}, {"href": "https://buildpacks.io/", "label": "Buildpacks standard"}, {"href": "https://github.com/simonw/buildpack-datasette-demo", "label": "simonw/buildpack-datasette-demo"}] |
contributing:contributing-documentation-cog | contributing | contributing-documentation-cog | Running Cog | Some pages of documentation (in particular the CLI reference ) are automatically updated using Cog . To update these pages, run the following command: cog -r docs/*.rst | ["Contributing", "Editing and building the documentation"] | [{"href": "https://github.com/nedbat/cog", "label": "Cog"}] |
plugins:plugins-configuration-secret | plugins | plugins-configuration-secret | Secret configuration values | Some plugins may need configuration that should stay secret - API keys for example. There are two ways in which you can store secret configuration values. As environment variables . If your secret lives in an environment variable that is available to the Datasette process, you can indicate that the configuration value should be read from that environment variable like so: [[[cog config_example(cog, { "plugins": { "datasette-auth-github": { "client_secret": { "$env": "GITHUB_CLIENT_SECRET" } } } }) ]]] [[[end]]] As values in separate files . Your secrets can also live in files on disk. To specify a secret should be read from a file, provide the full file path like this: [[[cog config_example(cog, { "plugins": { "datasette-auth-github": { "client_secret": { "$file": "/secrets/client-secret" } } } }) ]]] [[[end]]] If you are publishing your data using the datasette publish family of commands, you can use the --plugin-secret option to set these secrets at publish time. For example, using Heroku you might run the following command: datasette publish heroku my_database.db \ --name my-heroku-app-demo \ --install=datasette-auth-github \ --plugin-secret datasette-auth-github client_id your_client_id \ --plugin-secret datasette-auth-github client_secret your_client_secret This will set the necessary environment variables and add the following to the deployed metadata.yaml : [[[cog config_example(cog, { "plugins": { "datasette-auth-github": { "client_id": { "$env": "DATASETTE_AUTH_GITHUB_CLIENT_ID" }, "client_secret": { "$env": "DATASETTE_AUTH_GITHUB_CLIENT_SECRET" } … | ["Plugins", "Plugin configuration"] | [] |
sql_queries:fragment | sql_queries | fragment | fragment | Some plugins, such as datasette-vega , can be configured by including additional data in the fragment hash of the URL - the bit that comes after a # symbol. You can set a default fragment hash that will be included in the link to the canned query from the database index page using the "fragment" key. This example demonstrates both fragment and hide_sql : [[[cog config_example(cog, """ databases: fixtures: queries: neighborhood_search: fragment: fragment-goes-here hide_sql: true sql: |- select neighborhood, facet_cities.name, state from facetable join facet_cities on facetable.city_id = facet_cities.id where neighborhood like '%' || :text || '%' order by neighborhood; """) ]]] [[[end]]] See here for a demo of this in action. | ["Running SQL queries", "Canned queries", "Additional canned query options"] | [{"href": "https://github.com/simonw/datasette-vega", "label": "datasette-vega"}, {"href": "https://latest.datasette.io/fixtures#queries", "label": "See here"}] |
pages:databaseview-hidden | pages | databaseview-hidden | Hidden tables | Some tables listed on the database page are treated as hidden. Hidden tables are not completely invisible - they can be accessed through the "hidden tables" link at the bottom of the page. They are hidden because they represent low-level implementation details which are generally not useful to end-users of Datasette. The following tables are hidden by default: Any table with a name that starts with an underscore - this is a Datasette convention to help plugins easily hide their own internal tables. Tables that have been configured as "hidden": true using Hiding tables . *_fts tables that implement SQLite full-text search indexes. Tables relating to the inner workings of the SpatiaLite SQLite extension. sqlite_stat tables used to store statistics used by the query optimizer. | ["Pages and API endpoints", "Database"] | [] |
spatialite:installing-spatialite-on-linux | spatialite | installing-spatialite-on-linux | Installing SpatiaLite on Linux | SpatiaLite is packaged for most Linux distributions. apt install spatialite-bin libsqlite3-mod-spatialite Depending on your distribution, you should be able to run Datasette something like this: datasette --load-extension=/usr/lib/x86_64-linux-gnu/mod_spatialite.so If you are unsure of the location of the module, try running locate mod_spatialite and see what comes back. | ["SpatiaLite", "Installation"] | [] |
spatialite:making-use-of-a-spatial-index | spatialite | making-use-of-a-spatial-index | Making use of a spatial index | SpatiaLite spatial indexes are R*Trees. They allow you to run efficient bounding box queries using a sub-select, with a similar pattern to that used for Searches using custom SQL . In the above example, the resulting index will be called idx_museums_point_geom . This takes the form of a SQLite virtual table. You can inspect its contents using the following query: select * from idx_museums_point_geom limit 10; Here's a live example: timezones-api.datasette.io/timezones/idx_timezones_Geometry pkid xmin xmax ymin ymax 1 -8.601725578308105 -2.4930307865142822 4.162120819091797 10.74019718170166 2 … | ["SpatiaLite"] | [{"href": "https://timezones-api.datasette.io/timezones/idx_timezones_Geometry", "label": "timezones-api.datasette.io/timezones/idx_timezones_Geometry"}] |
changelog:id33 | changelog | id33 | 0.55 (2021-02-18) | Support for cross-database SQL queries and built-in support for serving via HTTPS. The new --crossdb command-line option causes Datasette to attach up to ten database files to the same /_memory database connection. This enables cross-database SQL queries, including the ability to use joins and unions to combine data from tables that exist in different database files. See Cross-database queries for details. ( #283 ) --ssl-keyfile and --ssl-certfile options can be used to specify a TLS certificate, allowing Datasette to serve traffic over https:// without needing to run it behind a separate proxy. ( #1221 ) The /:memory: page has been renamed (and redirected) to /_memory for consistency with the new /_internal database introduced in Datasette 0.54. ( #1205 ) Added plugin testing documentation on Using pdb for errors thrown inside Datasette . ( #1207 ) The official Datasette Docker image now uses Python 3.7.10, applying the latest security fix for that Python version. ( #1235 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/283", "label": "#283"}, {"href": "https://github.com/simonw/datasette/issues/1221", "label": "#1221"}, {"href": "https://github.com/simonw/datasette/issues/1205", "label": "#1205"}, {"href": "https://github.com/simonw/datasette/issues/1207", "label": "#1207"}, {"href": "https://hub.docker.com/r/datasetteproject/datasette", "label": "official Datasette Docker image"}, {"href": "https://www.python.org/downloads/release/python-3710/", "label": "the latest security fix"}, {"href": "https://github.com/simonw/datasette/issues/1235", "label": "#1235"}] |
full_text_search:full-text-search-table-view-api | full_text_search | full-text-search-table-view-api | The table page and table view API | Table views that support full-text search can be queried using the ?_search=TERMS query string parameter. This will run the search against content from all of the columns that have been included in the index. Try this example: fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort SQLite full-text search supports wildcards. This means you can easily implement prefix auto-complete by including an asterisk at the end of the search term - for example: /dbname/tablename/?_search=rob* This will return all records containing at least one word that starts with the letters rob . You can also run searches against just the content of a specific named column by using _search_COLNAME=TERMS - for example, this would search for just rows where the name column in the FTS index mentions Sarah : /dbname/tablename/?_search_name=Sarah | ["Full-text search"] | [{"href": "https://fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort", "label": "fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort"}] |
changelog:v1-0-a11 | changelog | v1-0-a11 | 1.0a11 (2024-02-19) | The "replace": true argument to the /db/table/-/insert API now requires the actor to have the update-row permission. ( #2279 ) Fixed some UI bugs in the interactive permissions debugging tool. ( #2278 ) The column action menu now aligns better with the cog icon, and positions itself taking into account the width of the browser window. ( #2263 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/2279", "label": "#2279"}, {"href": "https://github.com/simonw/datasette/issues/2278", "label": "#2278"}, {"href": "https://github.com/simonw/datasette/issues/2263", "label": "#2263"}] |
cli-reference:cli-datasette-get | cli-reference | cli-datasette-get | datasette --get | The --get option to datasette serve (or just datasette ) specifies the path to a page within Datasette and causes Datasette to output the content from that path without starting the web server. This means that all of Datasette's functionality can be accessed directly from the command-line. For example: datasette --get '/-/versions.json' | jq . { "python": { "version": "3.8.5", "full": "3.8.5 (default, Jul 21 2020, 10:48:26) \n[Clang 11.0.3 (clang-1103.0.32.62)]" }, "datasette": { "version": "0.46+15.g222a84a.dirty" }, "asgi": "3.0", "uvicorn": "0.11.8", "sqlite": { "version": "3.32.3", "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "extensions": { "json1": null }, "compile_options": [ "COMPILER=clang-11.0.3", "ENABLE_COLUMN_METADATA", "ENABLE_FTS3", "ENABLE_FTS3_PARENTHESIS", "ENABLE_FTS4", "ENABLE_FTS5", "ENABLE_GEOPOLY", "ENABLE_JSON1", "ENABLE_PREUPDATE_HOOK", "ENABLE_RTREE", "ENABLE_SESSION", "MAX_VARIABLE_NUMBER=250000", "THREADSAFE=1" ] } } You can use the --token TOKEN option to send an API token with the simulated request. Or you can make a request as a specific actor by passing a JSON representation of that actor to --actor : datasette --memory --actor '{"id": "root"}' --get '/-/actor.json' The exit code of datasette --get will be 0 if the request succeeds and 1 if the request produced an HTTP status code other than 200 - e.g. a 404 or 500 error. This lets you use datasette --get / to run tests against a Datasette application in a continuous integration environment such as GitHub Actions. | ["CLI reference", "datasette serve"] | [] |
binary_data:binary-linking | binary_data | binary-linking | Linking to binary downloads | The .blob output format is used to return binary data. It requires a _blob_column= query string argument specifying which BLOB column should be downloaded, for example: https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data This output format can also be used to return binary data from an arbitrary SQL query. Since such queries do not specify an exact row, an additional ?_blob_hash= parameter can be used to specify the SHA-256 hash of the value that is being linked to. Consider the query select data from binary_data - demonstrated here . That page links to the binary value downloads. Those links look like this: https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d These .blob links are also returned in the .csv exports Datasette provides for binary tables and queries, since the CSV format does not have a mechanism for representing binary data. | ["Binary data"] | [{"href": "https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data", "label": "https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data"}, {"href": "https://latest.datasette.io/fixtures?sql=select+data+from+binary_data", "label": "demonstrated here"}, {"href": "https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d", "label": "https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d"}] |
authentication:allowdebugview | authentication | allowdebugview | The /-/allow-debug tool | The /-/allow-debug tool lets you try out different "action" blocks against different "actor" JSON objects. You can try that out here: https://latest.datasette.io/-/allow-debug | ["Authentication and permissions", "Permissions"] | [{"href": "https://latest.datasette.io/-/allow-debug", "label": "https://latest.datasette.io/-/allow-debug"}] |
changelog:permissions-fix-for-the-upsert-api | changelog | permissions-fix-for-the-upsert-api | Permissions fix for the upsert API | The /database/table/-/upsert API had a minor permissions bug, only affecting Datasette instances that had configured the insert-row and update-row permissions to apply to a specific table rather than the database or instance as a whole. Full details in issue #2262 . To avoid similar mistakes in the future the datasette.permission_allowed() method now specifies default= as a keyword-only argument. | ["Changelog", "1.0a9 (2024-02-16)"] | [{"href": "https://github.com/simonw/datasette/issues/2262", "label": "#2262"}] |
writing_plugins:writing-plugins-tracing | writing_plugins | writing-plugins-tracing | Tracing plugin hooks | The DATASETTE_TRACE_PLUGINS environment variable turns on detailed tracing showing exactly which hooks are being run. This can be useful for understanding how Datasette is using your plugin. DATASETTE_TRACE_PLUGINS=1 datasette mydb.db Example output: actor_from_request: { 'datasette': <datasette.app.Datasette object at 0x100bc7220>, 'request': <asgi.Request method="GET" url="http://127.0.0.1:4433/">} Hook implementations: [ <HookImpl plugin_name='codespaces', plugin=<module 'datasette_codespaces' from '.../site-packages/datasette_codespaces/__init__.py'>>, <HookImpl plugin_name='datasette.actor_auth_cookie', plugin=<module 'datasette.actor_auth_cookie' from '.../datasette/datasette/actor_auth_cookie.py'>>, <HookImpl plugin_name='datasette.default_permissions', plugin=<module 'datasette.default_permissions' from '.../datasette/default_permissions.py'>>] Results: [{'id': 'root'}] | ["Writing plugins"] | [] |
internals:internals-database-introspection | internals | internals-database-introspection | Database introspection | The Database class also provides properties and methods for introspecting the database. db.name - string The name of the database - usually the filename without the .db prefix. db.size - integer The size of the database file in bytes. 0 for :memory: databases. db.mtime_ns - integer or None The last modification time of the database file in nanoseconds since the epoch. None for :memory: databases. db.is_mutable - boolean Is this database mutable, and allowed to accept writes? db.is_memory - boolean Is this database an in-memory database? await db.attached_databases() - list of named tuples Returns a list of additional databases that have been connected to this database using the SQLite ATTACH command. Each named tuple has fields seq , name and file . await db.table_exists(table) - boolean Check if a table called table exists. await db.view_exists(view) - boolean … | ["Internals for plugins", "Database class"] | [] |
internals:database-constructor | internals | database-constructor | Database(ds, path=None, is_mutable=True, is_memory=False, memory_name=None) | The Database() constructor can be used by plugins, in conjunction with .add_database(db, name=None, route=None) , to create and register new databases. The arguments are as follows: ds - Datasette class (required) The Datasette instance you are attaching this database to. path - string Path to a SQLite database file on disk. is_mutable - boolean Set this to False to cause Datasette to open the file in immutable mode. is_memory - boolean Use this to create non-shared memory connections. memory_name - string or None Use this to create a named in-memory database. Unlike regular memory databases these can be accessed by multiple threads and will persist an changes made to them for the lifetime of the Datasette server process. The first argument is the datasette instance you are attaching to, the second is a path= , then is_mutable and is_memory are both optional arguments. | ["Internals for plugins", "Database class"] | [] |
changelog:id67 | changelog | id67 | 0.38 (2020-03-08) | The Docker build of Datasette now uses SQLite 3.31.1, upgraded from 3.26. ( #695 ) datasette publish cloudrun now accepts an optional --memory=2Gi flag for setting the Cloud Run allocated memory to a value other than the default (256Mi). ( #694 ) Fixed bug where templates that shipped with plugins were sometimes not being correctly loaded. ( #697 ) | ["Changelog"] | [{"href": "https://hub.docker.com/r/datasetteproject/datasette", "label": "Docker build"}, {"href": "https://github.com/simonw/datasette/issues/695", "label": "#695"}, {"href": "https://github.com/simonw/datasette/issues/694", "label": "#694"}, {"href": "https://github.com/simonw/datasette/issues/697", "label": "#697"}] |
changelog:alter-table-support-for-create-insert-upsert-and-update | changelog | alter-table-support-for-create-insert-upsert-and-update | Alter table support for create, insert, upsert and update | The JSON write API can now be used to apply simple alter table schema changes, provided the acting actor has the new alter-table permission. ( #2101 ) The only alter operation supported so far is adding new columns to an existing table. The /db/-/create API now adds new columns during large operations to create a table based on incoming example "rows" , in the case where one of the later rows includes columns that were not present in the earlier batches. This requires the create-table but not the alter-table permission. When /db/-/create is called with rows in a situation where the table may have been already created, an "alter": true key can be included to indicate that any missing columns from the new rows should be added to the table. This requires the alter-table permission. /db/table/-/insert and /db/table/-/upsert and /db/table/row-pks/-/update all now also accept "alter": true , depending on the alter-table permission. Operations that alter a table now fire the new alter-table event . | ["Changelog", "1.0a9 (2024-02-16)"] | [{"href": "https://github.com/simonw/datasette/issues/2101", "label": "#2101"}] |
internals:internals-response | internals | internals-response | Response class | The Response class can be returned from view functions that have been registered using the register_routes(datasette) hook. The Response() constructor takes the following arguments: body - string The body of the response. status - integer (optional) The HTTP status - defaults to 200. headers - dictionary (optional) A dictionary of extra HTTP headers, e.g. {"x-hello": "world"} . content_type - string (optional) The content-type for the response. Defaults to text/plain . For example: from datasette.utils.asgi import Response response = Response( "<xml>This is XML</xml>", content_type="application/xml; charset=utf-8", ) The quickest way to create responses is using the Response.text(...) , Response.html(...) , Response.json(...) or Response.redirect(...) helper methods: from datasette.utils.asgi import Response html_response = Response.html("This is HTML") json_response = Response.json({"this_is": "json"}) text_response = Response.text( "This will become utf-8 encoded text" ) # Redirects are served as 302, unless you pass status=301: redirect_response = Response.redirect( "https://latest.datasette.io/" ) Each of these responses will use the correct corresponding content-type - text/html; charset=utf-8 , application/json; charset=utf-8 or text/plain; charset=utf-8 respectively. Each of the helper methods take optional status= and headers= argument… | ["Internals for plugins"] | [] |
changelog:improved-support-for-spatialite | changelog | improved-support-for-spatialite | Improved support for SpatiaLite | The SpatiaLite module for SQLite adds robust geospatial features to the database. Getting SpatiaLite working can be tricky, especially if you want to use the most recent alpha version (with support for K-nearest neighbor). Datasette now includes extensive documentation on SpatiaLite , and thanks to Ravi Kotecha our GitHub repo includes a Dockerfile that can build the latest SpatiaLite and configure it for use with Datasette. The datasette publish and datasette package commands now accept a new --spatialite argument which causes them to install and configure SpatiaLite as part of the container they deploy. | ["Changelog", "0.23 (2018-06-18)"] | [{"href": "https://www.gaia-gis.it/fossil/libspatialite/index", "label": "SpatiaLite module"}, {"href": "https://github.com/r4vi", "label": "Ravi Kotecha"}, {"href": "https://github.com/simonw/datasette/blob/master/Dockerfile", "label": "Dockerfile"}] |
spatialite:id1 | spatialite | id1 | SpatiaLite | The SpatiaLite module for SQLite adds features for handling geographic and spatial data. For an example of what you can do with it, see the tutorial Building a location to time zone API with SpatiaLite . To use it with Datasette, you need to install the mod_spatialite dynamic library. This can then be loaded into Datasette using the --load-extension command-line option. Datasette can look for SpatiaLite in common installation locations if you run it like this: datasette --load-extension=spatialite --setting default_allow_sql off If SpatiaLite is in another location, use the full path to the extension instead: datasette --setting default_allow_sql off \ --load-extension=/usr/local/lib/mod_spatialite.dylib | [] | [{"href": "https://www.gaia-gis.it/fossil/libspatialite/index", "label": "SpatiaLite module"}, {"href": "https://datasette.io/tutorials/spatialite", "label": "Building a location to time zone API with SpatiaLite"}] |
json_api:json-api-shapes | json_api | json-api-shapes | Different shapes | The _shape parameter can be used to access alternative formats for the rows key which may be more convenient for your application. There are three options: ?_shape=objects - "rows" is a list of JSON key/value objects - the default ?_shape=arrays - "rows" is a list of lists, where the order of values in each list matches the order of the columns ?_shape=array - a JSON array of objects - effectively just the "rows" key from the default representation ?_shape=array&_nl=on - a newline-separated list of JSON objects ?_shape=arrayfirst - a flat JSON array containing just the first value from each row ?_shape=object - a JSON object keyed using the primary keys of the rows _shape=arrays looks like this: { "ok": true, "next": null, "rows": [ [3, "Detroit"], [2, "Los Angeles"], [4, "Memnonia"], [1, "San Francisco"] ] } _shape=array looks like this: [ { "id": 3, "name": "Detroit" }, { "id": 2, "name": "Los Angeles" }, { "id": 4, "name": "Memnonia" }, { "id": 1, "name": "San Francisco" } ] _shape=array&_nl=on looks like this: {"id": 1, "value": "Myoporum laetum :: Myoporum"} {"id": 2, "value": "Metrosideros excelsa :: New Zealand Xmas Tree"} {"id": 3, "value": "Pinus radiata :: Monterey Pine"} _shape=arrayfirst looks like this: [1, 2, 3] _shape=object looks like this: { "1": { "id": 1, "value": "Myoporum laetum :: Myoporum" }, "2": { "id": 2, "value": "Metrosideros excelsa :… | ["JSON API"] | [] |
changelog:new-plugin-hook-asgi-wrapper | changelog | new-plugin-hook-asgi-wrapper | New plugin hook: asgi_wrapper | The asgi_wrapper(datasette) plugin hook allows plugins to entirely wrap the Datasette ASGI application in their own ASGI middleware. ( #520 ) Two new plugins take advantage of this hook: datasette-auth-github adds a authentication layer: users will have to sign in using their GitHub account before they can view data or interact with Datasette. You can also use it to restrict access to specific GitHub users, or to members of specified GitHub organizations or teams . datasette-cors allows you to configure CORS headers for your Datasette instance. You can use this to enable JavaScript running on a whitelisted set of domains to make fetch() calls to the JSON API provided by your Datasette instance. | ["Changelog", "0.29 (2019-07-07)"] | [{"href": "https://github.com/simonw/datasette/issues/520", "label": "#520"}, {"href": "https://github.com/simonw/datasette-auth-github", "label": "datasette-auth-github"}, {"href": "https://help.github.com/en/articles/about-organizations", "label": "organizations"}, {"href": "https://help.github.com/en/articles/organizing-members-into-teams", "label": "teams"}, {"href": "https://github.com/simonw/datasette-cors", "label": "datasette-cors"}, {"href": "https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS", "label": "CORS headers"}] |
changelog:running-datasette-behind-a-proxy | changelog | running-datasette-behind-a-proxy | Running Datasette behind a proxy | The base_url configuration option is designed to help run Datasette on a specific path behind a proxy - for example if you want to run an instance of Datasette at /my-datasette/ within your existing site's URL hierarchy, proxied behind nginx or Apache. Support for this configuration option has been greatly improved ( #1023 ), and guidelines for using it are now available in a new documentation section on Running Datasette behind a proxy . ( #1027 ) | ["Changelog", "0.51 (2020-10-31)"] | [{"href": "https://github.com/simonw/datasette/issues/1023", "label": "#1023"}, {"href": "https://github.com/simonw/datasette/issues/1027", "label": "#1027"}] |
contributing:contributing-formatting-blacken-docs | contributing | contributing-formatting-blacken-docs | blacken-docs | The blacken-docs command applies Black formatting rules to code examples in the documentation. Run it like this: blacken-docs -l 60 docs/*.rst | ["Contributing", "Code formatting"] | [{"href": "https://pypi.org/project/blacken-docs/", "label": "blacken-docs"}] |
cli-reference:id1 | cli-reference | id1 | CLI reference | The datasette CLI tool provides a number of commands. Running datasette without specifying a command runs the default command, datasette serve . See datasette serve for the full list of options for that command. [[[cog from datasette import cli from click.testing import CliRunner import textwrap def help(args): title = "datasette " + " ".join(args) cog.out("\n::\n\n") result = CliRunner().invoke(cli.cli, args) output = result.output.replace("Usage: cli ", "Usage: datasette ") cog.out(textwrap.indent(output, ' ')) cog.out("\n\n") ]]] [[[end]]] | [] | [] |
changelog:id70 | changelog | id70 | 0.36 (2020-02-21) | The datasette object passed to plugins now has API documentation: Datasette class . ( #576 ) New methods on datasette : .add_database() and .remove_database() - documentation . ( #671 ) prepare_connection() plugin hook now takes optional datasette and database arguments - prepare_connection(conn, database, datasette) . ( #678 ) Added three new plugins and one new conversion tool to the The Datasette Ecosystem . | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/576", "label": "#576"}, {"href": "https://github.com/simonw/datasette/issues/671", "label": "#671"}, {"href": "https://github.com/simonw/datasette/issues/678", "label": "#678"}] |
changelog:id53 | changelog | id53 | 0.47.3 (2020-08-15) | The datasette --get command-line mechanism now ensures any plugins using the startup() hook are correctly executed. ( #934 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/934", "label": "#934"}] |
settings:setting-publish-secrets | settings | setting-publish-secrets | Using secrets with datasette publish | The datasette publish and datasette package commands both generate a secret for you automatically when Datasette is deployed. This means that every time you deploy a new version of a Datasette project, a new secret will be generated. This will cause signed cookies to become invalid on every fresh deploy. You can fix this by creating a secret that will be used for multiple deploys and passing it using the --secret option: datasette publish cloudrun mydb.db --service=my-service --secret=cdb19e94283a20f9d42cca5 | ["Settings"] | [] |
plugins:deploying-plugins-using-datasette-publish | plugins | deploying-plugins-using-datasette-publish | Deploying plugins using datasette publish | The datasette publish and datasette package commands both take an optional --install argument. You can use this one or more times to tell Datasette to pip install specific plugins as part of the process: datasette publish cloudrun mydb.db --install=datasette-vega You can use the name of a package on PyPI or any of the other valid arguments to pip install such as a URL to a .zip file: datasette publish cloudrun mydb.db \ --install=https://url-to-my-package.zip | ["Plugins", "Installing plugins"] | [] |
custom_templates:publishing-static-assets | custom_templates | publishing-static-assets | Publishing static assets | The datasette publish command can be used to publish your static assets, using the same syntax as above: datasette publish cloudrun mydb.db --static assets:static-files/ This will upload the contents of the static-files/ directory as part of the deployment, and configure Datasette to correctly serve the assets from /assets/ . | ["Custom pages and templates"] | [] |
testing_plugins:testing-datasette-client | testing_plugins | testing-datasette-client | Using datasette.client in tests | The datasette.client mechanism is designed for use in tests. It provides access to a pre-configured HTTPX async client instance that can make GET, POST and other HTTP requests against a Datasette instance from inside a test. A simple test looks like this: @pytest.mark.asyncio async def test_homepage(): ds = Datasette(memory=True) response = await ds.client.get("/") html = response.text assert "<h1>" in html Or for a JSON API: @pytest.mark.asyncio async def test_actor_is_null(): ds = Datasette(memory=True) response = await ds.client.get("/-/actor.json") assert response.json() == {"actor": None} To make requests as an authenticated actor, create a signed ds_cookie using the datasette.client.actor_cookie() helper function and pass it in cookies= like this: @pytest.mark.asyncio async def test_signed_cookie_actor(): ds = Datasette(memory=True) cookies = {"ds_actor": ds.client.actor_cookie({"id": "root"})} response = await ds.client.get("/-/actor.json", cookies=cookies) assert response.json() == {"actor": {"id": "root"}} | ["Testing plugins"] | [{"href": "https://www.python-httpx.org/async/", "label": "HTTPX async client"}] |
changelog:permission-checks-now-consider-opinions-from-every-plugin | changelog | permission-checks-now-consider-opinions-from-every-plugin | Permission checks now consider opinions from every plugin | The datasette.permission_allowed() method previously consulted every plugin that implemented the permission_allowed() plugin hook and obeyed the opinion of the last plugin to return a value. ( #2275 ) Datasette now consults every plugin and checks to see if any of them returned False (the veto rule), and if none of them did, it then checks to see if any of them returned True . This is explained at length in the new documentation covering How permissions are resolved . | ["Changelog", "1.0a9 (2024-02-16)"] | [{"href": "https://github.com/simonw/datasette/issues/2275", "label": "#2275"}] |
authentication:authentication-permissions-explained | authentication | authentication-permissions-explained | How permissions are resolved | The datasette.permission_allowed(actor, action, resource=None, default=...) method is called to check if an actor is allowed to perform a specific action. This method asks every plugin that implements the permission_allowed(datasette, actor, action, resource) hook if the actor is allowed to perform the action. Each plugin can return True to indicate that the actor is allowed to perform the action, False if they are not allowed and None if the plugin has no opinion on the matter. False acts as a veto - if any plugin returns False then the permission check is denied. Otherwise, if any plugin returns True then the permission check is allowed. The resource argument can be used to specify a specific resource that the action is being performed against. Some permissions, such as view-instance , do not involve a resource. Others such as view-database have a resource that is a string naming the database. Permissions that take both a database name and the name of a table, view or canned query within that database use a resource that is a tuple of two strings, (database_name, resource_name) . Plugins that implement the permission_allowed() hook can decide if they are going to consider the provided resource or not. | ["Authentication and permissions", "Permissions"] | [] |
internals:internals-datasette-urls | internals | internals-datasette-urls | datasette.urls | The datasette.urls object contains methods for building URLs to pages within Datasette. Plugins should use this to link to pages, since these methods take into account any base_url configuration setting that might be in effect. datasette.urls.instance(format=None) Returns the URL to the Datasette instance root page. This is usually "/" . datasette.urls.path(path, format=None) Takes a path and returns the full path, taking base_url into account. For example, datasette.urls.path("-/logout") will return the path to the logout page, which will be "/-/logout" by default or /prefix-path/-/logout if base_url is set to /prefix-path/ datasette.urls.logout() Returns the URL to the logout page, usually "/-/logout" datasette.urls.static(path) Returns the URL of one of Datasette's default static assets, for example "/-/static/app.css" datasette.urls.static_plugins(plugin_name, path) Returns the URL of one of the static assets belonging to a plugin. datasette.urls.static_plugins("datasette_cluster_map", "datasette-cluster-map.js") would return "/-/static-plugins/datasette_cluster_map/datasette-cluster-map.js" datasette.urls.static(path) … | ["Internals for plugins", "Datasette class"] | [] |
internals:internals-utils | internals | internals-utils | The datasette.utils module | The datasette.utils module contains various utility functions used by Datasette. As a general rule you should consider anything in this module to be unstable - functions and classes here could change without warning or be removed entirely between Datasette releases, without being mentioned in the release notes. The exception to this rule is anything that is documented here. If you find a need for an undocumented utility function in your own work, consider opening an issue requesting that the function you are using be upgraded to documented and supported status. | ["Internals for plugins"] | [{"href": "https://github.com/simonw/datasette/issues/new", "label": "opening an issue"}] |
javascript_plugins:javascript-datasette-manager | javascript_plugins | javascript-datasette-manager | datasetteManager | The datasetteManager object VERSION - string The version of Datasette plugins - Map() A Map of currently loaded plugin names to plugin implementations registerPlugin(name, implementation) Call this to register a plugin, passing its name and implementation selectors - object An object providing named aliases to useful CSS selectors, listed below | ["JavaScript plugins"] | [] |
installation:loading-spatialite | installation | loading-spatialite | Loading SpatiaLite | The datasetteproject/datasette image includes a recent version of the SpatiaLite extension for SQLite. To load and enable that module, use the following command: docker run -p 8001:8001 -v `pwd`:/mnt \ datasetteproject/datasette \ datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db \ --load-extension=spatialite You can confirm that SpatiaLite is successfully loaded by visiting http://127.0.0.1:8001/-/versions | ["Installation", "Advanced installation options", "Using Docker"] | [{"href": "http://127.0.0.1:8001/-/versions", "label": "http://127.0.0.1:8001/-/versions"}] |
internals:database-results | internals | database-results | Results | The db.execute() method returns a single Results object. This can be used to access the rows returned by the query. Iterating over a Results object will yield SQLite Row objects . Each of these can be treated as a tuple or can be accessed using row["column"] syntax: info = [] results = await db.execute("select name from sqlite_master") for row in results: info.append(row["name"]) The Results object also has the following properties and methods: .truncated - boolean Indicates if this query was truncated - if it returned more results than the specified page_size . If this is true then the results object will only provide access to the first page_size rows in the query result. You can disable truncation by passing truncate=False to the db.query() method. .columns - list of strings A list of column names returned by the query. .rows - list of sqlite3.Row This property provides direct access to the list of rows returned by the database. You can access specific rows by index using results.rows[0] . .first() - row or None Returns the first row in the results, or None if no rows were returned. .single_value() Returns the value of the first column of the first row of results - but only if the query returned a single row wit… | ["Internals for plugins", "Database class"] | [{"href": "https://docs.python.org/3/library/sqlite3.html#row-objects", "label": "Row objects"}] |
changelog:log-out | changelog | log-out | Log out | The ds_actor cookie can be used by plugins (or by Datasette's --root mechanism ) to authenticate users. The new /-/logout page provides a way to clear that cookie. A "Log out" button now shows in the global navigation provided the user is authenticated using the ds_actor cookie. ( #840 ) | ["Changelog", "0.45 (2020-07-01)"] | [{"href": "https://github.com/simonw/datasette/issues/840", "label": "#840"}] |
changelog:new-plugin-hook-extra-template-vars | changelog | new-plugin-hook-extra-template-vars | New plugin hook: extra_template_vars | The extra_template_vars(template, database, table, columns, view_name, request, datasette) plugin hook allows plugins to inject their own additional variables into the Datasette template context. This can be used in conjunction with custom templates to customize the Datasette interface. datasette-auth-github uses this hook to add custom HTML to the new top navigation bar (which is designed to be modified by plugins, see #540 ). | ["Changelog", "0.29 (2019-07-07)"] | [{"href": "https://github.com/simonw/datasette-auth-github", "label": "datasette-auth-github"}, {"href": "https://github.com/simonw/datasette/issues/540", "label": "#540"}] |
changelog:plugin-hooks-and-internals | changelog | plugin-hooks-and-internals | Plugin hooks and internals | The prepare_jinja2_environment(env, datasette) plugin hook now accepts an optional datasette argument. Hook implementations can also now return an async function which will be awaited automatically. ( #1809 ) Database(is_mutable=) now defaults to True . ( #1808 ) The datasette.check_visibility() method now accepts an optional permissions= list, allowing it to take multiple permissions into account at once when deciding if something should be shown as public or private. This has been used to correctly display padlock icons in more places in the Datasette interface. ( #1829 ) Datasette no longer enforces upper bounds on its dependencies. ( #1800 ) | ["Changelog", "0.63 (2022-10-27)"] | [{"href": "https://github.com/simonw/datasette/issues/1809", "label": "#1809"}, {"href": "https://github.com/simonw/datasette/issues/1808", "label": "#1808"}, {"href": "https://github.com/simonw/datasette/issues/1829", "label": "#1829"}, {"href": "https://github.com/simonw/datasette/issues/1800", "label": "#1800"}] |
spatialite:importing-shapefiles-into-spatialite | spatialite | importing-shapefiles-into-spatialite | Importing shapefiles into SpatiaLite | The shapefile format is a common format for distributing geospatial data. You can use the spatialite command-line tool to create a new database table from a shapefile. Try it now with the North America shapefile available from the University of North Carolina Global River Database project. Download the file and unzip it (this will create files called narivs.dbf , narivs.prj , narivs.shp and narivs.shx in the current directory), then run the following: spatialite rivers-database.db SpatiaLite version ..: 4.3.0a Supported Extensions: ... spatialite> .loadshp narivs rivers CP1252 23032 ======== Loading shapefile at 'narivs' into SQLite table 'rivers' ... Inserted 467973 rows into 'rivers' from SHAPEFILE This will load the data from the narivs shapefile into a new database table called rivers . Exit out of spatialite (using Ctrl+D ) and run Datasette against your new database like this: datasette rivers-database.db \ --load-extension=/usr/local/lib/mod_spatialite.dylib If you browse to http://localhost:8001/rivers-database/rivers you will see the new table... but the Geometry column will contain unreadable binary data (SpatiaLite uses a custom format based on WKB ). The easiest way to turn this into semi-readable data is to use the SpatiaLite AsGeoJSON function. Try the following using the SQL query interface at http://localhost:8001/rivers-database : select *, AsGeoJSON(Geometry) from rivers limit 10; This will give you back an additional column of GeoJSON. You can copy and paste GeoJSON from this column into the debugging tool at geojson.io to visualize it on a map. To see a more interesting example, try ordering the records with the longest geometry first. Since there are 467,000 rows in the table you will first need to increase the SQL time limit imposed by Datasette: datasette rivers-database.db \ --load-e… | ["SpatiaLite"] | [{"href": "https://en.wikipedia.org/wiki/Shapefile", "label": "shapefile format"}, {"href": "http://gaia.geosci.unc.edu/rivers/", "label": "Global River Database"}, {"href": "https://www.gaia-gis.it/gaia-sins/BLOB-Geometry.html", "label": "a custom format based on WKB"}, {"href": "https://geojson.io/", "label": "geojson.io"}] |
csv_export:streaming-all-records | csv_export | streaming-all-records | Streaming all records | The stream all rows option is designed to be as efficient as possible - under the hood it takes advantage of Python 3 asyncio capabilities and Datasette's efficient pagination to stream back the full CSV file. Since databases can get pretty large, by default this option is capped at 100MB - if a table returns more than 100MB of data the last line of the CSV will be a truncation error message. You can increase or remove this limit using the max_csv_mb config setting. You can also disable the CSV export feature entirely using allow_csv_stream . | ["CSV export"] | [] |
spatialite:querying-polygons-using-within | spatialite | querying-polygons-using-within | Querying polygons using within() | The within() SQL function can be used to check if a point is within a geometry: select name from places where within(GeomFromText('POINT(-3.1724366 51.4704448)'), places.geom); The GeomFromText() function takes a string of well-known text. Note that the order used here is longitude then latitude . To run that same within() query in a way that benefits from the spatial index, use the following: select name from places where within(GeomFromText('POINT(-3.1724366 51.4704448)'), places.geom) and rowid in ( SELECT pkid FROM idx_places_geom where xmin < -3.1724366 and xmax > -3.1724366 and ymin < 51.4704448 and ymax > 51.4704448 ); | ["SpatiaLite"] | [] |
json_api:id2 | json_api | id2 | Table arguments | The Datasette table view takes a number of special query string arguments. | ["JSON API"] | [] |
pages:pages | pages | pages | Pages and API endpoints | The Datasette web application offers a number of different pages that can be accessed to explore the data in question, each of which is accompanied by an equivalent JSON API. | [] | [] |
spatialite:spatialite-warning | spatialite | spatialite-warning | Warning | The SpatiaLite extension adds a large number of additional SQL functions , some of which are not be safe for untrusted users to execute: they may cause the Datasette server to crash. You should not expose a SpatiaLite-enabled Datasette instance to the public internet without taking extra measures to secure it against potentially harmful SQL queries. The following steps are recommended: Disable arbitrary SQL queries by untrusted users. See Controlling the ability to execute arbitrary SQL for ways to do this. The easiest is to start Datasette with the datasette --setting default_allow_sql off option. Define Canned queries with the SQL queries that use SpatiaLite functions that you want people to be able to execute. The Datasette SpatiaLite tutorial includes detailed instructions for running SpatiaLite safely using these techniques | ["SpatiaLite"] | [{"href": "https://www.gaia-gis.it/gaia-sins/spatialite-sql-5.0.1.html", "label": "a large number of additional SQL functions"}, {"href": "https://datasette.io/tutorials/spatialite", "label": "Datasette SpatiaLite tutorial"}] |
testing_plugins:testing-plugins-datasette-test-instance | testing_plugins | testing-plugins-datasette-test-instance | Setting up a Datasette test instance | The above example shows the easiest way to start writing tests against a Datasette instance: from datasette.app import Datasette import pytest @pytest.mark.asyncio async def test_plugin_is_installed(): datasette = Datasette(memory=True) response = await datasette.client.get("/-/plugins.json") assert response.status_code == 200 Creating a Datasette() instance like this as useful shortcut in tests, but there is one detail you need to be aware of. It's important to ensure that the async method .invoke_startup() is called on that instance. You can do that like this: datasette = Datasette(memory=True) await datasette.invoke_startup() This method registers any startup(datasette) or prepare_jinja2_environment(env, datasette) plugins that might themselves need to make async calls. If you are using await datasette.client.get() and similar methods then you don't need to worry about this - Datasette automatically calls invoke_startup() the first time it handles a request. | ["Testing plugins"] | [] |
getting_started:getting-started-demo | getting_started | getting-started-demo | Play with a live demo | The best way to experience Datasette for the first time is with a demo: global-power-plants.datasettes.com provides a searchable database of power plants around the world, using data from the World Resources Institude rendered using the datasette-cluster-map plugin. fivethirtyeight.datasettes.com shows Datasette running against over 400 datasets imported from the FiveThirtyEight GitHub repository . | ["Getting started"] | [{"href": "https://global-power-plants.datasettes.com/global-power-plants/global-power-plants", "label": "global-power-plants.datasettes.com"}, {"href": "https://www.wri.org/publication/global-power-plant-database", "label": "World Resources Institude"}, {"href": "https://github.com/simonw/datasette-cluster-map", "label": "datasette-cluster-map"}, {"href": "https://fivethirtyeight.datasettes.com/fivethirtyeight", "label": "fivethirtyeight.datasettes.com"}, {"href": "https://github.com/fivethirtyeight/data", "label": "FiveThirtyEight GitHub repository"}] |
changelog:id118 | changelog | id118 | 0.22 (2018-05-20) | The big new feature in this release is Facets . Datasette can now apply faceted browse to any column in any table. It will also suggest possible facets. See the Datasette Facets announcement post for more details. In addition to the work on facets: Added docs for introspection endpoints New --config option, added --help-config , closes #274 Removed the --page_size= argument to datasette serve in favour of: datasette serve --config default_page_size:50 mydb.db Added new help section: datasette --help-config Config options: default_page_size Default page size for the table view (default=100) max_returned_rows Maximum rows that can be returned from a table or custom query (default=1000) sql_time_limit_ms Time limit for a SQL query in milliseconds (default=1000) default_facet_size Number of values to return for requested facets (default=30) facet_time_limit_ms Time limit for calculating a requested facet (default=200) facet_suggest_time_limit_ms Time limit for calculating a suggested facet (default=50) Only apply responsive table styles to .rows-and-column Otherwise they interfere with tables in the description, e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight/nba-elo%2Fnbaallelo Refactored views into new views/ modules, refs #256 Documentation for SQLite full-text search support, closes #25… | ["Changelog"] | [{"href": "https://simonwillison.net/2018/May/20/datasette-facets/", "label": "Datasette Facets"}, {"href": "https://docs.datasette.io/en/stable/introspection.html", "label": "docs for introspection endpoints"}, {"href": "https://github.com/simonw/datasette/issues/274", "label": "#274"}, {"href": "https://fivethirtyeight.datasettes.com/fivethirtyeight/nba-elo%2Fnbaallelo", "label": "https://fivethirtyeight.datasettes.com/fivethirtyeight/nba-elo%2Fnbaallelo"}, {"href": "https://github.com/simonw/datasette/issues/256", "label": "#256"}, {"href": "https://docs.datasette.io/en/stable/full_text_search.html", "label": "Documentation for SQLite full-text search"}, {"href": "https://github.com/simonw/datasette/issues/253", "label": "#253"}, {"href": "https://github.com/simonw/datasette/issues/252", "label": "#252"}] |
changelog:id161 | changelog | id161 | 0.15 (2018-04-09) | The biggest new feature in this release is the ability to sort by column. On the table page the column headers can now be clicked to apply sort (or descending sort), or you can specify ?_sort=column or ?_sort_desc=column directly in the URL. table_rows => table_rows_count , filtered_table_rows => filtered_table_rows_count Renamed properties. Closes #194 New sortable_columns option in metadata.json to control sort options. You can now explicitly set which columns in a table can be used for sorting using the _sort and _sort_desc arguments using metadata.json : { "databases": { "database1": { "tables": { "example_table": { "sortable_columns": [ "height", "weight" ] } } } } } Refs #189 Column headers now link to sort/desc sort - refs #189 _sort and _sort_desc parameters for table views Allows for paginated sorted results based on a specified column. Refs #189 Total row count now correct even if _next applied Use .custom_sql() for _group_count implementation (refs #150 ) Make HTML title more readable in query template ( #180 ) [Ryan Pitts] New ?_shape=objects/object/lists param for JSON API ( #192 ) New _shape= parameter repl… | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/194", "label": "#194"}, {"href": "https://github.com/simonw/datasette/issues/189", "label": "#189"}, {"href": "https://github.com/simonw/datasette/issues/189", "label": "#189"}, {"href": "https://github.com/simonw/datasette/issues/189", "label": "#189"}, {"href": "https://github.com/simonw/datasette/issues/150", "label": "#150"}, {"href": "https://github.com/simonw/datasette/issues/180", "label": "#180"}, {"href": "https://github.com/simonw/datasette/issues/192", "label": "#192"}, {"href": "https://github.com/simonw/datasette/issues/122", "label": "#122"}, {"href": "https://github.com/simonw/datasette/issues/190", "label": "#190"}, {"href": "https://github.com/simonw/datasette/issues/190", "label": "#190"}, {"href": "https://github.com/simonw/datasette/issues/185", "label": "#185"}, {"href": "https://github.com/simonw/datasette/issues/178", "label": "#178"}] |
introspection:messagesdebugview | introspection | messagesdebugview | /-/messages | The debug tool at /-/messages can be used to set flash messages to try out that feature. See .add_message(request, message, type=datasette.INFO) for details of this feature. | ["Introspection"] | [] |
authentication:permissionsdebugview | authentication | permissionsdebugview | The permissions debug tool | The debug tool at /-/permissions is only available to the authenticated root user (or any actor granted the permissions-debug action). It shows the thirty most recent permission checks that have been carried out by the Datasette instance. It also provides an interface for running hypothetical permission checks against a hypothetical actor. This is a useful way of confirming that your configured permissions work in the way you expect. This is designed to help administrators and plugin authors understand exactly how permission checks are being carried out, in order to effectively configure Datasette's permission system. | ["Authentication and permissions"] | [] |
json_api:json-api-pagination | json_api | json-api-pagination | Pagination | The default JSON representation includes a "next_url" key which can be used to access the next page of results. If that key is null or missing then it means you have reached the final page of results. Other representations include pagination information in the link HTTP header. That header will look something like this: link: <https://latest.datasette.io/fixtures/sortable.json?_next=d%2Cv>; rel="next" Here is an example Python function built using requests that returns a list of all of the paginated items from one of these API endpoints: def paginate(url): items = [] while url: response = requests.get(url) try: url = response.links.get("next").get("url") except AttributeError: url = None items.extend(response.json()) return items | ["JSON API"] | [{"href": "https://requests.readthedocs.io/", "label": "requests"}] |
json_api:json-api-default | json_api | json-api-default | Default representation | The default JSON representation of data from a SQLite table or custom query looks like this: { "ok": true, "rows": [ { "id": 3, "name": "Detroit" }, { "id": 2, "name": "Los Angeles" }, { "id": 4, "name": "Memnonia" }, { "id": 1, "name": "San Francisco" } ], "truncated": false } "ok" is always true if an error did not occur. The "rows" key is a list of objects, each one representing a row. The "truncated" key lets you know if the query was truncated. This can happen if a SQL query returns more than 1,000 results (or the max_returned_rows setting). For table pages, an additional key "next" may be present. This indicates that the next page in the pagination set can be retrieved using ?_next=VALUE . | ["JSON API"] | [] |
settings:setting-default-page-size | settings | setting-default-page-size | default_page_size | The default number of rows returned by the table page. You can over-ride this on a per-page basis using the ?_size=80 query string parameter, provided you do not specify a value higher than the max_returned_rows setting. You can set this default using --setting like so: datasette mydatabase.db --setting default_page_size 50 | ["Settings", "Settings"] | [] |
settings:setting-default-facet-size | settings | setting-default-facet-size | default_facet_size | The default number of unique rows returned by Facets is 30. You can customize it like this: datasette mydatabase.db --setting default_facet_size 50 | ["Settings", "Settings"] | [] |
contributing:contributing-continuous-deployment | contributing | contributing-continuous-deployment | Continuously deployed demo instances | The demo instance at latest.datasette.io is re-deployed automatically to Google Cloud Run for every push to main that passes the test suite. This is implemented by the GitHub Actions workflow at .github/workflows/deploy-latest.yml . Specific branches can also be set to automatically deploy by adding them to the on: push: branches block at the top of the workflow YAML file. Branches configured in this way will be deployed to a new Cloud Run service whether or not their tests pass. The Cloud Run URL for a branch demo can be found in the GitHub Actions logs. | ["Contributing"] | [{"href": "https://latest.datasette.io/", "label": "latest.datasette.io"}, {"href": "https://github.com/simonw/datasette/blob/main/.github/workflows/deploy-latest.yml", "label": ".github/workflows/deploy-latest.yml"}] |
spatialite:installing-spatialite-on-os-x | spatialite | installing-spatialite-on-os-x | Installing SpatiaLite on OS X | The easiest way to install SpatiaLite on OS X is to use Homebrew . brew update brew install spatialite-tools This will install the spatialite command-line tool and the mod_spatialite dynamic library. You can now run Datasette like so: datasette --load-extension=spatialite | ["SpatiaLite", "Installation"] | [{"href": "https://brew.sh/", "label": "Homebrew"}] |
internals:internals-shortcuts | internals | internals-shortcuts | Import shortcuts | The following commonly used symbols can be imported directly from the datasette module: from datasette import Response from datasette import Forbidden from datasette import NotFound from datasette import hookimpl from datasette import actor_matches_allow | ["Internals for plugins"] | [] |
configuration:configuration-reference | configuration | configuration-reference | The following example shows some of the valid configuration options that can exist inside datasette.yaml . [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ # Datasette settings block settings: default_page_size: 50 sql_time_limit_ms: 3500 max_returned_rows: 2000 # top-level plugin configuration plugins: datasette-my-plugin: key: valueA # Database and table-level configuration databases: your_db_name: # plugin configuration for the your_db_name database plugins: datasette-my-plugin: key: valueA tables: your_table_name: allow: # Only the root user can access this table id: root # plugin configuration for the your_table_name table # inside your_db_name database plugins: datasette-my-plugin: key: valueB """) ) ]]] [[[end]]] | ["Configuration"] | [] | |
settings:id2 | settings | id2 | Settings | The following options can be set using --setting name value , or by storing them in the settings.json file for use with Configuration directory mode . | ["Settings"] | [] |
csv_export:csv-export-url-parameters | csv_export | csv-export-url-parameters | URL parameters | The following options can be used to customize the CSVs returned by Datasette. ?_header=off This removes the first row of the CSV file specifying the headings - only the row data will be returned. ?_stream=on Stream all matching records, not just the first page of results. See below. ?_dl=on Causes Datasette to return a content-disposition: attachment; filename="filename.csv" header. | ["CSV export"] | [] |
plugin_hooks:plugin-hook-slots | plugin_hooks | plugin-hook-slots | Template slots | The following set of plugin hooks can be used to return extra HTML content that will be inserted into the corresponding page, directly below the <h1> heading. Multiple plugins can contribute content here. The order in which it is displayed can be controlled using Pluggy's call time order options . Each of these plugin hooks can return either a string or an awaitable function that returns a string. | ["Plugin hooks"] | [{"href": "https://pluggy.readthedocs.io/en/stable/#call-time-order", "label": "call time order options"}] |
changelog:id49 | changelog | id49 | 0.50 (2020-10-09) | The key new feature in this release is the column actions menu on the table page ( #891 ). This can be used to sort a column in ascending or descending order, facet data by that column or filter the table to just rows that have a value for that column. Plugin authors can use the new datasette.client object to make internal HTTP requests from their plugins, allowing them to make use of Datasette's JSON API. ( #943 ) New Deploying Datasette documentation with guides for deploying Datasette on a Linux server using systemd or to hosting providers that support buildpacks . ( #514 , #997 ) Other improvements in this release: Publishing to Google Cloud Run documentation now covers Google Cloud SDK options. Thanks, Geoffrey Hing. ( #995 ) New datasette -o option which opens your browser as soon as Datasette starts up. ( #970 ) Datasette now sets sqlite3.enable_callback_tracebacks(True) so that errors in custom SQL functions will display tracebacks. ( #891 ) Fixed two rendering bugs with column headers in portrait mobile view. ( #978 , #980 ) New db.table_column_details(table) introspection method for retrieving full details of the columns in a specific table, see Database introspection . Fixed a routing bug with custom page wildcard templates. ( #996 ) datasette publish heroku now deploys using Python 3.8.6. New datasette publish heroku --tar= option. ( #969 ) OPTIONS requests against HTML pages no longer return a 500 error. ( #1001 ) … | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/891", "label": "#891"}, {"href": "https://github.com/simonw/datasette/issues/943", "label": "#943"}, {"href": "https://github.com/simonw/datasette/issues/514", "label": "#514"}, {"href": "https://github.com/simonw/datasette/issues/997", "label": "#997"}, {"href": "https://github.com/simonw/datasette/pull/995", "label": "#995"}, {"href": "https://github.com/simonw/datasette/issues/970", "label": "#970"}, {"href": "https://github.com/simonw/datasette/issues/891", "label": "#891"}, {"href": "https://github.com/simonw/datasette/issues/978", "label": "#978"}, {"href": "https://github.com/simonw/datasette/issues/980", "label": "#980"}, {"href": "https://github.com/simonw/datasette/issues/996", "label": "#996"}, {"href": "https://github.com/simonw/datasette/issues/969", "label": "#969"}, {"href": "https://github.com/simonw/datasette/issues/1001", "label": "#1001"}, {"href": "https://simonwillison.net/2020/Oct/9/datasette-0-50/", "label": "Datasette 0.50: The annotated release notes"}] |
changelog:id62 | changelog | id62 | 0.43 (2020-05-28) | The main focus of this release is a major upgrade to the register_output_renderer(datasette) plugin hook, which allows plugins to provide new output formats for Datasette such as datasette-atom and datasette-ics . Redesign of register_output_renderer(datasette) to provide more context to the render callback and support an optional "can_render" callback that controls if a suggested link to the output format is provided. ( #581 , #770 ) Visually distinguish float and integer columns - useful for figuring out why order-by-column might be returning unexpected results. ( #729 ) The Request object , which is passed to several plugin hooks, is now documented. ( #706 ) New metadata.json option for setting a custom default page size for specific tables and views, see Setting a custom page size . ( #751 ) Canned queries can now be configured with a default URL fragment hash, useful when working with plugins such as datasette-vega , see Additional canned query options . ( #706 ) Fixed a bug in datasette publish when running on operating systems where the /tmp directory lives in a different volume, using a backport of the Python 3.8 shutil.copytree() function. ( #744 ) Every plugin hook is now covered by the unit tests, and a new unit test checks that each plugin hook has at least one corresponding test. ( #771 , #773 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette-atom", "label": "datasette-atom"}, {"href": "https://github.com/simonw/datasette-ics", "label": "datasette-ics"}, {"href": "https://github.com/simonw/datasette/issues/581", "label": "#581"}, {"href": "https://github.com/simonw/datasette/issues/770", "label": "#770"}, {"href": "https://github.com/simonw/datasette/issues/729", "label": "#729"}, {"href": "https://github.com/simonw/datasette/issues/706", "label": "#706"}, {"href": "https://github.com/simonw/datasette/issues/751", "label": "#751"}, {"href": "https://github.com/simonw/datasette-vega", "label": "datasette-vega"}, {"href": "https://github.com/simonw/datasette/issues/706", "label": "#706"}, {"href": "https://github.com/simonw/datasette/issues/744", "label": "#744"}, {"href": "https://github.com/simonw/datasette/issues/771", "label": "#771"}, {"href": "https://github.com/simonw/datasette/issues/773", "label": "#773"}] |
settings:setting-max-csv-mb | settings | setting-max-csv-mb | max_csv_mb | The maximum size of CSV that can be exported, in megabytes. Defaults to 100MB. You can disable the limit entirely by settings this to 0: datasette mydatabase.db --setting max_csv_mb 0 | ["Settings", "Settings"] | [] |
changelog:through-for-joins-through-many-to-many-tables | changelog | through-for-joins-through-many-to-many-tables | ?_through= for joins through many-to-many tables | The new ?_through={json} argument to the Table view allows records to be filtered based on a many-to-many relationship. See Special table arguments for full documentation - here's an example . ( #355 ) This feature was added to help support facet by many-to-many , which isn't quite ready yet but will be coming in the next Datasette release. | ["Changelog", "0.29 (2019-07-07)"] | [{"href": "https://latest.datasette.io/fixtures/roadside_attractions?_through={%22table%22:%22roadside_attraction_characteristics%22,%22column%22:%22characteristic_id%22,%22value%22:%221%22}", "label": "an example"}, {"href": "https://github.com/simonw/datasette/issues/355", "label": "#355"}, {"href": "https://github.com/simonw/datasette/issues/551", "label": "facet by many-to-many"}] |
changelog:other-changes | changelog | other-changes | Other changes | The new DATASETTE_TRACE_PLUGINS=1 environment variable turns on detailed trace output for every executed plugin hook, useful for debugging and understanding how the plugin system works at a low level. ( #2274 ) Datasette on Python 3.9 or above marks its non-cryptographic uses of the MD5 hash function as usedforsecurity=False , for compatibility with FIPS systems. ( #2270 ) SQL relating to Datasette's internal database now executes inside a transaction, avoiding a potential database locked error. ( #2273 ) The /-/threads debug page now identifies the database in the name associated with each dedicated write thread. ( #2265 ) The /db/-/create API now fires a insert-rows event if rows were inserted after the table was created. ( #2260 ) | ["Changelog", "1.0a9 (2024-02-16)"] | [{"href": "https://github.com/simonw/datasette/issues/2274", "label": "#2274"}, {"href": "https://github.com/simonw/datasette/issues/2270", "label": "#2270"}, {"href": "https://github.com/simonw/datasette/issues/2273", "label": "#2273"}, {"href": "https://github.com/simonw/datasette/issues/2265", "label": "#2265"}, {"href": "https://github.com/simonw/datasette/issues/2260", "label": "#2260"}] |
changelog:url-building | changelog | url-building | URL building | The new datasette.urls family of methods can be used to generate URLs to key pages within the Datasette interface, both within custom templates and Datasette plugins. See Building URLs within plugins for more details. ( #904 ) | ["Changelog", "0.51 (2020-10-31)"] | [{"href": "https://github.com/simonw/datasette/issues/904", "label": "#904"}] |
changelog:faceting | changelog | faceting | Faceting | The number of unique values in a facet is now always displayed. Previously it was only displayed if the user specified ?_facet_size=max . ( #1556 ) Facets of type date or array can now be configured in metadata.json , see Facets in metadata . Thanks, David Larlet. ( #1552 ) New ?_nosuggest=1 parameter for table views, which disables facet suggestion. ( #1557 ) Fixed bug where ?_facet_array=tags&_facet=tags would only display one of the two selected facets. ( #625 ) | ["Changelog", "0.60 (2022-01-13)"] | [{"href": "https://github.com/simonw/datasette/issues/1556", "label": "#1556"}, {"href": "https://github.com/simonw/datasette/issues/1552", "label": "#1552"}, {"href": "https://github.com/simonw/datasette/issues/1557", "label": "#1557"}, {"href": "https://github.com/simonw/datasette/issues/625", "label": "#625"}] |
changelog:v1-0-a10 | changelog | v1-0-a10 | 1.0a10 (2024-02-17) | The only changes in this alpha correspond to the way Datasette handles database transactions. ( #2277 ) The database.execute_write_fn() method has a new transaction=True parameter. This defaults to True which means all functions executed using this method are now automatically wrapped in a transaction - previously the functions needed to roll transaction handling on their own, and many did not. Pass transaction=False to execute_write_fn() if you want to manually handle transactions in your function. Several internal Datasette features, including parts of the JSON write API , had been failing to wrap their operations in a transaction. This has been fixed by the new transaction=True default. | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/2277", "label": "#2277"}] |
authentication:logoutview | authentication | logoutview | The /-/logout page | The page at /-/logout provides the ability to log out of a ds_actor cookie authentication session. | ["Authentication and permissions", "The ds_actor cookie"] | [] |
facets:speeding-up-facets-with-indexes | facets | speeding-up-facets-with-indexes | Speeding up facets with indexes | The performance of facets can be greatly improved by adding indexes on the columns you wish to facet by. Adding indexes can be performed using the sqlite3 command-line utility. Here's how to add an index on the state column in a table called Food_Trucks : sqlite3 mydatabase.db SQLite version 3.19.3 2017-06-27 16:48:08 Enter ".help" for usage hints. sqlite> CREATE INDEX Food_Trucks_state ON Food_Trucks("state"); Or using the sqlite-utils command-line utility: sqlite-utils create-index mydatabase.db Food_Trucks state | ["Facets"] | [{"href": "https://sqlite-utils.datasette.io/en/stable/cli.html#creating-indexes", "label": "sqlite-utils"}] |
changelog:better-plugin-documentation | changelog | better-plugin-documentation | Better plugin documentation | The plugin documentation has been re-arranged into four sections, including a brand new section on testing plugins. ( #687 ) Plugins introduces Datasette's plugin system and describes how to install and configure plugins. Writing plugins describes how to author plugins, from one-off single file plugins to packaged plugins that can be published to PyPI. It also describes how to start a plugin using the new datasette-plugin cookiecutter template. Plugin hooks is a full list of detailed documentation for every Datasette plugin hook. Testing plugins describes how to write tests for Datasette plugins, using pytest and HTTPX . | ["Changelog", "0.45 (2020-07-01)"] | [{"href": "https://github.com/simonw/datasette/issues/687", "label": "#687"}, {"href": "https://github.com/simonw/datasette-plugin", "label": "datasette-plugin"}, {"href": "https://docs.pytest.org/", "label": "pytest"}, {"href": "https://www.python-httpx.org/", "label": "HTTPX"}] |
deploying:deploying | deploying | deploying | Deploying Datasette | The quickest way to deploy a Datasette instance on the internet is to use the datasette publish command, described in Publishing data . This can be used to quickly deploy Datasette to a number of hosting providers including Heroku, Google Cloud Run and Vercel. You can deploy Datasette to other hosting providers using the instructions on this page. | [] | [] |
writing_plugins:writing-plugins-one-off | writing_plugins | writing-plugins-one-off | Writing one-off plugins | The quickest way to start writing a plugin is to create a my_plugin.py file and drop it into your plugins/ directory. Here is an example plugin, which adds a new custom SQL function called hello_world() which takes no arguments and returns the string Hello world! . from datasette import hookimpl @hookimpl def prepare_connection(conn): conn.create_function( "hello_world", 0, lambda: "Hello world!" ) If you save this in plugins/my_plugin.py you can then start Datasette like this: datasette serve mydb.db --plugins-dir=plugins/ Now you can navigate to http://localhost:8001/mydb and run this SQL: select hello_world(); To see the output of your plugin. | ["Writing plugins"] | [{"href": "http://localhost:8001/mydb", "label": "http://localhost:8001/mydb"}] |
configuration:configuration-cli | configuration | configuration-cli | Configuration via the command-line | The recommended way to configure Datasette is using a datasette.yaml file passed to -c/--config . You can also pass individual settings to Datasette using the -s/--setting option, which can be used multiple times: datasette mydatabase.db \ --setting settings.default_page_size 50 \ --setting settings.sql_time_limit_ms 3500 This option takes dotted-notation for the first argument and a value for the second argument. This means you can use it to set any configuration value that would be valid in a datasette.yaml file. It also works for plugin configuration, for example for datasette-cluster-map : datasette mydatabase.db \ --setting plugins.datasette-cluster-map.latitude_column xlat \ --setting plugins.datasette-cluster-map.longitude_column xlon If the value you provide is a valid JSON object or list it will be treated as nested data, allowing you to configure plugins that accept lists such as datasette-proxy-url : datasette mydatabase.db \ -s plugins.datasette-proxy-url.paths '[{"path": "/proxy", "backend": "http://example.com/"}]' This is equivalent to a datasette.yaml file containing the following: [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ plugins: datasette-proxy-url: paths: - path: /proxy backend: http://example.com/ """).strip() ) ]]] [[[end]]] | ["Configuration"] | [{"href": "https://datasette.io/plugins/datasette-cluster-map", "label": "datasette-cluster-map"}, {"href": "https://datasette.io/plugins/datasette-proxy-url", "label": "datasette-proxy-url"}] |
internals:internals-request | internals | internals-request | Request object | The request object is passed to various plugin hooks. It represents an incoming HTTP request. It has the following properties: .scope - dictionary The ASGI scope that was used to construct this request, described in the ASGI HTTP connection scope specification. .method - string The HTTP method for this request, usually GET or POST . .url - string The full URL for this request, e.g. https://latest.datasette.io/fixtures . .scheme - string The request scheme - usually https or http . .headers - dictionary (str -> str) A dictionary of incoming HTTP request headers. Header names have been converted to lowercase. .cookies - dictionary (str -> str) A dictionary of incoming cookies .host - string The host header from the incoming request, e.g. latest.datasette.io or localhost . .path - string The path of the request excluding the query string, e.g. /fixtures . .full_path - string The path of the… | ["Internals for plugins"] | [{"href": "https://asgi.readthedocs.io/en/latest/specs/www.html#connection-scope", "label": "ASGI HTTP connection scope"}] |
pages:indexview | pages | indexview | Top-level index | The root page of any Datasette installation is an index page that lists all of the currently attached databases. Some examples: fivethirtyeight.datasettes.com global-power-plants.datasettes.com register-of-members-interests.datasettes.com Add /.json to the end of the URL for the JSON version of the underlying data: fivethirtyeight.datasettes.com/.json global-power-plants.datasettes.com/.json register-of-members-interests.datasettes.com/.json | ["Pages and API endpoints"] | [{"href": "https://fivethirtyeight.datasettes.com/", "label": "fivethirtyeight.datasettes.com"}, {"href": "https://global-power-plants.datasettes.com/", "label": "global-power-plants.datasettes.com"}, {"href": "https://register-of-members-interests.datasettes.com/", "label": "register-of-members-interests.datasettes.com"}, {"href": "https://fivethirtyeight.datasettes.com/.json", "label": "fivethirtyeight.datasettes.com/.json"}, {"href": "https://global-power-plants.datasettes.com/.json", "label": "global-power-plants.datasettes.com/.json"}, {"href": "https://register-of-members-interests.datasettes.com/.json", "label": "register-of-members-interests.datasettes.com/.json"}] |
authentication:authentication-permissions-allow | authentication | authentication-permissions-allow | Defining permissions with "allow" blocks | The standard way to define permissions in Datasette is to use an "allow" block in the datasette.yaml file . This is a JSON document describing which actors are allowed to perform a permission. The most basic form of allow block is this ( allow demo , deny demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: id: root """).strip(), "YAML", "JSON" ) ]]] [[[end]]] This will match any actors with an "id" property of "root" - for example, an actor that looks like this: { "id": "root", "name": "Root User" } An allow block can specify "deny all" using false ( demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: false """).strip(), "YAML", "JSON" ) ]]] [[[end]]] An "allow" of true allows all access ( demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: true """).strip(), "YAML", "JSON" ) ]]] [[[end]]] Allow keys can provide a list of values. These will match any actor that has any of those values ( allow demo , deny demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: id: - simon - cleopaws """).strip(), "YAML", "JSON" ) ]]] [[[end]]] This will match any actor with an "id" of either "simon" or "cleopaws" . Actors can have properties that feature a list of values. These will be matched against the list of values in an allow block. Consider the following actor: { "id": "simon"… | ["Authentication and permissions", "Permissions"] | [{"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%22id%22%3A+%22root%22%7D&allow=%7B%0D%0A++++++++%22id%22%3A+%22root%22%0D%0A++++%7D", "label": "allow demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%22id%22%3A+%22trevor%22%7D&allow=%7B%0D%0A++++++++%22id%22%3A+%22root%22%0D%0A++++%7D", "label": "deny demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22root%22%0D%0A%7D&allow=false", "label": "demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22root%22%0D%0A%7D&allow=true", "label": "demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22cleopaws%22%0D%0A%7D&allow=%7B%0D%0A++++%22id%22%3A+%5B%0D%0A++++++++%22simon%22%2C%0D%0A++++++++%22cleopaws%22%0D%0A++++%5D%0D%0A%7D", "label": "allow demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22pancakes%22%0D%0A%7D&allow=%7B%0D%0A++++%22id%22%3A+%5B%0D%0A++++++++%22simon%22%2C%0D%0A++++++++%22cleopaws%22%0D%0A++++%5D%0D%0A%7D", "label": "deny demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22simon%22%2C%0D%0A++++%22roles%22%3A+%5B%0D%0A++++++++%22staff%22%2C%0D%0A++++++++%22developer%22%0D%0A++++%5D%0D%0A%7D&allow=%7B%0D%0A++++%22roles%22%3A+%5B%0D%0A++++++++%22developer%22%0D%0A++++%5D%0D%0A%7D", "label": "allow demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22cleopaws%22%2C%0D%0A++++%22roles%22%3A+%5B%22dog%22%5D%0D%0A%7D&allow=%7B%0D%0A++++%22roles%22%3A+%5B%0D%0A++++++++%22developer%22%0D%0A++++%5D%0D%0A%7D", "label": "deny demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22simon%22%0D%0A%7D&allow=%7B%0D%0A++++%22id%22%3A+%22*%22%0D%0A%7D", "label": "allow demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22bot%22%3A+%22readme-bot%22%0D%0A%7D&allow=%7B%0D%0A++++%22id%22%3A+%22*%22%0D%0A%7D", "label": "deny demo"… |
pages:tableview | pages | tableview | Table | The table page is the heart of Datasette: it allows users to interactively explore the contents of a database table, including sorting, filtering, Full-text search and applying Facets . The HTML interface is worth spending some time exploring. As with other pages, you can return the JSON data by appending .json to the URL path, before any ? query string arguments. The query string arguments are described in more detail here: Table arguments You can also use the table page to interactively construct a SQL query - by applying different filters and a sort order for example - and then click the "View and edit SQL" link to see the SQL query that was used for the page and edit and re-submit it. Some examples: ../items lists all of the line-items registered by UK MPs as potential conflicts of interest. It demonstrates Datasette's support for Full-text search . ../antiquities-act%2Factions_under_antiquities_act is an interface for exploring the "actions under the antiquities act" data table published by FiveThirtyEight. ../global-power-plants?country_long=United+Kingdom&primary_fuel=Gas is a filtered table page showing every Gas power plant in the United Kingdom. It includes some default facets (configured using its metadata.json ) and uses the datasette-cluster-map plugin to show a map of the results. | ["Pages and API endpoints"] | [{"href": "https://register-of-members-interests.datasettes.com/regmem/items", "label": "../items"}, {"href": "https://fivethirtyeight.datasettes.com/fivethirtyeight/antiquities-act%2Factions_under_antiquities_act", "label": "../antiquities-act%2Factions_under_antiquities_act"}, {"href": "https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_facet=primary_fuel&_facet=owner&_facet=country_long&country_long__exact=United+Kingdom&primary_fuel=Gas", "label": "../global-power-plants?country_long=United+Kingdom&primary_fuel=Gas"}, {"href": "https://global-power-plants.datasettes.com/-/metadata", "label": "its metadata.json"}, {"href": "https://github.com/simonw/datasette-cluster-map", "label": "datasette-cluster-map"}] |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [sections] ( [id] TEXT PRIMARY KEY, [page] TEXT, [ref] TEXT, [title] TEXT, [content] TEXT, [breadcrumbs] TEXT, [references] TEXT );