rowid,title,content,sections_fts,rank
720,Secret configuration values,"Some plugins may need configuration that should stay secret - API keys for example. There are two ways in which you can store secret configuration values.
As environment variables . If your secret lives in an environment variable that is available to the Datasette process, you can indicate that the configuration value should be read from that environment variable like so:
[[[cog
config_example(cog, {
""plugins"": {
""datasette-auth-github"": {
""client_secret"": {
""$env"": ""GITHUB_CLIENT_SECRET""
}
}
}
})
]]]
[[[end]]]
As values in separate files . Your secrets can also live in files on disk. To specify a secret should be read from a file, provide the full file path like this:
[[[cog
config_example(cog, {
""plugins"": {
""datasette-auth-github"": {
""client_secret"": {
""$file"": ""/secrets/client-secret""
}
}
}
})
]]]
[[[end]]]
If you are publishing your data using the datasette publish family of commands, you can use the --plugin-secret option to set these secrets at publish time. For example, using Heroku you might run the following command:
datasette publish heroku my_database.db \
--name my-heroku-app-demo \
--install=datasette-auth-github \
--plugin-secret datasette-auth-github client_id your_client_id \
--plugin-secret datasette-auth-github client_secret your_client_secret
This will set the necessary environment variables and add the following to the deployed metadata.yaml :
[[[cog
config_example(cog, {
""plugins"": {
""datasette-auth-github"": {
""client_id"": {
""$env"": ""DATASETTE_AUTH_GITHUB_CLIENT_ID""
},
""client_secret"": {
""$env"": ""DATASETTE_AUTH_GITHUB_CLIENT_SECRET""
}
}
}
})
]]]
[[[end]]]",1,
719,Plugin configuration,"Plugins can have their own configuration, embedded in a configuration file . Configuration options for plugins live within a ""plugins"" key in that file, which can be included at the root, database or table level.
Here is an example of some plugin configuration for a specific table:
[[[cog
from metadata_doc import config_example
config_example(cog, {
""databases"": {
""sf-trees"": {
""tables"": {
""Street_Tree_List"": {
""plugins"": {
""datasette-cluster-map"": {
""latitude_column"": ""lat"",
""longitude_column"": ""lng""
}
}
}
}
}
}
})
]]]
[[[end]]]
This tells the datasette-cluster-map column which latitude and longitude columns should be used for a table called Street_Tree_List inside a database file called sf-trees.db .",1,
718,Seeing what plugins are installed,"You can see a list of installed plugins by navigating to the /-/plugins page of your Datasette instance - for example: https://fivethirtyeight.datasettes.com/-/plugins
You can also use the datasette plugins command:
datasette plugins
Which outputs:
[
{
""name"": ""datasette_json_html"",
""static"": false,
""templates"": false,
""version"": ""0.4.0""
}
]
[[[cog
from datasette import cli
from click.testing import CliRunner
import textwrap, json
cog.out(""\n"")
result = CliRunner().invoke(cli.cli, [""plugins"", ""--all""])
# cog.out() with text containing newlines was unindenting for some reason
cog.outl(""If you run ``datasette plugins --all`` it will include default plugins that ship as part of Datasette:\n"")
cog.outl("".. code-block:: json\n"")
plugins = [p for p in json.loads(result.output) if p[""name""].startswith(""datasette."")]
indented = textwrap.indent(json.dumps(plugins, indent=4), "" "")
for line in indented.split(""\n""):
cog.outl(line)
cog.out(""\n\n"")
]]]
If you run datasette plugins --all it will include default plugins that ship as part of Datasette:
[
{
""name"": ""datasette.actor_auth_cookie"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""actor_from_request""
]
},
{
""name"": ""datasette.blob_renderer"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""register_output_renderer""
]
},
{
""name"": ""datasette.default_actions"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""register_actions""
]
},
{
""name"": ""datasette.default_column_types"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""register_column_types""
]
},
{
""name"": ""datasette.default_debug_menu"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""jump_items_sql""
]
},
{
""name"": ""datasette.default_jump_items"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""jump_items_sql""
]
},
{
""name"": ""datasette.default_magic_parameters"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""register_magic_parameters""
]
},
{
""name"": ""datasette.default_permissions"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""canned_queries"",
""permission_resources_sql""
]
},
{
""name"": ""datasette.default_permissions.tokens"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""actor_from_request"",
""register_token_handler""
]
},
{
""name"": ""datasette.events"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""register_events"",
""write_wrapper""
]
},
{
""name"": ""datasette.facets"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""register_facet_classes""
]
},
{
""name"": ""datasette.filters"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""filters_from_request""
]
},
{
""name"": ""datasette.forbidden"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""forbidden""
]
},
{
""name"": ""datasette.handle_exception"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""handle_exception""
]
},
{
""name"": ""datasette.publish.cloudrun"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""publish_subcommand""
]
},
{
""name"": ""datasette.publish.heroku"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""publish_subcommand""
]
},
{
""name"": ""datasette.sql_functions"",
""static"": false,
""templates"": false,
""version"": null,
""hooks"": [
""prepare_connection""
]
}
]
[[[end]]]
You can add the --plugins-dir= option to include any plugins found in that directory.
Add --requirements to output a list of installed plugins that can then be installed in another Datasette instance using datasette install -r requirements.txt :
datasette plugins --requirements
The output will look something like this:
datasette-codespaces==0.1.1
datasette-graphql==2.2
datasette-json-html==1.0.1
datasette-pretty-json==0.2.2
datasette-x-forwarded-host==0.1
To write that to a requirements.txt file, run this:
datasette plugins --requirements > requirements.txt",1,
717,Controlling which plugins are loaded,"Datasette defaults to loading every plugin that is installed in the same virtual environment as Datasette itself.
You can set the DATASETTE_LOAD_PLUGINS environment variable to a comma-separated list of plugin names to load a controlled subset of plugins instead.
For example, to load just the datasette-vega and datasette-cluster-map plugins, set DATASETTE_LOAD_PLUGINS to datasette-vega,datasette-cluster-map :
export DATASETTE_LOAD_PLUGINS='datasette-vega,datasette-cluster-map'
datasette mydb.db
Or:
DATASETTE_LOAD_PLUGINS='datasette-vega,datasette-cluster-map' \
datasette mydb.db
To disable the loading of all additional plugins, set DATASETTE_LOAD_PLUGINS to an empty string:
export DATASETTE_LOAD_PLUGINS=''
datasette mydb.db
A quick way to test this setting is to use it with the datasette plugins command:
DATASETTE_LOAD_PLUGINS='datasette-vega' datasette plugins
This should output the following:
[
{
""name"": ""datasette-vega"",
""static"": true,
""templates"": false,
""version"": ""0.6.2"",
""hooks"": [
""extra_css_urls"",
""extra_js_urls""
]
}
]",1,
716,Deploying plugins using datasette publish,"The datasette publish and datasette package commands both take an optional --install argument. You can use this one or more times to tell Datasette to pip install specific plugins as part of the process:
datasette publish cloudrun mydb.db --install=datasette-vega
You can use the name of a package on PyPI or any of the other valid arguments to pip install such as a URL to a .zip file:
datasette publish cloudrun mydb.db \
--install=https://url-to-my-package.zip",1,
715,One-off plugins using --plugins-dir,"You can also define one-off per-project plugins by saving them as plugin_name.py functions in a plugins/ folder and then passing that folder to datasette using the --plugins-dir option:
datasette mydb.db --plugins-dir=plugins/",1,
714,Installing plugins,"If a plugin has been packaged for distribution using setuptools you can use the plugin by installing it alongside Datasette in the same virtual environment or Docker container.
You can install plugins using the datasette install command:
datasette install datasette-vega
You can uninstall plugins with datasette uninstall :
datasette uninstall datasette-vega
You can upgrade plugins with datasette install --upgrade or datasette install -U :
datasette install -U datasette-vega
This command can also be used to upgrade Datasette itself to the latest released version:
datasette install -U datasette
You can install multiple plugins at once by listing them as lines in a requirements.txt file like this:
datasette-vega
datasette-cluster-map
Then pass that file to datasette install -r :
datasette install -r requirements.txt
The install and uninstall commands are thin wrappers around pip install and pip uninstall , which ensure that they run pip in the same virtual environment as Datasette itself.",1,
713,Plugins,"Datasette's plugin system allows additional features to be implemented as Python
code (or front-end JavaScript) which can be wrapped up in a separate Python
package. The underlying mechanism uses pluggy .
See the Datasette plugins directory for a list of existing plugins, or take a look at the
datasette-plugin topic on GitHub.
Things you can do with plugins include:
Add visualizations to Datasette, for example
datasette-cluster-map and
datasette-vega .
Make new custom SQL functions available for use within Datasette, for example
datasette-haversine and
datasette-jellyfish .
Define custom output formats with custom extensions, for example datasette-atom and
datasette-ics .
Add template functions that can be called within your Jinja custom templates,
for example datasette-render-markdown .
Customize how database values are rendered in the Datasette interface, for example
datasette-render-binary and
datasette-pretty-json .
Customize how Datasette's authentication and permissions systems work, for example datasette-auth-passwords and
datasette-permissions-sql .",1,
712,debug-menu,Controls if the various debug pages are displayed in the jump menu.,1,
711,permissions-debug,Actor is allowed to view the /-/permissions debug tools.,1,
710,execute-sql,"Actor is allowed to run arbitrary SQL queries against a specific database, e.g. https://latest.datasette.io/fixtures/-/query?sql=select+100
resource - datasette.resources.DatabaseResource(database)
database is the name of the database (string)
See also the default_allow_sql setting .",1,
709,drop-table,"Actor is allowed to drop a database table.
resource - datasette.resources.TableResource(database, table)
database is the name of the database (string)
table is the name of the table (string)",1,
708,set-column-type,"Actor is allowed to set assigned column types for columns in a table.
resource - datasette.resources.TableResource(database, table)
database is the name of the database (string)
table is the name of the table (string)",1,
707,alter-table,"Actor is allowed to alter a database table.
resource - datasette.resources.TableResource(database, table)
database is the name of the database (string)
table is the name of the table (string)",1,
706,create-table,"Actor is allowed to create a database table.
resource - datasette.resources.DatabaseResource(database)
database is the name of the database (string)",1,
705,update-row,"Actor is allowed to update rows in a table.
resource - datasette.resources.TableResource(database, table)
database is the name of the database (string)
table is the name of the table (string)",1,
704,delete-row,"Actor is allowed to delete rows from a table.
resource - datasette.resources.TableResource(database, table)
database is the name of the database (string)
table is the name of the table (string)",1,
703,insert-row,"Actor is allowed to insert rows into a table.
resource - datasette.resources.TableResource(database, table)
database is the name of the database (string)
table is the name of the table (string)",1,
702,view-query,"Actor is allowed to view (and execute) a canned query page, e.g. https://latest.datasette.io/fixtures/pragma_cache_size - this includes executing Writable canned queries .
resource - datasette.resources.QueryResource(database, query)
database is the name of the database (string)
query is the name of the canned query (string)",1,
701,view-table,"Actor is allowed to view a table (or view) page, e.g. https://latest.datasette.io/fixtures/complex_foreign_keys
resource - datasette.resources.TableResource(database, table)
database is the name of the database (string)
table is the name of the table (string)",1,
700,view-database-download,"Actor is allowed to download a database, e.g. https://latest.datasette.io/fixtures.db
resource - datasette.resources.DatabaseResource(database)
database is the name of the database (string)",1,
699,view-database,"Actor is allowed to view a database page, e.g. https://latest.datasette.io/fixtures
resource - datasette.permissions.DatabaseResource(database)
database is the name of the database (string)",1,
698,view-instance,"Top level permission - Actor is allowed to view any pages within this instance, starting at https://latest.datasette.io/",1,
697,Built-in actions,"This section lists all of the permission checks that are carried out by Datasette core, along with the resource if it was passed.",1,
696,The /-/logout page,The page at /-/logout provides the ability to log out of a ds_actor cookie authentication session.,1,
695,Including an expiry time,"ds_actor cookies can optionally include a signed expiry timestamp, after which the cookies will no longer be valid. Authentication plugins may chose to use this mechanism to limit the lifetime of the cookie. For example, if a plugin implements single-sign-on against another source it may decide to set short-lived cookies so that if the user is removed from the SSO system their existing Datasette cookies will stop working shortly afterwards.
To include an expiry pass expire_after= to datasette.set_actor_cookie() with a number of seconds. For example, to expire in 24 hours:
response = Response.redirect(""/"")
datasette.set_actor_cookie(
response, {""id"": ""cleopaws""}, expire_after=60 * 60 * 24
)
The resulting cookie will encode data that looks something like this:
{
""a"": {
""id"": ""cleopaws""
},
""e"": ""1jjSji""
}",1,
694,The ds_actor cookie,"Datasette includes a default authentication plugin which looks for a signed ds_actor cookie containing a JSON actor dictionary. This is how the root actor mechanism works.
Authentication plugins can set signed ds_actor cookies themselves like so:
response = Response.redirect(""/"")
datasette.set_actor_cookie(response, {""id"": ""cleopaws""})
The shape of data encoded in the cookie is as follows:
{
""a"": {
""id"": ""cleopaws""
}
}
To implement logout in a plugin, use the delete_actor_cookie() method:
response = Response.redirect(""/"")
datasette.delete_actor_cookie(response)",1,
693,Permission check view,"The /-/check endpoint evaluates a single action/resource pair and returns information indicating whether the access was allowed along with diagnostic information.
This endpoint provides an interactive HTML form interface. Add .json to the URL path (e.g. /-/check.json?action=view-instance ) to get the raw JSON response instead.
Pass ?action= to specify the action to check, and optional ?parent= and ?child= parameters to specify the resource.",1,
692,Permission rules view,"The /-/rules endpoint displays all permission rules (both allow and deny) for each candidate resource for the requested action.
This endpoint provides an interactive HTML form interface. Add .json to the URL path (e.g. /-/rules.json?action=view-table ) to get the raw JSON response instead.
Pass ?action= as a query parameter to specify which action to check.
This endpoint requires the permissions-debug permission.",1,
691,Allowed resources view,"The /-/allowed endpoint displays resources that the current actor can access for a specified action .
This endpoint provides an interactive HTML form interface. Add .json to the URL path (e.g. /-/allowed.json ) to get the raw JSON response instead.
Pass ?action=view-table (or another action) to select the action. Optional parent= and child= query parameters can narrow the results to a specific database/table pair.
This endpoint is publicly accessible to help users understand their own permissions. The potentially sensitive reason field is only shown to users with the permissions-debug permission - it shows the plugins and explanatory reasons that were responsible for each decision.",1,
690,Permissions debug tools,"The debug tool at /-/permissions is available to any actor with the permissions-debug permission. By default this is just the authenticated root user but you can open it up to all users by starting Datasette like this:
datasette -s permissions.permissions-debug true data.db
The page shows the permission checks that have been carried out by the Datasette instance.
It also provides an interface for running hypothetical permission checks against a hypothetical actor. This is a useful way of confirming that your configured permissions work in the way you expect.
This is designed to help administrators and plugin authors understand exactly how permission checks are being carried out, in order to effectively configure Datasette's permission system.",1,
689,actor_matches_allow(),"Plugins that wish to implement this same ""allow"" block permissions scheme can take advantage of the datasette.utils.actor_matches_allow(actor, allow) function:
from datasette.utils import actor_matches_allow
actor_matches_allow({""id"": ""root""}, {""id"": ""*""})
# returns True
The currently authenticated actor is made available to plugins as request.actor .",1,
688,Checking permissions in plugins,"Datasette plugins can check if an actor has permission to perform an action using await .allowed(*, action, resource, actor=None) —for example:
from datasette.resources import TableResource
can_edit = await datasette.allowed(
action=""update-row"",
resource=TableResource(database=""fixtures"", table=""facetable""),
actor=request.actor,
)
Use await .ensure_permission(action, resource=None, actor=None) when you need to enforce a permission and
raise a Forbidden error automatically.
Plugins that define new operations should return Action
objects from register_actions(datasette) and can supply additional allow/deny
rules by returning PermissionSQL objects from the
permission_resources_sql(datasette, actor, action) hook. Those rules are merged with
configuration allow blocks and actor restrictions to determine the final
result for each check.",1,
687,Restricting the actions that a token can perform,"Tokens created using datasette create-token ACTOR_ID will inherit all of the permissions of the actor that they are associated with.
You can pass additional options to create tokens that are restricted to a subset of that actor's permissions.
To restrict the token to just specific permissions against all available databases, use the --all option:
datasette create-token root --all insert-row --all update-row
This option can be passed as many times as you like. In the above example the token will only be allowed to insert and update rows.
You can also restrict permissions such that they can only be used within specific databases:
datasette create-token root --database mydatabase insert-row
The resulting token will only be able to insert rows, and only to tables in the mydatabase database.
Finally, you can restrict permissions to individual resources - tables, SQL views and named queries - within a specific database:
datasette create-token root --resource mydatabase mytable insert-row
These options have short versions: -a for --all , -d for --database and -r for --resource .
You can add --debug to see a JSON representation of the token that has been created. Here's a full example:
datasette create-token root \
--secret mysecret \
--all view-instance \
--all view-table \
--database docs view-query \
--resource docs documents insert-row \
--resource docs documents update-row \
--debug
This example outputs the following:
dstok_.eJxFizEKgDAMRe_y5w4qYrFXERGxDkVsMI0uxbubdjFL8l_ez1jhwEQCA6Fjjxp90qtkuHawzdjYrh8MFobLxZ_wBH0_gtnAF-hpS5VfmF8D_lnd97lHqUJgLd6sls4H1qwlhA.nH_7RecYHj5qSzvjhMU95iy0Xlc
Decoded:
{
""a"": ""root"",
""token"": ""dstok"",
""t"": 1670907246,
""_r"": {
""a"": [
""vi"",
""vt""
],
""d"": {
""docs"": [
""vq""
]
},
""r"": {
""docs"": {
""documents"": [
""ir"",
""ur""
]
}
}
}
}
Restrictions act as an allowlist layered on top of the actor's existing
permissions. They can only remove access the actor would otherwise have—they
cannot grant new access. If the underlying actor is denied by allow rules in
datasette.yaml or by a plugin, a token that lists that resource in its
""_r"" section will still be denied.
To create tokens with restrictions in Python code, use the TokenRestrictions builder and pass it to datasette.create_token() .",1,
686,datasette create-token,"You can also create tokens on the command line using the datasette create-token command.
This command takes one required argument - the ID of the actor to be associated with the created token.
You can specify a -e/--expires-after option in seconds. If omitted, the token will never expire.
The command will sign the token using the DATASETTE_SECRET environment variable, if available. You can also pass the secret using the --secret option.
This means you can run the command locally to create tokens for use with a deployed Datasette instance, provided you know that instance's secret.
To create a token for the root actor that will expire in one hour:
datasette create-token root --expires-after 3600
To create a token that never expires using a specific secret:
datasette create-token root --secret my-secret-goes-here",1,
685,API Tokens,"Datasette includes a default mechanism for generating API tokens that can be used to authenticate requests.
Authenticated users can create new API tokens using a form on the /-/create-token page.
Tokens created in this way can be further restricted to only allow access to specific actions, or to limit those actions to specific databases, tables or queries.
Created tokens can then be passed in the Authorization: Bearer $token header of HTTP requests to Datasette.
A token created by a user will include that user's ""id"" in the token payload, so any permissions granted to that user based on their ID can be made available to the token as well.
When one of these a token accompanies a request, the actor for that request will have the following shape:
{
""id"": ""user_id"",
""token"": ""dstok"",
""token_expires"": 1667717426
}
The ""id"" field duplicates the ID of the actor who first created the token.
The ""token"" field identifies that this actor was authenticated using a Datasette signed token ( dstok ).
The ""token_expires"" field, if present, indicates that the token will expire after that integer timestamp.
The /-/create-token page cannot be accessed by actors that are authenticated with a ""token"": ""some-value"" property. This is to prevent API tokens from being used to create more tokens.
Datasette plugins that implement their own form of API token authentication should follow this convention.
You can disable the signed token feature entirely using the allow_signed_tokens setting.",1,
684,Other permissions in ,"For all other permissions, you can use one or more ""permissions"" blocks in your datasette.yaml configuration file.
To grant access to the permissions debug tool to all signed in users, you can grant permissions-debug to any actor with an id matching the wildcard * by adding this a the root of your configuration:
[[[cog
config_example(cog, """"""
permissions:
debug-menu:
id: '*'
"""""")
]]]
[[[end]]]
To grant create-table to the user with id of editor for the docs database:
[[[cog
config_example(cog, """"""
databases:
docs:
permissions:
create-table:
id: editor
"""""")
]]]
[[[end]]]
Other table-scoped write permissions, including set-column-type , can be configured in the same place.
And for insert-row against the reports table in that docs database:
[[[cog
config_example(cog, """"""
databases:
docs:
tables:
reports:
permissions:
insert-row:
id: editor
"""""")
]]]
[[[end]]]
The permissions debug tool can be useful for helping test permissions that you have configured in this way.",1,
683,Controlling the ability to execute arbitrary SQL,"Datasette defaults to allowing any site visitor to execute their own custom SQL queries, for example using the form on the database page or by appending a ?_where= parameter to the table page like this .
Access to this ability is controlled by the execute-sql permission.
The easiest way to disable arbitrary SQL queries is using the default_allow_sql setting when you first start Datasette running.
You can alternatively use an ""allow_sql"" block to control who is allowed to execute arbitrary SQL queries.
To prevent any user from executing arbitrary SQL queries, use this:
[[[cog
config_example(cog, """"""
allow_sql: false
"""""")
]]]
[[[end]]]
To enable just the root user to execute SQL for all databases in your instance, use the following:
[[[cog
config_example(cog, """"""
allow_sql:
id: root
"""""")
]]]
[[[end]]]
To limit this ability for just one specific database, use this:
[[[cog
config_example(cog, """"""
databases:
mydatabase:
allow_sql:
id: root
"""""")
]]]
[[[end]]]",1,
682,Access to specific canned queries,"Canned queries allow you to configure named SQL queries in your datasette.yaml that can be executed by users. These queries can be set up to both read and write to the database, so controlling who can execute them can be important.
To limit access to the add_name canned query in your dogs.db database to just the root user :
[[[cog
config_example(cog, """"""
databases:
dogs:
queries:
add_name:
sql: INSERT INTO names (name) VALUES (:name)
write: true
allow:
id:
- root
"""""")
]]]
[[[end]]]",1,
681,Access to specific tables and views,"To limit access to the users table in your bakery.db database:
[[[cog
config_example(cog, """"""
databases:
bakery:
tables:
users:
allow:
id: '*'
"""""")
]]]
[[[end]]]
This works for SQL views as well - you can list their names in the ""tables"" block above in the same way as regular tables.
Restricting access to tables and views in this way will NOT prevent users from querying them using arbitrary SQL queries, like this for example.
If you are restricting access to specific tables you should also use the ""allow_sql"" block to prevent users from bypassing the limit with their own SQL queries - see Controlling the ability to execute arbitrary SQL .",1,
680,Access to specific databases,"To limit access to a specific private.db database to just authenticated users, use the ""allow"" block like this:
[[[cog
config_example(cog, """"""
databases:
private:
allow:
id: ""*""
"""""")
]]]
[[[end]]]",1,
679,Access to an instance,"Here's how to restrict access to your entire Datasette instance to just the ""id"": ""root"" user:
[[[cog
from metadata_doc import config_example
config_example(cog, """"""
title: My private Datasette instance
allow:
id: root
"""""")
]]]
[[[end]]]
To deny access to all users, you can use ""allow"": false :
[[[cog
config_example(cog, """"""
title: My entirely inaccessible instance
allow: false
"""""")
]]]
[[[end]]]
One reason to do this is if you are using a Datasette plugin - such as datasette-permissions-sql - to control permissions instead.",1,
678,Access permissions in ,"There are two ways to configure permissions using datasette.yaml (or datasette.json ).
For simple visibility permissions you can use ""allow"" blocks in the root, database, table and query sections.
For other permissions you can use a ""permissions"" block, described in the next section .
You can limit who is allowed to view different parts of your Datasette instance using ""allow"" keys in your Configuration .
You can control the following:
Access to the entire Datasette instance
Access to specific databases
Access to specific tables and views
Access to specific Canned queries
If a user has permission to view a table they will be able to view that table, independent of if they have permission to view the database or instance that the table exists within.",1,
677,The /-/allow-debug tool,"The /-/allow-debug tool lets you try out different ""action"" blocks against different ""actor"" JSON objects. You can try that out here: https://latest.datasette.io/-/allow-debug",1,
676,"Defining permissions with ""allow"" blocks","One way to define permissions in Datasette is to use an ""allow"" block in the datasette.yaml file . This is a JSON document describing which actors are allowed to perform an action against a specific resource.
Each allow block is compiled into SQL and combined with any
plugin-provided rules to produce
the cascading allow/deny decisions that power await .allowed(*, action, resource, actor=None) .
The most basic form of allow block is this ( allow demo , deny demo ):
[[[cog
from metadata_doc import config_example
import textwrap
config_example(cog, textwrap.dedent(
""""""
allow:
id: root
"""""").strip(),
""YAML"", ""JSON""
)
]]]
[[[end]]]
This will match any actors with an ""id"" property of ""root"" - for example, an actor that looks like this:
{
""id"": ""root"",
""name"": ""Root User""
}
An allow block can specify ""deny all"" using false ( demo ):
[[[cog
from metadata_doc import config_example
import textwrap
config_example(cog, textwrap.dedent(
""""""
allow: false
"""""").strip(),
""YAML"", ""JSON""
)
]]]
[[[end]]]
An ""allow"" of true allows all access ( demo ):
[[[cog
from metadata_doc import config_example
import textwrap
config_example(cog, textwrap.dedent(
""""""
allow: true
"""""").strip(),
""YAML"", ""JSON""
)
]]]
[[[end]]]
Allow keys can provide a list of values. These will match any actor that has any of those values ( allow demo , deny demo ):
[[[cog
from metadata_doc import config_example
import textwrap
config_example(cog, textwrap.dedent(
""""""
allow:
id:
- simon
- cleopaws
"""""").strip(),
""YAML"", ""JSON""
)
]]]
[[[end]]]
This will match any actor with an ""id"" of either ""simon"" or ""cleopaws"" .
Actors can have properties that feature a list of values. These will be matched against the list of values in an allow block. Consider the following actor:
{
""id"": ""simon"",
""roles"": [""staff"", ""developer""]
}
This allow block will provide access to any actor that has ""developer"" as one of their roles ( allow demo , deny demo ):
[[[cog
from metadata_doc import config_example
import textwrap
config_example(cog, textwrap.dedent(
""""""
allow:
roles:
- developer
"""""").strip(),
""YAML"", ""JSON""
)
]]]
[[[end]]]
Note that ""roles"" is not a concept that is baked into Datasette - it's a convention that plugins can choose to implement and act on.
If you want to provide access to any actor with a value for a specific key, use ""*"" . For example, to match any logged-in user specify the following ( allow demo , deny demo ):
[[[cog
from metadata_doc import config_example
import textwrap
config_example(cog, textwrap.dedent(
""""""
allow:
id: ""*""
"""""").strip(),
""YAML"", ""JSON""
)
]]]
[[[end]]]
You can specify that only unauthenticated actors (from anonymous HTTP requests) should be allowed access using the special ""unauthenticated"": true key in an allow block ( allow demo , deny demo ):
[[[cog
from metadata_doc import config_example
import textwrap
config_example(cog, textwrap.dedent(
""""""
allow:
unauthenticated: true
"""""").strip(),
""YAML"", ""JSON""
)
]]]
[[[end]]]
Allow keys act as an ""or"" mechanism. An actor will be able to execute the query if any of their JSON properties match any of the values in the corresponding lists in the allow block. The following block will allow users with either a role of ""ops"" OR users who have an id of ""simon"" or ""cleopaws"" :
[[[cog
from metadata_doc import config_example
import textwrap
config_example(cog, textwrap.dedent(
""""""
allow:
id:
- simon
- cleopaws
role: ops
"""""").strip(),
""YAML"", ""JSON""
)
]]]
[[[end]]]
Demo for cleopaws , demo for ops role , demo for an actor matching neither rule .",1,
675,How permissions are resolved,"Datasette performs permission checks using the internal await .allowed(*, action, resource, actor=None) , method which accepts keyword arguments for action , resource and an optional actor .
resource should be an instance of the appropriate Resource subclass from datasette.resources —for example InstanceResource() , DatabaseResource(database=""... )`` or TableResource(database=""..."", table=""..."") . This defaults to InstanceResource() if not specified.
When a check runs Datasette gathers allow/deny rules from multiple sources and
compiles them into a SQL query. The resulting query describes all of the
resources an actor may access for that action, together with the reasons those
resources were allowed or denied. The combined sources are:
allow blocks configured in datasette.yaml .
Actor restrictions encoded into the actor dictionary or API token.
The ""root"" user shortcut when --root (or Datasette.root_enabled ) is active, replying True to all permission chucks unless configuration rules deny them at a more specific level.
Any additional SQL provided by plugins implementing permission_resources_sql(datasette, actor, action) .
Datasette evaluates the SQL to determine if the requested resource is
included. Explicit deny rules returned by configuration or plugins will block
access even if other rules allowed it.",1,
674,Denying all permissions by default,"By default, Datasette allows unauthenticated access to view databases, tables, and execute SQL queries.
You may want to run Datasette in a mode where all access is denied by default, and you explicitly grant permissions only to authenticated users, either using the --root mechanism or through configuration file rules or plugins.
Use the --default-deny command-line option to run Datasette in this mode:
datasette --default-deny data.db --root
With --default-deny enabled:
Anonymous users are denied access to view the instance, databases, tables, and queries
Authenticated users are also denied access unless they're explicitly granted permissions
The root user (when using --root ) still has access to everything
You can grant permissions using configuration file rules or plugins
For example, to allow only a specific user to access your instance:
datasette --default-deny data.db --config datasette.yaml
Where datasette.yaml contains:
allow:
id: alice
This configuration will deny access to everyone except the user with id of alice .",1,
673,Permissions,"Datasette's permissions system is built around SQL queries. Datasette and its plugins construct SQL queries to resolve the list of resources that an actor cas access.
The key question the permissions system answers is this:
Is this actor allowed to perform this action , optionally against this particular resource ?
Actors are described above .
An action is a string describing the action the actor would like to perform. A full list is provided below - examples include view-table and execute-sql .
A resource is the item the actor wishes to interact with - for example a specific database or table. Some actions, such as permissions-debug , are not associated with a particular resource.
Datasette's built-in view actions ( view-database , view-table etc) are allowed by Datasette's default configuration: unless you configure additional permission rules unauthenticated users will be allowed to access content.
Other actions, including those introduced by plugins, will default to deny .",1,
672,"Using the ""root"" actor","Datasette currently leaves almost all forms of authentication to plugins - datasette-auth-github for example.
The one exception is the ""root"" account, which you can sign into while using Datasette on your local machine. The root user has all permissions - they can perform any action regardless of other permission rules.
The --root flag is designed for local development and testing. When you start Datasette with --root , the root user automatically receives every permission, including:
All view permissions ( view-instance , view-database , view-table , etc.)
All write permissions ( insert-row , update-row , delete-row , create-table , alter-table , set-column-type , drop-table )
Debug permissions ( permissions-debug , debug-menu )
Any custom permissions defined by plugins
If you add explicit deny rules in datasette.yaml those can still block the
root actor from specific databases or tables.
The --root flag sets an internal root_enabled switch—without it, a signed-in user with {""id"": ""root""} is treated like any other actor.
To sign in as root, start Datasette using the --root command-line option, like this:
datasette --root
Datasette will output a single-use-only login URL on startup:
http://127.0.0.1:8001/-/auth-token?token=786fc524e0199d70dc9a581d851f466244e114ca92f33aa3b42a139e9388daa7
INFO: Started server process [25801]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
Click on that link and then visit http://127.0.0.1:8001/-/actor to confirm that you are authenticated as an actor that looks like this:
{
""id"": ""root""
}",1,
671,Actors,"Through plugins, Datasette can support both authenticated users (with cookies) and authenticated API clients (via authentication tokens). The word ""actor"" is used to cover both of these cases.
Every request to Datasette has an associated actor value, available in the code as request.actor . This can be None for unauthenticated requests, or a JSON compatible Python dictionary for authenticated users or API clients.
The actor dictionary can be any shape - the design of that data structure is left up to the plugins. Actors should always include a unique ""id"" string, as demonstrated by the ""root"" actor below.
Plugins can use the actor_from_request(datasette, request) hook to implement custom logic for authenticating an actor based on the incoming HTTP request.",1,
670,Authentication and permissions,"Datasette doesn't require authentication by default. Any visitor to a Datasette instance can explore the full data and execute read-only SQL queries.
Datasette can be configured to only allow authenticated users, or to control which databases, tables, and queries can be accessed by the public or by specific users. Datasette's plugin system can be used to add many different styles of authentication, such as user accounts, single sign-on or API keys.",1,
669,Selectors,"These are available on the selectors property of the datasetteManager object.
const DOM_SELECTORS = {
/** Should have one match */
jsonExportLink: "".export-links a[href*=json]"",
/** Event listeners that go outside of the main table, e.g. existing scroll listener */
tableWrapper: "".table-wrapper"",
table: ""table.rows-and-columns"",
aboveTablePanel: "".above-table-panel"",
// These could have multiple matches
/** Used for selecting table headers. Use makeColumnActions if you want to add menu items. */
tableHeaders: `table.rows-and-columns th`,
/** Used to add ""where"" clauses to query using direct manipulation */
filterRows: "".filter-row"",
/** Used to show top available enum values for a column (""facets"") */
facetResults: "".facet-results [data-column]"",
};",1,
668,makeColumnActions(columnDetails),"This method, if present, will be called when Datasette is rendering the cog action menu icons that appear at the top of the table view. By default these include options like ""Sort ascending/descending"" and ""Facet by this"", but plugins can return additional actions to be included in this menu.
The method will be called with a columnDetails object with the following keys:
columnName - string
The name of the column
columnNotNull - boolean
True if the column is defined as NOT NULL
columnType - string
The SQLite data type of the column
isPk - boolean
True if the column is part of the primary key
It should return a JavaScript array of objects each with a label and onClick property:
label - string
The human-readable label for the action
onClick(evt) - function
A function that will be called when the action is clicked
The evt object passed to the onClick is the standard browser event object that triggered the click.
This example plugin adds two menu items - one to copy the column name to the clipboard and another that displays the column metadata in an alert() window:
document.addEventListener('datasette_init', function(ev) {
ev.detail.registerPlugin('column-name-plugin', {
version: 0.1,
makeColumnActions: (columnDetails) => {
return [
{
label: 'Copy column to clipboard',
onClick: async (evt) => {
await navigator.clipboard.writeText(columnDetails.columnName)
}
},
{
label: 'Alert column metadata',
onClick: () => alert(JSON.stringify(columnDetails, null, 2))
}
];
}
});
});",1,
667,makeAboveTablePanelConfigs(),"This method should return a JavaScript array of objects defining additional panels to be added to the top of the table page. Each object should have the following:
id - string
A unique string ID for the panel, for example map-panel
label - string
A human-readable label for the panel
render(node) - function
A function that will be called with a DOM node to render the panel into
This example shows how a plugin might define a single panel:
document.addEventListener('datasette_init', function(ev) {
ev.detail.registerPlugin('panel-plugin', {
version: 0.1,
makeAboveTablePanelConfigs: () => {
return [
{
id: 'first-panel',
label: 'First panel',
render: node => {
node.innerHTML = '
My custom panel
This is a custom panel that I added using a JavaScript plugin
';
}
}
]
}
});
});
When a page with a table loads, all registered plugins that implement makeAboveTablePanelConfigs() will be called and panels they return will be added to the top of the table page.",1,
666,makeJumpSections(),"This method should return a JavaScript array of objects defining additional sections to be added to the blank state of the / jump menu, before the user starts typing a search.
Each object should have the following:
id - string
A unique string ID for the section, for example agent-chat
render(node, context) - function
A function that will be called with a DOM node to render the section into
The context object has the following keys:
navigationSearch
The custom element instance.
This example shows how a plugin might add a button for starting a new chat:
document.addEventListener('datasette_init', function(ev) {
ev.detail.registerPlugin('agent-plugin', {
version: 0.1,
makeJumpSections: () => {
return [
{
id: 'agent-chat',
render: node => {
node.innerHTML = '';
node.querySelector('button').addEventListener('click', () => {
location.href = '/-/agent/new';
});
}
}
];
}
});
});",1,
665,JavaScript plugin objects,"JavaScript plugins are blocks of code that can be registered with Datasette using the registerPlugin() method on the datasetteManager object.
The implementation object passed to this method should include a version key defining the plugin version, and one or more of the following named functions providing the implementation of the plugin:",1,
664,datasetteManager,"The datasetteManager object
VERSION - string
The version of Datasette
plugins - Map()
A Map of currently loaded plugin names to plugin implementations
registerPlugin(name, implementation)
Call this to register a plugin, passing its name and implementation
selectors - object
An object providing named aliases to useful CSS selectors, listed below",1,
663,The datasette_init event,"Datasette emits a custom event called datasette_init when the page is loaded. This event is dispatched on the document object, and includes a detail object with a reference to the datasetteManager object.
Your JavaScript code can listen out for this event using document.addEventListener() like this:
document.addEventListener(""datasette_init"", function (evt) {
const manager = evt.detail;
console.log(""Datasette version:"", manager.VERSION);
});",1,
662,JavaScript plugins,"Datasette can run custom JavaScript in several different ways:
Datasette plugins written in Python can use the extra_js_urls() or extra_body_script() plugin hooks to inject JavaScript into a page
Datasette instances with custom templates can include additional JavaScript in those templates
The extra_js_urls key in datasette.yaml can be used to include extra JavaScript
There are no limitations on what this JavaScript can do. It is executed directly by the browser, so it can manipulate the DOM, fetch additional data and do anything else that JavaScript is capable of.
Custom JavaScript has security implications, especially for authenticated Datasette instances where the JavaScript might run in the context of the authenticated user. It's important to carefully review any JavaScript you run in your Datasette instance.",1,
661,Dropping tables,"To drop a table, make a POST to //
/-/drop . This requires the drop-table permission.
POST //
/-/drop
Content-Type: application/json
Authorization: Bearer dstok_
Without a POST body this will return a status 200 with a note about how many rows will be deleted:
{
""ok"": true,
""database"": """",
""table"": ""
"",
""row_count"": 5,
""message"": ""Pass \""confirm\"": true to confirm""
}
If you pass the following POST body:
{
""confirm"": true
}
Then the table will be dropped and a status 200 response of {""ok"": true} will be returned.
Any errors will return {""errors"": [""... descriptive message ...""], ""ok"": false} , and a 400 status code for a bad input or a 403 status code for an authentication or permission error.",1,
660,Setting a column type,"To set a column type for a table column, make a POST to //
/-/set-column-type . This requires the set-column-type permission.
POST //
/-/set-column-type
Content-Type: application/json
Authorization: Bearer dstok_
{
""column"": ""title"",
""column_type"": {
""type"": ""email""
}
}
This will return a 200 response like this:
{
""ok"": true,
""database"": ""data"",
""table"": ""posts"",
""column"": ""title"",
""column_type"": {
""type"": ""email"",
""config"": null
}
}
To provide column type configuration, include a config object:
{
""column"": ""title"",
""column_type"": {
""type"": ""url"",
""config"": {
""max_length"": 200
}
}
}
To clear an existing column type assignment, set column_type to null :
{
""column"": ""title"",
""column_type"": null
}
This API stores the assignment in Datasette's internal database, so it can be used with immutable databases as well as mutable ones.
Any errors will return {""errors"": [""... descriptive message ...""], ""ok"": false} , and a 400 status code for a bad input or a 403 status code for an authentication or permission error.",1,
659,Creating a table from example data,"Instead of specifying columns directly you can instead pass a single example row or a list of rows .
Datasette will create a table with a schema that matches those rows and insert them for you:
POST //-/create
Content-Type: application/json
Authorization: Bearer dstok_
{
""table"": ""creatures"",
""rows"": [
{
""id"": 1,
""name"": ""Tarantula""
},
{
""id"": 2,
""name"": ""Kākāpō""
}
],
""pk"": ""id""
}
Doing this requires both the create-table and insert-row permissions.
The 201 response here will be similar to the columns form, but will also include the number of rows that were inserted as row_count :
{
""ok"": true,
""database"": ""data"",
""table"": ""creatures"",
""table_url"": ""http://127.0.0.1:8001/data/creatures"",
""table_api_url"": ""http://127.0.0.1:8001/data/creatures.json"",
""schema"": ""CREATE TABLE [creatures] (\n [id] INTEGER PRIMARY KEY,\n [name] TEXT\n)"",
""row_count"": 2
}
You can call the create endpoint multiple times for the same table provided you are specifying the table using the rows or row option. New rows will be inserted into the table each time. This means you can use this API if you are unsure if the relevant table has been created yet.
If you pass a row to the create endpoint with a primary key that already exists you will get an error that looks like this:
{
""ok"": false,
""errors"": [
""UNIQUE constraint failed: creatures.id""
]
}
You can avoid this error by passing the same ""ignore"": true or ""replace"": true options to the create endpoint as you can to the insert endpoint .
To use the ""replace"": true option you will also need the update-row permission.
Pass ""alter"": true to automatically add any missing columns to the existing table that are present in the rows you are submitting. This requires the alter-table permission.",1,
658,Creating a table,"To create a table, make a POST to //-/create . This requires the create-table permission.
POST //-/create
Content-Type: application/json
Authorization: Bearer dstok_
{
""table"": ""name_of_new_table"",
""columns"": [
{
""name"": ""id"",
""type"": ""integer""
},
{
""name"": ""title"",
""type"": ""text""
}
],
""pk"": ""id""
}
The JSON here describes the table that will be created:
table is the name of the table to create. This field is required.
columns is a list of columns to create. Each column is a dictionary with name and type keys.
name is the name of the column. This is required.
type is the type of the column. This is optional - if not provided, text will be assumed. The valid types are text , integer , float and blob .
pk is the primary key for the table. This is optional - if not provided, Datasette will create a SQLite table with a hidden rowid column.
If the primary key is an integer column, it will be configured to automatically increment for each new record.
If you set this to id without including an id column in the list of columns , Datasette will create an auto-incrementing integer ID column for you.
pks can be used instead of pk to create a compound primary key. It should be a JSON list of column names to use in that primary key.
ignore can be set to true to ignore existing rows by primary key if the table already exists.
replace can be set to true to replace existing rows by primary key if the table already exists. This requires the update-row permission.
alter can be set to true if you want to automatically add any missing columns to the table. This requires the alter-table permission.
If the table is successfully created this will return a 201 status code and the following response:
{
""ok"": true,
""database"": ""data"",
""table"": ""name_of_new_table"",
""table_url"": ""http://127.0.0.1:8001/data/name_of_new_table"",
""table_api_url"": ""http://127.0.0.1:8001/data/name_of_new_table.json"",
""schema"": ""CREATE TABLE [name_of_new_table] (\n [id] INTEGER PRIMARY KEY,\n [title] TEXT\n)""
}",1,
657,Deleting a row,"To delete a row, make a POST to //
//-/delete . This requires the delete-row permission.
POST //
//-/delete
Content-Type: application/json
Authorization: Bearer dstok_ here is the tilde-encoded primary key value of the row to delete - or a comma-separated list of primary key values if the table has a composite primary key.
If successful, this will return a 200 status code and a {""ok"": true} response body.
Any errors will return {""errors"": [""... descriptive message ...""], ""ok"": false} , and a 400 status code for a bad input or a 403 status code for an authentication or permission error.",1,
656,Updating a row,"To update a row, make a POST to //
//-/update . This requires the update-row permission.
POST //
//-/update
Content-Type: application/json
Authorization: Bearer dstok_
{
""update"": {
""text_column"": ""New text string"",
""integer_column"": 3,
""float_column"": 3.14
}
}
here is the tilde-encoded primary key value of the row to update - or a comma-separated list of primary key values if the table has a composite primary key.
You only need to pass the columns you want to update. Any other columns will be left unchanged.
If successful, this will return a 200 status code and a {""ok"": true} response body.
Add ""return"": true to the request body to return the updated row:
{
""update"": {
""title"": ""New title""
},
""return"": true
}
The returned JSON will look like this:
{
""ok"": true,
""row"": {
""id"": 1,
""title"": ""New title"",
""other_column"": ""Will be present here too""
}
}
Any errors will return {""errors"": [""... descriptive message ...""], ""ok"": false} , and a 400 status code for a bad input or a 403 status code for an authentication or permission error.
Pass ""alter: true to automatically add any missing columns to the table. This requires the alter-table permission.",1,
655,Upserting rows,"An upsert is an insert or update operation. If a row with a matching primary key already exists it will be updated - otherwise a new row will be inserted.
The upsert API is mostly the same shape as the insert API . It requires both the insert-row and update-row permissions.
POST //
/-/upsert
Content-Type: application/json
Authorization: Bearer dstok_
{
""rows"": [
{
""id"": 1,
""title"": ""Updated title for 1"",
""description"": ""Updated description for 1""
},
{
""id"": 2,
""description"": ""Updated description for 2"",
},
{
""id"": 3,
""title"": ""Item 3"",
""description"": ""Description for 3""
}
]
}
Imagine a table with a primary key of id and which already has rows with id values of 1 and 2 .
The above example will:
Update the row with id of 1 to set both title and description to the new values
Update the row with id of 2 to set title to the new value - description will be left unchanged
Insert a new row with id of 3 and both title and description set to the new values
Similar to /-/insert , a row key with an object can be used instead of a rows array to upsert a single row.
If successful, this will return a 200 status code and a {""ok"": true} response body.
Add ""return"": true to the request body to return full copies of the affected rows after they have been inserted or updated:
{
""rows"": [
{
""id"": 1,
""title"": ""Updated title for 1"",
""description"": ""Updated description for 1""
},
{
""id"": 2,
""description"": ""Updated description for 2"",
},
{
""id"": 3,
""title"": ""Item 3"",
""description"": ""Description for 3""
}
],
""return"": true
}
This will return the following:
{
""ok"": true,
""rows"": [
{
""id"": 1,
""title"": ""Updated title for 1"",
""description"": ""Updated description for 1""
},
{
""id"": 2,
""title"": ""Item 2"",
""description"": ""Updated description for 2""
},
{
""id"": 3,
""title"": ""Item 3"",
""description"": ""Description for 3""
}
]
}
When using upsert you must provide the primary key column (or columns if the table has a compound primary key) for every row, or you will get a 400 error:
{
""ok"": false,
""errors"": [
""Row 0 is missing primary key column(s): \""id\""""
]
}
If your table does not have an explicit primary key you should pass the SQLite rowid key instead.
Pass ""alter: true to automatically add any missing columns to the table. This requires the alter-table permission.",1,
654,Inserting rows,"This requires the insert-row permission.
A single row can be inserted using the ""row"" key:
POST //
/-/insert
Content-Type: application/json
Authorization: Bearer dstok_
{
""row"": {
""column1"": ""value1"",
""column2"": ""value2""
}
}
If successful, this will return a 201 status code and the newly inserted row, for example:
{
""rows"": [
{
""id"": 1,
""column1"": ""value1"",
""column2"": ""value2""
}
]
}
To insert multiple rows at a time, use the same API method but send a list of dictionaries as the ""rows"" key:
POST //
/-/insert
Content-Type: application/json
Authorization: Bearer dstok_
{
""rows"": [
{
""column1"": ""value1"",
""column2"": ""value2""
},
{
""column1"": ""value3"",
""column2"": ""value4""
}
]
}
If successful, this will return a 201 status code and a {""ok"": true} response body.
The maximum number rows that can be submitted at once defaults to 100, but this can be changed using the max_insert_rows setting.
To return the newly inserted rows, add the ""return"": true key to the request body:
{
""rows"": [
{
""column1"": ""value1"",
""column2"": ""value2""
},
{
""column1"": ""value3"",
""column2"": ""value4""
}
],
""return"": true
}
This will return the same ""rows"" key as the single row example above. There is a small performance penalty for using this option.
If any of your rows have a primary key that is already in use, you will get an error and none of the rows will be inserted:
{
""ok"": false,
""errors"": [
""UNIQUE constraint failed: new_table.id""
]
}
Pass ""ignore"": true to ignore these errors and insert the other rows:
{
""rows"": [
{
""id"": 1,
""column1"": ""value1"",
""column2"": ""value2""
},
{
""id"": 2,
""column1"": ""value3"",
""column2"": ""value4""
}
],
""ignore"": true
}
Or you can pass ""replace"": true to replace any rows with conflicting primary keys with the new values. This requires the update-row permission.
Pass ""alter: true to automatically add any missing columns to the table. This requires the alter-table permission.",1,
653,The JSON write API,"Datasette provides a write API for JSON data. This is a POST-only API that requires an authenticated API token, see API Tokens . The token will need to have the specified Permissions .",1,
652,Enabling CORS,"If you start Datasette with the --cors option, each JSON endpoint will be
served with the following additional HTTP headers:
[[[cog
from datasette.utils import add_cors_headers
import textwrap
headers = {}
add_cors_headers(headers)
output = ""\n"".join(""{}: {}"".format(k, v) for k, v in headers.items())
cog.out(""\n::\n\n"")
cog.out(textwrap.indent(output, ' '))
cog.out(""\n\n"")
]]]
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type
Access-Control-Expose-Headers: Link
Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS
Access-Control-Max-Age: 3600
[[[end]]]
This allows JavaScript running on any domain to make cross-origin
requests to interact with the Datasette API.
If you start Datasette without the --cors option only JavaScript running on
the same domain as Datasette will be able to access the API.
Here's how to serve data.db with CORS enabled:
datasette data.db --cors",1,
651,Discovering the JSON for a page,"Most of the HTML pages served by Datasette provide a mechanism for discovering their JSON equivalents using the HTML link mechanism.
You can find this near the top of the source code of those pages, looking like this:
The JSON URL is also made available in a Link HTTP header for the page:
Link: ; rel=""alternate""; type=""application/json+datasette""",1,
650,Expanding foreign key references,"Datasette can detect foreign key relationships and resolve those references into
labels. The HTML interface does this by default for every detected foreign key
column - you can turn that off using ?_labels=off .
You can request foreign keys be expanded in JSON using the _labels=on or
_label=COLUMN special query string parameters. Here's what an expanded row
looks like:
[
{
""rowid"": 1,
""TreeID"": 141565,
""qLegalStatus"": {
""value"": 1,
""label"": ""Permitted Site""
},
""qSpecies"": {
""value"": 1,
""label"": ""Myoporum laetum :: Myoporum""
},
""qAddress"": ""501X Baker St"",
""SiteOrder"": 1
}
]
The column in the foreign key table that is used for the label can be specified
in datasette.yaml - see label_column .",1,
649,Special table arguments,"?_col=COLUMN1&_col=COLUMN2
List specific columns to display. These will be shown along with any primary keys.
?_nocol=COLUMN1&_nocol=COLUMN2
List specific columns to hide - any column not listed will be displayed. Primary keys cannot be hidden.
?_labels=on/off
Expand foreign key references for every possible column. See below.
?_label=COLUMN1&_label=COLUMN2
Expand foreign key references for one or more specified columns.
?_size=1000 or ?_size=max
Sets a custom page size. This cannot exceed the max_returned_rows limit
passed to datasette serve . Use max to get max_returned_rows .
?_sort=COLUMN
Sorts the results by the specified column.
?_sort_desc=COLUMN
Sorts the results by the specified column in descending order.
?_search=keywords
For SQLite tables that have been configured for
full-text search executes a search
with the provided keywords.
?_search_COLUMN=keywords
Like _search= but allows you to specify the column to be searched, as
opposed to searching all columns that have been indexed by FTS.
?_searchmode=raw
With this option, queries passed to ?_search= or ?_search_COLUMN= will
not have special characters escaped. This means you can make use of the full
set of advanced SQLite FTS syntax ,
though this could potentially result in errors if the wrong syntax is used.
?_where=SQL-fragment
If the execute-sql permission is enabled, this parameter
can be used to pass one or more additional SQL fragments to be used in the
WHERE clause of the SQL used to query the table.
This is particularly useful if you are building a JavaScript application
that needs to do something creative but still wants the other conveniences
provided by the table view (such as faceting) and hence would like not to
have to construct a completely custom SQL query.
Some examples:
facetable?_where=_neighborhood like ""%c%""&_where=_city_id=3
facetable?_where=_city_id in (select id from facet_cities where name != ""Detroit"")
?_through={json}
This can be used to filter rows via a join against another table.
The JSON parameter must include three keys: table , column and value .
table must be a table that the current table is related to via a foreign key relationship.
column must be a column in that other table.
value is the value that you want to match against.
For example, to filter roadside_attractions to just show the attractions that have a characteristic of ""museum"", you would construct this JSON:
{
""table"": ""roadside_attraction_characteristics"",
""column"": ""characteristic_id"",
""value"": ""1""
}
As a URL, that looks like this:
?_through={%22table%22:%22roadside_attraction_characteristics%22,%22column%22:%22characteristic_id%22,%22value%22:%221%22}
Here's an example .
?_next=TOKEN
Pagination by continuation token - pass the token that was returned in the
""next"" property by the previous page.
?_facet=column
Facet by column. Can be applied multiple times, see Facets . Only works on the default JSON output, not on any of the custom shapes.
?_facet_size=100
Increase the number of facet results returned for each facet. Use ?_facet_size=max for the maximum available size, determined by max_returned_rows .
?_nofacet=1
Disable all facets and facet suggestions for this page, including any defined by Facets in configuration .
?_nosuggest=1
Disable facet suggestions for this page.
?_nocount=1
Disable the select count(*) query used on this page - a count of None will be returned instead.",1,
648,Column filter arguments,"You can filter the data returned by the table based on column values using a query string argument.
?column__exact=value or ?_column=value
Returns rows where the specified column exactly matches the value.
?column__not=value
Returns rows where the column does not match the value.
?column__contains=value
Rows where the string column contains the specified value ( column like ""%value%"" in SQL).
?column__notcontains=value
Rows where the string column does not contain the specified value ( column not like ""%value%"" in SQL).
?column__endswith=value
Rows where the string column ends with the specified value ( column like ""%value"" in SQL).
?column__startswith=value
Rows where the string column starts with the specified value ( column like ""value%"" in SQL).
?column__gt=value
Rows which are greater than the specified value.
?column__gte=value
Rows which are greater than or equal to the specified value.
?column__lt=value
Rows which are less than the specified value.
?column__lte=value
Rows which are less than or equal to the specified value.
?column__like=value
Match rows with a LIKE clause, case insensitive and with % as the wildcard character.
?column__notlike=value
Match rows that do not match the provided LIKE clause.
?column__glob=value
Similar to LIKE but uses Unix wildcard syntax and is case sensitive.
?column__in=value1,value2,value3
Rows where column matches any of the provided values.
You can use a comma separated string, or you can use a JSON array.
The JSON array option is useful if one of your matching values itself contains a comma:
?column__in=[""value"",""value,with,commas""]
?column__notin=value1,value2,value3
Rows where column does not match any of the provided values. The inverse of __in= . Also supports JSON arrays.
?column__arraycontains=value
Works against columns that contain JSON arrays - matches if any of the values in that array match the provided value.
This is only available if the json1 SQLite extension is enabled.
?column__arraynotcontains=value
Works against columns that contain JSON arrays - matches if none of the values in that array match the provided value.
This is only available if the json1 SQLite extension is enabled.
?column__date=value
Column is a datestamp occurring on the specified YYYY-MM-DD date, e.g. 2018-01-02 .
?column__isnull=1
Matches rows where the column is null.
?column__notnull=1
Matches rows where the column is not null.
?column__isblank=1
Matches rows where the column is blank, meaning null or the empty string.
?column__notblank=1
Matches rows where the column is not blank.",1,
647,Table arguments,The Datasette table view takes a number of special query string arguments.,1,
646,Special JSON arguments,"Every Datasette endpoint that can return JSON also accepts the following
query string arguments:
?_shape=SHAPE
The shape of the JSON to return, documented above.
?_nl=on
When used with ?_shape=array produces newline-delimited JSON objects.
?_json=COLUMN1&_json=COLUMN2
If any of your SQLite columns contain JSON values, you can use one or more
_json= parameters to request that those columns be returned as regular
JSON. Without this argument those columns will be returned as JSON objects
that have been double-encoded into a JSON string value.
Compare this query without the argument to this query using the argument
?_json_infinity=on
If your data contains infinity or -infinity values, Datasette will replace
them with None when returning them as JSON. If you pass _json_infinity=1
Datasette will instead return them as Infinity or -Infinity which is
invalid JSON but can be processed by some custom JSON parsers.
?_timelimit=MS
Sets a custom time limit for the query in ms. You can use this for optimistic
queries where you would like Datasette to give up if the query takes too
long, for example if you want to implement autocomplete search but only if
it can be executed in less than 10ms.
?_ttl=SECONDS
For how many seconds should this response be cached by HTTP proxies? Use
?_ttl=0 to disable HTTP caching entirely for this request.
?_trace=1
Turns on tracing for this page: SQL queries executed during the request will
be gathered and included in the response, either in a new ""_traces"" key
for JSON responses or at the bottom of the page if the response is in HTML.
The structure of the data returned here should be considered highly unstable
and very likely to change.
Only available if the trace_debug setting is enabled.",1,
645,Pagination,"The default JSON representation includes a ""next_url"" key which can be used to access the next page of results. If that key is null or missing then it means you have reached the final page of results.
Other representations include pagination information in the link HTTP header. That header will look something like this:
link: ; rel=""next""
Here is an example Python function built using requests that returns a list of all of the paginated items from one of these API endpoints:
def paginate(url):
items = []
while url:
response = requests.get(url)
try:
url = response.links.get(""next"").get(""url"")
except AttributeError:
url = None
items.extend(response.json())
return items",1,
644,Different shapes,"The _shape parameter can be used to access alternative formats for the
rows key which may be more convenient for your application. There are three
options:
?_shape=objects - ""rows"" is a list of JSON key/value objects - the default
?_shape=arrays - ""rows"" is a list of lists, where the order of values in each list matches the order of the columns
?_shape=array - a JSON array of objects - effectively just the ""rows"" key from the default representation
?_shape=array&_nl=on - a newline-separated list of JSON objects
?_shape=arrayfirst - a flat JSON array containing just the first value from each row
?_shape=object - a JSON object keyed using the primary keys of the rows
_shape=arrays looks like this:
{
""ok"": true,
""next"": null,
""rows"": [
[3, ""Detroit""],
[2, ""Los Angeles""],
[4, ""Memnonia""],
[1, ""San Francisco""]
]
}
_shape=array looks like this:
[
{
""id"": 3,
""name"": ""Detroit""
},
{
""id"": 2,
""name"": ""Los Angeles""
},
{
""id"": 4,
""name"": ""Memnonia""
},
{
""id"": 1,
""name"": ""San Francisco""
}
]
_shape=array&_nl=on looks like this:
{""id"": 1, ""value"": ""Myoporum laetum :: Myoporum""}
{""id"": 2, ""value"": ""Metrosideros excelsa :: New Zealand Xmas Tree""}
{""id"": 3, ""value"": ""Pinus radiata :: Monterey Pine""}
_shape=arrayfirst looks like this:
[1, 2, 3]
_shape=object looks like this:
{
""1"": {
""id"": 1,
""value"": ""Myoporum laetum :: Myoporum""
},
""2"": {
""id"": 2,
""value"": ""Metrosideros excelsa :: New Zealand Xmas Tree""
},
""3"": {
""id"": 3,
""value"": ""Pinus radiata :: Monterey Pine""
}
]
The object shape is only available for queries against tables - custom SQL
queries and views do not have an obvious primary key so cannot be returned using
this format.
The object keys are always strings. If your table has a compound primary
key, the object keys will be a comma-separated string.",1,
643,Default representation,"The default JSON representation of data from a SQLite table or custom query
looks like this:
{
""ok"": true,
""rows"": [
{
""id"": 3,
""name"": ""Detroit""
},
{
""id"": 2,
""name"": ""Los Angeles""
},
{
""id"": 4,
""name"": ""Memnonia""
},
{
""id"": 1,
""name"": ""San Francisco""
}
],
""truncated"": false
}
""ok"" is always true if an error did not occur.
The ""rows"" key is a list of objects, each one representing a row.
The ""truncated"" key lets you know if the query was truncated. This can happen if a SQL query returns more than 1,000 results (or the max_returned_rows setting).
For table pages, an additional key ""next"" may be present. This indicates that the next page in the pagination set can be retrieved using ?_next=VALUE .",1,
642,JSON API,"Datasette provides a JSON API for your SQLite databases. Anything you can do
through the Datasette user interface can also be accessed as JSON via the API.
To access the API for a page, either click on the .json link on that page or
edit the URL and add a .json extension to it.",1,
641,Querying polygons using within(),"The within() SQL function can be used to check if a point is within a geometry:
select
name
from
places
where
within(GeomFromText('POINT(-3.1724366 51.4704448)'), places.geom);
The GeomFromText() function takes a string of well-known text. Note that the order used here is longitude then latitude .
To run that same within() query in a way that benefits from the spatial index, use the following:
select
name
from
places
where
within(GeomFromText('POINT(-3.1724366 51.4704448)'), places.geom)
and rowid in (
SELECT pkid FROM idx_places_geom
where xmin < -3.1724366
and xmax > -3.1724366
and ymin < 51.4704448
and ymax > 51.4704448
);",1,
640,Importing GeoJSON polygons using Shapely,"Another common form of polygon data is the GeoJSON format. This can be imported into SpatiaLite directly, or by using the Shapely Python library.
Who's On First is an excellent source of openly licensed GeoJSON polygons. Let's import the geographical polygon for Wales. First, we can use the Who's On First Spelunker tool to find the record for Wales:
spelunker.whosonfirst.org/id/404227475
That page includes a link to the GeoJSON record, which can be accessed here:
data.whosonfirst.org/404/227/475/404227475.geojson
Here's Python code to create a SQLite database, enable SpatiaLite, create a places table and then add a record for Wales:
import sqlite3
conn = sqlite3.connect(""places.db"")
# Enable SpatialLite extension
conn.enable_load_extension(True)
conn.load_extension(""/usr/local/lib/mod_spatialite.dylib"")
# Create the masic countries table
conn.execute(""select InitSpatialMetadata(1)"")
conn.execute(
""create table places (id integer primary key, name text);""
)
# Add a MULTIPOLYGON Geometry column
conn.execute(
""SELECT AddGeometryColumn('places', 'geom', 4326, 'MULTIPOLYGON', 2);""
)
# Add a spatial index against the new column
conn.execute(""SELECT CreateSpatialIndex('places', 'geom');"")
# Now populate the table
from shapely.geometry.multipolygon import MultiPolygon
from shapely.geometry import shape
import requests
geojson = requests.get(
""https://data.whosonfirst.org/404/227/475/404227475.geojson""
).json()
# Convert to ""Well Known Text"" format
wkt = shape(geojson[""geometry""]).wkt
# Insert and commit the record
conn.execute(
""INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))"",
(""Wales"", wkt),
)
conn.commit()",1,
639,Importing shapefiles into SpatiaLite,"The shapefile format is a common format for distributing geospatial data. You can use the spatialite command-line tool to create a new database table from a shapefile.
Try it now with the North America shapefile available from the University of North Carolina Global River Database project. Download the file and unzip it (this will create files called narivs.dbf , narivs.prj , narivs.shp and narivs.shx in the current directory), then run the following:
spatialite rivers-database.db
SpatiaLite version ..: 4.3.0a Supported Extensions:
...
spatialite> .loadshp narivs rivers CP1252 23032
========
Loading shapefile at 'narivs' into SQLite table 'rivers'
...
Inserted 467973 rows into 'rivers' from SHAPEFILE
This will load the data from the narivs shapefile into a new database table called rivers .
Exit out of spatialite (using Ctrl+D ) and run Datasette against your new database like this:
datasette rivers-database.db \
--load-extension=/usr/local/lib/mod_spatialite.dylib
If you browse to http://localhost:8001/rivers-database/rivers you will see the new table... but the Geometry column will contain unreadable binary data (SpatiaLite uses a custom format based on WKB ).
The easiest way to turn this into semi-readable data is to use the SpatiaLite AsGeoJSON function. Try the following using the SQL query interface at http://localhost:8001/rivers-database :
select *, AsGeoJSON(Geometry) from rivers limit 10;
This will give you back an additional column of GeoJSON. You can copy and paste GeoJSON from this column into the debugging tool at geojson.io to visualize it on a map.
To see a more interesting example, try ordering the records with the longest geometry first. Since there are 467,000 rows in the table you will first need to increase the SQL time limit imposed by Datasette:
datasette rivers-database.db \
--load-extension=/usr/local/lib/mod_spatialite.dylib \
--setting sql_time_limit_ms 10000
Now try the following query:
select *, AsGeoJSON(Geometry) from rivers
order by length(Geometry) desc limit 10;",1,
638,Making use of a spatial index,"SpatiaLite spatial indexes are R*Trees. They allow you to run efficient bounding box queries using a sub-select, with a similar pattern to that used for Searches using custom SQL .
In the above example, the resulting index will be called idx_museums_point_geom . This takes the form of a SQLite virtual table. You can inspect its contents using the following query:
select * from idx_museums_point_geom limit 10;
Here's a live example: timezones-api.datasette.io/timezones/idx_timezones_Geometry
pkid
xmin
xmax
ymin
ymax
1
-8.601725578308105
-2.4930307865142822
4.162120819091797
10.74019718170166
2
-3.2607860565185547
1.27329421043396
4.539252281188965
11.174856185913086
3
32.997581481933594
47.98238754272461
3.3974475860595703
14.894054412841797
4
-8.66890811920166
11.997337341308594
18.9681453704834
37.296207427978516
5
36.43336486816406
43.300174713134766
12.354820251464844
18.070993423461914
You can now construct efficient bounding box queries that will make use of the index like this:
select * from museums where museums.rowid in (
SELECT pkid FROM idx_museums_point_geom
-- left-hand-edge of point > left-hand-edge of bbox (minx)
where xmin > :bbox_minx
-- right-hand-edge of point < right-hand-edge of bbox (maxx)
and xmax < :bbox_maxx
-- bottom-edge of point > bottom-edge of bbox (miny)
and ymin > :bbox_miny
-- top-edge of point < top-edge of bbox (maxy)
and ymax < :bbox_maxy
);
Spatial indexes can be created against polygon columns as well as point columns, in which case they will represent the minimum bounding rectangle of that polygon. This is useful for accelerating within queries, as seen in the Timezones API example.",1,
637,Spatial indexing latitude/longitude columns,"Here's a recipe for taking a table with existing latitude and longitude columns, adding a SpatiaLite POINT geometry column to that table, populating the new column and then populating a spatial index:
import sqlite3
conn = sqlite3.connect(""museums.db"")
# Lead the spatialite extension:
conn.enable_load_extension(True)
conn.load_extension(""/usr/local/lib/mod_spatialite.dylib"")
# Initialize spatial metadata for this database:
conn.execute(""select InitSpatialMetadata(1)"")
# Add a geometry column called point_geom to our museums table:
conn.execute(
""SELECT AddGeometryColumn('museums', 'point_geom', 4326, 'POINT', 2);""
)
# Now update that geometry column with the lat/lon points
conn.execute(""""""
UPDATE museums SET
point_geom = GeomFromText('POINT('||""longitude""||' '||""latitude""||')',4326);
"""""")
# Now add a spatial index to that column
conn.execute(
'select CreateSpatialIndex(""museums"", ""point_geom"");'
)
# If you don't commit your changes will not be persisted:
conn.commit()
conn.close()",1,
636,Installing SpatiaLite on Linux,"SpatiaLite is packaged for most Linux distributions.
apt install spatialite-bin libsqlite3-mod-spatialite
Depending on your distribution, you should be able to run Datasette something like this:
datasette --load-extension=/usr/lib/x86_64-linux-gnu/mod_spatialite.so
If you are unsure of the location of the module, try running locate mod_spatialite and see what comes back.",1,
635,Installing SpatiaLite on OS X,"The easiest way to install SpatiaLite on OS X is to use Homebrew .
brew update
brew install spatialite-tools
This will install the spatialite command-line tool and the mod_spatialite dynamic library.
You can now run Datasette like so:
datasette --load-extension=spatialite",1,
634,Installation,,1,
633,Warning,"The SpatiaLite extension adds a large number of additional SQL functions , some of which are not be safe for untrusted users to execute: they may cause the Datasette server to crash.
You should not expose a SpatiaLite-enabled Datasette instance to the public internet without taking extra measures to secure it against potentially harmful SQL queries.
The following steps are recommended:
Disable arbitrary SQL queries by untrusted users. See Controlling the ability to execute arbitrary SQL for ways to do this. The easiest is to start Datasette with the datasette --setting default_allow_sql off option.
Define Canned queries with the SQL queries that use SpatiaLite functions that you want people to be able to execute.
The Datasette SpatiaLite tutorial includes detailed instructions for running SpatiaLite safely using these techniques",1,
632,SpatiaLite,"The SpatiaLite module for SQLite adds features for handling geographic and spatial data. For an example of what you can do with it, see the tutorial Building a location to time zone API with SpatiaLite .
To use it with Datasette, you need to install the mod_spatialite dynamic library. This can then be loaded into Datasette using the --load-extension command-line option.
Datasette can look for SpatiaLite in common installation locations if you run it like this:
datasette --load-extension=spatialite --setting default_allow_sql off
If SpatiaLite is in another location, use the full path to the extension instead:
datasette --setting default_allow_sql off \
--load-extension=/usr/local/lib/mod_spatialite.dylib",1,
631,datasette-hashed-urls,"If you open a database file in immutable mode using the -i option, you can be assured that the content of that database will not change for the lifetime of the Datasette server.
The datasette-hashed-urls plugin implements an optimization where your database is served with part of the SHA-256 hash of the database contents baked into the URL.
A database at /fixtures will instead be served at /fixtures-aa7318b , and a year-long cache expiry header will be returned with those pages.
This will then be cached by both browsers and caching proxies such as Cloudflare or Fastly, providing a potentially significant performance boost.
To install the plugin, run the following:
datasette install datasette-hashed-urls
Prior to Datasette 0.61 hashed URL mode was a core Datasette feature, enabled using the hash_urls setting. This implementation has now been removed in favor of the datasette-hashed-urls plugin.
Prior to Datasette 0.28 hashed URL mode was the default behaviour for Datasette, since all database files were assumed to be immutable and unchanging. From 0.28 onwards the default has been to treat database files as mutable unless explicitly configured otherwise.",1,
630,HTTP caching,"If your database is immutable and guaranteed not to change, you can gain major performance improvements from Datasette by enabling HTTP caching.
This can work at two different levels. First, it can tell browsers to cache the results of queries and serve future requests from the browser cache.
More significantly, it allows you to run Datasette behind a caching proxy such as Varnish or use a cache provided by a hosted service such as Fastly or Cloudflare . This can provide incredible speed-ups since a query only needs to be executed by Datasette the first time it is accessed - all subsequent hits can then be served by the cache.
Using a caching proxy in this way could enable a Datasette-backed visualization to serve thousands of hits a second while running Datasette itself on extremely inexpensive hosting.
Datasette's integration with HTTP caches can be enabled using a combination of configuration options and query string arguments.
The default_cache_ttl setting sets the default HTTP cache TTL for all Datasette pages. This is 5 seconds unless you change it - you can set it to 0 if you wish to disable HTTP caching entirely.
You can also change the cache timeout on a per-request basis using the ?_ttl=10 query string parameter. This can be useful when you are working with the Datasette JSON API - you may decide that a specific query can be cached for a longer time, or maybe you need to set ?_ttl=0 for some requests for example if you are running a SQL order by random() query.",1,
629,"Using ""datasette inspect""","Counting the rows in a table can be a very expensive operation on larger databases. In immutable mode Datasette performs this count only once and caches the results, but this can still cause server startup time to increase by several seconds or more.
If you know that a database is never going to change you can precalculate the table row counts once and store then in a JSON file, then use that file when you later start the server.
To create a JSON file containing the calculated row counts for a database, use the following:
datasette inspect data.db --inspect-file=counts.json
Then later you can start Datasette against the counts.json file and use it to skip the row counting step and speed up server startup:
datasette -i data.db --inspect-file=counts.json
You need to use the -i immutable mode against the database file here or the counts from the JSON file will be ignored.
You will rarely need to use this optimization in every-day use, but several of the datasette publish commands described in Publishing data use this optimization for better performance when deploying a database file to a hosting provider.",1,
628,Immutable mode,"If you can be certain that a SQLite database file will not be changed by another process you can tell Datasette to open that file in immutable mode .
Doing so will disable all locking and change detection, which can result in improved query performance.
This also enables further optimizations relating to HTTP caching, described below.
To open a file in immutable mode pass it to the datasette command using the -i option:
datasette -i data.db
When you open a file in immutable mode like this Datasette will also calculate and cache the row counts for each table in that database when it first starts up, further improving performance.",1,
627,Performance and caching,"Datasette runs on top of SQLite, and SQLite has excellent performance. For small databases almost any query should return in just a few milliseconds, and larger databases (100s of MBs or even GBs of data) should perform extremely well provided your queries make sensible use of database indexes.
That said, there are a number of tricks you can use to improve Datasette's performance.",1,
626,Using secrets with datasette publish,"The datasette publish and datasette package commands both generate a secret for you automatically when Datasette is deployed.
This means that every time you deploy a new version of a Datasette project, a new secret will be generated. This will cause signed cookies to become invalid on every fresh deploy.
You can fix this by creating a secret that will be used for multiple deploys and passing it using the --secret option:
datasette publish cloudrun mydb.db --service=my-service --secret=cdb19e94283a20f9d42cca5",1,
625,Configuring the secret,"Datasette uses a secret string to sign secure values such as cookies.
If you do not provide a secret, Datasette will create one when it starts up. This secret will reset every time the Datasette server restarts though, so things like authentication cookies and API tokens will not stay valid between restarts.
You can pass a secret to Datasette in two ways: with the --secret command-line option or by setting a DATASETTE_SECRET environment variable.
datasette mydb.db --secret=SECRET_VALUE_HERE
Or:
export DATASETTE_SECRET=SECRET_VALUE_HERE
datasette mydb.db
One way to generate a secure random secret is to use Python like this:
python3 -c 'import secrets; print(secrets.token_hex(32))'
cdb19e94283a20f9d42cca50c5a4871c0aa07392db308755d60a1a5b9bb0fa52
Plugin authors can make use of this signing mechanism in their plugins using the datasette.sign() and datasette.unsign() methods.",1,
624,base_url,"If you are running Datasette behind a proxy, it may be useful to change the root path used for the Datasette instance.
For example, if you are sending traffic from https://www.example.com/tools/datasette/ through to a proxied Datasette instance you may wish Datasette to use /tools/datasette/ as its root URL.
You can do that like so:
datasette mydatabase.db --setting base_url /tools/datasette/",1,
623,trace_debug,"This setting enables appending ?_trace=1 to any page in order to see the SQL queries and other trace information that was used to generate that page.
Enable it like this:
datasette mydatabase.db --setting trace_debug 1
Some examples:
https://latest.datasette.io/?_trace=1
https://latest.datasette.io/fixtures/roadside_attractions?_trace=1
See datasette.tracer for details on how to hook into this mechanism as a plugin author.",1,
622,template_debug,"This setting enables template context debug mode, which is useful to help understand what variables are available to custom templates when you are writing them.
Enable it like this:
datasette mydatabase.db --setting template_debug 1
Now you can add ?_context=1 or &_context=1 to any Datasette page to see the context that was passed to that template.
Some examples:
https://latest.datasette.io/?_context=1
https://latest.datasette.io/fixtures?_context=1
https://latest.datasette.io/fixtures/roadside_attractions?_context=1",1,
621,force_https_urls,"Forces self-referential URLs in the JSON output to always use the https://
protocol. This is useful for cases where the application itself is hosted using
HTTP but is served to the outside world via a proxy that enables HTTPS.
datasette mydatabase.db --setting force_https_urls 1",1,
620,truncate_cells_html,"In the HTML table view, truncate any strings that are longer than this value.
The full value will still be available in CSV, JSON and on the individual row
HTML page. Set this to 0 to disable truncation.
datasette mydatabase.db --setting truncate_cells_html 0",1,