rowid,title,content,sections_fts,rank
613,A note about extensions,"SQLite supports extensions, such as SpatiaLite for geospatial operations.
These can be loaded using the --load-extension argument, like so:
datasette --load-extension=/usr/local/lib/mod_spatialite.dylib
Some Python installations do not include support for SQLite extensions. If this is the case you will see the following error when you attempt to load an extension:
Your Python installation does not have the ability to load SQLite extensions.
In some cases you may see the following error message instead:
AttributeError: 'sqlite3.Connection' object has no attribute 'enable_load_extension'
On macOS the easiest fix for this is to install Datasette using Homebrew:
brew install datasette
Use which datasette to confirm that datasette will run that version. The output should look something like this:
/usr/local/opt/datasette/bin/datasette
If you get a different location here such as /Library/Frameworks/Python.framework/Versions/3.10/bin/datasette you can run the following command to cause datasette to execute the Homebrew version instead:
alias datasette=$(echo $(brew --prefix datasette)/bin/datasette)
You can undo this operation using:
unalias datasette
If you need to run SQLite with extension support for other Python code, you can do so by install Python itself using Homebrew:
brew install python
Then executing Python using:
/usr/local/opt/python@3/libexec/bin/python
A more convenient way to work with this version of Python may be to use it to create a virtual environment:
/usr/local/opt/python@3/libexec/bin/python -m venv datasette-venv
Then activate it like this:
source datasette-venv/bin/activate
Now running python and pip will work against a version of Python 3 that includes support for SQLite extensions:
pip install datasette
which datasette
datasette --version",37,
612,Installing plugins,"If you want to install plugins into your local Datasette Docker image you can do
so using the following recipe. This will install the plugins and then save a
brand new local image called datasette-with-plugins :
docker run datasetteproject/datasette \
pip install datasette-vega
docker commit $(docker ps -lq) datasette-with-plugins
You can now run the new custom image like so:
docker run -p 8001:8001 -v `pwd`:/mnt \
datasette-with-plugins \
datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db
You can confirm that the plugins are installed by visiting
http://127.0.0.1:8001/-/plugins
Some plugins such as datasette-ripgrep may need additional system packages. You can install these by running apt-get install inside the container:
docker run datasette-057a0 bash -c '
apt-get update &&
apt-get install ripgrep &&
pip install datasette-ripgrep'
docker commit $(docker ps -lq) datasette-with-ripgrep",37,
611,Loading SpatiaLite,"The datasetteproject/datasette image includes a recent version of the
SpatiaLite extension for SQLite. To load and enable that
module, use the following command:
docker run -p 8001:8001 -v `pwd`:/mnt \
datasetteproject/datasette \
datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db \
--load-extension=spatialite
You can confirm that SpatiaLite is successfully loaded by visiting
http://127.0.0.1:8001/-/versions",37,
610,Using Docker,"A Docker image containing the latest release of Datasette is published to Docker
Hub here: https://hub.docker.com/r/datasetteproject/datasette/
If you have Docker installed (for example with Docker for Mac on OS X) you can download and run this
image like so:
docker run -p 8001:8001 -v `pwd`:/mnt \
datasetteproject/datasette \
datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db
This will start an instance of Datasette running on your machine's port 8001,
serving the fixtures.db file in your current directory.
Now visit http://127.0.0.1:8001/ to access Datasette.
(You can download a copy of fixtures.db from
https://latest.datasette.io/fixtures.db )
To upgrade to the most recent release of Datasette, run the following:
docker pull datasetteproject/datasette",37,
609,Upgrading packages using pipx,"You can upgrade your pipx installation to the latest release of Datasette using pipx upgrade datasette :
pipx upgrade datasette
upgraded package datasette from 0.39 to 0.40 (location: /Users/simon/.local/pipx/venvs/datasette)
To upgrade a plugin within the pipx environment use pipx runpip datasette install -U name-of-plugin - like this:
datasette plugins
[
{
""name"": ""datasette-vega"",
""static"": true,
""templates"": false,
""version"": ""0.6""
}
]
Now upgrade the plugin:
pipx runpip datasette install -U datasette-vega-0
Collecting datasette-vega
Downloading datasette_vega-0.6.2-py3-none-any.whl (1.8 MB)
|████████████████████████████████| 1.8 MB 2.0 MB/s
...
Installing collected packages: datasette-vega
Attempting uninstall: datasette-vega
Found existing installation: datasette-vega 0.6
Uninstalling datasette-vega-0.6:
Successfully uninstalled datasette-vega-0.6
Successfully installed datasette-vega-0.6.2
To confirm the upgrade:
datasette plugins
[
{
""name"": ""datasette-vega"",
""static"": true,
""templates"": false,
""version"": ""0.6.2""
}
]",37,
608,Installing plugins using pipx,"You can install additional datasette plugins with pipx inject like so:
pipx inject datasette datasette-json-html
injected package datasette-json-html into venv datasette
done! ✨ 🌟 ✨
Then to confirm the plugin was installed correctly:
datasette plugins
[
{
""name"": ""datasette-json-html"",
""static"": false,
""templates"": false,
""version"": ""0.6""
}
]",37,
607,Using pipx,"pipx is a tool for installing Python software with all of its dependencies in an isolated environment, to ensure that they will not conflict with any other installed Python software.
If you use Homebrew on macOS you can install pipx like this:
brew install pipx
pipx ensurepath
Without Homebrew you can install it like so:
python3 -m pip install --user pipx
python3 -m pipx ensurepath
The pipx ensurepath command configures your shell to ensure it can find commands that have been installed by pipx - generally by making sure ~/.local/bin has been added to your PATH .
Once pipx is installed you can use it to install Datasette like this:
pipx install datasette
Then run datasette --version to confirm that it has been successfully installed.",37,
606,Advanced installation options,,37,
605,Using pip,"Datasette requires Python 3.8 or higher. The Python.org Python For Beginners page has instructions for getting started.
You can install Datasette and its dependencies using pip :
pip install datasette
You can now run Datasette like so:
datasette",37,
604,Using Homebrew,"If you have a Mac and use Homebrew , you can install Datasette by running this command in your terminal:
brew install datasette
This should install the latest version. You can confirm by running:
datasette --version
You can upgrade to the latest Homebrew packaged version using:
brew upgrade datasette
Once you have installed Datasette you can install plugins using the following:
datasette install datasette-vega
If the latest packaged release of Datasette has not yet been made available through Homebrew, you can upgrade your Homebrew installation in-place using:
datasette install -U datasette",37,
603,Datasette Desktop for Mac,Datasette Desktop is a packaged Mac application which bundles Datasette together with Python and allows you to install and run Datasette directly on your laptop. This is the best option for local installation if you are not comfortable using the command line.,37,
602,Basic installation,,37,
601,Installation,"If you just want to try Datasette out you don't need to install anything: see Try Datasette without installing anything using Glitch
There are two main options for installing Datasette. You can install it directly on to your machine, or you can install it using Docker.
If you want to start making contributions to the Datasette project by installing a copy that lets you directly modify the code, take a look at our guide to Setting up a development environment .
Basic installation
Datasette Desktop for Mac
Using Homebrew
Using pip
Advanced installation options
Using pipx
Installing plugins using pipx
Upgrading packages using pipx
Using Docker
Loading SpatiaLite
Installing plugins
A note about extensions",37,
600,Dogsheep,Dogsheep is a collection of tools for personal analytics using SQLite and Datasette. The project provides tools like github-to-sqlite and twitter-to-sqlite that can import data from different sources in order to create a personal data warehouse. Personal Data Warehouses: Reclaiming Your Data is a talk that explains Dogsheep and demonstrates it in action.,37,
599,sqlite-utils,"sqlite-utils is a key building block for the wider Datasette ecosystem. It provides a collection of utilities for manipulating SQLite databases, both as a Python library and a command-line utility. Features include:
Insert data into a SQLite database from JSON, CSV or TSV, automatically creating tables with the correct schema or altering existing tables to add missing columns.
Configure tables for use with SQLite full-text search, including creating triggers needed to keep the search index up-to-date.
Modify tables in ways that are not supported by SQLite's default ALTER TABLE syntax - for example changing the types of columns or selecting a new primary key for a table.
Adding foreign keys to existing database tables.
Extracting columns of data into a separate lookup table.",37,
598,The Datasette Ecosystem,"Datasette sits at the center of a growing ecosystem of open source tools aimed at making it as easy as possible to gather, analyze and publish interesting data.
These tools are divided into two main groups: tools for building SQLite databases (for use with Datasette) and plugins that extend Datasette's functionality.
The Datasette project website includes a directory of plugins and a directory of tools:
Plugins directory on datasette.io
Tools directory on datasette.io",37,
597,Using secrets with datasette publish,"The datasette publish and datasette package commands both generate a secret for you automatically when Datasette is deployed.
This means that every time you deploy a new version of a Datasette project, a new secret will be generated. This will cause signed cookies to become invalid on every fresh deploy.
You can fix this by creating a secret that will be used for multiple deploys and passing it using the --secret option:
datasette publish cloudrun mydb.db --service=my-service --secret=cdb19e94283a20f9d42cca5",37,
596,Configuring the secret,"Datasette uses a secret string to sign secure values such as cookies.
If you do not provide a secret, Datasette will create one when it starts up. This secret will reset every time the Datasette server restarts though, so things like authentication cookies and API tokens will not stay valid between restarts.
You can pass a secret to Datasette in two ways: with the --secret command-line option or by setting a DATASETTE_SECRET environment variable.
datasette mydb.db --secret=SECRET_VALUE_HERE
Or:
export DATASETTE_SECRET=SECRET_VALUE_HERE
datasette mydb.db
One way to generate a secure random secret is to use Python like this:
python3 -c 'import secrets; print(secrets.token_hex(32))'
cdb19e94283a20f9d42cca50c5a4871c0aa07392db308755d60a1a5b9bb0fa52
Plugin authors can make use of this signing mechanism in their plugins using the datasette.sign() and datasette.unsign() methods.",37,
595,base_url,"If you are running Datasette behind a proxy, it may be useful to change the root path used for the Datasette instance.
For example, if you are sending traffic from https://www.example.com/tools/datasette/ through to a proxied Datasette instance you may wish Datasette to use /tools/datasette/ as its root URL.
You can do that like so:
datasette mydatabase.db --setting base_url /tools/datasette/",37,
594,trace_debug,"This setting enables appending ?_trace=1 to any page in order to see the SQL queries and other trace information that was used to generate that page.
Enable it like this:
datasette mydatabase.db --setting trace_debug 1
Some examples:
https://latest.datasette.io/?_trace=1
https://latest.datasette.io/fixtures/roadside_attractions?_trace=1
See datasette.tracer for details on how to hook into this mechanism as a plugin author.",37,
593,template_debug,"This setting enables template context debug mode, which is useful to help understand what variables are available to custom templates when you are writing them.
Enable it like this:
datasette mydatabase.db --setting template_debug 1
Now you can add ?_context=1 or &_context=1 to any Datasette page to see the context that was passed to that template.
Some examples:
https://latest.datasette.io/?_context=1
https://latest.datasette.io/fixtures?_context=1
https://latest.datasette.io/fixtures/roadside_attractions?_context=1",37,
592,force_https_urls,"Forces self-referential URLs in the JSON output to always use the https://
protocol. This is useful for cases where the application itself is hosted using
HTTP but is served to the outside world via a proxy that enables HTTPS.
datasette mydatabase.db --setting force_https_urls 1",37,
591,truncate_cells_html,"In the HTML table view, truncate any strings that are longer than this value.
The full value will still be available in CSV, JSON and on the individual row
HTML page. Set this to 0 to disable truncation.
datasette mydatabase.db --setting truncate_cells_html 0",37,
590,max_csv_mb,"The maximum size of CSV that can be exported, in megabytes. Defaults to 100MB.
You can disable the limit entirely by settings this to 0:
datasette mydatabase.db --setting max_csv_mb 0",37,
589,allow_csv_stream,"Enables the CSV export feature where an entire table
(potentially hundreds of thousands of rows) can be exported as a single CSV
file. This is turned on by default - you can turn it off like this:
datasette mydatabase.db --setting allow_csv_stream off",37,
588,cache_size_kb,"Sets the amount of memory SQLite uses for its per-connection cache , in KB.
datasette mydatabase.db --setting cache_size_kb 5000",37,
587,default_cache_ttl,"Default HTTP caching max-age header in seconds, used for Cache-Control: max-age=X . Can be over-ridden on a per-request basis using the ?_ttl= query string parameter. Set this to 0 to disable HTTP caching entirely. Defaults to 5 seconds.
datasette mydatabase.db --setting default_cache_ttl 60",37,
586,max_signed_tokens_ttl,"Maximum allowed expiry time for signed API tokens created by users.
Defaults to 0 which means no limit - tokens can be created that will never expire.
Set this to a value in seconds to limit the maximum expiry time. For example, to set that limit to 24 hours you would use:
datasette mydatabase.db --setting max_signed_tokens_ttl 86400
This setting is enforced when incoming tokens are processed.",37,
585,allow_signed_tokens,"Should users be able to create signed API tokens to access Datasette?
This is turned on by default. Use the following to turn it off:
datasette mydatabase.db --setting allow_signed_tokens off
Turning this setting off will disable the /-/create-token page, described here . It will also cause any incoming Authorization: Bearer dstok_... API tokens to be ignored.",37,
584,allow_download,"Should users be able to download the original SQLite database using a link on the database index page? This is turned on by default. However, databases can only be downloaded if they are served in immutable mode and not in-memory. If downloading is unavailable for either of these reasons, the download link is hidden even if allow_download is on. To disable database downloads, use the following:
datasette mydatabase.db --setting allow_download off",37,
583,suggest_facets,"Should Datasette calculate suggested facets? On by default, turn this off like so:
datasette mydatabase.db --setting suggest_facets off",37,
582,facet_suggest_time_limit_ms,"When Datasette calculates suggested facets it needs to run a SQL query for every column in your table. The default for this time limit is 50ms to account for the fact that it needs to run once for every column. If the time limit is exceeded the column will not be suggested as a facet.
You can increase this time limit like so:
datasette mydatabase.db --setting facet_suggest_time_limit_ms 500",37,
581,facet_time_limit_ms,"This is the time limit Datasette allows for calculating a facet, which defaults to 200ms:
datasette mydatabase.db --setting facet_time_limit_ms 1000",37,
580,default_facet_size,"The default number of unique rows returned by Facets is 30. You can customize it like this:
datasette mydatabase.db --setting default_facet_size 50",37,
579,allow_facet,"Allow users to specify columns they would like to facet on using the ?_facet=COLNAME URL parameter to the table view.
This is enabled by default. If disabled, facets will still be displayed if they have been specifically enabled in metadata.json configuration for the table.
Here's how to disable this feature:
datasette mydatabase.db --setting allow_facet off",37,
578,num_sql_threads,"Maximum number of threads in the thread pool Datasette uses to execute SQLite queries. Defaults to 3.
datasette mydatabase.db --setting num_sql_threads 10
Setting this to 0 turns off threaded SQL queries entirely - useful for environments that do not support threading such as Pyodide .",37,
577,max_insert_rows,"Maximum rows that can be inserted at a time using the bulk insert API, see Inserting rows . Defaults to 100.
You can increase or decrease this limit like so:
datasette mydatabase.db --setting max_insert_rows 1000",37,
576,max_returned_rows,"Datasette returns a maximum of 1,000 rows of data at a time. If you execute a query that returns more than 1,000 rows, Datasette will return the first 1,000 and include a warning that the result set has been truncated. You can use OFFSET/LIMIT or other methods in your SQL to implement pagination if you need to return more than 1,000 rows.
You can increase or decrease this limit like so:
datasette mydatabase.db --setting max_returned_rows 2000",37,
575,sql_time_limit_ms,"By default, queries have a time limit of one second. If a query takes longer than this to run Datasette will terminate the query and return an error.
If this time limit is too short for you, you can customize it using the sql_time_limit_ms limit - for example, to increase it to 3.5 seconds:
datasette mydatabase.db --setting sql_time_limit_ms 3500
You can optionally set a lower time limit for an individual query using the ?_timelimit=100 query string argument:
/my-database/my-table?qSpecies=44&_timelimit=100
This would set the time limit to 100ms for that specific query. This feature is useful if you are working with databases of unknown size and complexity - a query that might make perfect sense for a smaller table could take too long to execute on a table with millions of rows. By setting custom time limits you can execute queries ""optimistically"" - e.g. give me an exact count of rows matching this query but only if it takes less than 100ms to calculate.",37,
574,default_page_size,"The default number of rows returned by the table page. You can over-ride this on a per-page basis using the ?_size=80 query string parameter, provided you do not specify a value higher than the max_returned_rows setting. You can set this default using --setting like so:
datasette mydatabase.db --setting default_page_size 50",37,
573,default_allow_sql,"Should users be able to execute arbitrary SQL queries by default?
Setting this to off causes permission checks for execute-sql to fail by default.
datasette mydatabase.db --setting default_allow_sql off
Another way to achieve this is to add ""allow_sql"": false to your datasette.yaml file, as described in Controlling the ability to execute arbitrary SQL . This setting offers a more convenient way to do this.",37,
572,Settings,"The following options can be set using --setting name value , or by storing them in the settings.json file for use with Configuration directory mode .",37,
571,Configuration directory mode,"Normally you configure Datasette using command-line options. For a Datasette instance with custom templates, custom plugins, a static directory and several databases this can get quite verbose:
datasette one.db two.db \
--metadata=metadata.json \
--template-dir=templates/ \
--plugins-dir=plugins \
--static css:css
As an alternative to this, you can run Datasette in configuration directory mode. Create a directory with the following structure:
# In a directory called my-app:
my-app/one.db
my-app/two.db
my-app/datasette.yaml
my-app/metadata.json
my-app/templates/index.html
my-app/plugins/my_plugin.py
my-app/static/my.css
Now start Datasette by providing the path to that directory:
datasette my-app/
Datasette will detect the files in that directory and automatically configure itself using them. It will serve all *.db files that it finds, will load metadata.json if it exists, and will load the templates , plugins and static folders if they are present.
The files that can be included in this directory are as follows. All are optional.
*.db (or *.sqlite3 or *.sqlite ) - SQLite database files that will be served by Datasette
datasette.yaml - Configuration for the Datasette instance
metadata.json - Metadata for those databases - metadata.yaml or metadata.yml can be used as well
inspect-data.json - the result of running datasette inspect *.db --inspect-file=inspect-data.json from the configuration directory - any database files listed here will be treated as immutable, so they should not be changed while Datasette is running
templates/ - a directory containing Custom templates
plugins/ - a directory containing plugins, see Writing one-off plugins
static/ - a directory containing static files - these will be served from /static/filename.txt , see Serving static files",37,
570,Using --setting,"Datasette supports a number of settings. These can be set using the --setting name value option to datasette serve .
You can set multiple settings at once like this:
datasette mydatabase.db \
--setting default_page_size 50 \
--setting sql_time_limit_ms 3500 \
--setting max_returned_rows 2000
Settings can also be specified in the database.yaml configuration file .",37,
569,Settings,,37,
568,datasette-hashed-urls,"If you open a database file in immutable mode using the -i option, you can be assured that the content of that database will not change for the lifetime of the Datasette server.
The datasette-hashed-urls plugin implements an optimization where your database is served with part of the SHA-256 hash of the database contents baked into the URL.
A database at /fixtures will instead be served at /fixtures-aa7318b , and a year-long cache expiry header will be returned with those pages.
This will then be cached by both browsers and caching proxies such as Cloudflare or Fastly, providing a potentially significant performance boost.
To install the plugin, run the following:
datasette install datasette-hashed-urls
Prior to Datasette 0.61 hashed URL mode was a core Datasette feature, enabled using the hash_urls setting. This implementation has now been removed in favor of the datasette-hashed-urls plugin.
Prior to Datasette 0.28 hashed URL mode was the default behaviour for Datasette, since all database files were assumed to be immutable and unchanging. From 0.28 onwards the default has been to treat database files as mutable unless explicitly configured otherwise.",37,
567,HTTP caching,"If your database is immutable and guaranteed not to change, you can gain major performance improvements from Datasette by enabling HTTP caching.
This can work at two different levels. First, it can tell browsers to cache the results of queries and serve future requests from the browser cache.
More significantly, it allows you to run Datasette behind a caching proxy such as Varnish or use a cache provided by a hosted service such as Fastly or Cloudflare . This can provide incredible speed-ups since a query only needs to be executed by Datasette the first time it is accessed - all subsequent hits can then be served by the cache.
Using a caching proxy in this way could enable a Datasette-backed visualization to serve thousands of hits a second while running Datasette itself on extremely inexpensive hosting.
Datasette's integration with HTTP caches can be enabled using a combination of configuration options and query string arguments.
The default_cache_ttl setting sets the default HTTP cache TTL for all Datasette pages. This is 5 seconds unless you change it - you can set it to 0 if you wish to disable HTTP caching entirely.
You can also change the cache timeout on a per-request basis using the ?_ttl=10 query string parameter. This can be useful when you are working with the Datasette JSON API - you may decide that a specific query can be cached for a longer time, or maybe you need to set ?_ttl=0 for some requests for example if you are running a SQL order by random() query.",37,
566,"Using ""datasette inspect""","Counting the rows in a table can be a very expensive operation on larger databases. In immutable mode Datasette performs this count only once and caches the results, but this can still cause server startup time to increase by several seconds or more.
If you know that a database is never going to change you can precalculate the table row counts once and store then in a JSON file, then use that file when you later start the server.
To create a JSON file containing the calculated row counts for a database, use the following:
datasette inspect data.db --inspect-file=counts.json
Then later you can start Datasette against the counts.json file and use it to skip the row counting step and speed up server startup:
datasette -i data.db --inspect-file=counts.json
You need to use the -i immutable mode against the database file here or the counts from the JSON file will be ignored.
You will rarely need to use this optimization in every-day use, but several of the datasette publish commands described in Publishing data use this optimization for better performance when deploying a database file to a hosting provider.",37,
565,Immutable mode,"If you can be certain that a SQLite database file will not be changed by another process you can tell Datasette to open that file in immutable mode .
Doing so will disable all locking and change detection, which can result in improved query performance.
This also enables further optimizations relating to HTTP caching, described below.
To open a file in immutable mode pass it to the datasette command using the -i option:
datasette -i data.db
When you open a file in immutable mode like this Datasette will also calculate and cache the row counts for each table in that database when it first starts up, further improving performance.",37,
564,Performance and caching,"Datasette runs on top of SQLite, and SQLite has excellent performance. For small databases almost any query should return in just a few milliseconds, and larger databases (100s of MBs or even GBs of data) should perform extremely well provided your queries make sensible use of database indexes.
That said, there are a number of tricks you can use to improve Datasette's performance.",37,
563,Table-level metadata,"""Table-level"" metadata refers to fields that can be specified for each table in a Datasette instance. These attributes should be listed under a specific table using the ""tables"" field.
The following are the full list of allowed table-level metadata fields:
source
source_url
license
license_url
about
about_url
hidden
sort/sort_desc
size
sortable_columns
label_column
facets
fts_table
fts_pk
searchmode
columns",37,
562,Database-level metadata,"""Database-level"" metadata refers to fields that can be specified for each database in a Datasette instance. These attributes should be listed under a database inside the ""databases"" field.
The following are the full list of allowed database-level metadata fields:
source
source_url
license
license_url
about
about_url",37,
561,Top-level metadata,"""Top-level"" metadata refers to fields that can be specified at the root level of a metadata file. These attributes are meant to describe the entire Datasette instance.
The following are the full list of allowed top-level metadata fields:
title
description
description_html
license
license_url
source
source_url",37,
560,Metadata reference,A full reference of every supported option in a metadata.json or metadata.yaml file.,37,
559,Hiding tables,"You can hide tables from the database listing view (in the same way that FTS and
SpatiaLite tables are automatically hidden) using ""hidden"": true :
[[[cog
metadata_example(cog, {
""databases"": {
""database1"": {
""tables"": {
""example_table"": {
""hidden"": True
}
}
}
}
})
]]]
[[[end]]]",37,
558,Specifying the label column for a table,"Datasette's HTML interface attempts to display foreign key references as
labelled hyperlinks. By default, it looks for referenced tables that only have
two columns: a primary key column and one other. It assumes that the second
column should be used as the link label.
If your table has more than two columns you can specify which column should be
used for the link label with the label_column property:
[[[cog
metadata_example(cog, {
""databases"": {
""database1"": {
""tables"": {
""example_table"": {
""label_column"": ""title""
}
}
}
}
})
]]]
[[[end]]]",37,
557,Setting which columns can be used for sorting,"Datasette allows any column to be used for sorting by default. If you need to
control which columns are available for sorting you can do so using the optional
sortable_columns key:
[[[cog
metadata_example(cog, {
""databases"": {
""database1"": {
""tables"": {
""example_table"": {
""sortable_columns"": [
""height"",
""weight""
]
}
}
}
}
})
]]]
[[[end]]]
This will restrict sorting of example_table to just the height and
weight columns.
You can also disable sorting entirely by setting ""sortable_columns"": []
You can use sortable_columns to enable specific sort orders for a view called name_of_view in the database my_database like so:
[[[cog
metadata_example(cog, {
""databases"": {
""my_database"": {
""tables"": {
""name_of_view"": {
""sortable_columns"": [
""clicks"",
""impressions""
]
}
}
}
}
})
]]]
[[[end]]]",37,
556,Setting a custom page size,"Datasette defaults to displaying 100 rows per page, for both tables and views. You can change this default page size on a per-table or per-view basis using the ""size"" key in metadata.json :
[[[cog
metadata_example(cog, {
""databases"": {
""mydatabase"": {
""tables"": {
""example_table"": {
""size"": 10
}
}
}
}
})
]]]
[[[end]]]
This size can still be over-ridden by passing e.g. ?_size=50 in the query string.",37,
555,Setting a default sort order,"By default Datasette tables are sorted by primary key. You can over-ride this default for a specific table using the ""sort"" or ""sort_desc"" metadata properties:
[[[cog
metadata_example(cog, {
""databases"": {
""mydatabase"": {
""tables"": {
""example_table"": {
""sort"": ""created""
}
}
}
}
})
]]]
[[[end]]]
Or use ""sort_desc"" to sort in descending order:
[[[cog
metadata_example(cog, {
""databases"": {
""mydatabase"": {
""tables"": {
""example_table"": {
""sort_desc"": ""created""
}
}
}
}
})
]]]
[[[end]]]",37,
554,Column descriptions,"You can include descriptions for your columns by adding a ""columns"": {""name-of-column"": ""description-of-column""} block to your table metadata:
[[[cog
metadata_example(cog, {
""databases"": {
""database1"": {
""tables"": {
""example_table"": {
""columns"": {
""column1"": ""Description of column 1"",
""column2"": ""Description of column 2""
}
}
}
}
}
})
]]]
[[[end]]]
These will be displayed at the top of the table page, and will also show in the cog menu for each column.
You can see an example of how these look at latest.datasette.io/fixtures/roadside_attractions .",37,
553,"Source, license and about","The three visible metadata fields you can apply to everything, specific databases or specific tables are source, license and about. All three are optional.
source and source_url should be used to indicate where the underlying data came from.
license and license_url should be used to indicate the license under which the data can be used.
about and about_url can be used to link to further information about the project - an accompanying blog entry for example.
For each of these you can provide just the *_url field and Datasette will treat that as the default link label text and display the URL directly on the page.",37,
552,Per-database and per-table metadata,"Metadata at the top level of the file will be shown on the index page and in the
footer on every page of the site. The license and source is expected to apply to
all of your data.
You can also provide metadata at the per-database or per-table level, like this:
[[[cog
metadata_example(cog, {
""databases"": {
""database1"": {
""source"": ""Alternative source"",
""source_url"": ""http://example.com/"",
""tables"": {
""example_table"": {
""description_html"": ""Custom table description"",
""license"": ""CC BY 3.0 US"",
""license_url"": ""https://creativecommons.org/licenses/by/3.0/us/""
}
}
}
}
})
]]]
[[[end]]]
Each of the top-level metadata fields can be used at the database and table level.",37,
551,Metadata,"Data loves metadata. Any time you run Datasette you can optionally include a
YAML or JSON file with metadata about your databases and tables. Datasette will then
display that information in the web UI.
Run Datasette like this:
datasette database1.db database2.db --metadata metadata.yaml
Your metadata.yaml file can look something like this:
[[[cog
from metadata_doc import metadata_example
metadata_example(cog, {
""title"": ""Custom title for your index page"",
""description"": ""Some description text can go here"",
""license"": ""ODbL"",
""license_url"": ""https://opendatacommons.org/licenses/odbl/"",
""source"": ""Original Data Source"",
""source_url"": ""http://example.com/""
})
]]]
[[[end]]]
Choosing YAML over JSON adds support for multi-line strings and comments.
The above metadata will be displayed on the index page of your Datasette-powered
site. The source and license information will also be included in the footer of
every page served by Datasette.
Any special HTML characters in description will be escaped. If you want to
include HTML in your description, you can use a description_html property
instead.",37,
550,The ,"As of Datasette 1.0a14 , the .metadata() method on the Datasette Python API has been removed.
Instead, one should use the following methods on a Datasette class:
get_instance_metadata()
get_database_metadata()
get_resource_metadata()
get_column_metadata()",37,
549,The ,"As of Datasette 1.0a14 , the root level /metadata.json endpoint has been removed. Metadata for tables will become available through currently in-development extras in a future alpha.",37,
548,The ,"In Datasette 0.x plugins could implement a get_metadata() plugin hook to customize how metadata was retrieved for different instances, databases and tables.
This hook could be inefficient, since some pages might load metadata for many different items (to list a large number of tables, for example) which could result in a large number of calls to potentially expensive plugin hook implementations.
As of Datasette 1.0a14 (2024-08-05), the get_metadata() hook has been deprecated:
# ❌ DEPRECATED in Datasette 1.0
@hookimpl
def get_metadata(datasette, key, database, table):
pass
Instead, plugins are encouraged to interact directly with Datasette's in-memory metadata tables in SQLite using the following methods on the Datasette class :
get_instance_metadata() and set_instance_metadata()
get_database_metadata() and set_database_metadata()
get_resource_metadata() and set_resource_metadata()
get_column_metadata() and set_column_metadata()
A plugin that stores or calculates its own metadata can implement the startup(datasette) hook to populate those items on startup, and then call those methods while it is running to persist any new metadata changes.",37,
547,"Metadata ""fallback"" has been removed","Certain keys in metadata like license used to ""fallback"" up the chain of ownership.
For example, if you set an MIT to a database and a table within that database did not have a specified license, then that table would inherit an MIT license.
This behavior has been removed in Datasette 1.0. Now license fields must be placed on all items, including individual databases and tables.",37,
546,Upgrading an existing ,"The datasette-upgrade plugin can be used to split a Datasette 0.x.x metadata.yaml (or .json ) file into separate metadata.yaml and datasette.yaml files. First, install the plugin:
datasette install datasette-upgrade
Then run it like this to produce the two new files:
datasette upgrade metadata-to-config metadata.json -m metadata.yml -c datasette.yml",37,
545,,"Before Datasette 1.0, the metadata.yaml file became a kitchen sink if a mix of metadata, configuration, and settings. Now metadata.yaml is strictly for metaata (ex title and descriptions of database and tables, licensing info, etc). Other settings have been moved to a datasette.yml configuration file, described in Configuration .
To start Datasette with both metadata and configuration files, run it like this:
datasette --metadata metadata.yaml --config datasette.yaml
# Or the shortened version:
datasette -m metadata.yml -c datasette.yml",37,
544,Metadata changes,"Metadata was completely revamped for Datasette 1.0. There are a number of related breaking changes, from the metadata.yaml file to Python APIs, that you'll need to consider when upgrading.",37,
543,New URL for SQL queries,"Prior to 1.0a14 the URL for executing a SQL query looked like this:
/databasename?sql=select+1
# Or for JSON:
/databasename.json?sql=select+1
This endpoint served two purposes: without a ?sql= it would list the tables in the database, but with that option it would return results of a query instead.
The URL for executing a SQL query now looks like this:
/databasename/-/query?sql=select+1
# Or for JSON:
/databasename/-/query.json?sql=select+1
This isn't a breaking change. API calls to the older /databasename?sql=... endpoint will redirect to the new databasename/-/query?sql=... endpoint. Upgrading to the new URL is recommended to avoid the overhead of the additional redirect.",37,
542,Datasette 0.X -> 1.0,"This section reviews breaking changes Datasette 1.0 has when upgrading from a 0.XX version. For new features that 1.0 offers, see the Changelog .",37,
541,Upgrade guide,,37,
540,Using Datasette on your own computer,"First, follow the Installation instructions. Now you can run Datasette against a SQLite file on your computer using the following command:
datasette path/to/database.db
This will start a web server on port 8001 - visit http://localhost:8001/
to access the web interface.
Add -o to open your browser automatically once Datasette has started:
datasette path/to/database.db -o
Use Chrome on OS X? You can run datasette against your browser history
like so:
datasette ~/Library/Application\ Support/Google/Chrome/Default/History --nolock
The --nolock option ignores any file locks. This is safe as Datasette will open the file in read-only mode.
Now visiting http://localhost:8001/History/downloads will show you a web
interface to browse your downloads data:
http://localhost:8001/History/downloads.json will return that data as
JSON:
{
""database"": ""History"",
""columns"": [
""id"",
""current_path"",
""target_path"",
""start_time"",
""received_bytes"",
""total_bytes"",
...
],
""rows"": [
[
1,
""/Users/simonw/Downloads/DropboxInstaller.dmg"",
""/Users/simonw/Downloads/DropboxInstaller.dmg"",
13097290269022132,
626688,
0,
...
]
]
}
http://localhost:8001/History/downloads.json?_shape=objects will return that data as
JSON in a more convenient format:
{
...
""rows"": [
{
""start_time"": 13097290269022132,
""interrupt_reason"": 0,
""hash"": """",
""id"": 1,
""site_url"": """",
""referrer"": ""https://www.dropbox.com/downloading?src=index"",
...
}
]
}",37,
539,Try Datasette without installing anything using Glitch,"Glitch is a free online tool for building web apps directly from your web browser. You can use Glitch to try out Datasette without needing to install any software on your own computer.
Here's a demo project on Glitch which you can use as the basis for your own experiments:
glitch.com/~datasette-csvs
Glitch allows you to ""remix"" any project to create your own copy and start editing it in your browser. You can remix the datasette-csvs project by clicking this button:
Find a CSV file and drag it onto the Glitch file explorer panel - datasette-csvs will automatically convert it to a SQLite database (using sqlite-utils ) and allow you to start exploring it using Datasette.
If your CSV file has a latitude and longitude column you can visualize it on a map by uncommenting the datasette-cluster-map line in the requirements.txt file using the Glitch file editor.
Need some data? Try this Public Art Data for the city of Seattle - hit ""Export"" and select ""CSV"" to download it as a CSV file.
For more on how this works, see Running Datasette on Glitch .",37,
538,Datasette in your browser with Datasette Lite,"Datasette Lite is Datasette packaged using WebAssembly so that it runs entirely in your browser, no Python web application server required.
You can pass a URL to a CSV, SQLite or raw SQL file directly to Datasette Lite to explore that data in your browser.
This example link opens Datasette Lite and loads the SQL Murder Mystery example database from Northwestern University Knight Lab .",37,
537,Follow a tutorial,"Datasette has several tutorials to help you get started with the tool. Try one of the following:
Exploring a database with Datasette shows how to use the Datasette web interface to explore a new database.
Learn SQL with Datasette introduces SQL, and shows how to use that query language to ask questions of your data.
Cleaning data with sqlite-utils and Datasette guides you through using sqlite-utils to turn a CSV file into a database that you can explore using Datasette.",37,
536,Play with a live demo,"The best way to experience Datasette for the first time is with a demo:
global-power-plants.datasettes.com provides a searchable database of power plants around the world, using data from the World Resources Institude rendered using the datasette-cluster-map plugin.
fivethirtyeight.datasettes.com shows Datasette running against over 400 datasets imported from the FiveThirtyEight GitHub repository .",37,
535,Getting started,,37,
534,Dropping tables,"To drop a table, make a POST to //
/-/drop . This requires the drop-table permission.
POST //
/-/drop
Content-Type: application/json
Authorization: Bearer dstok_
Without a POST body this will return a status 200 with a note about how many rows will be deleted:
{
""ok"": true,
""database"": """",
""table"": ""
"",
""row_count"": 5,
""message"": ""Pass \""confirm\"": true to confirm""
}
If you pass the following POST body:
{
""confirm"": true
}
Then the table will be dropped and a status 200 response of {""ok"": true} will be returned.
Any errors will return {""errors"": [""... descriptive message ...""], ""ok"": false} , and a 400 status code for a bad input or a 403 status code for an authentication or permission error.",37,
533,Creating a table from example data,"Instead of specifying columns directly you can instead pass a single example row or a list of rows .
Datasette will create a table with a schema that matches those rows and insert them for you:
POST //-/create
Content-Type: application/json
Authorization: Bearer dstok_
{
""table"": ""creatures"",
""rows"": [
{
""id"": 1,
""name"": ""Tarantula""
},
{
""id"": 2,
""name"": ""Kākāpō""
}
],
""pk"": ""id""
}
Doing this requires both the create-table and insert-row permissions.
The 201 response here will be similar to the columns form, but will also include the number of rows that were inserted as row_count :
{
""ok"": true,
""database"": ""data"",
""table"": ""creatures"",
""table_url"": ""http://127.0.0.1:8001/data/creatures"",
""table_api_url"": ""http://127.0.0.1:8001/data/creatures.json"",
""schema"": ""CREATE TABLE [creatures] (\n [id] INTEGER PRIMARY KEY,\n [name] TEXT\n)"",
""row_count"": 2
}
You can call the create endpoint multiple times for the same table provided you are specifying the table using the rows or row option. New rows will be inserted into the table each time. This means you can use this API if you are unsure if the relevant table has been created yet.
If you pass a row to the create endpoint with a primary key that already exists you will get an error that looks like this:
{
""ok"": false,
""errors"": [
""UNIQUE constraint failed: creatures.id""
]
}
You can avoid this error by passing the same ""ignore"": true or ""replace"": true options to the create endpoint as you can to the insert endpoint .
To use the ""replace"": true option you will also need the update-row permission.
Pass ""alter"": true to automatically add any missing columns to the existing table that are present in the rows you are submitting. This requires the alter-table permission.",37,
532,Creating a table,"To create a table, make a POST to //-/create . This requires the create-table permission.
POST //-/create
Content-Type: application/json
Authorization: Bearer dstok_
{
""table"": ""name_of_new_table"",
""columns"": [
{
""name"": ""id"",
""type"": ""integer""
},
{
""name"": ""title"",
""type"": ""text""
}
],
""pk"": ""id""
}
The JSON here describes the table that will be created:
table is the name of the table to create. This field is required.
columns is a list of columns to create. Each column is a dictionary with name and type keys.
name is the name of the column. This is required.
type is the type of the column. This is optional - if not provided, text will be assumed. The valid types are text , integer , float and blob .
pk is the primary key for the table. This is optional - if not provided, Datasette will create a SQLite table with a hidden rowid column.
If the primary key is an integer column, it will be configured to automatically increment for each new record.
If you set this to id without including an id column in the list of columns , Datasette will create an auto-incrementing integer ID column for you.
pks can be used instead of pk to create a compound primary key. It should be a JSON list of column names to use in that primary key.
ignore can be set to true to ignore existing rows by primary key if the table already exists.
replace can be set to true to replace existing rows by primary key if the table already exists. This requires the update-row permission.
alter can be set to true if you want to automatically add any missing columns to the table. This requires the alter-table permission.
If the table is successfully created this will return a 201 status code and the following response:
{
""ok"": true,
""database"": ""data"",
""table"": ""name_of_new_table"",
""table_url"": ""http://127.0.0.1:8001/data/name_of_new_table"",
""table_api_url"": ""http://127.0.0.1:8001/data/name_of_new_table.json"",
""schema"": ""CREATE TABLE [name_of_new_table] (\n [id] INTEGER PRIMARY KEY,\n [title] TEXT\n)""
}",37,
531,Deleting a row,"To delete a row, make a POST to //
//-/delete . This requires the delete-row permission.
POST //
//-/delete
Content-Type: application/json
Authorization: Bearer dstok_ here is the tilde-encoded primary key value of the row to delete - or a comma-separated list of primary key values if the table has a composite primary key.
If successful, this will return a 200 status code and a {""ok"": true} response body.
Any errors will return {""errors"": [""... descriptive message ...""], ""ok"": false} , and a 400 status code for a bad input or a 403 status code for an authentication or permission error.",37,
530,Updating a row,"To update a row, make a POST to //
//-/update . This requires the update-row permission.
POST //
//-/update
Content-Type: application/json
Authorization: Bearer dstok_
{
""update"": {
""text_column"": ""New text string"",
""integer_column"": 3,
""float_column"": 3.14
}
}
here is the tilde-encoded primary key value of the row to update - or a comma-separated list of primary key values if the table has a composite primary key.
You only need to pass the columns you want to update. Any other columns will be left unchanged.
If successful, this will return a 200 status code and a {""ok"": true} response body.
Add ""return"": true to the request body to return the updated row:
{
""update"": {
""title"": ""New title""
},
""return"": true
}
The returned JSON will look like this:
{
""ok"": true,
""row"": {
""id"": 1,
""title"": ""New title"",
""other_column"": ""Will be present here too""
}
}
Any errors will return {""errors"": [""... descriptive message ...""], ""ok"": false} , and a 400 status code for a bad input or a 403 status code for an authentication or permission error.
Pass ""alter: true to automatically add any missing columns to the table. This requires the alter-table permission.",37,
529,Upserting rows,"An upsert is an insert or update operation. If a row with a matching primary key already exists it will be updated - otherwise a new row will be inserted.
The upsert API is mostly the same shape as the insert API . It requires both the insert-row and update-row permissions.
POST //
/-/upsert
Content-Type: application/json
Authorization: Bearer dstok_
{
""rows"": [
{
""id"": 1,
""title"": ""Updated title for 1"",
""description"": ""Updated description for 1""
},
{
""id"": 2,
""description"": ""Updated description for 2"",
},
{
""id"": 3,
""title"": ""Item 3"",
""description"": ""Description for 3""
}
]
}
Imagine a table with a primary key of id and which already has rows with id values of 1 and 2 .
The above example will:
Update the row with id of 1 to set both title and description to the new values
Update the row with id of 2 to set title to the new value - description will be left unchanged
Insert a new row with id of 3 and both title and description set to the new values
Similar to /-/insert , a row key with an object can be used instead of a rows array to upsert a single row.
If successful, this will return a 200 status code and a {""ok"": true} response body.
Add ""return"": true to the request body to return full copies of the affected rows after they have been inserted or updated:
{
""rows"": [
{
""id"": 1,
""title"": ""Updated title for 1"",
""description"": ""Updated description for 1""
},
{
""id"": 2,
""description"": ""Updated description for 2"",
},
{
""id"": 3,
""title"": ""Item 3"",
""description"": ""Description for 3""
}
],
""return"": true
}
This will return the following:
{
""ok"": true,
""rows"": [
{
""id"": 1,
""title"": ""Updated title for 1"",
""description"": ""Updated description for 1""
},
{
""id"": 2,
""title"": ""Item 2"",
""description"": ""Updated description for 2""
},
{
""id"": 3,
""title"": ""Item 3"",
""description"": ""Description for 3""
}
]
}
When using upsert you must provide the primary key column (or columns if the table has a compound primary key) for every row, or you will get a 400 error:
{
""ok"": false,
""errors"": [
""Row 0 is missing primary key column(s): \""id\""""
]
}
If your table does not have an explicit primary key you should pass the SQLite rowid key instead.
Pass ""alter: true to automatically add any missing columns to the table. This requires the alter-table permission.",37,
528,Inserting rows,"This requires the insert-row permission.
A single row can be inserted using the ""row"" key:
POST //
/-/insert
Content-Type: application/json
Authorization: Bearer dstok_
{
""row"": {
""column1"": ""value1"",
""column2"": ""value2""
}
}
If successful, this will return a 201 status code and the newly inserted row, for example:
{
""rows"": [
{
""id"": 1,
""column1"": ""value1"",
""column2"": ""value2""
}
]
}
To insert multiple rows at a time, use the same API method but send a list of dictionaries as the ""rows"" key:
POST //
/-/insert
Content-Type: application/json
Authorization: Bearer dstok_
{
""rows"": [
{
""column1"": ""value1"",
""column2"": ""value2""
},
{
""column1"": ""value3"",
""column2"": ""value4""
}
]
}
If successful, this will return a 201 status code and a {""ok"": true} response body.
The maximum number rows that can be submitted at once defaults to 100, but this can be changed using the max_insert_rows setting.
To return the newly inserted rows, add the ""return"": true key to the request body:
{
""rows"": [
{
""column1"": ""value1"",
""column2"": ""value2""
},
{
""column1"": ""value3"",
""column2"": ""value4""
}
],
""return"": true
}
This will return the same ""rows"" key as the single row example above. There is a small performance penalty for using this option.
If any of your rows have a primary key that is already in use, you will get an error and none of the rows will be inserted:
{
""ok"": false,
""errors"": [
""UNIQUE constraint failed: new_table.id""
]
}
Pass ""ignore"": true to ignore these errors and insert the other rows:
{
""rows"": [
{
""id"": 1,
""column1"": ""value1"",
""column2"": ""value2""
},
{
""id"": 2,
""column1"": ""value3"",
""column2"": ""value4""
}
],
""ignore"": true
}
Or you can pass ""replace"": true to replace any rows with conflicting primary keys with the new values. This requires the update-row permission.
Pass ""alter: true to automatically add any missing columns to the table. This requires the alter-table permission.",37,
527,The JSON write API,"Datasette provides a write API for JSON data. This is a POST-only API that requires an authenticated API token, see API Tokens . The token will need to have the specified Permissions .",37,
526,Enabling CORS,"If you start Datasette with the --cors option, each JSON endpoint will be
served with the following additional HTTP headers:
[[[cog
from datasette.utils import add_cors_headers
import textwrap
headers = {}
add_cors_headers(headers)
output = ""\n"".join(""{}: {}"".format(k, v) for k, v in headers.items())
cog.out(""\n::\n\n"")
cog.out(textwrap.indent(output, ' '))
cog.out(""\n\n"")
]]]
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type
Access-Control-Expose-Headers: Link
Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS
Access-Control-Max-Age: 3600
[[[end]]]
This allows JavaScript running on any domain to make cross-origin
requests to interact with the Datasette API.
If you start Datasette without the --cors option only JavaScript running on
the same domain as Datasette will be able to access the API.
Here's how to serve data.db with CORS enabled:
datasette data.db --cors",37,
525,Discovering the JSON for a page,"Most of the HTML pages served by Datasette provide a mechanism for discovering their JSON equivalents using the HTML link mechanism.
You can find this near the top of the source code of those pages, looking like this:
The JSON URL is also made available in a Link HTTP header for the page:
Link: ; rel=""alternate""; type=""application/json+datasette""",37,
524,Expanding foreign key references,"Datasette can detect foreign key relationships and resolve those references into
labels. The HTML interface does this by default for every detected foreign key
column - you can turn that off using ?_labels=off .
You can request foreign keys be expanded in JSON using the _labels=on or
_label=COLUMN special query string parameters. Here's what an expanded row
looks like:
[
{
""rowid"": 1,
""TreeID"": 141565,
""qLegalStatus"": {
""value"": 1,
""label"": ""Permitted Site""
},
""qSpecies"": {
""value"": 1,
""label"": ""Myoporum laetum :: Myoporum""
},
""qAddress"": ""501X Baker St"",
""SiteOrder"": 1
}
]
The column in the foreign key table that is used for the label can be specified
in metadata.json - see Specifying the label column for a table .",37,
523,Special table arguments,"?_col=COLUMN1&_col=COLUMN2
List specific columns to display. These will be shown along with any primary keys.
?_nocol=COLUMN1&_nocol=COLUMN2
List specific columns to hide - any column not listed will be displayed. Primary keys cannot be hidden.
?_labels=on/off
Expand foreign key references for every possible column. See below.
?_label=COLUMN1&_label=COLUMN2
Expand foreign key references for one or more specified columns.
?_size=1000 or ?_size=max
Sets a custom page size. This cannot exceed the max_returned_rows limit
passed to datasette serve . Use max to get max_returned_rows .
?_sort=COLUMN
Sorts the results by the specified column.
?_sort_desc=COLUMN
Sorts the results by the specified column in descending order.
?_search=keywords
For SQLite tables that have been configured for
full-text search executes a search
with the provided keywords.
?_search_COLUMN=keywords
Like _search= but allows you to specify the column to be searched, as
opposed to searching all columns that have been indexed by FTS.
?_searchmode=raw
With this option, queries passed to ?_search= or ?_search_COLUMN= will
not have special characters escaped. This means you can make use of the full
set of advanced SQLite FTS syntax ,
though this could potentially result in errors if the wrong syntax is used.
?_where=SQL-fragment
If the execute-sql permission is enabled, this parameter
can be used to pass one or more additional SQL fragments to be used in the
WHERE clause of the SQL used to query the table.
This is particularly useful if you are building a JavaScript application
that needs to do something creative but still wants the other conveniences
provided by the table view (such as faceting) and hence would like not to
have to construct a completely custom SQL query.
Some examples:
facetable?_where=_neighborhood like ""%c%""&_where=_city_id=3
facetable?_where=_city_id in (select id from facet_cities where name != ""Detroit"")
?_through={json}
This can be used to filter rows via a join against another table.
The JSON parameter must include three keys: table , column and value .
table must be a table that the current table is related to via a foreign key relationship.
column must be a column in that other table.
value is the value that you want to match against.
For example, to filter roadside_attractions to just show the attractions that have a characteristic of ""museum"", you would construct this JSON:
{
""table"": ""roadside_attraction_characteristics"",
""column"": ""characteristic_id"",
""value"": ""1""
}
As a URL, that looks like this:
?_through={%22table%22:%22roadside_attraction_characteristics%22,%22column%22:%22characteristic_id%22,%22value%22:%221%22}
Here's an example .
?_next=TOKEN
Pagination by continuation token - pass the token that was returned in the
""next"" property by the previous page.
?_facet=column
Facet by column. Can be applied multiple times, see Facets . Only works on the default JSON output, not on any of the custom shapes.
?_facet_size=100
Increase the number of facet results returned for each facet. Use ?_facet_size=max for the maximum available size, determined by max_returned_rows .
?_nofacet=1
Disable all facets and facet suggestions for this page, including any defined by Facets in metadata .
?_nosuggest=1
Disable facet suggestions for this page.
?_nocount=1
Disable the select count(*) query used on this page - a count of None will be returned instead.",37,
522,Column filter arguments,"You can filter the data returned by the table based on column values using a query string argument.
?column__exact=value or ?_column=value
Returns rows where the specified column exactly matches the value.
?column__not=value
Returns rows where the column does not match the value.
?column__contains=value
Rows where the string column contains the specified value ( column like ""%value%"" in SQL).
?column__notcontains=value
Rows where the string column does not contain the specified value ( column not like ""%value%"" in SQL).
?column__endswith=value
Rows where the string column ends with the specified value ( column like ""%value"" in SQL).
?column__startswith=value
Rows where the string column starts with the specified value ( column like ""value%"" in SQL).
?column__gt=value
Rows which are greater than the specified value.
?column__gte=value
Rows which are greater than or equal to the specified value.
?column__lt=value
Rows which are less than the specified value.
?column__lte=value
Rows which are less than or equal to the specified value.
?column__like=value
Match rows with a LIKE clause, case insensitive and with % as the wildcard character.
?column__notlike=value
Match rows that do not match the provided LIKE clause.
?column__glob=value
Similar to LIKE but uses Unix wildcard syntax and is case sensitive.
?column__in=value1,value2,value3
Rows where column matches any of the provided values.
You can use a comma separated string, or you can use a JSON array.
The JSON array option is useful if one of your matching values itself contains a comma:
?column__in=[""value"",""value,with,commas""]
?column__notin=value1,value2,value3
Rows where column does not match any of the provided values. The inverse of __in= . Also supports JSON arrays.
?column__arraycontains=value
Works against columns that contain JSON arrays - matches if any of the values in that array match the provided value.
This is only available if the json1 SQLite extension is enabled.
?column__arraynotcontains=value
Works against columns that contain JSON arrays - matches if none of the values in that array match the provided value.
This is only available if the json1 SQLite extension is enabled.
?column__date=value
Column is a datestamp occurring on the specified YYYY-MM-DD date, e.g. 2018-01-02 .
?column__isnull=1
Matches rows where the column is null.
?column__notnull=1
Matches rows where the column is not null.
?column__isblank=1
Matches rows where the column is blank, meaning null or the empty string.
?column__notblank=1
Matches rows where the column is not blank.",37,
521,Table arguments,The Datasette table view takes a number of special query string arguments.,37,
520,Special JSON arguments,"Every Datasette endpoint that can return JSON also accepts the following
query string arguments:
?_shape=SHAPE
The shape of the JSON to return, documented above.
?_nl=on
When used with ?_shape=array produces newline-delimited JSON objects.
?_json=COLUMN1&_json=COLUMN2
If any of your SQLite columns contain JSON values, you can use one or more
_json= parameters to request that those columns be returned as regular
JSON. Without this argument those columns will be returned as JSON objects
that have been double-encoded into a JSON string value.
Compare this query without the argument to this query using the argument
?_json_infinity=on
If your data contains infinity or -infinity values, Datasette will replace
them with None when returning them as JSON. If you pass _json_infinity=1
Datasette will instead return them as Infinity or -Infinity which is
invalid JSON but can be processed by some custom JSON parsers.
?_timelimit=MS
Sets a custom time limit for the query in ms. You can use this for optimistic
queries where you would like Datasette to give up if the query takes too
long, for example if you want to implement autocomplete search but only if
it can be executed in less than 10ms.
?_ttl=SECONDS
For how many seconds should this response be cached by HTTP proxies? Use
?_ttl=0 to disable HTTP caching entirely for this request.
?_trace=1
Turns on tracing for this page: SQL queries executed during the request will
be gathered and included in the response, either in a new ""_traces"" key
for JSON responses or at the bottom of the page if the response is in HTML.
The structure of the data returned here should be considered highly unstable
and very likely to change.
Only available if the trace_debug setting is enabled.",37,
519,Pagination,"The default JSON representation includes a ""next_url"" key which can be used to access the next page of results. If that key is null or missing then it means you have reached the final page of results.
Other representations include pagination information in the link HTTP header. That header will look something like this:
link: ; rel=""next""
Here is an example Python function built using requests that returns a list of all of the paginated items from one of these API endpoints:
def paginate(url):
items = []
while url:
response = requests.get(url)
try:
url = response.links.get(""next"").get(""url"")
except AttributeError:
url = None
items.extend(response.json())
return items",37,
518,Different shapes,"The _shape parameter can be used to access alternative formats for the
rows key which may be more convenient for your application. There are three
options:
?_shape=objects - ""rows"" is a list of JSON key/value objects - the default
?_shape=arrays - ""rows"" is a list of lists, where the order of values in each list matches the order of the columns
?_shape=array - a JSON array of objects - effectively just the ""rows"" key from the default representation
?_shape=array&_nl=on - a newline-separated list of JSON objects
?_shape=arrayfirst - a flat JSON array containing just the first value from each row
?_shape=object - a JSON object keyed using the primary keys of the rows
_shape=arrays looks like this:
{
""ok"": true,
""next"": null,
""rows"": [
[3, ""Detroit""],
[2, ""Los Angeles""],
[4, ""Memnonia""],
[1, ""San Francisco""]
]
}
_shape=array looks like this:
[
{
""id"": 3,
""name"": ""Detroit""
},
{
""id"": 2,
""name"": ""Los Angeles""
},
{
""id"": 4,
""name"": ""Memnonia""
},
{
""id"": 1,
""name"": ""San Francisco""
}
]
_shape=array&_nl=on looks like this:
{""id"": 1, ""value"": ""Myoporum laetum :: Myoporum""}
{""id"": 2, ""value"": ""Metrosideros excelsa :: New Zealand Xmas Tree""}
{""id"": 3, ""value"": ""Pinus radiata :: Monterey Pine""}
_shape=arrayfirst looks like this:
[1, 2, 3]
_shape=object looks like this:
{
""1"": {
""id"": 1,
""value"": ""Myoporum laetum :: Myoporum""
},
""2"": {
""id"": 2,
""value"": ""Metrosideros excelsa :: New Zealand Xmas Tree""
},
""3"": {
""id"": 3,
""value"": ""Pinus radiata :: Monterey Pine""
}
]
The object shape is only available for queries against tables - custom SQL
queries and views do not have an obvious primary key so cannot be returned using
this format.
The object keys are always strings. If your table has a compound primary
key, the object keys will be a comma-separated string.",37,
517,Default representation,"The default JSON representation of data from a SQLite table or custom query
looks like this:
{
""ok"": true,
""rows"": [
{
""id"": 3,
""name"": ""Detroit""
},
{
""id"": 2,
""name"": ""Los Angeles""
},
{
""id"": 4,
""name"": ""Memnonia""
},
{
""id"": 1,
""name"": ""San Francisco""
}
],
""truncated"": false
}
""ok"" is always true if an error did not occur.
The ""rows"" key is a list of objects, each one representing a row.
The ""truncated"" key lets you know if the query was truncated. This can happen if a SQL query returns more than 1,000 results (or the max_returned_rows setting).
For table pages, an additional key ""next"" may be present. This indicates that the next page in the pagination set can be retrieved using ?_next=VALUE .",37,
516,JSON API,"Datasette provides a JSON API for your SQLite databases. Anything you can do
through the Datasette user interface can also be accessed as JSON via the API.
To access the API for a page, either click on the .json link on that page or
edit the URL and add a .json extension to it.",37,
515,Binary plugins,"Several Datasette plugins are available that change the way Datasette treats binary data.
datasette-render-binary modifies Datasette's default interface to show an automatic guess at what type of binary data is being stored, along with a visual representation of the binary value that displays ASCII strings directly in the interface.
datasette-render-images detects common image formats and renders them as images directly in the Datasette interface.
datasette-media allows Datasette interfaces to be configured to serve binary files from configured SQL queries, and includes the ability to resize images directly before serving them.",37,
514,Linking to binary downloads,"The .blob output format is used to return binary data. It requires a _blob_column= query string argument specifying which BLOB column should be downloaded, for example:
https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data
This output format can also be used to return binary data from an arbitrary SQL query. Since such queries do not specify an exact row, an additional ?_blob_hash= parameter can be used to specify the SHA-256 hash of the value that is being linked to.
Consider the query select data from binary_data - demonstrated here .
That page links to the binary value downloads. Those links look like this:
https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d
These .blob links are also returned in the .csv exports Datasette provides for binary tables and queries, since the CSV format does not have a mechanism for representing binary data.",37,
513,Binary data,"SQLite tables can contain binary data in BLOB columns.
Datasette includes special handling for these binary values. The Datasette interface detects binary values and provides a link to download their content, for example on https://latest.datasette.io/fixtures/binary_data
Binary data is represented in .json exports using Base64 encoding.
https://latest.datasette.io/fixtures/binary_data.json?_shape=array
[
{
""rowid"": 1,
""data"": {
""$base64"": true,
""encoded"": ""FRwCx60F/g==""
}
},
{
""rowid"": 2,
""data"": {
""$base64"": true,
""encoded"": ""FRwDx60F/g==""
}
},
{
""rowid"": 3,
""data"": null
}
]",37,