sections
582 rows sorted by ref
This data as json, CSV (advanced)
id | page | ref ▼ | title | content | breadcrumbs | references |
---|---|---|---|---|---|---|
internals:datasette-render-template | internals | datasette-render-template | await .render_template(template, context=None, request=None) | template - string, list of strings or jinja2.Template The template file to be rendered, e.g. my_plugin.html . Datasette will search for this file first in the --template-dir= location, if it was specified - then in the plugin's bundled templates and finally in Datasette's set of default templates. If this is a list of template file names then the first one that exists will be loaded and rendered. If this is a Jinja Template object it will be used directly. context - None or a Python dictionary The context variables to pass to the template. request - request object or None If you pass a Datasette request object here it will be made available to the template. Renders a Jinja template using Datasette's preconfigured instance of Jinja and returns the resulting string. The template will have access to Datasette's default template functions and any functions that have been made available by other plugins. | ["Internals for plugins", "Datasette class"] | [{"href": "https://jinja.palletsprojects.com/en/2.11.x/api/#jinja2.Template", "label": "Template object"}, {"href": "https://jinja.palletsprojects.com/en/2.11.x/", "label": "Jinja template"}] |
internals:datasette-resolve-database | internals | datasette-resolve-database | .resolve_database(request) | request - Request object A request object If you are implementing your own custom views, you may need to resolve the database that the user is requesting based on a URL path. If the regular expression for your route declares a database named group, you can use this method to resolve the database object. This returns a Database instance. If the database cannot be found, it raises a datasette.utils.asgi.DatabaseNotFound exception - which is a subclass of datasette.utils.asgi.NotFound with a .database_name attribute set to the name of the database that was requested. | ["Internals for plugins", "Datasette class"] | [] |
internals:datasette-resolve-row | internals | datasette-resolve-row | .resolve_row(request) | request - Request object A request object This method assumes your route declares named groups for database , table and pks . It returns a ResolvedRow named tuple instance with the following fields: db - Database The database object table - string The name of the table sql - string SQL snippet that can be used in a WHERE clause to select the row params - dict Parameters that should be passed to the SQL query pks - list List of primary key column names pk_values - list List of primary key values decoded from the URL row - sqlite3.Row The row itself If the database or table cannot be found it raises a datasette.utils.asgi.DatabaseNotFound exception. If the table does not exist it raises a datasette.utils.asgi.TableNotFound … | ["Internals for plugins", "Datasette class"] | [] |
internals:datasette-resolve-table | internals | datasette-resolve-table | .resolve_table(request) | request - Request object A request object This assumes that the regular expression for your route declares both a database and a table named group. It returns a ResolvedTable named tuple instance with the following fields: db - Database The database object table - string The name of the table (or view) is_view - boolean True if this is a view, False if it is a table If the database or table cannot be found it raises a datasette.utils.asgi.DatabaseNotFound exception. If the table does not exist it raises a datasette.utils.asgi.TableNotFound exception - a subclass of datasette.utils.asgi.NotFound with .database_name and .table attributes. | ["Internals for plugins", "Datasette class"] | [] |
internals:datasette-setting | internals | datasette-setting | .setting(key) | key - string The name of the setting, e.g. base_url . Returns the configured value for the specified setting . This can be a string, boolean or integer depending on the requested setting. For example: downloads_are_allowed = datasette.setting("allow_download") | ["Internals for plugins", "Datasette class"] | [] |
internals:datasette-sign | internals | datasette-sign | .sign(value, namespace="default") | value - any serializable type The value to be signed. namespace - string, optional An alternative namespace, see the itsdangerous salt documentation . Utility method for signing values, such that you can safely pass data to and from an untrusted environment. This is a wrapper around the itsdangerous library. This method returns a signed string, which can be decoded and verified using .unsign(value, namespace="default") . | ["Internals for plugins", "Datasette class"] | [{"href": "https://itsdangerous.palletsprojects.com/en/1.1.x/serializer/#the-salt", "label": "itsdangerous salt documentation"}, {"href": "https://itsdangerous.palletsprojects.com/", "label": "itsdangerous"}] |
internals:datasette-track-event | internals | datasette-track-event | await .track_event(event) | event - Event An instance of a subclass of datasette.events.Event . Plugins can call this to track events, using classes they have previously registered. See Event tracking for details. The event will then be passed to all plugins that have registered to receive events using the track_event(datasette, event) hook. Example usage, assuming the plugin has previously registered the BanUserEvent class: await datasette.track_event( BanUserEvent(user={"id": 1, "username": "cleverbot"}) ) | ["Internals for plugins", "Datasette class"] | [] |
internals:datasette-unsign | internals | datasette-unsign | .unsign(value, namespace="default") | signed - any serializable type The signed string that was created using .sign(value, namespace="default") . namespace - string, optional The alternative namespace, if one was used. Returns the original, decoded object that was passed to .sign(value, namespace="default") . If the signature is not valid this raises a itsdangerous.BadSignature exception. | ["Internals for plugins", "Datasette class"] | [] |
deploying:deploying | deploying | deploying | Deploying Datasette | The quickest way to deploy a Datasette instance on the internet is to use the datasette publish command, described in Publishing data . This can be used to quickly deploy Datasette to a number of hosting providers including Heroku, Google Cloud Run and Vercel. You can deploy Datasette to other hosting providers using the instructions on this page. | [] | [] |
deploying:deploying-buildpacks | deploying | deploying-buildpacks | Deploying using buildpacks | Some hosting providers such as Heroku , DigitalOcean App Platform and Scalingo support the Buildpacks standard for deploying Python web applications. Deploying Datasette on these platforms requires two files: requirements.txt and Procfile . The requirements.txt file lets the platform know which Python packages should be installed. It should contain datasette at a minimum, but can also list any Datasette plugins you wish to install - for example: datasette datasette-vega The Procfile lets the hosting platform know how to run the command that serves web traffic. It should look like this: web: datasette . -h 0.0.0.0 -p $PORT --cors The $PORT environment variable is provided by the hosting platform. --cors enables CORS requests from JavaScript running on other websites to your domain - omit this if you don't want to allow CORS. You can add additional Datasette Settings options here too. These two files should be enough to deploy Datasette on any host that supports buildpacks. Datasette will serve any SQLite files that are included in the root directory of the application. If you want to build SQLite files or download them as part of the deployment process you can do so using a bin/post_compile file. For example, the following bin/post_compile will download an example database that will then be served by Datasette: wget https://fivethirtyeight.datasettes.com/fivethirtyeight.db simonw/buildpack-datasette-demo is an example GitHub repository showing a Datasette configuration that can be deployed to a buildpack-supporting host. | ["Deploying Datasette"] | [{"href": "https://www.heroku.com/", "label": "Heroku"}, {"href": "https://www.digitalocean.com/docs/app-platform/", "label": "DigitalOcean App Platform"}, {"href": "https://scalingo.com/", "label": "Scalingo"}, {"href": "https://buildpacks.io/", "label": "Buildpacks standard"}, {"href": "https://github.com/simonw/buildpack-datasette-demo", "label": "simonw/buildpack-datasette-demo"}] |
deploying:deploying-fundamentals | deploying | deploying-fundamentals | Deployment fundamentals | Datasette can be deployed as a single datasette process that listens on a port. Datasette is not designed to be run as root, so that process should listen on a higher port such as port 8000. If you want to serve Datasette on port 80 (the HTTP default port) or port 443 (for HTTPS) you should run it behind a proxy server, such as nginx, Apache or HAProxy. The proxy server can listen on port 80/443 and forward traffic on to Datasette. | ["Deploying Datasette"] | [] |
deploying:deploying-openrc | deploying | deploying-openrc | Running Datasette using OpenRC | OpenRC is the service manager on non-systemd Linux distributions like Alpine Linux and Gentoo . Create an init script at /etc/init.d/datasette with the following contents: #!/sbin/openrc-run name="datasette" command="datasette" command_args="serve -h 0.0.0.0 /path/to/db.db" command_background=true pidfile="/run/${RC_SVCNAME}.pid" You then need to configure the service to run at boot and start it: rc-update add datasette rc-service datasette start | ["Deploying Datasette"] | [{"href": "https://www.alpinelinux.org/", "label": "Alpine Linux"}, {"href": "https://www.gentoo.org/", "label": "Gentoo"}] |
plugins:deploying-plugins-using-datasette-publish | plugins | deploying-plugins-using-datasette-publish | Deploying plugins using datasette publish | The datasette publish and datasette package commands both take an optional --install argument. You can use this one or more times to tell Datasette to pip install specific plugins as part of the process: datasette publish cloudrun mydb.db --install=datasette-vega You can use the name of a package on PyPI or any of the other valid arguments to pip install such as a URL to a .zip file: datasette publish cloudrun mydb.db \ --install=https://url-to-my-package.zip | ["Plugins", "Installing plugins"] | [] |
deploying:deploying-proxy | deploying | deploying-proxy | Running Datasette behind a proxy | You may wish to run Datasette behind an Apache or nginx proxy, using a path within your existing site. You can use the base_url configuration setting to tell Datasette to serve traffic with a specific URL prefix. For example, you could run Datasette like this: datasette my-database.db --setting base_url /my-datasette/ -p 8009 This will run Datasette with the following URLs: http://127.0.0.1:8009/my-datasette/ - the Datasette homepage http://127.0.0.1:8009/my-datasette/my-database - the page for the my-database.db database http://127.0.0.1:8009/my-datasette/my-database/some_table - the page for the some_table table You can now set your nginx or Apache server to proxy the /my-datasette/ path to this Datasette instance. | ["Deploying Datasette"] | [] |
deploying:deploying-systemd | deploying | deploying-systemd | Running Datasette using systemd | You can run Datasette on Ubuntu or Debian systems using systemd . First, ensure you have Python 3 and pip installed. On Ubuntu you can use sudo apt-get install python3 python3-pip . You can install Datasette into a virtual environment, or you can install it system-wide. To install system-wide, use sudo pip3 install datasette . Now create a folder for your Datasette databases, for example using mkdir /home/ubuntu/datasette-root . You can copy a test database into that folder like so: cd /home/ubuntu/datasette-root curl -O https://latest.datasette.io/fixtures.db Create a file at /etc/systemd/system/datasette.service with the following contents: [Unit] Description=Datasette After=network.target [Service] Type=simple User=ubuntu Environment=DATASETTE_SECRET= WorkingDirectory=/home/ubuntu/datasette-root ExecStart=datasette serve . -h 127.0.0.1 -p 8000 Restart=on-failure [Install] WantedBy=multi-user.target Add a random value for the DATASETTE_SECRET - this will be used to sign Datasette cookies such as the CSRF token cookie. You can generate a suitable value like so: python3 -c 'import secrets; print(secrets.token_hex(32))' This configuration will run Datasette against all database files contained in the /home/ubuntu/datasette-root directory. If that directory contains a metadata.yml (or .json ) file or a templates/ or plugins/ sub-directory those will automatically be loaded by Datasette - see Configuration directory mode for details. You can start the Datasette process running using the following: sudo systemctl daemon-reload sudo systemctl start datasette.service You will need to restart the Datasette service after making changes to its metadata.json configuration or adding a new database file to that directory. You can do that using: sudo systemctl restart datasette.service Once the … | ["Deploying Datasette"] | [] |
contributing:devenvironment | contributing | devenvironment | Setting up a development environment | If you have Python 3.8 or higher installed on your computer (on OS X the quickest way to do this is using homebrew ) you can install an editable copy of Datasette using the following steps. If you want to use GitHub to publish your changes, first create a fork of datasette under your own GitHub account. Now clone that repository somewhere on your computer: git clone git@github.com:YOURNAME/datasette If you want to get started without creating your own fork, you can do this instead: git clone git@github.com:simonw/datasette The next step is to create a virtual environment for your project and use it to install Datasette's dependencies: cd datasette # Create a virtual environment in ./venv python3 -m venv ./venv # Now activate the virtual environment, so pip can install into it source venv/bin/activate # Install Datasette and its testing dependencies python3 -m pip install -e '.[test]' That last line does most of the work: pip install -e means "install this package in a way that allows me to edit the source code in place". The .[test] option means "use the setup.py in this directory and install the optional testing dependencies as well". | ["Contributing"] | [{"href": "https://docs.python-guide.org/starting/install3/osx/", "label": "is using homebrew"}, {"href": "https://github.com/simonw/datasette/fork", "label": "create a fork of datasette"}] |
changelog:documentation | changelog | documentation | Documentation | Documentation describing how to write tests that use signed actor cookies using datasette.client.actor_cookie() . ( #1830 ) Documentation on how to register a plugin for the duration of a test . ( #2234 ) The configuration documentation now shows examples of both YAML and JSON for each setting. | ["Changelog", "1.0a8 (2024-02-07)"] | [{"href": "https://github.com/simonw/datasette/issues/1830", "label": "#1830"}, {"href": "https://github.com/simonw/datasette/issues/2234", "label": "#2234"}] |
ecosystem:dogsheep | ecosystem | dogsheep | Dogsheep | Dogsheep is a collection of tools for personal analytics using SQLite and Datasette. The project provides tools like github-to-sqlite and twitter-to-sqlite that can import data from different sources in order to create a personal data warehouse. Personal Data Warehouses: Reclaiming Your Data is a talk that explains Dogsheep and demonstrates it in action. | ["The Datasette Ecosystem"] | [{"href": "https://dogsheep.github.io/", "label": "Dogsheep"}, {"href": "https://datasette.io/tools/github-to-sqlite", "label": "github-to-sqlite"}, {"href": "https://datasette.io/tools/twitter-to-sqlite", "label": "twitter-to-sqlite"}, {"href": "https://simonwillison.net/2020/Nov/14/personal-data-warehouses/", "label": "Personal Data Warehouses: Reclaiming Your Data"}] |
ecosystem:ecosystem | ecosystem | ecosystem | The Datasette Ecosystem | Datasette sits at the center of a growing ecosystem of open source tools aimed at making it as easy as possible to gather, analyze and publish interesting data. These tools are divided into two main groups: tools for building SQLite databases (for use with Datasette) and plugins that extend Datasette's functionality. The Datasette project website includes a directory of plugins and a directory of tools: Plugins directory on datasette.io Tools directory on datasette.io | [] | [{"href": "https://datasette.io/", "label": "Datasette project website"}, {"href": "https://datasette.io/plugins", "label": "Plugins directory on datasette.io"}, {"href": "https://datasette.io/tools", "label": "Tools directory on datasette.io"}] |
json_api:expand-foreign-keys | json_api | expand-foreign-keys | Expanding foreign key references | Datasette can detect foreign key relationships and resolve those references into labels. The HTML interface does this by default for every detected foreign key column - you can turn that off using ?_labels=off . You can request foreign keys be expanded in JSON using the _labels=on or _label=COLUMN special query string parameters. Here's what an expanded row looks like: [ { "rowid": 1, "TreeID": 141565, "qLegalStatus": { "value": 1, "label": "Permitted Site" }, "qSpecies": { "value": 1, "label": "Myoporum laetum :: Myoporum" }, "qAddress": "501X Baker St", "SiteOrder": 1 } ] The column in the foreign key table that is used for the label can be specified in metadata.json - see Specifying the label column for a table . | ["JSON API"] | [] |
changelog:facet-by-date | changelog | facet-by-date | Facet by date | If a column contains datetime values, Datasette can now facet that column by date. ( #481 ) | ["Changelog", "0.29 (2019-07-07)"] | [{"href": "https://github.com/simonw/datasette/issues/481", "label": "#481"}] |
changelog:faceting | changelog | faceting | Faceting | The number of unique values in a facet is now always displayed. Previously it was only displayed if the user specified ?_facet_size=max . ( #1556 ) Facets of type date or array can now be configured in metadata.json , see Facets in metadata . Thanks, David Larlet. ( #1552 ) New ?_nosuggest=1 parameter for table views, which disables facet suggestion. ( #1557 ) Fixed bug where ?_facet_array=tags&_facet=tags would only display one of the two selected facets. ( #625 ) | ["Changelog", "0.60 (2022-01-13)"] | [{"href": "https://github.com/simonw/datasette/issues/1556", "label": "#1556"}, {"href": "https://github.com/simonw/datasette/issues/1552", "label": "#1552"}, {"href": "https://github.com/simonw/datasette/issues/1557", "label": "#1557"}, {"href": "https://github.com/simonw/datasette/issues/625", "label": "#625"}] |
facets:facets-in-query-strings | facets | facets-in-query-strings | Facets in query strings | To turn on faceting for specific columns on a Datasette table view, add one or more _facet=COLUMN parameters to the URL. For example, if you want to turn on facets for the city_id and state columns, construct a URL that looks like this: /dbname/tablename?_facet=state&_facet=city_id This works for both the HTML interface and the .json view. When enabled, facets will cause a facet_results block to be added to the JSON output, looking something like this: { "state": { "name": "state", "results": [ { "value": "CA", "label": "CA", "count": 10, "toggle_url": "http://...?_facet=city_id&_facet=state&state=CA", "selected": false }, { "value": "MI", "label": "MI", "count": 4, "toggle_url": "http://...?_facet=city_id&_facet=state&state=MI", "selected": false }, { "value": "MC", "label": "MC", "count": 1, "toggle_url": "http://...?_facet=city_id&_facet=state&state=MC", "selected": false } ], "truncated": false } "city_id": { "name": "city_id", "results": [ { "value": 1, "label": "San Francisco", "count": 6, "toggle_url": "http://...?_facet=city_id&_facet=state&city_id=1", "selected": false }, { "value": 2, "label": "Los Angeles", "count": 4, "toggle_url": "http://...?_facet=city_id&_facet=state&city_id=2", "selected": false }, { "value": 3, "label": "Detroit", "count": 4, "toggle_url": "http://...?_facet=city_id&_facet=state&city_id=3", "selected": false }, { "value": 4, "label": "Memnonia", "count": 1, "toggle_url": "http://...?_facet=city_id&_facet=state&city_id=4", "selected": false } ], "truncated": false } } If Datasette detect… | ["Facets"] | [] |
facets:facets-metadata | facets | facets-metadata | Facets in metadata | You can turn facets on by default for specific tables by adding them to a "facets" key in a Datasette Metadata file. Here's an example that turns on faceting by default for the qLegalStatus column in the Street_Tree_List table in the sf-trees database: [[[cog from metadata_doc import metadata_example metadata_example(cog, { "databases": { "sf-trees": { "tables": { "Street_Tree_List": { "facets": ["qLegalStatus"] } } } } }) ]]] [[[end]]] Facets defined in this way will always be shown in the interface and returned in the API, regardless of the _facet arguments passed to the view. You can specify array or date facets in metadata using JSON objects with a single key of array or date and a value specifying the column, like this: [[[cog metadata_example(cog, { "facets": [ {"array": "tags"}, {"date": "created"} ] }) ]]] [[[end]]] You can change the default facet size (the number of results shown for each facet) for a table using facet_size : [[[cog metadata_example(cog, { "databases": { "sf-trees": { "tables": { "Street_Tree_List": { "facets": ["qLegalStatus"], "facet_size": 10 } } } } }) ]]] [[[end]]] | ["Facets"] | [] |
changelog:features | changelog | features | Features | Now tested against Python 3.11. Docker containers used by datasette publish and datasette package both now use that version of Python. ( #1853 ) --load-extension option now supports entrypoints. Thanks, Alex Garcia. ( #1789 ) Facet size can now be set per-table with the new facet_size table metadata option. ( #1804 ) The truncate_cells_html setting now also affects long URLs in columns. ( #1805 ) The non-JavaScript SQL editor textarea now increases height to fit the SQL query. ( #1786 ) Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. ( #1794 ) The settings.json file used in Configuration directory mode is now validated on startup. ( #1816 ) SQL queries can now include leading SQL comments, using /* ... */ or -- ... syntax. Thanks, Charles Nepote. ( #1860 ) SQL query is now re-displayed when terminated with a time limit error. ( #1819 ) The inspect data mechanism is now used to speed up server startup - thanks, Forest Gregg. ( #1834 ) In Configuration directory mode databases with filenames ending in .sqlite or .sqlite3 are now automatically added to the Datasette instance. ( #1646 ) Breadcrumb navigation display now respects the current user's permissions. ( #1831 ) | ["Changelog", "0.63 (2022-10-27)"] | [{"href": "https://github.com/simonw/datasette/issues/1853", "label": "#1853"}, {"href": "https://github.com/simonw/datasette/pull/1789", "label": "#1789"}, {"href": "https://github.com/simonw/datasette/issues/1804", "label": "#1804"}, {"href": "https://github.com/simonw/datasette/issues/1805", "label": "#1805"}, {"href": "https://github.com/simonw/datasette/issues/1786", "label": "#1786"}, {"href": "https://github.com/simonw/datasette/pull/1794", "label": "#1794"}, {"href": "https://github.com/simonw/datasette/issues/1816", "label": "#1816"}, {"href": "https://github.com/simonw/datasette/issues/1860", "label": "#1860"}, {"href": "https://github.com/simonw/datasette/issues/1819", "label": "#1819"}, {"href": "https://github.com/simonw/datasette/issues/1834", "label": "#1834"}, {"href": "https://github.com/simonw/datasette/issues/1646", "label": "#1646"}, {"href": "https://github.com/simonw/datasette/issues/1831", "label": "#1831"}] |
changelog:flash-messages | changelog | flash-messages | Flash messages | Writable canned queries needed a mechanism to let the user know that the query has been successfully executed. The new flash messaging system ( #790 ) allows messages to persist in signed cookies which are then displayed to the user on the next page that they visit. Plugins can use this mechanism to display their own messages, see .add_message(request, message, type=datasette.INFO) for details. You can try out the new messages using the /-/messages debug tool, for example at https://latest.datasette.io/-/messages | ["Changelog", "0.44 (2020-06-11)"] | [{"href": "https://github.com/simonw/datasette/issues/790", "label": "#790"}, {"href": "https://latest.datasette.io/-/messages", "label": "https://latest.datasette.io/-/messages"}] |
changelog:foreign-key-expansions | changelog | foreign-key-expansions | Foreign key expansions | When Datasette detects a foreign key reference it attempts to resolve a label for that reference (automatically or using the Specifying the label column for a table metadata option) so it can display a link to the associated row. This expansion is now also available for JSON and CSV representations of the table, using the new _labels=on query string option. See Expanding foreign key references for more details. | ["Changelog", "0.23 (2018-06-18)"] | [] |
sql_queries:fragment | sql_queries | fragment | fragment | Some plugins, such as datasette-vega , can be configured by including additional data in the fragment hash of the URL - the bit that comes after a # symbol. You can set a default fragment hash that will be included in the link to the canned query from the database index page using the "fragment" key. This example demonstrates both fragment and hide_sql : [[[cog config_example(cog, """ databases: fixtures: queries: neighborhood_search: fragment: fragment-goes-here hide_sql: true sql: |- select neighborhood, facet_cities.name, state from facetable join facet_cities on facetable.city_id = facet_cities.id where neighborhood like '%' || :text || '%' order by neighborhood; """) ]]] [[[end]]] See here for a demo of this in action. | ["Running SQL queries", "Canned queries", "Additional canned query options"] | [{"href": "https://github.com/simonw/datasette-vega", "label": "datasette-vega"}, {"href": "https://latest.datasette.io/fixtures#queries", "label": "See here"}] |
full_text_search:full-text-search-advanced-queries | full_text_search | full-text-search-advanced-queries | Advanced SQLite search queries | SQLite full-text search includes support for a variety of advanced queries , including AND , OR , NOT and NEAR . By default Datasette disables these features to ensure they do not cause errors or confusion for users who are not aware of them. You can disable this escaping and use the advanced queries by adding &_searchmode=raw to the table page query string. If you want to enable these operators by default for a specific table, you can do so by adding "searchmode": "raw" to the metadata configuration for that table, see Configuring full-text search for a table or view . If that option has been specified in the table metadata but you want to over-ride it and return to the default behavior you can append &_searchmode=escaped to the query string. | ["Full-text search"] | [{"href": "https://www.sqlite.org/fts5.html#full_text_query_syntax", "label": "a variety of advanced queries"}] |
full_text_search:full-text-search-custom-sql | full_text_search | full-text-search-custom-sql | Searches using custom SQL | You can include full-text search results in custom SQL queries. The general pattern with SQLite search is to run the search as a sub-select that returns rowid values, then include those rowids in another part of the query. You can see the syntax for a basic search by running that search on a table page and then clicking "View and edit SQL" to see the underlying SQL. For example, consider this search for manafort is the US FARA database : /fara/FARA_All_ShortForms?_search=manafort If you click View and edit SQL you'll see that the underlying SQL looks like this: select rowid, Short_Form_Termination_Date, Short_Form_Date, Short_Form_Last_Name, Short_Form_First_Name, Registration_Number, Registration_Date, Registrant_Name, Address_1, Address_2, City, State, Zip from FARA_All_ShortForms where rowid in ( select rowid from FARA_All_ShortForms_fts where FARA_All_ShortForms_fts match escape_fts(:search) ) order by rowid limit 101 | ["Full-text search"] | [{"href": "https://fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort", "label": "manafort is the US FARA database"}, {"href": "https://fara.datasettes.com/fara?sql=select%0D%0A++rowid%2C%0D%0A++Short_Form_Termination_Date%2C%0D%0A++Short_Form_Date%2C%0D%0A++Short_Form_Last_Name%2C%0D%0A++Short_Form_First_Name%2C%0D%0A++Registration_Number%2C%0D%0A++Registration_Date%2C%0D%0A++Registrant_Name%2C%0D%0A++Address_1%2C%0D%0A++Address_2%2C%0D%0A++City%2C%0D%0A++State%2C%0D%0A++Zip%0D%0Afrom%0D%0A++FARA_All_ShortForms%0D%0Awhere%0D%0A++rowid+in+%28%0D%0A++++select%0D%0A++++++rowid%0D%0A++++from%0D%0A++++++FARA_All_ShortForms_fts%0D%0A++++where%0D%0A++++++FARA_All_ShortForms_fts+match+escape_fts%28%3Asearch%29%0D%0A++%29%0D%0Aorder+by%0D%0A++rowid%0D%0Alimit%0D%0A++101&search=manafort", "label": "View and edit SQL"}] |
full_text_search:full-text-search-enabling | full_text_search | full-text-search-enabling | Enabling full-text search for a SQLite table | Datasette takes advantage of the external content mechanism in SQLite, which allows a full-text search virtual table to be associated with the contents of another SQLite table. To set up full-text search for a table, you need to do two things: Create a new FTS virtual table associated with your table Populate that FTS table with the data that you would like to be able to run searches against | ["Full-text search"] | [{"href": "https://www.sqlite.org/fts3.html#_external_content_fts4_tables_", "label": "external content"}] |
full_text_search:full-text-search-fts-versions | full_text_search | full-text-search-fts-versions | FTS versions | There are three different versions of the SQLite FTS module: FTS3, FTS4 and FTS5. You can tell which versions are supported by your instance of Datasette by checking the /-/versions page. FTS5 is the most advanced module but may not be available in the SQLite version that is bundled with your Python installation. Most importantly, FTS5 is the only version that has the ability to order by search relevance without needing extra code. If you can't be sure that FTS5 will be available, you should use FTS4. | ["Full-text search"] | [] |
full_text_search:full-text-search-table-or-view | full_text_search | full-text-search-table-or-view | Configuring full-text search for a table or view | If a table has a corresponding FTS table set up using the content= argument to CREATE VIRTUAL TABLE shown below, Datasette will detect it automatically and add a search interface to the table page for that table. You can also manually configure which table should be used for full-text search using query string parameters or Metadata . You can set the associated FTS table for a specific table and you can also set one for a view - if you do that, the page for that SQL view will offer a search option. Use ?_fts_table=x to over-ride the FTS table for a specific page. If the primary key was something other than rowid you can use ?_fts_pk=col to set that as well. This is particularly useful for views, for example: https://latest.datasette.io/fixtures/searchable_view?_fts_table=searchable_fts&_fts_pk=pk The fts_table metadata property can be used to specify an associated FTS table. If the primary key column in your table which was used to populate the FTS table is something other than rowid , you can specify the column to use with the fts_pk property. The "searchmode": "raw" property can be used to default the table to accepting SQLite advanced search operators, as described in Advanced SQLite search queries . Here is an example which enables full-text search (with SQLite advanced search operators) for a display_ads view which is defined against the ads table and hence needs to run FTS against the ads_fts table, using the id as the primary key: [[[cog from metadata_doc import metadata_example metadata_example(cog, { "databases": { "russian-ads": { "tables": { "display_ads": { "fts_table": "ads_fts", "fts_pk": "id", "searchmode": "raw" } } } } }) ]]] [[[end]]] | ["Full-text search"] | [{"href": "https://latest.datasette.io/fixtures/searchable_view?_fts_table=searchable_fts&_fts_pk=pk", "label": "https://latest.datasette.io/fixtures/searchable_view?_fts_table=searchable_fts&_fts_pk=pk"}] |
full_text_search:full-text-search-table-view-api | full_text_search | full-text-search-table-view-api | The table page and table view API | Table views that support full-text search can be queried using the ?_search=TERMS query string parameter. This will run the search against content from all of the columns that have been included in the index. Try this example: fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort SQLite full-text search supports wildcards. This means you can easily implement prefix auto-complete by including an asterisk at the end of the search term - for example: /dbname/tablename/?_search=rob* This will return all records containing at least one word that starts with the letters rob . You can also run searches against just the content of a specific named column by using _search_COLNAME=TERMS - for example, this would search for just rows where the name column in the FTS index mentions Sarah : /dbname/tablename/?_search_name=Sarah | ["Full-text search"] | [{"href": "https://fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort", "label": "fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort"}] |
contributing:general-guidelines | contributing | general-guidelines | General guidelines | main should always be releasable . Incomplete features should live in branches. This ensures that any small bug fixes can be quickly released. The ideal commit should bundle together the implementation, unit tests and associated documentation updates. The commit message should link to an associated issue. New plugin hooks should only be shipped if accompanied by a separate release of a non-demo plugin that uses them. | ["Contributing"] | [] |
getting_started:getting-started | getting_started | getting-started | Getting started | [] | [] | |
getting_started:getting-started-datasette-lite | getting_started | getting-started-datasette-lite | Datasette in your browser with Datasette Lite | Datasette Lite is Datasette packaged using WebAssembly so that it runs entirely in your browser, no Python web application server required. You can pass a URL to a CSV, SQLite or raw SQL file directly to Datasette Lite to explore that data in your browser. This example link opens Datasette Lite and loads the SQL Murder Mystery example database from Northwestern University Knight Lab . | ["Getting started"] | [{"href": "https://lite.datasette.io/", "label": "Datasette Lite"}, {"href": "https://lite.datasette.io/?url=https%3A%2F%2Fraw.githubusercontent.com%2FNUKnightLab%2Fsql-mysteries%2Fmaster%2Fsql-murder-mystery.db#/sql-murder-mystery", "label": "example link"}, {"href": "https://github.com/NUKnightLab/sql-mysteries", "label": "Northwestern University Knight Lab"}] |
getting_started:getting-started-demo | getting_started | getting-started-demo | Play with a live demo | The best way to experience Datasette for the first time is with a demo: global-power-plants.datasettes.com provides a searchable database of power plants around the world, using data from the World Resources Institude rendered using the datasette-cluster-map plugin. fivethirtyeight.datasettes.com shows Datasette running against over 400 datasets imported from the FiveThirtyEight GitHub repository . | ["Getting started"] | [{"href": "https://global-power-plants.datasettes.com/global-power-plants/global-power-plants", "label": "global-power-plants.datasettes.com"}, {"href": "https://www.wri.org/publication/global-power-plant-database", "label": "World Resources Institude"}, {"href": "https://github.com/simonw/datasette-cluster-map", "label": "datasette-cluster-map"}, {"href": "https://fivethirtyeight.datasettes.com/fivethirtyeight", "label": "fivethirtyeight.datasettes.com"}, {"href": "https://github.com/fivethirtyeight/data", "label": "FiveThirtyEight GitHub repository"}] |
getting_started:getting-started-glitch | getting_started | getting-started-glitch | Try Datasette without installing anything using Glitch | Glitch is a free online tool for building web apps directly from your web browser. You can use Glitch to try out Datasette without needing to install any software on your own computer. Here's a demo project on Glitch which you can use as the basis for your own experiments: glitch.com/~datasette-csvs Glitch allows you to "remix" any project to create your own copy and start editing it in your browser. You can remix the datasette-csvs project by clicking this button: Find a CSV file and drag it onto the Glitch file explorer panel - datasette-csvs will automatically convert it to a SQLite database (using sqlite-utils ) and allow you to start exploring it using Datasette. If your CSV file has a latitude and longitude column you can visualize it on a map by uncommenting the datasette-cluster-map line in the requirements.txt file using the Glitch file editor. Need some data? Try this Public Art Data for the city of Seattle - hit "Export" and select "CSV" to download it as a CSV file. For more on how this works, see Running Datasette on Glitch . | ["Getting started"] | [{"href": "https://glitch.com/", "label": "Glitch"}, {"href": "https://glitch.com/~datasette-csvs", "label": "glitch.com/~datasette-csvs"}, {"href": "https://glitch.com/edit/#!/remix/datasette-csvs", "label": null}, {"href": "https://github.com/simonw/sqlite-utils", "label": "sqlite-utils"}, {"href": "https://data.seattle.gov/Community/Public-Art-Data/j7sn-tdzk", "label": "Public Art Data"}, {"href": "https://simonwillison.net/2019/Apr/23/datasette-glitch/", "label": "Running Datasette on Glitch"}] |
getting_started:getting-started-tutorial | getting_started | getting-started-tutorial | Follow a tutorial | Datasette has several tutorials to help you get started with the tool. Try one of the following: Exploring a database with Datasette shows how to use the Datasette web interface to explore a new database. Learn SQL with Datasette introduces SQL, and shows how to use that query language to ask questions of your data. Cleaning data with sqlite-utils and Datasette guides you through using sqlite-utils to turn a CSV file into a database that you can explore using Datasette. | ["Getting started"] | [{"href": "https://datasette.io/tutorials", "label": "tutorials"}, {"href": "https://datasette.io/tutorials/explore", "label": "Exploring a database with Datasette"}, {"href": "https://datasette.io/tutorials/learn-sql", "label": "Learn SQL with Datasette"}, {"href": "https://datasette.io/tutorials/clean-data", "label": "Cleaning data with sqlite-utils and Datasette"}, {"href": "https://sqlite-utils.datasette.io/", "label": "sqlite-utils"}] |
getting_started:getting-started-your-computer | getting_started | getting-started-your-computer | Using Datasette on your own computer | First, follow the Installation instructions. Now you can run Datasette against a SQLite file on your computer using the following command: datasette path/to/database.db This will start a web server on port 8001 - visit http://localhost:8001/ to access the web interface. Add -o to open your browser automatically once Datasette has started: datasette path/to/database.db -o Use Chrome on OS X? You can run datasette against your browser history like so: datasette ~/Library/Application\ Support/Google/Chrome/Default/History --nolock The --nolock option ignores any file locks. This is safe as Datasette will open the file in read-only mode. Now visiting http://localhost:8001/History/downloads will show you a web interface to browse your downloads data: http://localhost:8001/History/downloads.json will return that data as JSON: { "database": "History", "columns": [ "id", "current_path", "target_path", "start_time", "received_bytes", "total_bytes", ... ], "rows": [ [ 1, "/Users/simonw/Downloads/DropboxInstaller.dmg", "/Users/simonw/Downloads/DropboxInstaller.dmg", 13097290269022132, 626688, 0, ... ] ] } http://localhost:8001/History/downloads.json?_shape=objects will return that data as JSON in a more convenient format: { ... "rows": [ { "start_time": 13097290269022132, "interrupt_reason": 0, "hash": "", "id": 1, "site_url": "", "referrer": "https://www.dropbox.com/downloading?src=index", ... } ] } | ["Getting started"] | [{"href": "http://localhost:8001/", "label": "http://localhost:8001/"}, {"href": "http://localhost:8001/History/downloads", "label": "http://localhost:8001/History/downloads"}, {"href": "http://localhost:8001/History/downloads.json", "label": "http://localhost:8001/History/downloads.json"}, {"href": "http://localhost:8001/History/downloads.json?_shape=objects", "label": "http://localhost:8001/History/downloads.json?_shape=objects"}] |
sql_queries:hide-sql | sql_queries | hide-sql | hide_sql | Canned queries default to displaying their SQL query at the top of the page. If the query is extremely long you may want to hide it by default, with a "show" link that can be used to make it visible. Add the "hide_sql": true option to hide the SQL query by default. | ["Running SQL queries", "Canned queries", "Additional canned query options"] | [] |
performance:http-caching | performance | http-caching | HTTP caching | If your database is immutable and guaranteed not to change, you can gain major performance improvements from Datasette by enabling HTTP caching. This can work at two different levels. First, it can tell browsers to cache the results of queries and serve future requests from the browser cache. More significantly, it allows you to run Datasette behind a caching proxy such as Varnish or use a cache provided by a hosted service such as Fastly or Cloudflare . This can provide incredible speed-ups since a query only needs to be executed by Datasette the first time it is accessed - all subsequent hits can then be served by the cache. Using a caching proxy in this way could enable a Datasette-backed visualization to serve thousands of hits a second while running Datasette itself on extremely inexpensive hosting. Datasette's integration with HTTP caches can be enabled using a combination of configuration options and query string arguments. The default_cache_ttl setting sets the default HTTP cache TTL for all Datasette pages. This is 5 seconds unless you change it - you can set it to 0 if you wish to disable HTTP caching entirely. You can also change the cache timeout on a per-request basis using the ?_ttl=10 query string parameter. This can be useful when you are working with the Datasette JSON API - you may decide that a specific query can be cached for a longer time, or maybe you need to set ?_ttl=0 for some requests for example if you are running a SQL order by random() query. | ["Performance and caching"] | [{"href": "https://varnish-cache.org/", "label": "Varnish"}, {"href": "https://www.fastly.com/", "label": "Fastly"}, {"href": "https://www.cloudflare.com/", "label": "Cloudflare"}] |
authentication:id1 | authentication | id1 | Built-in permissions | This section lists all of the permission checks that are carried out by Datasette core, along with the resource if it was passed. | ["Authentication and permissions"] | [] |
changelog:id1 | changelog | id1 | Changelog | [] | [] | |
cli-reference:id1 | cli-reference | id1 | CLI reference | The datasette CLI tool provides a number of commands. Running datasette without specifying a command runs the default command, datasette serve . See datasette serve for the full list of options for that command. [[[cog from datasette import cli from click.testing import CliRunner import textwrap def help(args): title = "datasette " + " ".join(args) cog.out("\n::\n\n") result = CliRunner().invoke(cli.cli, args) output = result.output.replace("Usage: cli ", "Usage: datasette ") cog.out(textwrap.indent(output, ' ')) cog.out("\n\n") ]]] [[[end]]] | [] | [] |
configuration:id1 | configuration | id1 | Configuration | Datasette offers several ways to configure your Datasette instances: server settings, plugin configuration, authentication, and more. Most configuration can be handled using a datasette.yaml configuration file, passed to datasette using the -c/--config flag: datasette mydatabase.db --config datasette.yaml This file can also use JSON, as datasette.json . YAML is recommended over JSON due to its support for comments and multi-line strings. | [] | [] |
contributing:id1 | contributing | id1 | Contributing | Datasette is an open source project. We welcome contributions! This document describes how to contribute to Datasette core. You can also contribute to the wider Datasette ecosystem by creating new Plugins . | [] | [] |
csv_export:id1 | csv_export | id1 | CSV export | Any Datasette table, view or custom SQL query can be exported as CSV. To obtain the CSV representation of the table you are looking, click the "this data as CSV" link. You can also use the advanced export form for more control over the resulting file, which looks like this and has the following options: download file - instead of displaying CSV in your browser, this forces your browser to download the CSV to your downloads directory. expand labels - if your table has any foreign key references this option will cause the CSV to gain additional COLUMN_NAME_label columns with a label for each foreign key derived from the linked table. In this example the city_id column is accompanied by a city_id_label column. stream all rows - by default CSV files only contain the first max_returned_rows records. This option will cause Datasette to loop through every matching record and return them as a single CSV file. You can try that out on https://latest.datasette.io/fixtures/facetable?_size=4 | [] | [{"href": "https://latest.datasette.io/fixtures/facetable.csv?_labels=on&_size=max", "label": "In this example"}, {"href": "https://latest.datasette.io/fixtures/facetable?_size=4", "label": "https://latest.datasette.io/fixtures/facetable?_size=4"}] |
custom_templates:id1 | custom_templates | id1 | Custom pages | You can add templated pages to your Datasette instance by creating HTML files in a pages directory within your templates directory. For example, to add a custom page that is served at http://localhost/about you would create a file in templates/pages/about.html , then start Datasette like this: datasette mydb.db --template-dir=templates/ You can nest directories within pages to create a nested structure. To create a http://localhost:8001/about/map page you would create templates/pages/about/map.html . | ["Custom pages and templates", "Publishing static assets"] | [] |
events:id1 | events | id1 | Events | Datasette includes a mechanism for tracking events that occur while the software is running. This is primarily intended to be used by plugins, which can both trigger events and listen for events. The core Datasette application triggers events when certain things happen. This page describes those events. Plugins can listen for events using the track_event(datasette, event) plugin hook, which will be called with instances of the following classes - or additional classes registered by other plugins . class datasette.events. LoginEvent actor : dict | None Event name: login A user (represented by event.actor ) has logged in. class datasette.events. LogoutEvent actor : dict | None Event name: logout A user (represented by event.actor ) has logged out. class datasette.events. CreateTokenEvent actor : dict | None expires_after : int | None restrict_all : list restrict_database : dict restrict_resource : dict Event name: create-token A user created an API token. Variables expires_after -- Number of seconds after which this token will expire. restrict_all -- Restricted permissions for this token. restrict_database -- Restricted database permissions for this token. … | [] | [] |
facets:id1 | facets | id1 | Facets | Datasette facets can be used to add a faceted browse interface to any database table. With facets, tables are displayed along with a summary showing the most common values in specified columns. These values can be selected to further filter the table. Here's an example : Facets can be specified in two ways: using query string parameters, or in metadata.json configuration for the table. | [] | [{"href": "https://congress-legislators.datasettes.com/legislators/legislator_terms?_facet=type&_facet=party&_facet=state&_facet_size=10", "label": "an example"}] |
full_text_search:id1 | full_text_search | id1 | Full-text search | SQLite includes a powerful mechanism for enabling full-text search against SQLite records. Datasette can detect if a table has had full-text search configured for it in the underlying database and display a search interface for filtering that table. Here's an example search : Datasette automatically detects which tables have been configured for full-text search. | [] | [{"href": "https://www.sqlite.org/fts3.html", "label": "a powerful mechanism for enabling full-text search"}, {"href": "https://register-of-members-interests.datasettes.com/regmem/items?_search=hamper&_sort_desc=date", "label": "an example search"}] |
installation:id1 | installation | id1 | Installation | If you just want to try Datasette out you don't need to install anything: see Try Datasette without installing anything using Glitch There are two main options for installing Datasette. You can install it directly on to your machine, or you can install it using Docker. If you want to start making contributions to the Datasette project by installing a copy that lets you directly modify the code, take a look at our guide to Setting up a development environment . Basic installation Datasette Desktop for Mac Using Homebrew Using pip Advanced installation options Using pipx Installing plugins using pipx Upgrading packages using pipx Using Docker Loading SpatiaLite Installing plugins A note about extensions | [] | [] |
internals:id1 | internals | id1 | .get_internal_database() | Returns a database object for reading and writing to the private internal database . | ["Internals for plugins", "Datasette class"] | [] |
introspection:id1 | introspection | id1 | Introspection | Datasette includes some pages and JSON API endpoints for introspecting the current instance. These can be used to understand some of the internals of Datasette and to see how a particular instance has been configured. Each of these pages can be viewed in your browser. Add .json to the URL to get back the contents as JSON. | [] | [] |
javascript_plugins:id1 | javascript_plugins | id1 | JavaScript plugins | Datasette can run custom JavaScript in several different ways: Datasette plugins written in Python can use the extra_js_urls() or extra_body_script() plugin hooks to inject JavaScript into a page Datasette instances with custom templates can include additional JavaScript in those templates The extra_js_urls key in datasette.yaml can be used to include extra JavaScript There are no limitations on what this JavaScript can do. It is executed directly by the browser, so it can manipulate the DOM, fetch additional data and do anything else that JavaScript is capable of. Custom JavaScript has security implications, especially for authenticated Datasette instances where the JavaScript might run in the context of the authenticated user. It's important to carefully review any JavaScript you run in your Datasette instance. | [] | [] |
json_api:id1 | json_api | id1 | JSON API | Datasette provides a JSON API for your SQLite databases. Anything you can do through the Datasette user interface can also be accessed as JSON via the API. To access the API for a page, either click on the .json link on that page or edit the URL and add a .json extension to it. | [] | [] |
metadata:id1 | metadata | id1 | Metadata | Data loves metadata. Any time you run Datasette you can optionally include a YAML or JSON file with metadata about your databases and tables. Datasette will then display that information in the web UI. Run Datasette like this: datasette database1.db database2.db --metadata metadata.yaml Your metadata.yaml file can look something like this: [[[cog from metadata_doc import metadata_example metadata_example(cog, { "title": "Custom title for your index page", "description": "Some description text can go here", "license": "ODbL", "license_url": "https://opendatacommons.org/licenses/odbl/", "source": "Original Data Source", "source_url": "http://example.com/" }) ]]] [[[end]]] Choosing YAML over JSON adds support for multi-line strings and comments. The above metadata will be displayed on the index page of your Datasette-powered site. The source and license information will also be included in the footer of every page served by Datasette. Any special HTML characters in description will be escaped. If you want to include HTML in your description, you can use a description_html property instead. | [] | [] |
plugin_hooks:id1 | plugin_hooks | id1 | Plugin hooks | Datasette plugins use plugin hooks to customize Datasette's behavior. These hooks are powered by the pluggy plugin system. Each plugin can implement one or more hooks using the @hookimpl decorator against a function named that matches one of the hooks documented on this page. When you implement a plugin hook you can accept any or all of the parameters that are documented as being passed to that hook. For example, you can implement the render_cell plugin hook like this even though the full documented hook signature is render_cell(row, value, column, table, database, datasette) : @hookimpl def render_cell(value, column): if column == "stars": return "*" * int(value) List of plugin hooks prepare_connection(conn, database, datasette) prepare_jinja2_environment(env, datasette) Page extras extra_template_vars(template, database, table, columns, view_name, request, datasette) extra_css_urls(template, database, table, columns, view_name, request, datasette) extra_js_urls(template, database, table, columns, view_name, request, datasette) extra_body_script(template, database, table, columns, view_name, request, datasette) publish_subcommand(publish) render_cell(row, value, column, table, database, datasette, request) register_output_re… | [] | [{"href": "https://pluggy.readthedocs.io/", "label": "pluggy"}] |
plugins:id1 | plugins | id1 | Plugins | Datasette's plugin system allows additional features to be implemented as Python code (or front-end JavaScript) which can be wrapped up in a separate Python package. The underlying mechanism uses pluggy . See the Datasette plugins directory for a list of existing plugins, or take a look at the datasette-plugin topic on GitHub. Things you can do with plugins include: Add visualizations to Datasette, for example datasette-cluster-map and datasette-vega . Make new custom SQL functions available for use within Datasette, for example datasette-haversine and datasette-jellyfish . Define custom output formats with custom extensions, for example datasette-atom and datasette-ics . Add template functions that can be called within your Jinja custom templates, for example datasette-render-markdown . Customize how database values are rendered in the Datasette interface, for example datasette-render-binary and datasette-pretty-json . Customize how Datasette's authentication and permissions systems work, for example datasette-auth-passwords and datasette-permissions-sql . | [] | [{"href": "https://pluggy.readthedocs.io/", "label": "pluggy"}, {"href": "https://datasette.io/plugins", "label": "Datasette plugins directory"}, {"href": "https://github.com/topics/datasette-plugin", "label": "datasette-plugin"}, {"href": "https://github.com/simonw/datasette-cluster-map", "label": "datasette-cluster-map"}, {"href": "https://github.com/simonw/datasette-vega", "label": "datasette-vega"}, {"href": "https://github.com/simonw/datasette-haversine", "label": "datasette-haversine"}, {"href": "https://github.com/simonw/datasette-jellyfish", "label": "datasette-jellyfish"}, {"href": "https://github.com/simonw/datasette-atom", "label": "datasette-atom"}, {"href": "https://github.com/simonw/datasette-ics", "label": "datasette-ics"}, {"href": "https://github.com/simonw/datasette-render-markdown#markdown-in-templates", "label": "datasette-render-markdown"}, {"href": "https://github.com/simonw/datasette-render-binary", "label": "datasette-render-binary"}, {"href": "https://github.com/simonw/datasette-pretty-json", "label": "datasette-pretty-json"}, {"href": "https://github.com/simonw/datasette-auth-passwords", "label": "datasette-auth-passwords"}, {"href": "https://github.com/simonw/datasette-permissions-sql", "label": "datasette-permissions-sql"}] |
settings:id1 | settings | id1 | Settings | [] | [] | |
spatialite:id1 | spatialite | id1 | SpatiaLite | The SpatiaLite module for SQLite adds features for handling geographic and spatial data. For an example of what you can do with it, see the tutorial Building a location to time zone API with SpatiaLite . To use it with Datasette, you need to install the mod_spatialite dynamic library. This can then be loaded into Datasette using the --load-extension command-line option. Datasette can look for SpatiaLite in common installation locations if you run it like this: datasette --load-extension=spatialite --setting default_allow_sql off If SpatiaLite is in another location, use the full path to the extension instead: datasette --setting default_allow_sql off \ --load-extension=/usr/local/lib/mod_spatialite.dylib | [] | [{"href": "https://www.gaia-gis.it/fossil/libspatialite/index", "label": "SpatiaLite module"}, {"href": "https://datasette.io/tutorials/spatialite", "label": "Building a location to time zone API with SpatiaLite"}] |
sql_queries:id1 | sql_queries | id1 | Canned queries | As an alternative to adding views to your database, you can define canned queries inside your datasette.yaml file. Here's an example: [[[cog from metadata_doc import config_example, config_example config_example(cog, { "databases": { "sf-trees": { "queries": { "just_species": { "sql": "select qSpecies from Street_Tree_List" } } } } }) ]]] [[[end]]] Then run Datasette like this: datasette sf-trees.db -m metadata.json Each canned query will be listed on the database index page, and will also get its own URL at: /database-name/canned-query-name For the above example, that URL would be: /sf-trees/just_species You can optionally include "title" and "description" keys to show a title and description on the canned query page. As with regular table metadata you can alternatively specify "description_html" to have your description rendered as HTML (rather than having HTML special characters escaped). | ["Running SQL queries"] | [] |
testing_plugins:id1 | testing_plugins | id1 | Testing plugins | We recommend using pytest to write automated tests for your plugins. If you use the template described in Starting an installable plugin using cookiecutter your plugin will start with a single test in your tests/ directory that looks like this: from datasette.app import Datasette import pytest @pytest.mark.asyncio async def test_plugin_is_installed(): datasette = Datasette(memory=True) response = await datasette.client.get("/-/plugins.json") assert response.status_code == 200 installed_plugins = {p["name"] for p in response.json()} assert ( "datasette-plugin-template-demo" in installed_plugins ) This test uses the datasette.client object to exercise a test instance of Datasette. datasette.client is a wrapper around the HTTPX Python library which can imitate HTTP requests using ASGI. This is the recommended way to write tests against a Datasette instance. This test also uses the pytest-asyncio package to add support for async def test functions running under pytest. You can install these packages like so: pip install pytest pytest-asyncio If you are building an installable package you can add them as test dependencies to your setup.py module like this: setup( name="datasette-my-plugin", # ... extras_require={"test": ["pytest", "pytest-asyncio"]}, tests_require=["datasette-my-plugin[test]"], ) You can then install the test dependencies like so: pip install -e '.[test]' Then run the tests using pytest like so: pytest | [] | [{"href": "https://docs.pytest.org/", "label": "pytest"}, {"href": "https://www.python-httpx.org/", "label": "HTTPX"}, {"href": "https://pypi.org/project/pytest-asyncio/", "label": "pytest-asyncio"}] |
writing_plugins:id1 | writing_plugins | id1 | Writing plugins | You can write one-off plugins that apply to just one Datasette instance, or you can write plugins which can be installed using pip and can be shipped to the Python Package Index ( PyPI ) for other people to install. Want to start by looking at an example? The Datasette plugins directory lists more than 90 open source plugins with code you can explore. The plugin hooks page includes links to example plugins for each of the documented hooks. | [] | [{"href": "https://pypi.org/", "label": "PyPI"}, {"href": "https://datasette.io/plugins", "label": "Datasette plugins directory"}] |
changelog:id10 | changelog | id10 | 0.63.1 (2022-11-10) | Fixed a bug where Datasette's table filter form would not redirect correctly when run behind a proxy using the base_url setting. ( #1883 ) SQL query is now shown wrapped in a <textarea> if a query exceeds a time limit. ( #1876 ) Fixed an intermittent "Too many open files" error while running the test suite. ( #1843 ) New db.close() internal method. | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/1883", "label": "#1883"}, {"href": "https://github.com/simonw/datasette/issues/1876", "label": "#1876"}, {"href": "https://github.com/simonw/datasette/issues/1843", "label": "#1843"}] |
changelog:id103 | changelog | id103 | 0.23.2 (2018-07-07) | Minor bugfix and documentation release. CSV export now respects --cors , fixes #326 Installation instructions , including docker image - closes #328 Fix for row pages for tables with / in, closes #325 | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/326", "label": "#326"}, {"href": "https://github.com/simonw/datasette/issues/328", "label": "#328"}, {"href": "https://github.com/simonw/datasette/issues/325", "label": "#325"}] |
changelog:id107 | changelog | id107 | 0.23.1 (2018-06-21) | Minor bugfix release. Correctly display empty strings in HTML table, closes #314 Allow "." in database filenames, closes #302 404s ending in slash redirect to remove that slash, closes #309 Fixed incorrect display of compound primary keys with foreign key references. Closes #319 Docs + example of canned SQL query using || concatenation. Closes #321 Correctly display facets with value of 0 - closes #318 Default 'expand labels' to checked in CSV advanced export | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/314", "label": "#314"}, {"href": "https://github.com/simonw/datasette/issues/302", "label": "#302"}, {"href": "https://github.com/simonw/datasette/issues/309", "label": "#309"}, {"href": "https://github.com/simonw/datasette/issues/319", "label": "#319"}, {"href": "https://github.com/simonw/datasette/issues/321", "label": "#321"}, {"href": "https://github.com/simonw/datasette/issues/318", "label": "#318"}] |
changelog:id11 | changelog | id11 | 0.63 (2022-10-27) | See Datasette 0.63: The annotated release notes for more background on the changes in this release. | ["Changelog"] | [{"href": "https://simonwillison.net/2022/Oct/27/datasette-0-63/", "label": "Datasette 0.63: The annotated release notes"}] |
changelog:id114 | changelog | id114 | 0.23 (2018-06-18) | This release features CSV export, improved options for foreign key expansions, new configuration settings and improved support for SpatiaLite. See datasette/compare/0.22.1...0.23 for a full list of commits added since the last release. | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/compare/0.22.1...0.23", "label": "datasette/compare/0.22.1...0.23"}] |
changelog:id115 | changelog | id115 | 0.22.1 (2018-05-23) | Bugfix release, plus we now use versioneer for our version numbers. Faceting no longer breaks pagination, fixes #282 Add __version_info__ derived from __version__ [Robert Gieseke] This might be tuple of more than two values (major and minor version) if commits have been made after a release. Add version number support with Versioneer. [Robert Gieseke] Versioneer Licence: Public Domain (CC0-1.0) Closes #273 Refactor inspect logic [Russ Garrett] | ["Changelog"] | [{"href": "https://github.com/warner/python-versioneer", "label": "versioneer"}, {"href": "https://github.com/simonw/datasette/issues/282", "label": "#282"}, {"href": "https://github.com/simonw/datasette/issues/273", "label": "#273"}] |
changelog:id118 | changelog | id118 | 0.22 (2018-05-20) | The big new feature in this release is Facets . Datasette can now apply faceted browse to any column in any table. It will also suggest possible facets. See the Datasette Facets announcement post for more details. In addition to the work on facets: Added docs for introspection endpoints New --config option, added --help-config , closes #274 Removed the --page_size= argument to datasette serve in favour of: datasette serve --config default_page_size:50 mydb.db Added new help section: datasette --help-config Config options: default_page_size Default page size for the table view (default=100) max_returned_rows Maximum rows that can be returned from a table or custom query (default=1000) sql_time_limit_ms Time limit for a SQL query in milliseconds (default=1000) default_facet_size Number of values to return for requested facets (default=30) facet_time_limit_ms Time limit for calculating a requested facet (default=200) facet_suggest_time_limit_ms Time limit for calculating a suggested facet (default=50) Only apply responsive table styles to .rows-and-column Otherwise they interfere with tables in the description, e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight/nba-elo%2Fnbaallelo Refactored views into new views/ modules, refs #256 Documentation for SQLite full-text search support, closes #25… | ["Changelog"] | [{"href": "https://simonwillison.net/2018/May/20/datasette-facets/", "label": "Datasette Facets"}, {"href": "https://docs.datasette.io/en/stable/introspection.html", "label": "docs for introspection endpoints"}, {"href": "https://github.com/simonw/datasette/issues/274", "label": "#274"}, {"href": "https://fivethirtyeight.datasettes.com/fivethirtyeight/nba-elo%2Fnbaallelo", "label": "https://fivethirtyeight.datasettes.com/fivethirtyeight/nba-elo%2Fnbaallelo"}, {"href": "https://github.com/simonw/datasette/issues/256", "label": "#256"}, {"href": "https://docs.datasette.io/en/stable/full_text_search.html", "label": "Documentation for SQLite full-text search"}, {"href": "https://github.com/simonw/datasette/issues/253", "label": "#253"}, {"href": "https://github.com/simonw/datasette/issues/252", "label": "#252"}] |
changelog:id12 | changelog | id12 | Documentation | New tutorial: Cleaning data with sqlite-utils and Datasette . Screenshots in the documentation are now maintained using shot-scraper , as described in Automating screenshots for the Datasette documentation using shot-scraper . ( #1844 ) More detailed command descriptions on the CLI reference page. ( #1787 ) New documentation on Running Datasette using OpenRC - thanks, Adam Simpson. ( #1825 ) | ["Changelog", "0.63 (2022-10-27)"] | [{"href": "https://datasette.io/tutorials/clean-data", "label": "Cleaning data with sqlite-utils and Datasette"}, {"href": "https://shot-scraper.datasette.io/", "label": "shot-scraper"}, {"href": "https://simonwillison.net/2022/Oct/14/automating-screenshots/", "label": "Automating screenshots for the Datasette documentation using shot-scraper"}, {"href": "https://github.com/simonw/datasette/issues/1844", "label": "#1844"}, {"href": "https://github.com/simonw/datasette/issues/1787", "label": "#1787"}, {"href": "https://github.com/simonw/datasette/pull/1825", "label": "#1825"}] |
changelog:id123 | changelog | id123 | 0.21 (2018-05-05) | New JSON _shape= options, the ability to set table _size= and a mechanism for searching within specific columns. Default tests to using a longer timelimit Every now and then a test will fail in Travis CI on Python 3.5 because it hit the default 20ms SQL time limit. Test fixtures now default to a 200ms time limit, and we only use the 20ms time limit for the specific test that tests query interruption. This should make our tests on Python 3.5 in Travis much more stable. Support _search_COLUMN=text searches, closes #237 Show version on /-/plugins page, closes #248 ?_size=max option, closes #249 Added /-/versions and /-/versions.json , closes #244 Sample output: { "python": { "version": "3.6.3", "full": "3.6.3 (default, Oct 4 2017, 06:09:38) \n[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]" }, "datasette": { "version": "0.20" }, "sqlite": { "version": "3.23.1", "extensions": { "json1": null, "spatialite": "4.3.0a" } } } Renamed ?_sql_time_limit_ms= to ?_timelimit , closes #242 New ?_shape=array option + tweaks to _shape , closes #245 Default is now ?_shape=arrays (renamed from lists ) New ?_shape=array returns an array of objects as the root object … | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/237", "label": "#237"}, {"href": "https://github.com/simonw/datasette/issues/248", "label": "#248"}, {"href": "https://github.com/simonw/datasette/issues/249", "label": "#249"}, {"href": "https://github.com/simonw/datasette/issues/244", "label": "#244"}, {"href": "https://github.com/simonw/datasette/issues/242", "label": "#242"}, {"href": "https://github.com/simonw/datasette/issues/245", "label": "#245"}, {"href": "https://github.com/simonw/datasette/issues/240", "label": "#240"}, {"href": "https://github.com/simonw/datasette/issues/229", "label": "#229"}, {"href": "https://github.com/simonw/datasette/issues/230", "label": "#230"}, {"href": "https://github.com/simonw/datasette/issues/239", "label": "#239"}, {"href": "https://github.com/simonw/datasette/issues/228", "label": "#228"}, {"href": "https://github.com/simonw/datasette/issues/234", "label": "#234"}] |
changelog:id13 | changelog | id13 | 0.62 (2022-08-14) | Datasette can now run entirely in your browser using WebAssembly. Try out Datasette Lite , take a look at the code or read more about it in Datasette Lite: a server-side Python web application running in a browser . Datasette now has a Discord community for questions and discussions about Datasette and its ecosystem of projects. | ["Changelog"] | [{"href": "https://lite.datasette.io/", "label": "Datasette Lite"}, {"href": "https://github.com/simonw/datasette-lite", "label": "at the code"}, {"href": "https://simonwillison.net/2022/May/4/datasette-lite/", "label": "Datasette Lite: a server-side Python web application running in a browser"}, {"href": "https://datasette.io/discord", "label": "Discord community"}] |
changelog:id136 | changelog | id136 | 0.20 (2018-04-20) | Mostly new work on the Plugins mechanism: plugins can now bundle static assets and custom templates, and datasette publish has a new --install=name-of-plugin option. Add col-X classes to HTML table on custom query page Fixed out-dated template in documentation Plugins can now bundle custom templates, #224 Added /-/metadata /-/plugins /-/inspect, #225 Documentation for --install option, refs #223 Datasette publish/package --install option, #223 Fix for plugins in Python 3.5, #222 New plugin hooks: extra_css_urls() and extra_js_urls(), #214 /-/static-plugins/PLUGIN_NAME/ now serves static/ from plugins <th> now gets class="col-X" - plus added col-X documentation Use to_css_class for table cell column classes This ensures that columns with spaces in the name will still generate usable CSS class names. Refs #209 Add column name classes to <td>s, make PK bold [Russ Garrett] Don't duplicate simple primary keys in the link column [Russ Garrett] When there's a simple (single-column) primary key, it looks weird to duplicate it in the link column. This change removes the second PK column and treats the link column as if it were the PK colu… | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/224", "label": "#224"}, {"href": "https://github.com/simonw/datasette/issues/225", "label": "#225"}, {"href": "https://github.com/simonw/datasette/issues/223", "label": "#223"}, {"href": "https://github.com/simonw/datasette/issues/223", "label": "#223"}, {"href": "https://github.com/simonw/datasette/issues/222", "label": "#222"}, {"href": "https://github.com/simonw/datasette/issues/214", "label": "#214"}, {"href": "https://github.com/simonw/datasette/issues/209", "label": "#209"}, {"href": "https://github.com/simonw/datasette/issues/209", "label": "#209"}] |
changelog:id14 | changelog | id14 | Features | Datasette is now compatible with Pyodide . This is the enabling technology behind Datasette Lite . ( #1733 ) Database file downloads now implement conditional GET using ETags. ( #1739 ) HTML for facet results and suggested results has been extracted out into new templates _facet_results.html and _suggested_facets.html . Thanks, M. Nasimul Haque. ( #1759 ) Datasette now runs some SQL queries in parallel. This has limited impact on performance, see this research issue for details. New --nolock option for ignoring file locks when opening read-only databases. ( #1744 ) Spaces in the database names in URLs are now encoded as + rather than ~20 . ( #1701 ) <Binary: 2427344 bytes> is now displayed as <Binary: 2,427,344 bytes> and is accompanied by tooltip showing "2.3MB". ( #1712 ) The base Docker image used by datasette publish cloudrun , datasette package and the official Datasette image has been upgraded to 3.10.6-slim-bullseye . ( #1768 ) Canned writable queries against immutable databases now show a warning message. ( #1728 ) datasette publish cloudrun has a new --timeout option which can be used to increase the time limit applied by the Google Cloud build environment. Thanks, Tim Sherratt. ( #1717 ) datasette publish cloudrun has new --min-instances and --max-instances options. ( #1779 ) | ["Changelog", "0.62 (2022-08-14)"] | [{"href": "https://pyodide.org/", "label": "Pyodide"}, {"href": "https://lite.datasette.io/", "label": "Datasette Lite"}, {"href": "https://github.com/simonw/datasette/issues/1733", "label": "#1733"}, {"href": "https://github.com/simonw/datasette/issues/1739", "label": "#1739"}, {"href": "https://github.com/simonw/datasette/pull/1759", "label": "#1759"}, {"href": "https://github.com/simonw/datasette/issues/1727", "label": "this research issue"}, {"href": "https://github.com/simonw/datasette/issues/1744", "label": "#1744"}, {"href": "https://github.com/simonw/datasette/issues/1701", "label": "#1701"}, {"href": "https://github.com/simonw/datasette/issues/1712", "label": "#1712"}, {"href": "https://hub.docker.com/datasetteproject/datasette", "label": "official Datasette image"}, {"href": "https://github.com/simonw/datasette/issues/1768", "label": "#1768"}, {"href": "https://github.com/simonw/datasette/issues/1728", "label": "#1728"}, {"href": "https://github.com/simonw/datasette/pull/1717", "label": "#1717"}, {"href": "https://github.com/simonw/datasette/issues/1779", "label": "#1779"}] |
changelog:id145 | changelog | id145 | 0.19 (2018-04-16) | This is the first preview of the new Datasette plugins mechanism. Only two plugin hooks are available so far - for custom SQL functions and custom template filters. There's plenty more to come - read the documentation and get involved in the tracking ticket if you have feedback on the direction so far. Fix for _sort_desc=sortable_with_nulls test, refs #216 Fixed #216 - paginate correctly when sorting by nullable column Initial documentation for plugins, closes #213 https://docs.datasette.io/en/stable/plugins.html New --plugins-dir=plugins/ option ( #212 ) New option causing Datasette to load and evaluate all of the Python files in the specified directory and register any plugins that are defined in those files. This new option is available for the following commands: datasette serve mydb.db --plugins-dir=plugins/ datasette publish now/heroku mydb.db --plugins-dir=plugins/ datasette package mydb.db --plugins-dir=plugins/ Start of the plugin system, based on pluggy ( #210 ) Uses https://pluggy.readthedocs.io/ originally created for the py.test project We're starting with two plugin hooks: prepare_connection(conn) This is called when a new SQLite connection is created. It can be used to register custom SQL functions. prepare_jinja2_environment(env) This is called with the Jinja2 environment. It can be used to register custom template tags and filters. An example plugin which… | ["Changelog"] | [{"href": "https://docs.datasette.io/en/stable/plugins.html", "label": "the documentation"}, {"href": "https://github.com/simonw/datasette/issues/14", "label": "the tracking ticket"}, {"href": "https://github.com/simonw/datasette/issues/216", "label": "#216"}, {"href": "https://github.com/simonw/datasette/issues/216", "label": "#216"}, {"href": "https://github.com/simonw/datasette/issues/213", "label": "#213"}, {"href": "https://docs.datasette.io/en/stable/plugins.html", "label": "https://docs.datasette.io/en/stable/plugins.html"}, {"href": "https://github.com/simonw/datasette/issues/212", "label": "#212"}, {"href": "https://github.com/simonw/datasette/issues/14", "label": "#210"}, {"href": "https://pluggy.readthedocs.io/", "label": "https://pluggy.readthedocs.io/"}, {"href": "https://github.com/simonw/datasette-plugin-demos", "label": "https://github.com/simonw/datasette-plugin-demos"}, {"href": "https://github.com/simonw/datasette/issues/14", "label": "#14"}] |
changelog:id15 | changelog | id15 | Plugin hooks | New plugin hook: handle_exception() , for custom handling of exceptions caught by Datasette. ( #1770 ) The render_cell() plugin hook is now also passed a row argument, representing the sqlite3.Row object that is being rendered. ( #1300 ) The configuration directory is now stored in datasette.config_dir , making it available to plugins. Thanks, Chris Amico. ( #1766 ) | ["Changelog", "0.62 (2022-08-14)"] | [{"href": "https://github.com/simonw/datasette/issues/1770", "label": "#1770"}, {"href": "https://github.com/simonw/datasette/issues/1300", "label": "#1300"}, {"href": "https://github.com/simonw/datasette/pull/1766", "label": "#1766"}] |
changelog:id152 | changelog | id152 | 0.18 (2018-04-14) | This release introduces support for units , contributed by Russ Garrett ( #203 ). You can now optionally specify the units for specific columns using metadata.json . Once specified, units will be displayed in the HTML view of your table. They also become available for use in filters - if a column is configured with a unit of distance, you can request all rows where that column is less than 50 meters or more than 20 feet for example. Link foreign keys which don't have labels. [Russ Garrett] This renders unlabeled FKs as simple links. Also includes bonus fixes for two minor issues: In foreign key link hrefs the primary key was escaped using HTML escaping rather than URL escaping. This broke some non-integer PKs. Print tracebacks to console when handling 500 errors. Fix SQLite error when loading rows with no incoming FKs. [Russ Garrett] This fixes an error caused by an invalid query when loading incoming FKs. The error was ignored due to async but it still got printed to the console. Allow custom units to be registered with Pint. [Russ Garrett] Support units in filters. [Russ Garrett] Tidy up units support. [Russ Garrett] Add units to exported JSON … | ["Changelog"] | [{"href": "https://docs.datasette.io/en/stable/metadata.html#specifying-units-for-a-column", "label": "support for units"}, {"href": "https://github.com/simonw/datasette/issues/203", "label": "#203"}, {"href": "https://pint.readthedocs.io/en/latest/", "label": "pint"}] |
changelog:id154 | changelog | id154 | 0.17 (2018-04-13) | Release 0.17 to fix issues with PyPI | ["Changelog"] | [] |
changelog:id155 | changelog | id155 | 0.16 (2018-04-13) | Better mechanism for handling errors; 404s for missing table/database New error mechanism closes #193 404s for missing tables/databases closes #184 long_description in markdown for the new PyPI Hide SpatiaLite system tables. [Russ Garrett] Allow explain select / explain query plan select #201 Datasette inspect now finds primary_keys #195 Ability to sort using form fields (for mobile portrait mode) #199 We now display sort options as a select box plus a descending checkbox, which means you can apply sort orders even in portrait mode on a mobile phone where the column headers are hidden. | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/193", "label": "#193"}, {"href": "https://github.com/simonw/datasette/issues/184", "label": "#184"}, {"href": "https://github.com/simonw/datasette/issues/201", "label": "#201"}, {"href": "https://github.com/simonw/datasette/issues/195", "label": "#195"}, {"href": "https://github.com/simonw/datasette/issues/199", "label": "#199"}] |
changelog:id16 | changelog | id16 | Documentation | Examples in the documentation now include a copy-to-clipboard button. ( #1748 ) Documentation now uses the Furo Sphinx theme. ( #1746 ) Code examples in the documentation are now all formatted using Black. ( #1718 ) Request.fake() method is now documented, see Request object . New documentation for plugin authors: Registering a plugin for the duration of a test . ( #903 ) | ["Changelog", "0.62 (2022-08-14)"] | [{"href": "https://github.com/simonw/datasette/issues/1748", "label": "#1748"}, {"href": "https://github.com/pradyunsg/furo", "label": "Furo"}, {"href": "https://github.com/simonw/datasette/issues/1746", "label": "#1746"}, {"href": "https://github.com/simonw/datasette/issues/1718", "label": "#1718"}, {"href": "https://github.com/simonw/datasette/issues/903", "label": "#903"}] |
changelog:id161 | changelog | id161 | 0.15 (2018-04-09) | The biggest new feature in this release is the ability to sort by column. On the table page the column headers can now be clicked to apply sort (or descending sort), or you can specify ?_sort=column or ?_sort_desc=column directly in the URL. table_rows => table_rows_count , filtered_table_rows => filtered_table_rows_count Renamed properties. Closes #194 New sortable_columns option in metadata.json to control sort options. You can now explicitly set which columns in a table can be used for sorting using the _sort and _sort_desc arguments using metadata.json : { "databases": { "database1": { "tables": { "example_table": { "sortable_columns": [ "height", "weight" ] } } } } } Refs #189 Column headers now link to sort/desc sort - refs #189 _sort and _sort_desc parameters for table views Allows for paginated sorted results based on a specified column. Refs #189 Total row count now correct even if _next applied Use .custom_sql() for _group_count implementation (refs #150 ) Make HTML title more readable in query template ( #180 ) [Ryan Pitts] New ?_shape=objects/object/lists param for JSON API ( #192 ) New _shape= parameter repl… | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/194", "label": "#194"}, {"href": "https://github.com/simonw/datasette/issues/189", "label": "#189"}, {"href": "https://github.com/simonw/datasette/issues/189", "label": "#189"}, {"href": "https://github.com/simonw/datasette/issues/189", "label": "#189"}, {"href": "https://github.com/simonw/datasette/issues/150", "label": "#150"}, {"href": "https://github.com/simonw/datasette/issues/180", "label": "#180"}, {"href": "https://github.com/simonw/datasette/issues/192", "label": "#192"}, {"href": "https://github.com/simonw/datasette/issues/122", "label": "#122"}, {"href": "https://github.com/simonw/datasette/issues/190", "label": "#190"}, {"href": "https://github.com/simonw/datasette/issues/190", "label": "#190"}, {"href": "https://github.com/simonw/datasette/issues/185", "label": "#185"}, {"href": "https://github.com/simonw/datasette/issues/178", "label": "#178"}] |
changelog:id17 | changelog | id17 | 0.61.1 (2022-03-23) | Fixed a bug where databases with a different route from their name (as used by the datasette-hashed-urls plugin ) returned errors when executing custom SQL queries. ( #1682 ) | ["Changelog"] | [{"href": "https://datasette.io/plugins/datasette-hashed-urls", "label": "datasette-hashed-urls plugin"}, {"href": "https://github.com/simonw/datasette/issues/1682", "label": "#1682"}] |
changelog:id174 | changelog | id174 | 0.14 (2017-12-09) | The theme of this release is customization: Datasette now allows every aspect of its presentation to be customized either using additional CSS or by providing entirely new templates. Datasette's metadata.json format has also been expanded, to allow per-database and per-table metadata. A new datasette skeleton command can be used to generate a skeleton JSON file ready to be filled in with per-database and per-table details. The metadata.json file can also be used to define canned queries , as a more powerful alternative to SQL views. extra_css_urls / extra_js_urls in metadata A mechanism in the metadata.json format for adding custom CSS and JS urls. Create a metadata.json file that looks like this: { "extra_css_urls": [ "https://simonwillison.net/static/css/all.bf8cd891642c.css" ], "extra_js_urls": [ "https://code.jquery.com/jquery-3.2.1.slim.min.js" ] } Then start datasette like this: datasette mydb.db --metadata=metadata.json The CSS and JavaScript files will be linked in the <head> of every page. You can also specify a SRI (subresource integrity hash) for these assets: { "extra_css_urls": [ { "url": "https://simonwillison.net/static/css/all.bf8cd891642c.css", "sri": "sha384-9qIZekWUyjCyDIf2YK1FRoKiPJq4PHt6tp/ulnuuyRBvazd0hG7pWbE99zvwSznI" } ], "extra_js_urls": [ { "url": "https://code.jquery.com/jquery-3.2.1.slim.min.js", "sri": "sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g=" } ] } Modern browsers will only execute the stylesheet or JavaScript if the SRI hash … | ["Changelog"] | [{"href": "https://docs.datasette.io/en/stable/custom_templates.html", "label": "to be customized"}, {"href": "https://docs.datasette.io/en/stable/metadata.html", "label": "metadata.json format"}, {"href": "https://docs.datasette.io/en/stable/sql_queries.html#canned-queries", "label": "canned queries"}, {"href": "https://www.srihash.org/", "label": "https://www.srihash.org/"}, {"href": "https://github.com/simonw/datasette/issues/153", "label": "#153"}, {"href": "https://github.com/simonw/datasette/issues/153", "label": "#153"}, {"href": "https://github.com/simonw/datasette/issues/160", "label": "#160"}, {"href": "https://github.com/simonw/datasette/issues/164", "label": "#164"}, {"href": "https://github.com/simonw/datasette/issues/165", "label": "#165"}, {"href": "https://github.com/simonw/datasette/issues/130", "label": "#130"}, {"href": "https://github.com/simonw/datasette/issues/168", "label": "#168"}, {"href": "https://github.com/channelcat/sanic/releases/tag/0.7.0", "label": "https://github.com/channelcat/sanic/releases/tag/0.7.0"}, {"href": "https://github.com/simonw/datasette/issues/171", "label": "#171"}] |
changelog:id18 | changelog | id18 | 0.61 (2022-03-23) | In preparation for Datasette 1.0, this release includes two potentially backwards-incompatible changes. Hashed URL mode has been moved to a separate plugin, and the way Datasette generates URLs to databases and tables with special characters in their name such as / and . has changed. Datasette also now requires Python 3.7 or higher. URLs within Datasette now use a different encoding scheme for tables or databases that include "special" characters outside of the range of a-zA-Z0-9_- . This scheme is explained here: Tilde encoding . ( #1657 ) Removed hashed URL mode from Datasette. The new datasette-hashed-urls plugin can be used to achieve the same result, see datasette-hashed-urls for details. ( #1661 ) Databases can now have a custom path within the Datasette instance that is independent of the database name, using the db.route property. ( #1668 ) Datasette is now covered by a Code of Conduct . ( #1654 ) Python 3.6 is no longer supported. ( #1577 ) Tests now run against Python 3.11-dev. ( #1621 ) New datasette.ensure_permissions(actor, permissions) internal method for checking multiple permissions at once. ( #1675 ) New datasette.check_visibility(actor, action, resource=None) internal method for checking if a user can see a resource that would otherwise be invisible to unauthenticated users. ( #1678 ) Table and row HTML pages now include a <link rel="alternate" type="application/json+datasette" href="..."> element and return a Link: URL; rel="alternate"; type="applicatio… | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/1657", "label": "#1657"}, {"href": "https://github.com/simonw/datasette/issues/1661", "label": "#1661"}, {"href": "https://github.com/simonw/datasette/issues/1668", "label": "#1668"}, {"href": "https://github.com/simonw/datasette/blob/main/CODE_OF_CONDUCT.md", "label": "Code of Conduct"}, {"href": "https://github.com/simonw/datasette/issues/1654", "label": "#1654"}, {"href": "https://github.com/simonw/datasette/issues/1577", "label": "#1577"}, {"href": "https://github.com/simonw/datasette/issues/1621", "label": "#1621"}, {"href": "https://github.com/simonw/datasette/issues/1675", "label": "#1675"}, {"href": "https://github.com/simonw/datasette/issues/1678", "label": "#1678"}, {"href": "https://github.com/simonw/datasette/issues/1533", "label": "#1533"}, {"href": "https://github.com/simonw/datasette/issues/1612", "label": "#1612"}, {"href": "https://github.com/simonw/datasette/issues/1603", "label": "#1603"}, {"href": "https://github.com/simonw/datasette/issues/1587", "label": "#1587"}, {"href": "https://github.com/simonw/datasette/issues/1601", "label": "#1601"}, {"href": "https://github.com/simonw/datasette/issues/1576", "label": "#1576"}, {"href": "https://github.com/simonw/datasette/issues/957", "label": "#957"}, {"href": "https://github.com/simonw/datasette/issues/1607", "label": "#1607"}, {"href": "https://datasette.io/tutorials", "label": "Datasette Tutorials"}, {"href": "https://github.com/simonw/datasette/pull/1649", "label": "#1649"}, {"href": "https://github.com/simonw/datasette/issues/1545", "label": "#1545"}, {"href": "https://github.com/simonw/datasette/issues/1228", "label": "#1228"}] |
changelog:id183 | changelog | id183 | 0.13 (2017-11-24) | Search now applies to current filters. Combined search into the same form as filters. Closes #133 Much tidier design for table view header. Closes #147 Added ?column__not=blah filter. Closes #148 Row page now resolves foreign keys. Closes #132 Further tweaks to select/input filter styling. Refs #86 - thanks for the help, @natbat! Show linked foreign key in table cells. Added UI for editing table filters. Refs #86 Hide FTS-created tables on index pages. Closes #129 Add publish to heroku support [Jacob Kaplan-Moss] datasette publish heroku mydb.db Pull request #104 Initial implementation of ?_group_count=column . URL shortcut for counting rows grouped by one or more columns. ?_group_count=column1&_group_count=column2 works as well. SQL generated looks like this: select "qSpecies", count(*) as "count" from Street_Tree_List group by "qSpecies" order by "count" desc limit 100 Or for two columns like this: select "qSpecies", "qSiteInfo", count(*) as "count" from Street_Tree_List group by "qSpecies", "qSiteInfo" order by "count" desc limit 100 Refs #44 Added --build=mas… | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/133", "label": "#133"}, {"href": "https://github.com/simonw/datasette/issues/147", "label": "#147"}, {"href": "https://github.com/simonw/datasette/issues/148", "label": "#148"}, {"href": "https://github.com/simonw/datasette/issues/132", "label": "#132"}, {"href": "https://github.com/simonw/datasette/issues/86", "label": "#86"}, {"href": "https://github.com/simonw/datasette/issues/86", "label": "#86"}, {"href": "https://github.com/simonw/datasette/issues/129", "label": "#129"}, {"href": "https://github.com/simonw/datasette/issues/104", "label": "#104"}, {"href": "https://github.com/simonw/datasette/issues/44", "label": "#44"}, {"href": "https://github.com/simonw/datasette/issues/131", "label": "#131"}, {"href": "https://github.com/simonw/datasette/issues/117", "label": "#117"}, {"href": "https://github.com/simonw/datasette/issues/115", "label": "#115"}, {"href": "https://github.com/simonw/datasette/issues/115", "label": "#115"}, {"href": "https://github.com/simonw/datasette/issues/107", "label": "#107"}, {"href": "https://github.com/simonw/datasette/issues/114", "label": "#114"}, {"href": "https://github.com/simonw/datasette/issues/110", "label": "#110"}] |
changelog:id19 | changelog | id19 | 0.60.2 (2022-02-07) | Fixed a bug where Datasette would open the same file twice with two different database names if you ran datasette file.db file.db . ( #1632 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/1632", "label": "#1632"}] |
changelog:id198 | changelog | id198 | 0.12 (2017-11-16) | Added __version__ , now displayed as tooltip in page footer ( #108 ). Added initial docs, including a changelog ( #99 ). Turned on auto-escaping in Jinja. Added a UI for editing named parameters ( #96 ). You can now construct a custom SQL statement using SQLite named parameters (e.g. :name ) and datasette will display form fields for editing those parameters. Here’s an example which lets you see the most popular names for dogs of different species registered through various dog registration schemes in Australia. Pin to specific Jinja version. ( #100 ). Default to 127.0.0.1 not 0.0.0.0. ( #98 ). Added extra metadata options to publish and package commands. ( #92 ). You can now run these commands like so: datasette now publish mydb.db \ --title="My Title" \ --source="Source" \ --source_url="http://www.example.com/" \ --license="CC0" \ --license_url="https://creativecommons.org/publicdomain/zero/1.0/" This will write those values into the metadata.json that is packaged with the app. If you also pass --metadata=metadata.json that file will be updated with the extra values before being written into the Docker image. Added production-ready Dockerfile ( #94 ) [Andrew Cutler] New ?_sql_time_limit_ms=10 argument to database and table page ( #95 ) … | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/108", "label": "#108"}, {"href": "https://github.com/simonw/datasette/issues/99", "label": "#99"}, {"href": "https://github.com/simonw/datasette/issues/96", "label": "#96"}, {"href": "https://australian-dogs.now.sh/australian-dogs-3ba9628?sql=select+name%2C+count%28*%29+as+n+from+%28%0D%0A%0D%0Aselect+upper%28%22Animal+name%22%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2013%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2014%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2015%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22AnimalName%22%29+as+name+from+%5BCity-of-Port-Adelaide-Enfield-Dog_Registrations_2016%5D+where+AnimalBreed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5BMitcham-dog-registrations-2015%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22DOG_NAME%22%29+as+name+from+%5Bburnside-dog-registrations-2015%5D+where+DOG_BREED+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28%22Animal_Name%22%29+as+name+from+%5Bcity-of-playford-2015-dog-registration%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5Bcity-of-prospect-dog-registration-details-2016%5D+where%22Breed+Description%22+like+%3Abreed%0D%0A%0D%0A%29+group+by+name+order+by+n+desc%3B&breed=pug", "label": "Here’s an example"}, {"href": "https://github.com/simonw/datasette/issues/100", "label": "#100"}, {"href": "https://github.com/simonw/datasette/issues/98", "label": "#98"}, {"href": "https://github.com/simonw/datasette/issues/92", "label": "#92"}, {"href": "https://github.com/simonw/datasette/issues/94", "label": "#94"}, {"href": "https://github.com/simonw/datasette/i… |
changelog:id2 | changelog | id2 | 0.64.6 (2023-12-22) | Fixed a bug where CSV export with expanded labels could fail if a foreign key reference did not correctly resolve. ( #2214 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/2214", "label": "#2214"}] |
facets:id2 | facets | id2 | Facet by JSON array | If your SQLite installation provides the json1 extension (you can check using /-/versions ) Datasette will automatically detect columns that contain JSON arrays of values and offer a faceting interface against those columns. This is useful for modelling things like tags without needing to break them out into a new table. Example here: latest.datasette.io/fixtures/facetable?_facet_array=tags | ["Facets"] | [{"href": "https://latest.datasette.io/fixtures/facetable?_facet_array=tags", "label": "latest.datasette.io/fixtures/facetable?_facet_array=tags"}] |
javascript_plugins:id2 | javascript_plugins | id2 | JavaScript plugin objects | JavaScript plugins are blocks of code that can be registered with Datasette using the registerPlugin() method on the datasetteManager object. The implementation object passed to this method should include a version key defining the plugin version, and one or more of the following named functions providing the implementation of the plugin: | ["JavaScript plugins"] | [] |
json_api:id2 | json_api | id2 | Table arguments | The Datasette table view takes a number of special query string arguments. | ["JSON API"] | [] |
metadata:id2 | metadata | id2 | Metadata reference | A full reference of every supported option in a metadata.json or metadata.yaml file. | ["Metadata"] | [] |
settings:id2 | settings | id2 | Settings | The following options can be set using --setting name value , or by storing them in the settings.json file for use with Configuration directory mode . | ["Settings"] | [] |
sql_queries:id2 | sql_queries | id2 | Pagination | Datasette's default table pagination is designed to be extremely efficient. SQL OFFSET/LIMIT pagination can have a significant performance penalty once you get into multiple thousands of rows, as each page still requires the database to scan through every preceding row to find the correct offset. When paginating through tables, Datasette instead orders the rows in the table by their primary key and performs a WHERE clause against the last seen primary key for the previous page. For example: select rowid, * from Tree_List where rowid > 200 order by rowid limit 101 This represents page three for this particular table, with a page size of 100. Note that we request 101 items in the limit clause rather than 100. This allows us to detect if we are on the last page of the results: if the query returns less than 101 rows we know we have reached the end of the pagination set. Datasette will only return the first 100 rows - the 101st is used purely to detect if there should be another page. Since the where clause acts against the index on the primary key, the query is extremely fast even for records that are a long way into the overall pagination set. | ["Running SQL queries"] | [] |
changelog:id20 | changelog | id20 | 0.60.1 (2022-01-20) | Fixed a bug where installation on Python 3.6 stopped working due to a change to an underlying dependency. This release can now be installed on Python 3.6, but is the last release of Datasette that will support anything less than Python 3.7. ( #1609 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/1609", "label": "#1609"}] |
changelog:id208 | changelog | id208 | 0.11 (2017-11-14) | Added datasette publish now --force option. This calls now with --force - useful as it means you get a fresh copy of datasette even if Now has already cached that docker layer. Enable --cors by default when running in a container. | ["Changelog"] | [] |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [sections] ( [id] TEXT PRIMARY KEY, [page] TEXT, [ref] TEXT, [title] TEXT, [content] TEXT, [breadcrumbs] TEXT, [references] TEXT );