id,page,ref,title,content,breadcrumbs,references writing_plugins:id1,writing_plugins,id1,Writing plugins,"You can write one-off plugins that apply to just one Datasette instance, or you can write plugins which can be installed using pip and can be shipped to the Python Package Index ( PyPI ) for other people to install. Want to start by looking at an example? The Datasette plugins directory lists more than 90 open source plugins with code you can explore. The plugin hooks page includes links to example plugins for each of the documented hooks.",[],"[{""href"": ""https://pypi.org/"", ""label"": ""PyPI""}, {""href"": ""https://datasette.io/plugins"", ""label"": ""Datasette plugins directory""}]" upgrade_guide:id1,upgrade_guide,id1,Upgrade guide,,[],[] ecosystem:ecosystem,ecosystem,ecosystem,The Datasette Ecosystem,"Datasette sits at the center of a growing ecosystem of open source tools aimed at making it as easy as possible to gather, analyze and publish interesting data. These tools are divided into two main groups: tools for building SQLite databases (for use with Datasette) and plugins that extend Datasette's functionality. The Datasette project website includes a directory of plugins and a directory of tools: Plugins directory on datasette.io Tools directory on datasette.io",[],"[{""href"": ""https://datasette.io/"", ""label"": ""Datasette project website""}, {""href"": ""https://datasette.io/plugins"", ""label"": ""Plugins directory on datasette.io""}, {""href"": ""https://datasette.io/tools"", ""label"": ""Tools directory on datasette.io""}]" testing_plugins:id1,testing_plugins,id1,Testing plugins,"We recommend using pytest to write automated tests for your plugins. If you use the template described in Starting an installable plugin using cookiecutter your plugin will start with a single test in your tests/ directory that looks like this: from datasette.app import Datasette import pytest @pytest.mark.asyncio async def test_plugin_is_installed(): datasette = Datasette(memory=True) response = await datasette.client.get(""/-/plugins.json"") assert response.status_code == 200 installed_plugins = {p[""name""] for p in response.json()} assert ( ""datasette-plugin-template-demo"" in installed_plugins ) This test uses the datasette.client object to exercise a test instance of Datasette. datasette.client is a wrapper around the HTTPX Python library which can imitate HTTP requests using ASGI. This is the recommended way to write tests against a Datasette instance. This test also uses the pytest-asyncio package to add support for async def test functions running under pytest. You can install these packages like so: pip install pytest pytest-asyncio If you are building an installable package you can add them as test dependencies to your setup.py module like this: setup( name=""datasette-my-plugin"", # ... extras_require={""test"": [""pytest"", ""pytest-asyncio""]}, tests_require=[""datasette-my-plugin[test]""], ) You can then install the test dependencies like so: pip install -e '.[test]' Then run the tests using pytest like so: pytest",[],"[{""href"": ""https://docs.pytest.org/"", ""label"": ""pytest""}, {""href"": ""https://www.python-httpx.org/"", ""label"": ""HTTPX""}, {""href"": ""https://pypi.org/project/pytest-asyncio/"", ""label"": ""pytest-asyncio""}]" spatialite:id1,spatialite,id1,SpatiaLite,"The SpatiaLite module for SQLite adds features for handling geographic and spatial data. For an example of what you can do with it, see the tutorial Building a location to time zone API with SpatiaLite . To use it with Datasette, you need to install the mod_spatialite dynamic library. This can then be loaded into Datasette using the --load-extension command-line option. Datasette can look for SpatiaLite in common installation locations if you run it like this: datasette --load-extension=spatialite --setting default_allow_sql off If SpatiaLite is in another location, use the full path to the extension instead: datasette --setting default_allow_sql off \ --load-extension=/usr/local/lib/mod_spatialite.dylib",[],"[{""href"": ""https://www.gaia-gis.it/fossil/libspatialite/index"", ""label"": ""SpatiaLite module""}, {""href"": ""https://datasette.io/tutorials/spatialite"", ""label"": ""Building a location to time zone API with SpatiaLite""}]" settings:id1,settings,id1,Settings,,[],[] sql_queries:sql,sql_queries,sql,Running SQL queries,"Datasette treats SQLite database files as read-only and immutable. This means it is not possible to execute INSERT or UPDATE statements using Datasette, which allows us to expose SELECT statements to the outside world without needing to worry about SQL injection attacks. The easiest way to execute custom SQL against Datasette is through the web UI. The database index page includes a SQL editor that lets you run any SELECT query you like. You can also construct queries using the filter interface on the tables page, then click ""View and edit SQL"" to open that query in the custom SQL editor. Note that this interface is only available if the execute-sql permission is allowed. See Controlling the ability to execute arbitrary SQL . Any Datasette SQL query is reflected in the URL of the page, allowing you to bookmark them, share them with others and navigate through previous queries using your browser back button. You can also retrieve the results of any query as JSON by adding .json to the base URL.",[],[] publish:publishing,publish,publishing,Publishing data,Datasette includes tools for publishing and deploying your data to the internet. The datasette publish command will deploy a new Datasette instance containing your databases directly to a Heroku or Google Cloud hosting account. You can also use datasette package to create a Docker image that bundles your databases together with the datasette application that is used to serve them.,[],[] plugins:id1,plugins,id1,Plugins,"Datasette's plugin system allows additional features to be implemented as Python code (or front-end JavaScript) which can be wrapped up in a separate Python package. The underlying mechanism uses pluggy . See the Datasette plugins directory for a list of existing plugins, or take a look at the datasette-plugin topic on GitHub. Things you can do with plugins include: Add visualizations to Datasette, for example datasette-cluster-map and datasette-vega . Make new custom SQL functions available for use within Datasette, for example datasette-haversine and datasette-jellyfish . Define custom output formats with custom extensions, for example datasette-atom and datasette-ics . Add template functions that can be called within your Jinja custom templates, for example datasette-render-markdown . Customize how database values are rendered in the Datasette interface, for example datasette-render-binary and datasette-pretty-json . Customize how Datasette's authentication and permissions systems work, for example datasette-auth-passwords and datasette-permissions-sql .",[],"[{""href"": ""https://pluggy.readthedocs.io/"", ""label"": ""pluggy""}, {""href"": ""https://datasette.io/plugins"", ""label"": ""Datasette plugins directory""}, {""href"": ""https://github.com/topics/datasette-plugin"", ""label"": ""datasette-plugin""}, {""href"": ""https://github.com/simonw/datasette-cluster-map"", ""label"": ""datasette-cluster-map""}, {""href"": ""https://github.com/simonw/datasette-vega"", ""label"": ""datasette-vega""}, {""href"": ""https://github.com/simonw/datasette-haversine"", ""label"": ""datasette-haversine""}, {""href"": ""https://github.com/simonw/datasette-jellyfish"", ""label"": ""datasette-jellyfish""}, {""href"": ""https://github.com/simonw/datasette-atom"", ""label"": ""datasette-atom""}, {""href"": ""https://github.com/simonw/datasette-ics"", ""label"": ""datasette-ics""}, {""href"": ""https://github.com/simonw/datasette-render-markdown#markdown-in-templates"", ""label"": ""datasette-render-markdown""}, {""href"": ""https://github.com/simonw/datasette-render-binary"", ""label"": ""datasette-render-binary""}, {""href"": ""https://github.com/simonw/datasette-pretty-json"", ""label"": ""datasette-pretty-json""}, {""href"": ""https://github.com/simonw/datasette-auth-passwords"", ""label"": ""datasette-auth-passwords""}, {""href"": ""https://github.com/simonw/datasette-permissions-sql"", ""label"": ""datasette-permissions-sql""}]" plugin_hooks:id1,plugin_hooks,id1,Plugin hooks,"Datasette plugins use plugin hooks to customize Datasette's behavior. These hooks are powered by the pluggy plugin system. Each plugin can implement one or more hooks using the @hookimpl decorator against a function named that matches one of the hooks documented on this page. When you implement a plugin hook you can accept any or all of the parameters that are documented as being passed to that hook. For example, you can implement the render_cell plugin hook like this even though the full documented hook signature is render_cell(row, value, column, table, database, datasette) : @hookimpl def render_cell(value, column): if column == ""stars"": return ""*"" * int(value) List of plugin hooks prepare_connection(conn, database, datasette) prepare_jinja2_environment(env, datasette) Page extras extra_template_vars(template, database, table, columns, view_name, request, datasette) extra_css_urls(template, database, table, columns, view_name, request, datasette) extra_js_urls(template, database, table, columns, view_name, request, datasette) extra_body_script(template, database, table, columns, view_name, request, datasette) publish_subcommand(publish) render_cell(row, value, column, table, database, datasette, request) register_output_renderer(datasette) register_routes(datasette) register_commands(cli) register_facet_classes() register_permissions(datasette) asgi_wrapper(datasette) startup(datasette) canned_queries(datasette, database, actor) actor_from_request(datasette, request) actors_from_ids(datasette, actor_ids) jinja2_environment_from_request(datasette, request, env) filters_from_request(request, database, table, datasette) permission_allowed(datasette, actor, action, resource) register_magic_parameters(datasette) forbidden(datasette, request, message) handle_exception(datasette, request, exception) skip_csrf(datasette, scope) menu_links(datasette, actor, request) Action hooks table_actions(datasette, actor, database, table, request) view_actions(datasette, actor, database, view, request) query_actions(datasette, actor, database, query_name, request, sql, params) row_actions(datasette, actor, request, database, table, row) database_actions(datasette, actor, database, request) homepage_actions(datasette, actor, request) Template slots top_homepage(datasette, request) top_database(datasette, request, database) top_table(datasette, request, database, table) top_row(datasette, request, database, table, row) top_query(datasette, request, database, sql) top_canned_query(datasette, request, database, query_name) Event tracking track_event(datasette, event) register_events(datasette)",[],"[{""href"": ""https://pluggy.readthedocs.io/"", ""label"": ""pluggy""}]" performance:performance,performance,performance,Performance and caching,"Datasette runs on top of SQLite, and SQLite has excellent performance. For small databases almost any query should return in just a few milliseconds, and larger databases (100s of MBs or even GBs of data) should perform extremely well provided your queries make sensible use of database indexes. That said, there are a number of tricks you can use to improve Datasette's performance.",[],[] pages:pages,pages,pages,Pages and API endpoints,"The Datasette web application offers a number of different pages that can be accessed to explore the data in question, each of which is accompanied by an equivalent JSON API.",[],[] metadata:id1,metadata,id1,Metadata,"Data loves metadata. Any time you run Datasette you can optionally include a YAML or JSON file with metadata about your databases and tables. Datasette will then display that information in the web UI. Run Datasette like this: datasette database1.db database2.db --metadata metadata.yaml Your metadata.yaml file can look something like this: [[[cog from metadata_doc import metadata_example metadata_example(cog, { ""title"": ""Custom title for your index page"", ""description"": ""Some description text can go here"", ""license"": ""ODbL"", ""license_url"": ""https://opendatacommons.org/licenses/odbl/"", ""source"": ""Original Data Source"", ""source_url"": ""http://example.com/"" }) ]]] [[[end]]] Choosing YAML over JSON adds support for multi-line strings and comments. The above metadata will be displayed on the index page of your Datasette-powered site. The source and license information will also be included in the footer of every page served by Datasette. Any special HTML characters in description will be escaped. If you want to include HTML in your description, you can use a description_html property instead.",[],[] javascript_plugins:id1,javascript_plugins,id1,JavaScript plugins,"Datasette can run custom JavaScript in several different ways: Datasette plugins written in Python can use the extra_js_urls() or extra_body_script() plugin hooks to inject JavaScript into a page Datasette instances with custom templates can include additional JavaScript in those templates The extra_js_urls key in datasette.yaml can be used to include extra JavaScript There are no limitations on what this JavaScript can do. It is executed directly by the browser, so it can manipulate the DOM, fetch additional data and do anything else that JavaScript is capable of. Custom JavaScript has security implications, especially for authenticated Datasette instances where the JavaScript might run in the context of the authenticated user. It's important to carefully review any JavaScript you run in your Datasette instance.",[],[] json_api:id1,json_api,id1,JSON API,"Datasette provides a JSON API for your SQLite databases. Anything you can do through the Datasette user interface can also be accessed as JSON via the API. To access the API for a page, either click on the .json link on that page or edit the URL and add a .json extension to it.",[],[] introspection:id1,introspection,id1,Introspection,"Datasette includes some pages and JSON API endpoints for introspecting the current instance. These can be used to understand some of the internals of Datasette and to see how a particular instance has been configured. Each of these pages can be viewed in your browser. Add .json to the URL to get back the contents as JSON.",[],[] internals:internals,internals,internals,Internals for plugins,Many Plugin hooks are passed objects that provide access to internal Datasette functionality. The interface to these objects should not be considered stable with the exception of methods that are documented here.,[],[] installation:id1,installation,id1,Installation,"If you just want to try Datasette out you don't need to install anything: see Try Datasette without installing anything using Glitch There are two main options for installing Datasette. You can install it directly on to your machine, or you can install it using Docker. If you want to start making contributions to the Datasette project by installing a copy that lets you directly modify the code, take a look at our guide to Setting up a development environment . Basic installation Datasette Desktop for Mac Using Homebrew Using pip Advanced installation options Using pipx Installing plugins using pipx Upgrading packages using pipx Using Docker Loading SpatiaLite Installing plugins A note about extensions",[],[] getting_started:getting-started,getting_started,getting-started,Getting started,,[],[] full_text_search:id1,full_text_search,id1,Full-text search,"SQLite includes a powerful mechanism for enabling full-text search against SQLite records. Datasette can detect if a table has had full-text search configured for it in the underlying database and display a search interface for filtering that table. Here's an example search : Datasette automatically detects which tables have been configured for full-text search.",[],"[{""href"": ""https://www.sqlite.org/fts3.html"", ""label"": ""a powerful mechanism for enabling full-text search""}, {""href"": ""https://register-of-members-interests.datasettes.com/regmem/items?_search=hamper&_sort_desc=date"", ""label"": ""an example search""}]" facets:id1,facets,id1,Facets,"Datasette facets can be used to add a faceted browse interface to any database table. With facets, tables are displayed along with a summary showing the most common values in specified columns. These values can be selected to further filter the table. Here's an example : Facets can be specified in two ways: using query string parameters, or in metadata.json configuration for the table.",[],"[{""href"": ""https://congress-legislators.datasettes.com/legislators/legislator_terms?_facet=type&_facet=party&_facet=state&_facet_size=10"", ""label"": ""an example""}]" events:id1,events,id1,Events,"Datasette includes a mechanism for tracking events that occur while the software is running. This is primarily intended to be used by plugins, which can both trigger events and listen for events. The core Datasette application triggers events when certain things happen. This page describes those events. Plugins can listen for events using the track_event(datasette, event) plugin hook, which will be called with instances of the following classes - or additional classes registered by other plugins . class datasette.events. LoginEvent actor : dict | None Event name: login A user (represented by event.actor ) has logged in. class datasette.events. LogoutEvent actor : dict | None Event name: logout A user (represented by event.actor ) has logged out. class datasette.events. CreateTokenEvent actor : dict | None expires_after : int | None restrict_all : list restrict_database : dict restrict_resource : dict Event name: create-token A user created an API token. Variables expires_after -- Number of seconds after which this token will expire. restrict_all -- Restricted permissions for this token. restrict_database -- Restricted database permissions for this token. restrict_resource -- Restricted resource permissions for this token. class datasette.events. CreateTableEvent actor : dict | None database : str table : str schema : str Event name: create-table A new table has been created in the database. Variables database -- The name of the database where the table was created. table -- The name of the table that was created schema -- The SQL schema definition for the new table. class datasette.events. DropTableEvent actor : dict | None database : str table : str Event name: drop-table A table has been dropped from the database. Variables database -- The name of the database where the table was dropped. table -- The name of the table that was dropped class datasette.events. AlterTableEvent actor : dict | None database : str table : str before_schema : str after_schema : str Event name: alter-table A table has been altered. Variables database -- The name of the database where the table was altered table -- The name of the table that was altered before_schema -- The table's SQL schema before the alteration after_schema -- The table's SQL schema after the alteration class datasette.events. InsertRowsEvent actor : dict | None database : str table : str num_rows : int ignore : bool replace : bool Event name: insert-rows Rows were inserted into a table. Variables database -- The name of the database where the rows were inserted. table -- The name of the table where the rows were inserted. num_rows -- The number of rows that were requested to be inserted. ignore -- Was ignore set? replace -- Was replace set? class datasette.events. UpsertRowsEvent actor : dict | None database : str table : str num_rows : int Event name: upsert-rows Rows were upserted into a table. Variables database -- The name of the database where the rows were inserted. table -- The name of the table where the rows were inserted. num_rows -- The number of rows that were requested to be inserted. class datasette.events. UpdateRowEvent actor : dict | None database : str table : str pks : list Event name: update-row A row was updated in a table. Variables database -- The name of the database where the row was updated. table -- The name of the table where the row was updated. pks -- The primary key values of the updated row. class datasette.events. DeleteRowEvent actor : dict | None database : str table : str pks : list Event name: delete-row A row was deleted from a table. Variables database -- The name of the database where the row was deleted. table -- The name of the table where the row was deleted. pks -- The primary key values of the deleted row.",[],[] deploying:deploying,deploying,deploying,Deploying Datasette,"The quickest way to deploy a Datasette instance on the internet is to use the datasette publish command, described in Publishing data . This can be used to quickly deploy Datasette to a number of hosting providers including Heroku, Google Cloud Run and Vercel. You can deploy Datasette to other hosting providers using the instructions on this page.",[],[] index:datasette,index,datasette,Datasette,"An open source multi-tool for exploring and publishing data Datasette is a tool for exploring and publishing data. It helps people take data of any shape or size and publish that as an interactive, explorable website and accompanying API. Datasette is aimed at data journalists, museum curators, archivists, local governments and anyone else who has data that they wish to share with the world. It is part of a wider ecosystem of tools and plugins dedicated to making working with structured data as productive as possible. Explore a demo , watch a presentation about the project or Try Datasette without installing anything using Glitch . Interested in learning Datasette? Start with the official tutorials . Support questions, feedback? Join the Datasette Discord .",[],"[{""href"": ""https://pypi.org/project/datasette/"", ""label"": null}, {""href"": ""https://docs.datasette.io/en/stable/changelog.html"", ""label"": null}, {""href"": ""https://pypi.org/project/datasette/"", ""label"": null}, {""href"": ""https://github.com/simonw/datasette/actions?query=workflow%3ATest"", ""label"": null}, {""href"": ""https://github.com/simonw/datasette/blob/main/LICENSE"", ""label"": null}, {""href"": ""https://hub.docker.com/r/datasetteproject/datasette"", ""label"": null}, {""href"": ""https://datasette.io/discord"", ""label"": null}, {""href"": ""https://pypi.org/project/datasette/"", ""label"": null}, {""href"": ""https://docs.datasette.io/en/stable/changelog.html"", ""label"": null}, {""href"": ""https://pypi.org/project/datasette/"", ""label"": null}, {""href"": ""https://github.com/simonw/datasette/actions?query=workflow%3ATest"", ""label"": null}, {""href"": ""https://github.com/simonw/datasette/blob/main/LICENSE"", ""label"": null}, {""href"": ""https://hub.docker.com/r/datasetteproject/datasette"", ""label"": null}, {""href"": ""https://datasette.io/discord"", ""label"": null}, {""href"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight"", ""label"": ""Explore a demo""}, {""href"": ""https://static.simonwillison.net/static/2018/pybay-datasette/"", ""label"": ""a presentation about the project""}, {""href"": ""https://datasette.io/tutorials"", ""label"": ""the official tutorials""}, {""href"": ""https://datasette.io/discord"", ""label"": ""Datasette Discord""}]" custom_templates:customization,custom_templates,customization,Custom pages and templates,Datasette provides a number of ways of customizing the way data is displayed.,[],[] contributing:id1,contributing,id1,Contributing,"Datasette is an open source project. We welcome contributions! This document describes how to contribute to Datasette core. You can also contribute to the wider Datasette ecosystem by creating new Plugins .",[],[] configuration:id1,configuration,id1,Configuration,"Datasette offers several ways to configure your Datasette instances: server settings, plugin configuration, authentication, and more. Most configuration can be handled using a datasette.yaml configuration file, passed to datasette using the -c/--config flag: datasette mydatabase.db --config datasette.yaml This file can also use JSON, as datasette.json . YAML is recommended over JSON due to its support for comments and multi-line strings.",[],[] changelog:id1,changelog,id1,Changelog,,[],[] csv_export:id1,csv_export,id1,CSV export,"Any Datasette table, view or custom SQL query can be exported as CSV. To obtain the CSV representation of the table you are looking, click the ""this data as CSV"" link. You can also use the advanced export form for more control over the resulting file, which looks like this and has the following options: download file - instead of displaying CSV in your browser, this forces your browser to download the CSV to your downloads directory. expand labels - if your table has any foreign key references this option will cause the CSV to gain additional COLUMN_NAME_label columns with a label for each foreign key derived from the linked table. In this example the city_id column is accompanied by a city_id_label column. stream all rows - by default CSV files only contain the first max_returned_rows records. This option will cause Datasette to loop through every matching record and return them as a single CSV file. You can try that out on https://latest.datasette.io/fixtures/facetable?_size=4",[],"[{""href"": ""https://latest.datasette.io/fixtures/facetable.csv?_labels=on&_size=max"", ""label"": ""In this example""}, {""href"": ""https://latest.datasette.io/fixtures/facetable?_size=4"", ""label"": ""https://latest.datasette.io/fixtures/facetable?_size=4""}]" cli-reference:id1,cli-reference,id1,CLI reference,"The datasette CLI tool provides a number of commands. Running datasette without specifying a command runs the default command, datasette serve . See datasette serve for the full list of options for that command. [[[cog from datasette import cli from click.testing import CliRunner import textwrap def help(args): title = ""datasette "" + "" "".join(args) cog.out(""\n::\n\n"") result = CliRunner().invoke(cli.cli, args) output = result.output.replace(""Usage: cli "", ""Usage: datasette "") cog.out(textwrap.indent(output, ' ')) cog.out(""\n\n"") ]]] [[[end]]]",[],[] binary_data:binary,binary_data,binary,Binary data,"SQLite tables can contain binary data in BLOB columns. Datasette includes special handling for these binary values. The Datasette interface detects binary values and provides a link to download their content, for example on https://latest.datasette.io/fixtures/binary_data Binary data is represented in .json exports using Base64 encoding. https://latest.datasette.io/fixtures/binary_data.json?_shape=array [ { ""rowid"": 1, ""data"": { ""$base64"": true, ""encoded"": ""FRwCx60F/g=="" } }, { ""rowid"": 2, ""data"": { ""$base64"": true, ""encoded"": ""FRwDx60F/g=="" } }, { ""rowid"": 3, ""data"": null } ]",[],"[{""href"": ""https://latest.datasette.io/fixtures/binary_data"", ""label"": ""https://latest.datasette.io/fixtures/binary_data""}, {""href"": ""https://latest.datasette.io/fixtures/binary_data.json?_shape=array"", ""label"": ""https://latest.datasette.io/fixtures/binary_data.json?_shape=array""}]" authentication:authentication,authentication,authentication,Authentication and permissions,"Datasette doesn't require authentication by default. Any visitor to a Datasette instance can explore the full data and execute read-only SQL queries. Datasette's plugin system can be used to add many different styles of authentication, such as user accounts, single sign-on or API keys.",[],[]