rowid,title,content,sections_fts,rank 792,Import shortcuts,"The following commonly used symbols can be imported directly from the datasette module: from datasette import Response from datasette import Forbidden from datasette import NotFound from datasette import hookimpl from datasette import actor_matches_allow",95, 791,Tracing child tasks,"If your code uses a mechanism such as asyncio.gather() to execute code in additional tasks you may find that some of the traces are missing from the display. You can use the trace_child_tasks() context manager to ensure these child tasks are correctly handled. from datasette import tracer with tracer.trace_child_tasks(): results = await asyncio.gather( # ... async tasks here ) This example uses the register_routes() plugin hook to add a page at /parallel-queries which executes two SQL queries in parallel using asyncio.gather() and returns their results. from datasette import hookimpl from datasette import tracer @hookimpl def register_routes(): async def parallel_queries(datasette): db = datasette.get_database() with tracer.trace_child_tasks(): one, two = await asyncio.gather( db.execute(""select 1""), db.execute(""select 2""), ) return Response.json( { ""one"": one.single_value(), ""two"": two.single_value(), } ) return [ (r""/parallel-queries$"", parallel_queries), ] Note that running parallel SQL queries in this way has been known to cause problems in the past , so treat this example with caution. Adding ?_trace=1 will show that the trace covers both of those child tasks.",95, 790,datasette.tracer,"Running Datasette with --setting trace_debug 1 enables trace debug output, which can then be viewed by adding ?_trace=1 to the query string for any page. You can see an example of this at the bottom of latest.datasette.io/fixtures/facetable?_trace=1 . The JSON output shows full details of every SQL query that was executed to generate the page. The datasette-pretty-traces plugin can be installed to provide a more readable display of this information. You can see a demo of that here . You can add your own custom traces to the JSON output using the trace() context manager. This takes a string that identifies the type of trace being recorded, and records any keyword arguments as additional JSON keys on the resulting trace object. The start and end time, duration and a traceback of where the trace was executed will be automatically attached to the JSON object. This example uses trace to record the start, end and duration of any HTTP GET requests made using the function: from datasette.tracer import trace import httpx async def fetch_url(url): with trace(""fetch-url"", url=url): async with httpx.AsyncClient() as client: return await client.get(url)",95, 789,JSON encoding,"class datasette.utils. CustomJSONEncoder * skipkeys = False ensure_ascii = True check_circular = True allow_nan = True sort_keys = False indent = None separators = None default = None The CustomJSONEncoder class handles serialization for objects commonly used by Datasette, including SQLite cursors and binary blobs. Datasette uses it internally to serve .json endpoints, and plugins that return JSON can use it to match Datasette's own handling. Built-in types (text, numbers, lists, etc) are encoded the same as Python's built-in json module. sqlite3.Row becomes a tuple sqlite3.Cursor becomes a list Binary blobs are encoded as an object, with the actual data base64-encoded, like so: { ""$base64"": True, ""encoded"": ..., } Example: https://latest.datasette.io/fixtures/binary_data.json",95, 788,"await async_call_with_supported_arguments(fn, **kwargs)","Async version of call_with_supported_arguments . Use this for async def callback functions. async datasette.utils. async_call_with_supported_arguments fn ** kwargs Async version of call_with_supported_arguments() . Calls await fn(...) with the subset of **kwargs matching its signature. Parameters fn -- An async callable kwargs -- All available keyword arguments Returns The return value of await fn(...)",95, 787,"call_with_supported_arguments(fn, **kwargs)","Call fn , passing it only those keyword arguments that match its function signature. This implements a dependency injection pattern - the caller provides all available arguments, and the function receives only the ones it declares as parameters. This is useful in plugins that want to define callback functions that only declare the arguments they need. For example: from datasette.utils import call_with_supported_arguments def my_callback(request, datasette): ... # This will pass only request and datasette, ignoring other kwargs: call_with_supported_arguments( my_callback, request=request, datasette=datasette, database=database, table=table, ) datasette.utils. call_with_supported_arguments fn ** kwargs Call fn with the subset of **kwargs matching its signature. This implements dependency injection: the caller provides all available keyword arguments and the function receives only the ones it declares as parameters. Parameters fn -- A callable (sync function) kwargs -- All available keyword arguments Returns The return value of fn",95, 786,Tilde encoding,"Datasette uses a custom encoding scheme in some places, called tilde encoding . This is primarily used for table names and row primary keys, to avoid any confusion between / characters in those values and the Datasette URLs that reference them. Tilde encoding uses the same algorithm as URL percent-encoding , but with the ~ tilde character used in place of % . Any character other than ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz0123456789_- will be replaced by the numeric equivalent preceded by a tilde. For example: / becomes ~2F . becomes ~2E % becomes ~25 ~ becomes ~7E Space becomes + polls/2022.primary becomes polls~2F2022~2Eprimary Note that the space character is a special case: it will be replaced with a + symbol. datasette.utils. tilde_encode s : str str Returns tilde-encoded string - for example /foo/bar -> ~2Ffoo~2Fbar datasette.utils. tilde_decode s : str str Decodes a tilde-encoded string, so ~2Ffoo~2Fbar -> /foo/bar",95, 785,named_parameters(sql),"Derive the list of :named parameters referenced in a SQL query. datasette.utils. named_parameters sql : str list [ str ] Given a SQL statement, return a list of named parameters that are used in the statement e.g. for select * from foo where id=:id this would return [""id""]",95, 784,await_me_maybe(value),"Utility function for calling await on a return value if it is awaitable, otherwise returning the value. This is used by Datasette to support plugin hooks that can optionally return awaitable functions. Read more about this function in The “await me maybe” pattern for Python asyncio . async datasette.utils. await_me_maybe value : Any Any If value is callable, call it. If awaitable, await it. Otherwise return it.",95, 783,parse_metadata(content),"This function accepts a string containing either JSON or YAML, expected to be of the format described in Metadata . It returns a nested Python dictionary representing the parsed data from that string. If the metadata cannot be parsed as either JSON or YAML the function will raise a utils.BadMetadataError exception. datasette.utils. parse_metadata content : str dict Detects if content is JSON or YAML and parses it appropriately.",95, 782,The datasette.utils module,"The datasette.utils module contains various utility functions used by Datasette. As a general rule you should consider anything in this module to be unstable - functions and classes here could change without warning or be removed entirely between Datasette releases, without being mentioned in the release notes. The exception to this rule is anything that is documented here. If you find a need for an undocumented utility function in your own work, consider opening an issue requesting that the function you are using be upgraded to documented and supported status.",95, 781,Internal database schema,"The internal database schema is as follows: [[[cog from metadata_doc import internal_schema internal_schema(cog) ]]] CREATE TABLE ""_sqlite_migrations"" ( ""id"" INTEGER PRIMARY KEY, ""migration_set"" TEXT, ""name"" TEXT, ""applied_at"" TEXT ); CREATE UNIQUE INDEX ""idx__sqlite_migrations_migration_set_name"" ON ""_sqlite_migrations"" (""migration_set"", ""name""); CREATE TABLE catalog_databases ( database_name TEXT PRIMARY KEY, path TEXT, is_memory INTEGER, schema_version INTEGER ); CREATE TABLE catalog_tables ( database_name TEXT, table_name TEXT, rootpage INTEGER, sql TEXT, PRIMARY KEY (database_name, table_name), FOREIGN KEY (database_name) REFERENCES catalog_databases(database_name) ); CREATE TABLE catalog_views ( database_name TEXT, view_name TEXT, rootpage INTEGER, sql TEXT, PRIMARY KEY (database_name, view_name), FOREIGN KEY (database_name) REFERENCES catalog_databases(database_name) ); CREATE TABLE catalog_columns ( database_name TEXT, table_name TEXT, cid INTEGER, name TEXT, type TEXT, ""notnull"" INTEGER, default_value TEXT, -- renamed from dflt_value is_pk INTEGER, -- renamed from pk hidden INTEGER, PRIMARY KEY (database_name, table_name, name), FOREIGN KEY (database_name) REFERENCES catalog_databases(database_name), FOREIGN KEY (database_name, table_name) REFERENCES catalog_tables(database_name, table_name) ); CREATE TABLE catalog_indexes ( database_name TEXT, table_name TEXT, seq INTEGER, name TEXT, ""unique"" INTEGER, origin TEXT, partial INTEGER, PRIMARY KEY (database_name, table_name, name), FOREIGN KEY (database_name) REFERENCES catalog_databases(database_name), FOREIGN KEY (database_name, table_name) REFERENCES catalog_tables(database_name, table_name) ); CREATE TABLE catalog_foreign_keys ( database_name TEXT, table_name TEXT, id INTEGER, seq INTEGER, ""table"" TEXT, ""from"" TEXT, ""to"" TEXT, on_update TEXT, on_delete TEXT, match TEXT, PRIMARY KEY (database_name, table_name, id, seq), FOREIGN KEY (database_name) REFERENCES catalog_databases(database_name), FOREIGN KEY (database_name, table_name) REFERENCES catalog_tables(database_name, table_name) ); CREATE TABLE metadata_instance ( key text, value text, unique(key) ); CREATE TABLE metadata_databases ( database_name text, key text, value text, unique(database_name, key) ); CREATE TABLE metadata_resources ( database_name text, resource_name text, key text, value text, unique(database_name, resource_name, key) ); CREATE TABLE metadata_columns ( database_name text, resource_name text, column_name text, key text, value text, unique(database_name, resource_name, column_name, key) ); CREATE TABLE column_types ( database_name TEXT NOT NULL, resource_name TEXT NOT NULL, column_name TEXT NOT NULL, column_type TEXT NOT NULL, config TEXT, PRIMARY KEY (database_name, resource_name, column_name) ); CREATE TABLE queries ( database_name TEXT NOT NULL, name TEXT NOT NULL, sql TEXT NOT NULL, title TEXT, description TEXT, description_html TEXT, options TEXT NOT NULL DEFAULT '{}', parameters TEXT NOT NULL DEFAULT '[]', is_write INTEGER NOT NULL DEFAULT 0 CHECK (is_write IN (0, 1)), is_private INTEGER NOT NULL DEFAULT 0 CHECK (is_private IN (0, 1)), is_trusted INTEGER NOT NULL DEFAULT 0 CHECK (is_trusted IN (0, 1)), source TEXT NOT NULL DEFAULT 'user', owner_id TEXT, created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (database_name, name) ); CREATE INDEX queries_owner_idx ON queries(owner_id); [[[end]]]",95, 780,Datasette's internal database,"Datasette maintains an ""internal"" SQLite database used for configuration, caching, and storage. Plugins can store configuration, settings, and other data inside this database. By default, Datasette will use a temporary in-memory SQLite database as the internal database, which is created at startup and destroyed at shutdown. Users of Datasette can optionally pass in a --internal flag to specify the path to a SQLite database to use as the internal database, which will persist internal data across Datasette instances. Datasette maintains tables called catalog_databases , catalog_tables , catalog_views , catalog_columns , catalog_indexes , catalog_foreign_keys with details of the attached databases and their schemas. These tables should not be considered a stable API - they may change between Datasette releases. Metadata is stored in tables metadata_instance , metadata_databases , metadata_resources and metadata_columns . Plugins can interact with these tables via the get_*_metadata() and set_*_metadata() methods . The internal database is not exposed in the Datasette application by default, which means private data can safely be stored without worry of accidentally leaking information through the default Datasette interface and API. However, other plugins do have full read and write access to the internal database. Plugins can access this database by calling internal_db = datasette.get_internal_database() and then executing queries using the Database API . Plugin authors are asked to practice good etiquette when using the internal database, as all plugins use the same database to store data. For example: Use a unique prefix when creating tables, indices, and triggers in the internal database. If your plugin is called datasette-xyz , then prefix names with datasette_xyz_* . Avoid long-running write statements that may stall or block other plugins that are trying to write at the same time. Use temporary tables or shared in-memory attached databases when possible. Avoid implementing features that could expose private data stored in the internal database by other plugins.",95, 779,CSRF protection,"Datasette protects against Cross-Site Request Forgery by inspecting the browser-set Sec-Fetch-Site and Origin headers on every unsafe (non- GET / HEAD / OPTIONS ) request, following the approach described in Filippo Valsorda's article and implemented in Go 1.25's http.CrossOriginProtection . A request is rejected with a 403 response if: It carries Sec-Fetch-Site with any value other than same-origin or none , or It has no Sec-Fetch-Site header but does carry an Origin header whose host does not match the request Host . Requests from non-browser clients ( curl , server-to-server scripts, etc.) do not send Sec-Fetch-Site or Origin and pass through unchanged - CSRF is a browser-only attack. No token, cookie, or hidden form field is needed. Any