sections
582 rows sorted by references
This data as json, CSV (advanced)
id | page | ref | title | content | breadcrumbs | references ▼ |
---|---|---|---|---|---|---|
changelog:id83 | changelog | id83 | 0.29.1 (2019-07-11) | Fixed bug with static mounts using relative paths which could lead to traversal exploits ( #555 ) - thanks Abdussamet Kocak! Datasette can now be run as a module: python -m datasette ( #556 ) - thanks, Abdussamet Kocak! | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/555", "label": "#555"}, {"href": "https://github.com/simonw/datasette/issues/556", "label": "#556"}] |
changelog:id81 | changelog | id81 | 0.29.3 (2019-09-02) | Fixed implementation of CodeMirror on database page ( #560 ) Documentation typo fixes - thanks, Min ho Kim ( #561 ) Mechanism for detecting if a table has FTS enabled now works if the table name used alternative escaping mechanisms ( #570 ) - for compatibility with a recent change to sqlite-utils . | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/560", "label": "#560"}, {"href": "https://github.com/simonw/datasette/pull/561", "label": "#561"}, {"href": "https://github.com/simonw/datasette/issues/570", "label": "#570"}, {"href": "https://github.com/simonw/sqlite-utils/pull/57", "label": "a recent change to sqlite-utils"}] |
changelog:id70 | changelog | id70 | 0.36 (2020-02-21) | The datasette object passed to plugins now has API documentation: Datasette class . ( #576 ) New methods on datasette : .add_database() and .remove_database() - documentation . ( #671 ) prepare_connection() plugin hook now takes optional datasette and database arguments - prepare_connection(conn, database, datasette) . ( #678 ) Added three new plugins and one new conversion tool to the The Datasette Ecosystem . | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/576", "label": "#576"}, {"href": "https://github.com/simonw/datasette/issues/671", "label": "#671"}, {"href": "https://github.com/simonw/datasette/issues/678", "label": "#678"}] |
changelog:id80 | changelog | id80 | 0.30 (2019-10-18) | Added /-/threads debugging page Allow EXPLAIN WITH... ( #583 ) Button to format SQL - thanks, Tobias Kunze ( #136 ) Sort databases on homepage by argument order - thanks, Tobias Kunze ( #585 ) Display metadata footer on custom SQL queries - thanks, Tobias Kunze ( #589 ) Use --platform=managed for publish cloudrun ( #587 ) Fixed bug returning non-ASCII characters in CSV ( #584 ) Fix for /foo v.s. /foo-bar bug ( #601 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/583", "label": "#583"}, {"href": "https://github.com/simonw/datasette/issues/136", "label": "#136"}, {"href": "https://github.com/simonw/datasette/issues/585", "label": "#585"}, {"href": "https://github.com/simonw/datasette/pull/589", "label": "#589"}, {"href": "https://github.com/simonw/datasette/issues/587", "label": "#587"}, {"href": "https://github.com/simonw/datasette/issues/584", "label": "#584"}, {"href": "https://github.com/simonw/datasette/issues/601", "label": "#601"}] |
changelog:id79 | changelog | id79 | 0.30.1 (2019-10-30) | Fixed bug where ?_where= parameter was not persisted in hidden form fields ( #604 ) Fixed bug with .JSON representation of row pages - thanks, Chris Shaw ( #603 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/604", "label": "#604"}, {"href": "https://github.com/simonw/datasette/issues/603", "label": "#603"}] |
changelog:id78 | changelog | id78 | 0.30.2 (2019-11-02) | /-/plugins page now uses distribution name e.g. datasette-cluster-map instead of the name of the underlying Python package ( datasette_cluster_map ) ( #606 ) Array faceting is now only suggested for columns that contain arrays of strings ( #562 ) Better documentation for the --host argument ( #574 ) Don't show None with a broken link for the label on a nullable foreign key ( #406 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/606", "label": "#606"}, {"href": "https://github.com/simonw/datasette/issues/562", "label": "#562"}, {"href": "https://github.com/simonw/datasette/issues/574", "label": "#574"}, {"href": "https://github.com/simonw/datasette/issues/406", "label": "#406"}] |
changelog:new-features | changelog | new-features | New features | If an error occurs while executing a user-provided SQL query, that query is now re-displayed in an editable form along with the error message. ( #619 ) New ?_col= and ?_nocol= parameters to show and hide columns in a table, plus an interface for hiding and showing columns in the column cog menu. ( #615 ) A new ?_facet_size= parameter for customizing the number of facet results returned on a table or view page. ( #1332 ) ?_facet_size=max sets that to the maximum, which defaults to 1,000 and is controlled by the the max_returned_rows setting. If facet results are truncated the … at the bottom of the facet list now links to this parameter. ( #1337 ) ?_nofacet=1 option to disable all facet calculations on a page, used as a performance optimization for CSV exports and ?_shape=array/object . ( #1349 , #263 ) ?_nocount=1 option to disable full query result counts. ( #1353 ) ?_trace=1 debugging option is now controlled by the new trace_debug setting, which is turned off by default. ( #1359 ) | ["Changelog", "0.57 (2021-06-05)"] | [{"href": "https://github.com/simonw/datasette/issues/619", "label": "#619"}, {"href": "https://github.com/simonw/datasette/issues/615", "label": "#615"}, {"href": "https://github.com/simonw/datasette/issues/1332", "label": "#1332"}, {"href": "https://github.com/simonw/datasette/issues/1337", "label": "#1337"}, {"href": "https://github.com/simonw/datasette/issues/1349", "label": "#1349"}, {"href": "https://github.com/simonw/datasette/issues/263", "label": "#263"}, {"href": "https://github.com/simonw/datasette/issues/1353", "label": "#1353"}, {"href": "https://github.com/simonw/datasette/issues/1359", "label": "#1359"}] |
changelog:id75 | changelog | id75 | 0.31.2 (2019-11-13) | Fixed a bug where datasette publish heroku applications failed to start ( #633 ) Fix for datasette publish with just --source_url - thanks, Stanley Zheng ( #572 ) Deployments to Heroku now use Python 3.8.0 ( #632 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/633", "label": "#633"}, {"href": "https://github.com/simonw/datasette/issues/572", "label": "#572"}, {"href": "https://github.com/simonw/datasette/issues/632", "label": "#632"}] |
changelog:id73 | changelog | id73 | 0.33 (2019-12-22) | rowid is now included in dropdown menus for filtering tables ( #636 ) Columns are now only suggested for faceting if they have at least one value with more than one record ( #638 ) Queries with no results now display "0 results" ( #637 ) Improved documentation for the --static option ( #641 ) asyncio task information is now included on the /-/threads debug page Bumped Uvicorn dependency 0.11 You can now use --port 0 to listen on an available port New template_debug setting for debugging templates, e.g. https://latest.datasette.io/fixtures/roadside_attractions?_context=1 ( #654 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/636", "label": "#636"}, {"href": "https://github.com/simonw/datasette/issues/638", "label": "#638"}, {"href": "https://github.com/simonw/datasette/issues/637", "label": "#637"}, {"href": "https://github.com/simonw/datasette/issues/641", "label": "#641"}, {"href": "https://latest.datasette.io/fixtures/roadside_attractions?_context=1", "label": "https://latest.datasette.io/fixtures/roadside_attractions?_context=1"}, {"href": "https://github.com/simonw/datasette/issues/654", "label": "#654"}] |
changelog:id64 | changelog | id64 | 0.41 (2020-05-06) | You can now create custom pages within your Datasette instance using a custom template file. For example, adding a template file called templates/pages/about.html will result in a new page being served at /about on your instance. See the custom pages documentation for full details, including how to return custom HTTP headers, redirects and status codes. ( #648 ) Configuration directory mode ( #731 ) allows you to define a custom Datasette instance as a directory. So instead of running the following: datasette one.db two.db \ --metadata=metadata.json \ --template-dir=templates/ \ --plugins-dir=plugins \ --static css:css You can instead arrange your files in a single directory called my-project and run this: datasette my-project/ Also in this release: New NOT LIKE table filter: ?colname__notlike=expression . ( #750 ) Datasette now has a pattern portfolio at /-/patterns - e.g. https://latest.datasette.io/-/patterns . This is a page that shows every Datasette user interface component in one place, to aid core development and people building custom CSS themes. ( #151 ) SQLite PRAGMA functions such as pragma_table_info(tablename) are now allowed in Datasette SQL queries. ( #761 ) Datasette pages now consistently return a content-type of text/html; charset=utf-8" . ( #752 ) Datasette now handles an ASGI raw_path value of None , which should allow compatibility with the Mangum adapter for running ASGI apps on AWS Lambda. Thanks, Colin Dellow. ( #719 ) Installation documentation now covers how to Using pipx . ( #756 ) I… | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/648", "label": "#648"}, {"href": "https://github.com/simonw/datasette/issues/731", "label": "#731"}, {"href": "https://github.com/simonw/datasette/issues/750", "label": "#750"}, {"href": "https://latest.datasette.io/-/patterns", "label": "https://latest.datasette.io/-/patterns"}, {"href": "https://github.com/simonw/datasette/issues/151", "label": "#151"}, {"href": "https://www.sqlite.org/pragma.html#pragfunc", "label": "PRAGMA functions"}, {"href": "https://github.com/simonw/datasette/issues/761", "label": "#761"}, {"href": "https://github.com/simonw/datasette/issues/752", "label": "#752"}, {"href": "https://github.com/erm/mangum", "label": "Mangum"}, {"href": "https://github.com/simonw/datasette/pull/719", "label": "#719"}, {"href": "https://github.com/simonw/datasette/issues/756", "label": "#756"}, {"href": "https://github.com/simonw/datasette/issues/748", "label": "#748"}] |
changelog:id72 | changelog | id72 | 0.34 (2020-01-29) | _search= queries are now correctly escaped using a new escape_fts() custom SQL function. This means you can now run searches for strings like park. without seeing errors. ( #651 ) Google Cloud Run is no longer in beta, so datasette publish cloudrun has been updated to work even if the user has not installed the gcloud beta components package. Thanks, Katie McLaughlin ( #660 ) datasette package now accepts a --port option for specifying which port the resulting Docker container should listen on. ( #661 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/651", "label": "#651"}, {"href": "https://cloud.google.com/run/", "label": "Google Cloud Run"}, {"href": "https://github.com/simonw/datasette/pull/660", "label": "#660"}, {"href": "https://github.com/simonw/datasette/issues/661", "label": "#661"}] |
changelog:id69 | changelog | id69 | 0.37 (2020-02-25) | Plugins now have a supported mechanism for writing to a database, using the new .execute_write() and .execute_write_fn() methods. Documentation . ( #682 ) Immutable databases that have had their rows counted using the inspect command now use the calculated count more effectively - thanks, Kevin Keogh. ( #666 ) --reload no longer restarts the server if a database file is modified, unless that database was opened immutable mode with -i . ( #494 ) New ?_searchmode=raw option turns off escaping for FTS queries in ?_search= allowing full use of SQLite's FTS5 query syntax . ( #676 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/682", "label": "#682"}, {"href": "https://github.com/simonw/datasette/pull/666", "label": "#666"}, {"href": "https://github.com/simonw/datasette/issues/494", "label": "#494"}, {"href": "https://www.sqlite.org/fts5.html#full_text_query_syntax", "label": "FTS5 query syntax"}, {"href": "https://github.com/simonw/datasette/issues/676", "label": "#676"}] |
changelog:id63 | changelog | id63 | 0.42 (2020-05-08) | A small release which provides improved internal methods for use in plugins, along with documentation. See #685 . Added documentation for db.execute() , see await db.execute(sql, ...) . Renamed db.execute_against_connection_in_thread() to db.execute_fn() and made it a documented method, see await db.execute_fn(fn) . New results.first() and results.single_value() methods, plus documentation for the Results class - see Results . | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/685", "label": "#685"}] |
changelog:better-plugin-documentation | changelog | better-plugin-documentation | Better plugin documentation | The plugin documentation has been re-arranged into four sections, including a brand new section on testing plugins. ( #687 ) Plugins introduces Datasette's plugin system and describes how to install and configure plugins. Writing plugins describes how to author plugins, from one-off single file plugins to packaged plugins that can be published to PyPI. It also describes how to start a plugin using the new datasette-plugin cookiecutter template. Plugin hooks is a full list of detailed documentation for every Datasette plugin hook. Testing plugins describes how to write tests for Datasette plugins, using pytest and HTTPX . | ["Changelog", "0.45 (2020-07-01)"] | [{"href": "https://github.com/simonw/datasette/issues/687", "label": "#687"}, {"href": "https://github.com/simonw/datasette-plugin", "label": "datasette-plugin"}, {"href": "https://docs.pytest.org/", "label": "pytest"}, {"href": "https://www.python-httpx.org/", "label": "HTTPX"}] |
changelog:id68 | changelog | id68 | 0.37.1 (2020-03-02) | Don't attempt to count table rows to display on the index page for databases > 100MB. ( #688 ) Print exceptions if they occur in the write thread rather than silently swallowing them. Handle the possibility of scope["path"] being a string rather than bytes Better documentation for the extra_template_vars(template, database, table, columns, view_name, request, datasette) plugin hook. | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/688", "label": "#688"}] |
changelog:id212 | changelog | id212 | 0.8 (2017-11-13) | V0.8 - added PyPI metadata, ready to ship. Implemented offset/limit pagination for views ( #70 ). Improved pagination. ( #78 ) Limit on max rows returned, controlled by --max_returned_rows option. ( #69 ) If someone executes 'select * from table' against a table with a million rows in it, we could run into problems: just serializing that much data as JSON is likely to lock up the server. Solution: we now have a hard limit on the maximum number of rows that can be returned by a query. If that limit is exceeded, the server will return a "truncated": true field in the JSON. This limit can be optionally controlled by the new --max_returned_rows option. Setting that option to 0 disables the limit entirely. | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/70", "label": "#70"}, {"href": "https://github.com/simonw/datasette/issues/78", "label": "#78"}, {"href": "https://github.com/simonw/datasette/issues/69", "label": "#69"}] |
changelog:id61 | changelog | id61 | Smaller changes | New internals documentation for Request object and Response class . ( #706 ) request.url now respects the force_https_urls config setting. closes ( #781 ) request.args.getlist() returns [] if missing. Removed request.raw_args entirely. ( #774 ) New datasette.get_database() method. Added _ prefix to many private, undocumented methods of the Datasette class. ( #576 ) Removed the db.get_outbound_foreign_keys() method which duplicated the behaviour of db.foreign_keys_for_table() . New await datasette.permission_allowed() method. /-/actor debugging endpoint for viewing the currently authenticated actor. New request.cookies property. /-/plugins endpoint now shows a list of hooks implemented by each plugin, e.g. https://latest.datasette.io/-/plugins?all=1 request.post_vars() method no longer discards empty values. New "params" canned query key for explicitly setting named parameters, see Canned query parameters . ( #797 ) request.args is now a MultiParams object. Fixed a bug with the datasette plugins command. ( #802 ) Nicer pa… | ["Changelog", "0.44 (2020-06-11)"] | [{"href": "https://github.com/simonw/datasette/issues/706", "label": "#706"}, {"href": "https://github.com/simonw/datasette/issues/781", "label": "#781"}, {"href": "https://github.com/simonw/datasette/issues/774", "label": "#774"}, {"href": "https://github.com/simonw/datasette/issues/576", "label": "#576"}, {"href": "https://latest.datasette.io/-/plugins?all=1", "label": "https://latest.datasette.io/-/plugins?all=1"}, {"href": "https://github.com/simonw/datasette/issues/797", "label": "#797"}, {"href": "https://github.com/simonw/datasette/issues/802", "label": "#802"}, {"href": "https://github.com/simonw/datasette/issues/395", "label": "#395"}, {"href": "https://github.com/simonw/datasette/issues/777", "label": "#777"}, {"href": "https://github.com/simonw/datasette/issues/822", "label": "#822"}, {"href": "https://github.com/simonw/datasette/issues/804", "label": "#804"}, {"href": "https://github.com/simonw/datasette/issues/830", "label": "#830"}, {"href": "https://github.com/simonw/datasette/issues/837", "label": "#837"}] |
changelog:id65 | changelog | id65 | 0.40 (2020-04-21) | Datasette Metadata can now be provided as a YAML file as an optional alternative to JSON. ( #713 ) Removed support for datasette publish now , which used the the now-retired Zeit Now v1 hosting platform. A new plugin, datasette-publish-now , can be installed to publish data to Zeit ( now Vercel ) Now v2. ( #710 ) Fixed a bug where the extra_template_vars(request, view_name) plugin hook was not receiving the correct view_name . ( #716 ) Variables added to the template context by the extra_template_vars() plugin hook are now shown in the ?_context=1 debugging mode (see template_debug ). ( #693 ) Fixed a bug where the "templates considered" HTML comment was no longer being displayed. ( #689 ) Fixed a datasette publish bug where --plugin-secret would over-ride plugin configuration in the provided metadata.json file. ( #724 ) Added a new CSS class for customizing the canned query page. ( #727 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/713", "label": "#713"}, {"href": "https://github.com/simonw/datasette-publish-now", "label": "datasette-publish-now"}, {"href": "https://vercel.com/blog/zeit-is-now-vercel", "label": "now Vercel"}, {"href": "https://github.com/simonw/datasette/issues/710", "label": "#710"}, {"href": "https://github.com/simonw/datasette/issues/716", "label": "#716"}, {"href": "https://github.com/simonw/datasette/issues/693", "label": "#693"}, {"href": "https://github.com/simonw/datasette/issues/689", "label": "#689"}, {"href": "https://github.com/simonw/datasette/issues/724", "label": "#724"}, {"href": "https://github.com/simonw/datasette/issues/727", "label": "#727"}] |
sql_queries:sql-parameters | sql_queries | sql-parameters | Named parameters | Datasette has special support for SQLite named parameters. Consider a SQL query like this: select * from Street_Tree_List where "PermitNotes" like :notes and "qSpecies" = :species If you execute this query using the custom query editor, Datasette will extract the two named parameters and use them to construct form fields for you to provide values. You can also provide values for these fields by constructing a URL: /mydatabase?sql=select...&species=44 SQLite string escaping rules will be applied to values passed using named parameters - they will be wrapped in quotes and their content will be correctly escaped. Values from named parameters are treated as SQLite strings. If you need to perform numeric comparisons on them you should cast them to an integer or float first using cast(:name as integer) or cast(:name as real) , for example: select * from Street_Tree_List where latitude > cast(:min_latitude as real) and latitude < cast(:max_latitude as real) Datasette disallows custom SQL queries containing the string PRAGMA (with a small number of exceptions ) as SQLite pragma statements can be used to change database settings at runtime. If you need to include the string "pragma" in a query you can do so safely using a named parameter. | ["Running SQL queries"] | [{"href": "https://github.com/simonw/datasette/issues/761", "label": "of exceptions"}] |
changelog:v1-0-a3 | changelog | v1-0-a3 | 1.0a3 (2023-08-09) | This alpha release previews the updated design for Datasette's default JSON API. ( #782 ) The new default JSON representation for both table pages ( /dbname/table.json ) and arbitrary SQL queries ( /dbname.json?sql=... ) is now shaped like this: { "ok": true, "rows": [ { "id": 3, "name": "Detroit" }, { "id": 2, "name": "Los Angeles" }, { "id": 4, "name": "Memnonia" }, { "id": 1, "name": "San Francisco" } ], "truncated": false } Tables will include an additional "next" key for pagination, which can be passed to ?_next= to fetch the next page of results. The various ?_shape= options continue to work as before - see Different shapes for details. A new ?_extra= mechanism is available for tables, but has not yet been stabilized or documented. Details on that are available in #262 . | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/782", "label": "#782"}, {"href": "https://github.com/simonw/datasette/issues/262", "label": "#262"}] |
changelog:permissions | changelog | permissions | Permissions | Datasette also now has a built-in concept of Permissions . The permissions system answers the following question: Is this actor allowed to perform this action , optionally against this particular resource ? You can use the new "allow" block syntax in metadata.json (or metadata.yaml ) to set required permissions at the instance, database, table or canned query level. For example, to restrict access to the fixtures.db database to the "root" user: { "databases": { "fixtures": { "allow": { "id" "root" } } } } See Defining permissions with "allow" blocks for more details. Plugins can implement their own custom permission checks using the new permission_allowed(datasette, actor, action, resource) hook. A new debug page at /-/permissions shows recent permission checks, to help administrators and plugin authors understand exactly what checks are being performed. This tool defaults to only being available to the root user, but can be exposed to other users by plugins that respond to the permissions-debug permission. ( #788 ) | ["Changelog", "0.44 (2020-06-11)"] | [{"href": "https://github.com/simonw/datasette/issues/788", "label": "#788"}] |
changelog:flash-messages | changelog | flash-messages | Flash messages | Writable canned queries needed a mechanism to let the user know that the query has been successfully executed. The new flash messaging system ( #790 ) allows messages to persist in signed cookies which are then displayed to the user on the next page that they visit. Plugins can use this mechanism to display their own messages, see .add_message(request, message, type=datasette.INFO) for details. You can try out the new messages using the /-/messages debug tool, for example at https://latest.datasette.io/-/messages | ["Changelog", "0.44 (2020-06-11)"] | [{"href": "https://github.com/simonw/datasette/issues/790", "label": "#790"}, {"href": "https://latest.datasette.io/-/messages", "label": "https://latest.datasette.io/-/messages"}] |
changelog:csrf-protection | changelog | csrf-protection | CSRF protection | Since writable canned queries are built using POST forms, Datasette now ships with CSRF protection ( #798 ). This applies automatically to any POST request, which means plugins need to include a csrftoken in any POST forms that they render. They can do that like so: <input type="hidden" name="csrftoken" value="{{ csrftoken() }}"> | ["Changelog", "0.44 (2020-06-11)"] | [{"href": "https://github.com/simonw/datasette/issues/798", "label": "#798"}] |
changelog:register-routes-plugin-hooks | changelog | register-routes-plugin-hooks | register_routes() plugin hooks | Plugins can now register new views and routes via the register_routes(datasette) plugin hook ( #819 ). View functions can be defined that accept any of the current datasette object, the current request , or the ASGI scope , send and receive objects. | ["Changelog", "0.44 (2020-06-11)"] | [{"href": "https://github.com/simonw/datasette/issues/819", "label": "#819"}] |
changelog:id209 | changelog | id209 | 0.10 (2017-11-14) | Fixed #83 - 500 error on individual row pages. Stop using sqlite WITH RECURSIVE in our tests. The version of Python 3 running in Travis CI doesn't support this. | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/83", "label": "#83"}] |
plugin_hooks:plugin-register-facet-classes | plugin_hooks | plugin-register-facet-classes | register_facet_classes() | Return a list of additional Facet subclasses to be registered. The design of this plugin hook is unstable and may change. See issue 830 . Each Facet subclass implements a new type of facet operation. The class should look like this: class SpecialFacet(Facet): # This key must be unique across all facet classes: type = "special" async def suggest(self): # Use self.sql and self.params to suggest some facets suggested_facets = [] suggested_facets.append( { "name": column, # Or other unique name # Construct the URL that will enable this facet: "toggle_url": self.ds.absolute_url( self.request, path_with_added_args( self.request, {"_facet": column} ), ), } ) return suggested_facets async def facet_results(self): # This should execute the facet operation and return results, again # using self.sql and self.params as the starting point facet_results = [] facets_timed_out = [] facet_size = self.get_facet_size() # Do some calculations here... for column in columns_selected_for_facet: try: facet_results_values = [] # More calculations... facet_results_values.append( { "value": value, "label": label, "count": count, "toggle_url": self.ds.absolute_url( self.request, toggle_path ), "selected": selected, } ) facet_results.append( { "name": column, "results": facet_results_values, "trunc… | ["Plugin hooks"] | [{"href": "https://github.com/simonw/datasette/issues/830", "label": "issue 830"}, {"href": "https://github.com/simonw/datasette/blob/main/datasette/facets.py", "label": "datasette/facets.py"}] |
changelog:id59 | changelog | id59 | Smaller changes | Cascading view permissions - so if a user has view-table they can view the table page even if they do not have view-database or view-instance . ( #832 ) CSRF protection no longer applies to Authentication: Bearer token requests or requests without cookies. ( #835 ) datasette.add_message() now works inside plugins. ( #864 ) Workaround for "Too many open files" error in test runs. ( #846 ) Respect existing scope["actor"] if already set by ASGI middleware. ( #854 ) New process for shipping Alpha and beta releases . ( #807 ) {{ csrftoken() }} now works when plugins render a template using datasette.render_template(..., request=request) . ( #863 ) Datasette now creates a single Request object and uses it throughout the lifetime of the current HTTP request. ( #870 ) | ["Changelog", "0.45 (2020-07-01)"] | [{"href": "https://github.com/simonw/datasette/issues/832", "label": "#832"}, {"href": "https://github.com/simonw/datasette/issues/835", "label": "#835"}, {"href": "https://github.com/simonw/datasette/issues/864", "label": "#864"}, {"href": "https://github.com/simonw/datasette/issues/846", "label": "#846"}, {"href": "https://github.com/simonw/datasette/issues/854", "label": "#854"}, {"href": "https://github.com/simonw/datasette/issues/807", "label": "#807"}, {"href": "https://github.com/simonw/datasette/issues/863", "label": "#863"}, {"href": "https://github.com/simonw/datasette/issues/870", "label": "#870"}] |
changelog:log-out | changelog | log-out | Log out | The ds_actor cookie can be used by plugins (or by Datasette's --root mechanism ) to authenticate users. The new /-/logout page provides a way to clear that cookie. A "Log out" button now shows in the global navigation provided the user is authenticated using the ds_actor cookie. ( #840 ) | ["Changelog", "0.45 (2020-07-01)"] | [{"href": "https://github.com/simonw/datasette/issues/840", "label": "#840"}] |
changelog:magic-parameters-for-canned-queries | changelog | magic-parameters-for-canned-queries | Magic parameters for canned queries | Canned queries now support Magic parameters , which can be used to insert or select automatically generated values. For example: insert into logs (user_id, timestamp) values (:_actor_id, :_now_datetime_utc) This inserts the currently authenticated actor ID and the current datetime. ( #842 ) | ["Changelog", "0.45 (2020-07-01)"] | [{"href": "https://github.com/simonw/datasette/issues/842", "label": "#842"}] |
changelog:id49 | changelog | id49 | 0.50 (2020-10-09) | The key new feature in this release is the column actions menu on the table page ( #891 ). This can be used to sort a column in ascending or descending order, facet data by that column or filter the table to just rows that have a value for that column. Plugin authors can use the new datasette.client object to make internal HTTP requests from their plugins, allowing them to make use of Datasette's JSON API. ( #943 ) New Deploying Datasette documentation with guides for deploying Datasette on a Linux server using systemd or to hosting providers that support buildpacks . ( #514 , #997 ) Other improvements in this release: Publishing to Google Cloud Run documentation now covers Google Cloud SDK options. Thanks, Geoffrey Hing. ( #995 ) New datasette -o option which opens your browser as soon as Datasette starts up. ( #970 ) Datasette now sets sqlite3.enable_callback_tracebacks(True) so that errors in custom SQL functions will display tracebacks. ( #891 ) Fixed two rendering bugs with column headers in portrait mobile view. ( #978 , #980 ) New db.table_column_details(table) introspection method for retrieving full details of the columns in a specific table, see Database introspection . Fixed a routing bug with custom page wildcard templates. ( #996 ) datasette publish heroku now deploys using Python 3.8.6. New datasette publish heroku --tar= option. ( #969 ) OPTIONS requests against HTML pages no longer return a 500 error. ( #1001 ) … | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/891", "label": "#891"}, {"href": "https://github.com/simonw/datasette/issues/943", "label": "#943"}, {"href": "https://github.com/simonw/datasette/issues/514", "label": "#514"}, {"href": "https://github.com/simonw/datasette/issues/997", "label": "#997"}, {"href": "https://github.com/simonw/datasette/pull/995", "label": "#995"}, {"href": "https://github.com/simonw/datasette/issues/970", "label": "#970"}, {"href": "https://github.com/simonw/datasette/issues/891", "label": "#891"}, {"href": "https://github.com/simonw/datasette/issues/978", "label": "#978"}, {"href": "https://github.com/simonw/datasette/issues/980", "label": "#980"}, {"href": "https://github.com/simonw/datasette/issues/996", "label": "#996"}, {"href": "https://github.com/simonw/datasette/issues/969", "label": "#969"}, {"href": "https://github.com/simonw/datasette/issues/1001", "label": "#1001"}, {"href": "https://simonwillison.net/2020/Oct/9/datasette-0-50/", "label": "Datasette 0.50: The annotated release notes"}] |
changelog:url-building | changelog | url-building | URL building | The new datasette.urls family of methods can be used to generate URLs to key pages within the Datasette interface, both within custom templates and Datasette plugins. See Building URLs within plugins for more details. ( #904 ) | ["Changelog", "0.51 (2020-10-31)"] | [{"href": "https://github.com/simonw/datasette/issues/904", "label": "#904"}] |
changelog:id57 | changelog | id57 | 0.46 (2020-08-09) | This release contains a security fix related to authenticated writable canned queries. If you are using this feature you should upgrade as soon as possible. Security fix: CSRF tokens were incorrectly included in read-only canned query forms, which could allow them to be leaked to a sophisticated attacker. See issue 918 for details. Datasette now supports GraphQL via the new datasette-graphql plugin - see GraphQL in Datasette with the new datasette-graphql plugin . Principle git branch has been renamed from master to main . ( #849 ) New debugging tool: /-/allow-debug tool ( demo here ) helps test allow blocks against actors, as described in Defining permissions with "allow" blocks . ( #908 ) New logo for the documentation, and a new project tagline: "An open source multi-tool for exploring and publishing data". Whitespace in column values is now respected on display, using white-space: pre-wrap . ( #896 ) New await request.post_body() method for accessing the raw POST body, see Request object . ( #897 ) Database file downloads now include a content-length HTTP header, enabling download progress bars. ( #905 ) File downloads now also correctly set the suggested file name using a content-disposition HTTP header. ( #909 ) tests are now excluded from the Datasette package properly - thanks, abeyerpath. ( #456 ) The Datasette package published to PyPI now include… | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/918", "label": "issue 918"}, {"href": "https://github.com/simonw/datasette-graphql", "label": "datasette-graphql"}, {"href": "https://simonwillison.net/2020/Aug/7/datasette-graphql/", "label": "GraphQL in Datasette with the new datasette-graphql plugin"}, {"href": "https://github.com/simonw/datasette/issues/849", "label": "#849"}, {"href": "https://latest.datasette.io/-/allow-debug", "label": "demo here"}, {"href": "https://github.com/simonw/datasette/issues/908", "label": "#908"}, {"href": "https://github.com/simonw/datasette/issues/896", "label": "#896"}, {"href": "https://github.com/simonw/datasette/issues/897", "label": "#897"}, {"href": "https://github.com/simonw/datasette/issues/905", "label": "#905"}, {"href": "https://github.com/simonw/datasette/issues/909", "label": "#909"}, {"href": "https://github.com/simonw/datasette/issues/456", "label": "#456"}, {"href": "https://github.com/simonw/datasette/issues/887", "label": "#887"}, {"href": "https://github.com/simonw/datasette/pull/890", "label": "#890"}] |
changelog:id55 | changelog | id55 | 0.47.1 (2020-08-11) | Fixed a bug where the sdist distribution of Datasette was not correctly including the template files. ( #930 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/930", "label": "#930"}] |
changelog:id53 | changelog | id53 | 0.47.3 (2020-08-15) | The datasette --get command-line mechanism now ensures any plugins using the startup() hook are correctly executed. ( #934 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/934", "label": "#934"}] |
changelog:id26 | changelog | id26 | 0.59 (2021-10-14) | Columns can now have associated metadata descriptions in metadata.json , see Column descriptions . ( #942 ) New register_commands() plugin hook allows plugins to register additional Datasette CLI commands, e.g. datasette mycommand file.db . ( #1449 ) Adding ?_facet_size=max to a table page now shows the number of unique values in each facet. ( #1423 ) Upgraded dependency httpx 0.20 - the undocumented allow_redirects= parameter to datasette.client is now follow_redirects= , and defaults to False where it previously defaulted to True . ( #1488 ) The --cors option now causes Datasette to return the Access-Control-Allow-Headers: Authorization header, in addition to Access-Control-Allow-Origin: * . ( #1467 ) Code that figures out which named parameters a SQL query takes in order to display form fields for them is no longer confused by strings that contain colon characters. ( #1421 ) Renamed --help-config option to --help-settings . ( #1431 ) datasette.databases property is now a documented API. ( #1443 ) The base.html template now wraps everything other than the <footer> in a <div class="not-footer"> element, to help with advanced CSS customization. ( #1446 ) The render_cell() plugin hook can now return an awaitable function. This means the hook can execute SQL queries. ( #1425 ) register_routes(datasette) plugin hook now accepts an optional datasette argument. ( #1404 ) … | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/942", "label": "#942"}, {"href": "https://github.com/simonw/datasette/issues/1449", "label": "#1449"}, {"href": "https://github.com/simonw/datasette/issues/1423", "label": "#1423"}, {"href": "https://github.com/encode/httpx/releases/tag/0.20.0", "label": "httpx 0.20"}, {"href": "https://github.com/simonw/datasette/issues/1488", "label": "#1488"}, {"href": "https://github.com/simonw/datasette/pull/1467", "label": "#1467"}, {"href": "https://github.com/simonw/datasette/issues/1421", "label": "#1421"}, {"href": "https://github.com/simonw/datasette/issues/1431", "label": "#1431"}, {"href": "https://github.com/simonw/datasette/issues/1443", "label": "#1443"}, {"href": "https://github.com/simonw/datasette/issues/1446", "label": "#1446"}, {"href": "https://github.com/simonw/datasette/issues/1425", "label": "#1425"}, {"href": "https://github.com/simonw/datasette/issues/1404", "label": "#1404"}, {"href": "https://github.com/simonw/datasette/issues/1422", "label": "#1422"}, {"href": "https://github.com/simonw/datasette/issues/1420", "label": "#1420"}, {"href": "https://github.com/willmcgugan/rich", "label": "Rich"}, {"href": "https://github.com/simonw/datasette/issues/1416", "label": "#1416"}, {"href": "https://datasette.io/plugins/datasette-remote-metadata", "label": "datasette-remote-metadata plugin"}, {"href": "https://github.com/simonw/datasette/issues/1405", "label": "#1405"}, {"href": "https://github.com/simonw/datasette/issues/1470", "label": "#1470"}, {"href": "https://github.com/simonw/datasette/issues/1469", "label": "#1469"}] |
changelog:id50 | changelog | id50 | 0.49.1 (2020-09-15) | Fixed a bug with writable canned queries that use magic parameters but accept no non-magic arguments. ( #967 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/967", "label": "#967"}] |
changelog:id43 | changelog | id43 | 0.52 (2020-11-28) | This release includes a number of changes relating to an internal rebranding effort: Datasette's configuration mechanism (things like datasette --config default_page_size:10 ) has been renamed to settings . New --setting default_page_size 10 option as a replacement for --config default_page_size:10 (note the lack of a colon). The --config option is deprecated but will continue working until Datasette 1.0. ( #992 ) The /-/config introspection page is now /-/settings , and the previous page redirects to the new one. ( #1103 ) The config.json file in Configuration directory mode is now called settings.json . ( #1104 ) The undocumented datasette.config() internal method has been replaced by a documented .setting(key) method. ( #1107 ) Also in this release: New plugin hook: database_actions(datasette, actor, database, request) , which adds menu items to a new cog menu shown at the top of the database page. ( #1077 ) datasette publish cloudrun has a new --apt-get-install option that can be used to install additional Ubuntu packages as part of the deployment. This is useful for deploying the new datasette-ripgrep plugin . ( #1110 ) Swept the documentation to remove words that minimize involved difficulty. ( #1089 ) And some bug fixes: Foreign keys linking to rows with blank label columns now display as a hyphen, allowing those links to be clicked. ( #1086 ) Fixed bug where row pages could sometimes 500 … | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/issues/992", "label": "#992"}, {"href": "https://github.com/simonw/datasette/issues/1103", "label": "#1103"}, {"href": "https://github.com/simonw/datasette/issues/1104", "label": "#1104"}, {"href": "https://github.com/simonw/datasette/issues/1107", "label": "#1107"}, {"href": "https://github.com/simonw/datasette/issues/1077", "label": "#1077"}, {"href": "https://github.com/simonw/datasette-ripgrep", "label": "datasette-ripgrep plugin"}, {"href": "https://github.com/simonw/datasette/issues/1110", "label": "#1110"}, {"href": "https://github.com/simonw/datasette/issues/1089", "label": "#1089"}, {"href": "https://github.com/simonw/datasette/issues/1086", "label": "#1086"}, {"href": "https://github.com/simonw/datasette/issues/1088", "label": "#1088"}, {"href": "https://github.com/simonw/datasette/issues/1084", "label": "#1084"}] |
changelog:id46 | changelog | id46 | Smaller changes | Wide tables shown within Datasette now scroll horizontally ( #998 ). This is achieved using a new <div class="table-wrapper"> element which may impact the implementation of some plugins (for example this change to datasette-cluster-map ). New debug-menu permission. ( #1068 ) Removed --debug option, which didn't do anything. ( #814 ) Link: HTTP header pagination. ( #1014 ) x button for clearing filters. ( #1016 ) Edit SQL button on canned queries, ( #1019 ) --load-extension=spatialite shortcut. ( #1028 ) scale-in animation for column action menu. ( #1039 ) Option to pass a list of templates to .render_template() is now documented. ( #1045 ) New datasette.urls.static_plugins() method. ( #1033 ) datasette -o option now opens the most relevant page. ( #976 ) datasette --cors option now enables access to /database.db downloads. ( #1057 ) Database file downloads now implement cascading permissions, so you can download a database if you have view-database-download permission even if you do not have permission to access the Datasette instance. ( #1058 ) New documentation on Designing URLs for your plugin . (… | ["Changelog", "0.51 (2020-10-31)"] | [{"href": "https://github.com/simonw/datasette/issues/998", "label": "#998"}, {"href": "https://github.com/simonw/datasette-cluster-map/commit/fcb4abbe7df9071c5ab57defd39147de7145b34e", "label": "this change to datasette-cluster-map"}, {"href": "https://github.com/simonw/datasette/issues/1068", "label": "#1068"}, {"href": "https://github.com/simonw/datasette/issues/814", "label": "#814"}, {"href": "https://github.com/simonw/datasette/issues/1014", "label": "#1014"}, {"href": "https://github.com/simonw/datasette/issues/1016", "label": "#1016"}, {"href": "https://github.com/simonw/datasette/issues/1019", "label": "#1019"}, {"href": "https://github.com/simonw/datasette/issues/1028", "label": "#1028"}, {"href": "https://github.com/simonw/datasette/issues/1039", "label": "#1039"}, {"href": "https://github.com/simonw/datasette/issues/1045", "label": "#1045"}, {"href": "https://github.com/simonw/datasette/issues/1033", "label": "#1033"}, {"href": "https://github.com/simonw/datasette/issues/976", "label": "#976"}, {"href": "https://github.com/simonw/datasette/issues/1057", "label": "#1057"}, {"href": "https://github.com/simonw/datasette/issues/1058", "label": "#1058"}, {"href": "https://github.com/simonw/datasette/issues/1053", "label": "#1053"}] |
internals:internals-utils | internals | internals-utils | The datasette.utils module | The datasette.utils module contains various utility functions used by Datasette. As a general rule you should consider anything in this module to be unstable - functions and classes here could change without warning or be removed entirely between Datasette releases, without being mentioned in the release notes. The exception to this rule is anything that is documented here. If you find a need for an undocumented utility function in your own work, consider opening an issue requesting that the function you are using be upgraded to documented and supported status. | ["Internals for plugins"] | [{"href": "https://github.com/simonw/datasette/issues/new", "label": "opening an issue"}] |
changelog:the-road-to-datasette-1-0 | changelog | the-road-to-datasette-1-0 | The road to Datasette 1.0 | I've assembled a milestone for Datasette 1.0 . The focus of the 1.0 release will be the following: Signify confidence in the quality/stability of Datasette Give plugin authors confidence that their plugins will work for the whole 1.x release cycle Provide the same confidence to developers building against Datasette JSON APIs If you have thoughts about what you would like to see for Datasette 1.0 you can join the conversation on issue #519 . | ["Changelog", "0.44 (2020-06-11)"] | [{"href": "https://github.com/simonw/datasette/milestone/7", "label": "milestone for Datasette 1.0"}, {"href": "https://github.com/simonw/datasette/issues/519", "label": "the conversation on issue #519"}] |
changelog:v0-28-register-output-renderer | changelog | v0-28-register-output-renderer | register_output_renderer plugins | Russ Garrett implemented a new Datasette plugin hook called register_output_renderer ( #441 ) which allows plugins to create additional output renderers in addition to Datasette's default .json and .csv . Russ's in-development datasette-geo plugin includes an example of this hook being used to output .geojson automatically converted from SpatiaLite. | ["Changelog", "0.28 (2019-05-19)"] | [{"href": "https://github.com/simonw/datasette/pull/441", "label": "#441"}, {"href": "https://github.com/russss/datasette-geo", "label": "datasette-geo"}, {"href": "https://github.com/russss/datasette-geo/blob/d4cecc020848bbde91e9e17bf352f7c70bc3dccf/datasette_plugin_geo/geojson.py", "label": "an example"}] |
changelog:id77 | changelog | id77 | 0.31 (2019-11-11) | This version adds compatibility with Python 3.8 and breaks compatibility with Python 3.5. If you are still running Python 3.5 you should stick with 0.30.2 , which you can install like this: pip install datasette==0.30.2 Format SQL button now works with read-only SQL queries - thanks, Tobias Kunze ( #602 ) New ?column__notin=x,y,z filter for table views ( #614 ) Table view now uses select col1, col2, col3 instead of select * Database filenames can now contain spaces - thanks, Tobias Kunze ( #590 ) Removed obsolete ?_group_count=col feature ( #504 ) Improved user interface and documentation for datasette publish cloudrun ( #608 ) Tables with indexes now show the CREATE INDEX statements on the table page ( #618 ) Current version of uvicorn is now shown on /-/versions Python 3.8 is now supported! ( #622 ) Python 3.5 is no longer supported. | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/pull/602", "label": "#602"}, {"href": "https://github.com/simonw/datasette/issues/614", "label": "#614"}, {"href": "https://github.com/simonw/datasette/pull/590", "label": "#590"}, {"href": "https://github.com/simonw/datasette/issues/504", "label": "#504"}, {"href": "https://github.com/simonw/datasette/issues/608", "label": "#608"}, {"href": "https://github.com/simonw/datasette/issues/618", "label": "#618"}, {"href": "https://www.uvicorn.org/", "label": "uvicorn"}, {"href": "https://github.com/simonw/datasette/issues/622", "label": "#622"}] |
changelog:id76 | changelog | id76 | 0.31.1 (2019-11-12) | Deployments created using datasette publish now use python:3.8 base Docker image ( #629 ) | ["Changelog"] | [{"href": "https://github.com/simonw/datasette/pull/629", "label": "#629"}] |
contributing:contributing-bug-fix-branch | contributing | contributing-bug-fix-branch | Releasing bug fixes from a branch | If it's necessary to publish a bug fix release without shipping new features that have landed on main a release branch can be used. Create it from the relevant last tagged release like so: git branch 0.52.x 0.52.4 git checkout 0.52.x Next cherry-pick the commits containing the bug fixes: git cherry-pick COMMIT Write the release notes in the branch, and update the version number in version.py . Then push the branch: git push -u origin 0.52.x Once the tests have completed, publish the release from that branch target using the GitHub Draft a new release form. Finally, cherry-pick the commit with the release notes and version number bump across to main : git checkout main git cherry-pick COMMIT git push | ["Contributing"] | [{"href": "https://github.com/simonw/datasette/releases/new", "label": "Draft a new release"}] |
plugin_hooks:plugin-hook-publish-subcommand | plugin_hooks | plugin-hook-publish-subcommand | publish_subcommand(publish) | publish - Click publish command group The Click command group for the datasette publish subcommand This hook allows you to create new providers for the datasette publish command. Datasette uses this hook internally to implement the default cloudrun and heroku subcommands, so you can read their source to see examples of this hook in action. Let's say you want to build a plugin that adds a datasette publish my_hosting_provider --api_key=xxx mydatabase.db publish command. Your implementation would start like this: from datasette import hookimpl from datasette.publish.common import ( add_common_publish_arguments_and_options, ) import click @hookimpl def publish_subcommand(publish): @publish.command() @add_common_publish_arguments_and_options @click.option( "-k", "--api_key", help="API key for talking to my hosting provider", ) def my_hosting_provider( files, metadata, extra_options, branch, template_dir, plugins_dir, static, install, plugin_secret, version_note, secret, title, license, license_url, source, source_url, about, about_url, api_key, ): ... Examples: datasette-publish-fly , datasette-publish-vercel | ["Plugin hooks"] | [{"href": "https://github.com/simonw/datasette/tree/main/datasette/publish", "label": "their source"}, {"href": "https://datasette.io/plugins/datasette-publish-fly", "label": "datasette-publish-fly"}, {"href": "https://datasette.io/plugins/datasette-publish-vercel", "label": "datasette-publish-vercel"}] |
changelog:id115 | changelog | id115 | 0.22.1 (2018-05-23) | Bugfix release, plus we now use versioneer for our version numbers. Faceting no longer breaks pagination, fixes #282 Add __version_info__ derived from __version__ [Robert Gieseke] This might be tuple of more than two values (major and minor version) if commits have been made after a release. Add version number support with Versioneer. [Robert Gieseke] Versioneer Licence: Public Domain (CC0-1.0) Closes #273 Refactor inspect logic [Russ Garrett] | ["Changelog"] | [{"href": "https://github.com/warner/python-versioneer", "label": "versioneer"}, {"href": "https://github.com/simonw/datasette/issues/282", "label": "#282"}, {"href": "https://github.com/simonw/datasette/issues/273", "label": "#273"}] |
getting_started:getting-started-glitch | getting_started | getting-started-glitch | Try Datasette without installing anything using Glitch | Glitch is a free online tool for building web apps directly from your web browser. You can use Glitch to try out Datasette without needing to install any software on your own computer. Here's a demo project on Glitch which you can use as the basis for your own experiments: glitch.com/~datasette-csvs Glitch allows you to "remix" any project to create your own copy and start editing it in your browser. You can remix the datasette-csvs project by clicking this button: Find a CSV file and drag it onto the Glitch file explorer panel - datasette-csvs will automatically convert it to a SQLite database (using sqlite-utils ) and allow you to start exploring it using Datasette. If your CSV file has a latitude and longitude column you can visualize it on a map by uncommenting the datasette-cluster-map line in the requirements.txt file using the Glitch file editor. Need some data? Try this Public Art Data for the city of Seattle - hit "Export" and select "CSV" to download it as a CSV file. For more on how this works, see Running Datasette on Glitch . | ["Getting started"] | [{"href": "https://glitch.com/", "label": "Glitch"}, {"href": "https://glitch.com/~datasette-csvs", "label": "glitch.com/~datasette-csvs"}, {"href": "https://glitch.com/edit/#!/remix/datasette-csvs", "label": null}, {"href": "https://github.com/simonw/sqlite-utils", "label": "sqlite-utils"}, {"href": "https://data.seattle.gov/Community/Public-Art-Data/j7sn-tdzk", "label": "Public Art Data"}, {"href": "https://simonwillison.net/2019/Apr/23/datasette-glitch/", "label": "Running Datasette on Glitch"}] |
getting_started:getting-started-demo | getting_started | getting-started-demo | Play with a live demo | The best way to experience Datasette for the first time is with a demo: global-power-plants.datasettes.com provides a searchable database of power plants around the world, using data from the World Resources Institude rendered using the datasette-cluster-map plugin. fivethirtyeight.datasettes.com shows Datasette running against over 400 datasets imported from the FiveThirtyEight GitHub repository . | ["Getting started"] | [{"href": "https://global-power-plants.datasettes.com/global-power-plants/global-power-plants", "label": "global-power-plants.datasettes.com"}, {"href": "https://www.wri.org/publication/global-power-plant-database", "label": "World Resources Institude"}, {"href": "https://github.com/simonw/datasette-cluster-map", "label": "datasette-cluster-map"}, {"href": "https://fivethirtyeight.datasettes.com/fivethirtyeight", "label": "fivethirtyeight.datasettes.com"}, {"href": "https://github.com/fivethirtyeight/data", "label": "FiveThirtyEight GitHub repository"}] |
deploying:apache-proxy-configuration | deploying | apache-proxy-configuration | Apache proxy configuration | For Apache , you can use the ProxyPass directive. First make sure the following lines are uncommented: LoadModule proxy_module lib/httpd/modules/mod_proxy.so LoadModule proxy_http_module lib/httpd/modules/mod_proxy_http.so Then add these directives to proxy traffic: ProxyPass /my-datasette/ http://127.0.0.1:8009/my-datasette/ ProxyPreserveHost On A live demo of Datasette running behind Apache using this proxy setup can be seen at datasette-apache-proxy-demo.datasette.io/prefix/ . The code for that demo can be found in the demos/apache-proxy directory. Using --uds you can use Unix domain sockets similar to the nginx example: ProxyPass /my-datasette/ unix:/tmp/datasette.sock|http://localhost/my-datasette/ The ProxyPreserveHost On directive ensures that the original Host: header from the incoming request is passed through to Datasette. Datasette needs this to correctly assemble links to other pages using the .absolute_url(request, path) method. | ["Deploying Datasette", "Running Datasette behind a proxy"] | [{"href": "https://httpd.apache.org/", "label": "Apache"}, {"href": "https://datasette-apache-proxy-demo.datasette.io/prefix/", "label": "datasette-apache-proxy-demo.datasette.io/prefix/"}, {"href": "https://github.com/simonw/datasette/tree/main/demos/apache-proxy", "label": "demos/apache-proxy"}, {"href": "https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypreservehost", "label": "ProxyPreserveHost On"}] |
changelog:id67 | changelog | id67 | 0.38 (2020-03-08) | The Docker build of Datasette now uses SQLite 3.31.1, upgraded from 3.26. ( #695 ) datasette publish cloudrun now accepts an optional --memory=2Gi flag for setting the Cloud Run allocated memory to a value other than the default (256Mi). ( #694 ) Fixed bug where templates that shipped with plugins were sometimes not being correctly loaded. ( #697 ) | ["Changelog"] | [{"href": "https://hub.docker.com/r/datasetteproject/datasette", "label": "Docker build"}, {"href": "https://github.com/simonw/datasette/issues/695", "label": "#695"}, {"href": "https://github.com/simonw/datasette/issues/694", "label": "#694"}, {"href": "https://github.com/simonw/datasette/issues/697", "label": "#697"}] |
changelog:id54 | changelog | id54 | 0.47.2 (2020-08-12) | Fixed an issue with the Docker image published to Docker Hub . ( #931 ) | ["Changelog"] | [{"href": "https://hub.docker.com/r/datasetteproject/datasette", "label": "published to Docker Hub"}, {"href": "https://github.com/simonw/datasette/issues/931", "label": "#931"}] |
installation:installation-docker | installation | installation-docker | Using Docker | A Docker image containing the latest release of Datasette is published to Docker Hub here: https://hub.docker.com/r/datasetteproject/datasette/ If you have Docker installed (for example with Docker for Mac on OS X) you can download and run this image like so: docker run -p 8001:8001 -v `pwd`:/mnt \ datasetteproject/datasette \ datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db This will start an instance of Datasette running on your machine's port 8001, serving the fixtures.db file in your current directory. Now visit http://127.0.0.1:8001/ to access Datasette. (You can download a copy of fixtures.db from https://latest.datasette.io/fixtures.db ) To upgrade to the most recent release of Datasette, run the following: docker pull datasetteproject/datasette | ["Installation", "Advanced installation options"] | [{"href": "https://hub.docker.com/r/datasetteproject/datasette/", "label": "https://hub.docker.com/r/datasetteproject/datasette/"}, {"href": "https://www.docker.com/docker-mac", "label": "Docker for Mac"}, {"href": "http://127.0.0.1:8001/", "label": "http://127.0.0.1:8001/"}, {"href": "https://latest.datasette.io/fixtures.db", "label": "https://latest.datasette.io/fixtures.db"}] |
internals:datasette-sign | internals | datasette-sign | .sign(value, namespace="default") | value - any serializable type The value to be signed. namespace - string, optional An alternative namespace, see the itsdangerous salt documentation . Utility method for signing values, such that you can safely pass data to and from an untrusted environment. This is a wrapper around the itsdangerous library. This method returns a signed string, which can be decoded and verified using .unsign(value, namespace="default") . | ["Internals for plugins", "Datasette class"] | [{"href": "https://itsdangerous.palletsprojects.com/en/1.1.x/serializer/#the-salt", "label": "itsdangerous salt documentation"}, {"href": "https://itsdangerous.palletsprojects.com/", "label": "itsdangerous"}] |
changelog:id71 | changelog | id71 | 0.35 (2020-02-04) | Added five new plugins and one new conversion tool to the The Datasette Ecosystem . The Datasette class has a new render_template() method which can be used by plugins to render templates using Datasette's pre-configured Jinja templating library. You can now execute SQL queries that start with a -- comment - thanks, Jay Graves ( #653 ) | ["Changelog"] | [{"href": "https://jinja.palletsprojects.com/", "label": "Jinja"}, {"href": "https://github.com/simonw/datasette/pull/653", "label": "#653"}] |
changelog:id74 | changelog | id74 | 0.32 (2019-11-14) | Datasette now renders templates using Jinja async mode . This means plugins can provide custom template functions that perform asynchronous actions, for example the new datasette-template-sql plugin which allows custom templates to directly execute SQL queries and render their results. ( #628 ) | ["Changelog"] | [{"href": "https://jinja.palletsprojects.com/en/2.10.x/api/#async-support", "label": "Jinja async mode"}, {"href": "https://github.com/simonw/datasette-template-sql", "label": "datasette-template-sql"}, {"href": "https://github.com/simonw/datasette/issues/628", "label": "#628"}] |
plugin_hooks:plugin-hook-extra-template-vars | plugin_hooks | plugin-hook-extra-template-vars | extra_template_vars(template, database, table, columns, view_name, request, datasette) | Extra template variables that should be made available in the rendered template context. template - string The template that is being rendered, e.g. database.html database - string or None The name of the database, or None if the page does not correspond to a database (e.g. the root page) table - string or None The name of the table, or None if the page does not correct to a table columns - list of strings or None The names of the database columns that will be displayed on this page. None if the page does not contain a table. view_name - string The name of the view being displayed. ( index , database , table , and row are the most important ones.) request - Request object or None The current HTTP request. This can be None if the request object is not available. datasette - Datasette class You can use this to access plugin configuration options via datasette.plugin_config(your_plugin_name) This… | ["Plugin hooks", "Page extras"] | [{"href": "https://jinja.palletsprojects.com/en/2.10.x/api/#async-support", "label": "async mode"}, {"href": "https://datasette.io/plugins/datasette-search-all", "label": "datasette-search-all"}, {"href": "https://datasette.io/plugins/datasette-template-sql", "label": "datasette-template-sql"}] |
internals:datasette-render-template | internals | datasette-render-template | await .render_template(template, context=None, request=None) | template - string, list of strings or jinja2.Template The template file to be rendered, e.g. my_plugin.html . Datasette will search for this file first in the --template-dir= location, if it was specified - then in the plugin's bundled templates and finally in Datasette's set of default templates. If this is a list of template file names then the first one that exists will be loaded and rendered. If this is a Jinja Template object it will be used directly. context - None or a Python dictionary The context variables to pass to the template. request - request object or None If you pass a Datasette request object here it will be made available to the template. Renders a Jinja template using Datasette's preconfigured instance of Jinja and returns the resulting string. The template will have access to Datasette's default template functions and any functions that have been made available by other plugins. | ["Internals for plugins", "Datasette class"] | [{"href": "https://jinja.palletsprojects.com/en/2.11.x/api/#jinja2.Template", "label": "Template object"}, {"href": "https://jinja.palletsprojects.com/en/2.11.x/", "label": "Jinja template"}] |
plugin_hooks:plugin-hook-jinja2-environment-from-request | plugin_hooks | plugin-hook-jinja2-environment-from-request | jinja2_environment_from_request(datasette, request, env) | datasette - Datasette class A Datasette instance. request - Request object or None The current HTTP request, if one is available. env - Environment The Jinja2 environment that will be used to render the current page. This hook can be used to return a customized Jinja environment based on the incoming request. If you want to run a single Datasette instance that serves different content for different domains, you can do so like this: from datasette import hookimpl from jinja2 import ChoiceLoader, FileSystemLoader @hookimpl def jinja2_environment_from_request(request, env): if request and request.host == "www.niche-museums.com": return env.overlay( loader=ChoiceLoader( [ FileSystemLoader( "/mnt/niche-museums/templates" ), env.loader, ] ), enable_async=True, ) return env This uses the Jinja overlay() method to create a new environment identical to the default environment except for having a different template loader, which first looks in the /mnt/niche-museums/templates directory before falling back on the default loader. | ["Plugin hooks"] | [{"href": "https://jinja.palletsprojects.com/en/3.0.x/api/#jinja2.Environment", "label": "Jinja environment"}, {"href": "https://jinja.palletsprojects.com/en/3.0.x/api/#jinja2.Environment.overlay", "label": "overlay() method"}] |
changelog:latest-datasette-io | changelog | latest-datasette-io | latest.datasette.io | Every commit to Datasette master is now automatically deployed by Travis CI to https://latest.datasette.io/ - ensuring there is always a live demo of the latest version of the software. The demo uses the fixtures from our unit tests, ensuring it demonstrates the same range of functionality that is covered by the tests. You can see how the deployment mechanism works in our .travis.yml file. | ["Changelog", "0.23 (2018-06-18)"] | [{"href": "https://latest.datasette.io/", "label": "https://latest.datasette.io/"}, {"href": "https://github.com/simonw/datasette/blob/master/tests/fixtures.py", "label": "the fixtures"}, {"href": "https://github.com/simonw/datasette/blob/master/.travis.yml", "label": ".travis.yml"}] |
authentication:permissions-view-instance | authentication | permissions-view-instance | view-instance | Top level permission - Actor is allowed to view any pages within this instance, starting at https://latest.datasette.io/ Default allow . | ["Authentication and permissions", "Built-in permissions"] | [{"href": "https://latest.datasette.io/", "label": "https://latest.datasette.io/"}] |
contributing:contributing-continuous-deployment | contributing | contributing-continuous-deployment | Continuously deployed demo instances | The demo instance at latest.datasette.io is re-deployed automatically to Google Cloud Run for every push to main that passes the test suite. This is implemented by the GitHub Actions workflow at .github/workflows/deploy-latest.yml . Specific branches can also be set to automatically deploy by adding them to the on: push: branches block at the top of the workflow YAML file. Branches configured in this way will be deployed to a new Cloud Run service whether or not their tests pass. The Cloud Run URL for a branch demo can be found in the GitHub Actions logs. | ["Contributing"] | [{"href": "https://latest.datasette.io/", "label": "latest.datasette.io"}, {"href": "https://github.com/simonw/datasette/blob/main/.github/workflows/deploy-latest.yml", "label": ".github/workflows/deploy-latest.yml"}] |
authentication:allowdebugview | authentication | allowdebugview | The /-/allow-debug tool | The /-/allow-debug tool lets you try out different "action" blocks against different "actor" JSON objects. You can try that out here: https://latest.datasette.io/-/allow-debug | ["Authentication and permissions", "Permissions"] | [{"href": "https://latest.datasette.io/-/allow-debug", "label": "https://latest.datasette.io/-/allow-debug"}] |
authentication:authentication-permissions-allow | authentication | authentication-permissions-allow | Defining permissions with "allow" blocks | The standard way to define permissions in Datasette is to use an "allow" block in the datasette.yaml file . This is a JSON document describing which actors are allowed to perform a permission. The most basic form of allow block is this ( allow demo , deny demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: id: root """).strip(), "YAML", "JSON" ) ]]] [[[end]]] This will match any actors with an "id" property of "root" - for example, an actor that looks like this: { "id": "root", "name": "Root User" } An allow block can specify "deny all" using false ( demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: false """).strip(), "YAML", "JSON" ) ]]] [[[end]]] An "allow" of true allows all access ( demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: true """).strip(), "YAML", "JSON" ) ]]] [[[end]]] Allow keys can provide a list of values. These will match any actor that has any of those values ( allow demo , deny demo ): [[[cog from metadata_doc import config_example import textwrap config_example(cog, textwrap.dedent( """ allow: id: - simon - cleopaws """).strip(), "YAML", "JSON" ) ]]] [[[end]]] This will match any actor with an "id" of either "simon" or "cleopaws" . Actors can have properties that feature a list of values. These will be matched against the list of values in an allow block. Consider the following actor: { "id": "simon"… | ["Authentication and permissions", "Permissions"] | [{"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%22id%22%3A+%22root%22%7D&allow=%7B%0D%0A++++++++%22id%22%3A+%22root%22%0D%0A++++%7D", "label": "allow demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%22id%22%3A+%22trevor%22%7D&allow=%7B%0D%0A++++++++%22id%22%3A+%22root%22%0D%0A++++%7D", "label": "deny demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22root%22%0D%0A%7D&allow=false", "label": "demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22root%22%0D%0A%7D&allow=true", "label": "demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22cleopaws%22%0D%0A%7D&allow=%7B%0D%0A++++%22id%22%3A+%5B%0D%0A++++++++%22simon%22%2C%0D%0A++++++++%22cleopaws%22%0D%0A++++%5D%0D%0A%7D", "label": "allow demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22pancakes%22%0D%0A%7D&allow=%7B%0D%0A++++%22id%22%3A+%5B%0D%0A++++++++%22simon%22%2C%0D%0A++++++++%22cleopaws%22%0D%0A++++%5D%0D%0A%7D", "label": "deny demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22simon%22%2C%0D%0A++++%22roles%22%3A+%5B%0D%0A++++++++%22staff%22%2C%0D%0A++++++++%22developer%22%0D%0A++++%5D%0D%0A%7D&allow=%7B%0D%0A++++%22roles%22%3A+%5B%0D%0A++++++++%22developer%22%0D%0A++++%5D%0D%0A%7D", "label": "allow demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22cleopaws%22%2C%0D%0A++++%22roles%22%3A+%5B%22dog%22%5D%0D%0A%7D&allow=%7B%0D%0A++++%22roles%22%3A+%5B%0D%0A++++++++%22developer%22%0D%0A++++%5D%0D%0A%7D", "label": "deny demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22id%22%3A+%22simon%22%0D%0A%7D&allow=%7B%0D%0A++++%22id%22%3A+%22*%22%0D%0A%7D", "label": "allow demo"}, {"href": "https://latest.datasette.io/-/allow-debug?actor=%7B%0D%0A++++%22bot%22%3A+%22readme-bot%22%0D%0A%7D&allow=%7B%0D%0A++++%22id%22%3A+%22*%22%0D%0A%7D", "label": "deny demo"… |
introspection:jsondataview-config | introspection | jsondataview-config | /-/config | Shows the configuration for this instance of Datasette. This is generally the contents of the datasette.yaml or datasette.json file, which can include plugin configuration as well. Config example : { "settings": { "template_debug": true, "trace_debug": true, "force_https_urls": true } } Any keys that include the one of the following substrings in their names will be returned as redacted *** output, to help avoid accidentally leaking private configuration information: secret , key , password , token , hash , dsn . | ["Introspection"] | [{"href": "https://latest.datasette.io/-/config", "label": "Config example"}] |
introspection:jsondataview-databases | introspection | jsondataview-databases | /-/databases | Shows currently attached databases. Databases example : [ { "hash": null, "is_memory": false, "is_mutable": true, "name": "fixtures", "path": "fixtures.db", "size": 225280 } ] | ["Introspection"] | [{"href": "https://latest.datasette.io/-/databases", "label": "Databases example"}] |
introspection:jsondataview-threads | introspection | jsondataview-threads | /-/threads | Shows details of threads and asyncio tasks. Threads example : { "num_threads": 2, "threads": [ { "daemon": false, "ident": 4759197120, "name": "MainThread" }, { "daemon": true, "ident": 123145319682048, "name": "Thread-1" }, ], "num_tasks": 3, "tasks": [ "<Task pending coro=<RequestResponseCycle.run_asgi() running at uvicorn/protocols/http/httptools_impl.py:385> cb=[set.discard()]>", "<Task pending coro=<Server.serve() running at uvicorn/main.py:361> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10365c3d0>()]> cb=[run_until_complete.<locals>.<lambda>()]>", "<Task pending coro=<LifespanOn.main() running at uvicorn/lifespan/on.py:48> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10364f050>()]>>" ] } | ["Introspection"] | [{"href": "https://latest.datasette.io/-/threads", "label": "Threads example"}] |
introspection:jsondataview-versions | introspection | jsondataview-versions | /-/versions | Shows the version of Datasette, Python and SQLite. Versions example : { "datasette": { "version": "0.60" }, "python": { "full": "3.8.12 (default, Dec 21 2021, 10:45:09) \n[GCC 10.2.1 20210110]", "version": "3.8.12" }, "sqlite": { "extensions": { "json1": null }, "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "compile_options": [ "COMPILER=gcc-6.3.0 20170516", "ENABLE_FTS3", "ENABLE_FTS4", "ENABLE_FTS5", "ENABLE_JSON1", "ENABLE_RTREE", "THREADSAFE=1" ], "version": "3.37.0" } } | ["Introspection"] | [{"href": "https://latest.datasette.io/-/versions", "label": "Versions example"}] |
settings:setting-template-debug | settings | setting-template-debug | template_debug | This setting enables template context debug mode, which is useful to help understand what variables are available to custom templates when you are writing them. Enable it like this: datasette mydatabase.db --setting template_debug 1 Now you can add ?_context=1 or &_context=1 to any Datasette page to see the context that was passed to that template. Some examples: https://latest.datasette.io/?_context=1 https://latest.datasette.io/fixtures?_context=1 https://latest.datasette.io/fixtures/roadside_attractions?_context=1 | ["Settings", "Settings"] | [{"href": "https://latest.datasette.io/?_context=1", "label": "https://latest.datasette.io/?_context=1"}, {"href": "https://latest.datasette.io/fixtures?_context=1", "label": "https://latest.datasette.io/fixtures?_context=1"}, {"href": "https://latest.datasette.io/fixtures/roadside_attractions?_context=1", "label": "https://latest.datasette.io/fixtures/roadside_attractions?_context=1"}] |
settings:setting-trace-debug | settings | setting-trace-debug | trace_debug | This setting enables appending ?_trace=1 to any page in order to see the SQL queries and other trace information that was used to generate that page. Enable it like this: datasette mydatabase.db --setting trace_debug 1 Some examples: https://latest.datasette.io/?_trace=1 https://latest.datasette.io/fixtures/roadside_attractions?_trace=1 See datasette.tracer for details on how to hook into this mechanism as a plugin author. | ["Settings", "Settings"] | [{"href": "https://latest.datasette.io/?_trace=1", "label": "https://latest.datasette.io/?_trace=1"}, {"href": "https://latest.datasette.io/fixtures/roadside_attractions?_trace=1", "label": "https://latest.datasette.io/fixtures/roadside_attractions?_trace=1"}] |
sql_queries:id3 | sql_queries | id3 | Cross-database queries | SQLite has the ability to run queries that join across multiple databases. Up to ten databases can be attached to a single SQLite connection and queried together. Datasette can execute joins across multiple databases if it is started with the --crossdb option: datasette fixtures.db extra_database.db --crossdb If it is started in this way, the /_memory page can be used to execute queries that join across multiple databases. References to tables in attached databases should be preceded by the database name and a period. For example, this query will show a list of tables across both of the above databases: select 'fixtures' as database, * from [fixtures].sqlite_master union select 'extra_database' as database, * from [extra_database].sqlite_master Try that out here . | ["Running SQL queries"] | [{"href": "https://latest.datasette.io/_memory?sql=select%0D%0A++%27fixtures%27+as+database%2C+*%0D%0Afrom%0D%0A++%5Bfixtures%5D.sqlite_master%0D%0Aunion%0D%0Aselect%0D%0A++%27extra_database%27+as+database%2C+*%0D%0Afrom%0D%0A++%5Bextra_database%5D.sqlite_master", "label": "Try that out here"}] |
authentication:permissions-view-database | authentication | permissions-view-database | view-database | Actor is allowed to view a database page, e.g. https://latest.datasette.io/fixtures resource - string The name of the database Default allow . | ["Authentication and permissions", "Built-in permissions"] | [{"href": "https://latest.datasette.io/fixtures", "label": "https://latest.datasette.io/fixtures"}] |
authentication:authentication-permissions-execute-sql | authentication | authentication-permissions-execute-sql | Controlling the ability to execute arbitrary SQL | Datasette defaults to allowing any site visitor to execute their own custom SQL queries, for example using the form on the database page or by appending a ?_where= parameter to the table page like this . Access to this ability is controlled by the execute-sql permission. The easiest way to disable arbitrary SQL queries is using the default_allow_sql setting when you first start Datasette running. You can alternatively use an "allow_sql" block to control who is allowed to execute arbitrary SQL queries. To prevent any user from executing arbitrary SQL queries, use this: [[[cog config_example(cog, """ allow_sql: false """) ]]] [[[end]]] To enable just the root user to execute SQL for all databases in your instance, use the following: [[[cog config_example(cog, """ allow_sql: id: root """) ]]] [[[end]]] To limit this ability for just one specific database, use this: [[[cog config_example(cog, """ databases: mydatabase: allow_sql: id: root """) ]]] [[[end]]] | ["Authentication and permissions", "Access permissions in "] | [{"href": "https://latest.datasette.io/fixtures", "label": "the database page"}, {"href": "https://latest.datasette.io/fixtures/facetable?_where=_city_id=1", "label": "like this"}] |
authentication:permissions-view-database-download | authentication | permissions-view-database-download | view-database-download | Actor is allowed to download a database, e.g. https://latest.datasette.io/fixtures.db resource - string The name of the database Default allow . | ["Authentication and permissions", "Built-in permissions"] | [{"href": "https://latest.datasette.io/fixtures.db", "label": "https://latest.datasette.io/fixtures.db"}] |
changelog:miscellaneous | changelog | miscellaneous | Miscellaneous | Got JSON data in one of your columns? Use the new ?_json=COLNAME argument to tell Datasette to return that JSON value directly rather than encoding it as a string. If you just want an array of the first value of each row, use the new ?_shape=arrayfirst option - example . | ["Changelog", "0.23 (2018-06-18)"] | [{"href": "https://latest.datasette.io/fixtures.json?sql=select+neighborhood+from+facetable+order+by+pk+limit+101&_shape=arrayfirst", "label": "example"}] |
binary_data:binary | binary_data | binary | Binary data | SQLite tables can contain binary data in BLOB columns. Datasette includes special handling for these binary values. The Datasette interface detects binary values and provides a link to download their content, for example on https://latest.datasette.io/fixtures/binary_data Binary data is represented in .json exports using Base64 encoding. https://latest.datasette.io/fixtures/binary_data.json?_shape=array [ { "rowid": 1, "data": { "$base64": true, "encoded": "FRwCx60F/g==" } }, { "rowid": 2, "data": { "$base64": true, "encoded": "FRwDx60F/g==" } }, { "rowid": 3, "data": null } ] | [] | [{"href": "https://latest.datasette.io/fixtures/binary_data", "label": "https://latest.datasette.io/fixtures/binary_data"}, {"href": "https://latest.datasette.io/fixtures/binary_data.json?_shape=array", "label": "https://latest.datasette.io/fixtures/binary_data.json?_shape=array"}] |
binary_data:binary-linking | binary_data | binary-linking | Linking to binary downloads | The .blob output format is used to return binary data. It requires a _blob_column= query string argument specifying which BLOB column should be downloaded, for example: https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data This output format can also be used to return binary data from an arbitrary SQL query. Since such queries do not specify an exact row, an additional ?_blob_hash= parameter can be used to specify the SHA-256 hash of the value that is being linked to. Consider the query select data from binary_data - demonstrated here . That page links to the binary value downloads. Those links look like this: https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d These .blob links are also returned in the .csv exports Datasette provides for binary tables and queries, since the CSV format does not have a mechanism for representing binary data. | ["Binary data"] | [{"href": "https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data", "label": "https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data"}, {"href": "https://latest.datasette.io/fixtures?sql=select+data+from+binary_data", "label": "demonstrated here"}, {"href": "https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d", "label": "https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d"}] |
authentication:permissions-view-table | authentication | permissions-view-table | view-table | Actor is allowed to view a table (or view) page, e.g. https://latest.datasette.io/fixtures/complex_foreign_keys resource - tuple: (string, string) The name of the database, then the name of the table Default allow . | ["Authentication and permissions", "Built-in permissions"] | [{"href": "https://latest.datasette.io/fixtures/complex_foreign_keys", "label": "https://latest.datasette.io/fixtures/complex_foreign_keys"}] |
csv_export:id1 | csv_export | id1 | CSV export | Any Datasette table, view or custom SQL query can be exported as CSV. To obtain the CSV representation of the table you are looking, click the "this data as CSV" link. You can also use the advanced export form for more control over the resulting file, which looks like this and has the following options: download file - instead of displaying CSV in your browser, this forces your browser to download the CSV to your downloads directory. expand labels - if your table has any foreign key references this option will cause the CSV to gain additional COLUMN_NAME_label columns with a label for each foreign key derived from the linked table. In this example the city_id column is accompanied by a city_id_label column. stream all rows - by default CSV files only contain the first max_returned_rows records. This option will cause Datasette to loop through every matching record and return them as a single CSV file. You can try that out on https://latest.datasette.io/fixtures/facetable?_size=4 | [] | [{"href": "https://latest.datasette.io/fixtures/facetable.csv?_labels=on&_size=max", "label": "In this example"}, {"href": "https://latest.datasette.io/fixtures/facetable?_size=4", "label": "https://latest.datasette.io/fixtures/facetable?_size=4"}] |
facets:id2 | facets | id2 | Facet by JSON array | If your SQLite installation provides the json1 extension (you can check using /-/versions ) Datasette will automatically detect columns that contain JSON arrays of values and offer a faceting interface against those columns. This is useful for modelling things like tags without needing to break them out into a new table. Example here: latest.datasette.io/fixtures/facetable?_facet_array=tags | ["Facets"] | [{"href": "https://latest.datasette.io/fixtures/facetable?_facet_array=tags", "label": "latest.datasette.io/fixtures/facetable?_facet_array=tags"}] |
facets:id3 | facets | id3 | Facet by date | If Datasette finds any columns that contain dates in the first 100 values, it will offer a faceting interface against the dates of those values. This works especially well against timestamp values such as 2019-03-01 12:44:00 . Example here: latest.datasette.io/fixtures/facetable?_facet_date=created | ["Facets"] | [{"href": "https://latest.datasette.io/fixtures/facetable?_facet_date=created", "label": "latest.datasette.io/fixtures/facetable?_facet_date=created"}] |
internals:internals-tracer | internals | internals-tracer | datasette.tracer | Running Datasette with --setting trace_debug 1 enables trace debug output, which can then be viewed by adding ?_trace=1 to the query string for any page. You can see an example of this at the bottom of latest.datasette.io/fixtures/facetable?_trace=1 . The JSON output shows full details of every SQL query that was executed to generate the page. The datasette-pretty-traces plugin can be installed to provide a more readable display of this information. You can see a demo of that here . You can add your own custom traces to the JSON output using the trace() context manager. This takes a string that identifies the type of trace being recorded, and records any keyword arguments as additional JSON keys on the resulting trace object. The start and end time, duration and a traceback of where the trace was executed will be automatically attached to the JSON object. This example uses trace to record the start, end and duration of any HTTP GET requests made using the function: from datasette.tracer import trace import httpx async def fetch_url(url): with trace("fetch-url", url=url): async with httpx.AsyncClient() as client: return await client.get(url) | ["Internals for plugins"] | [{"href": "https://latest.datasette.io/fixtures/facetable?_trace=1", "label": "latest.datasette.io/fixtures/facetable?_trace=1"}, {"href": "https://datasette.io/plugins/datasette-pretty-traces", "label": "datasette-pretty-traces"}, {"href": "https://latest-with-plugins.datasette.io/github/commits?_trace=1", "label": "a demo of that here"}] |
changelog:writable-canned-queries | changelog | writable-canned-queries | Writable canned queries | Datasette's Canned queries feature lets you define SQL queries in metadata.json which can then be executed by users visiting a specific URL. https://latest.datasette.io/fixtures/neighborhood_search for example. Canned queries were previously restricted to SELECT , but Datasette 0.44 introduces the ability for canned queries to execute INSERT or UPDATE queries as well, using the new "write": true property ( #800 ): { "databases": { "dogs": { "queries": { "add_name": { "sql": "INSERT INTO names (name) VALUES (:name)", "write": true } } } } } See Writable canned queries for more details. | ["Changelog", "0.44 (2020-06-11)"] | [{"href": "https://latest.datasette.io/fixtures/neighborhood_search", "label": "https://latest.datasette.io/fixtures/neighborhood_search"}, {"href": "https://github.com/simonw/datasette/issues/800", "label": "#800"}] |
sql_queries:canned-queries-named-parameters | sql_queries | canned-queries-named-parameters | Canned query parameters | Canned queries support named parameters, so if you include those in the SQL you will then be able to enter them using the form fields on the canned query page or by adding them to the URL. This means canned queries can be used to create custom JSON APIs based on a carefully designed SQL statement. Here's an example of a canned query with a named parameter: select neighborhood, facet_cities.name, state from facetable join facet_cities on facetable.city_id = facet_cities.id where neighborhood like '%' || :text || '%' order by neighborhood; In the canned query configuration looks like this: [[[cog config_example(cog, """ databases: fixtures: queries: neighborhood_search: title: Search neighborhoods sql: |- select neighborhood, facet_cities.name, state from facetable join facet_cities on facetable.city_id = facet_cities.id where neighborhood like '%' || :text || '%' order by neighborhood """) ]]] [[[end]]] Note that we are using SQLite string concatenation here - the || operator - to add wildcard % characters to the string provided by the user. You can try this canned query out here: https://latest.datasette.io/fixtures/neighborhood_search?text=town In this example the :text named parameter is automatically extracted from the query using a regular expression. You can alternatively provide an explicit list of named parameters using the "params" key, like this: [[[cog config_example(cog, """ databases: fixtures: queries: neighborhood_search: title: Search neighborhoods params: - text sql: |- select neighborhood, facet_cities.name, state from facetable join facet_cities on facetable.city_id = facet_cities.id where neighborhood like … | ["Running SQL queries", "Canned queries"] | [{"href": "https://latest.datasette.io/fixtures/neighborhood_search?text=town", "label": "https://latest.datasette.io/fixtures/neighborhood_search?text=town"}] |
authentication:permissions-view-query | authentication | permissions-view-query | view-query | Actor is allowed to view (and execute) a canned query page, e.g. https://latest.datasette.io/fixtures/pragma_cache_size - this includes executing Writable canned queries . resource - tuple: (string, string) The name of the database, then the name of the canned query Default allow . | ["Authentication and permissions", "Built-in permissions"] | [{"href": "https://latest.datasette.io/fixtures/pragma_cache_size", "label": "https://latest.datasette.io/fixtures/pragma_cache_size"}] |
metadata:metadata-column-descriptions | metadata | metadata-column-descriptions | Column descriptions | You can include descriptions for your columns by adding a "columns": {"name-of-column": "description-of-column"} block to your table metadata: [[[cog metadata_example(cog, { "databases": { "database1": { "tables": { "example_table": { "columns": { "column1": "Description of column 1", "column2": "Description of column 2" } } } } } }) ]]] [[[end]]] These will be displayed at the top of the table page, and will also show in the cog menu for each column. You can see an example of how these look at latest.datasette.io/fixtures/roadside_attractions . | ["Metadata"] | [{"href": "https://latest.datasette.io/fixtures/roadside_attractions", "label": "latest.datasette.io/fixtures/roadside_attractions"}] |
changelog:through-for-joins-through-many-to-many-tables | changelog | through-for-joins-through-many-to-many-tables | ?_through= for joins through many-to-many tables | The new ?_through={json} argument to the Table view allows records to be filtered based on a many-to-many relationship. See Special table arguments for full documentation - here's an example . ( #355 ) This feature was added to help support facet by many-to-many , which isn't quite ready yet but will be coming in the next Datasette release. | ["Changelog", "0.29 (2019-07-07)"] | [{"href": "https://latest.datasette.io/fixtures/roadside_attractions?_through={%22table%22:%22roadside_attraction_characteristics%22,%22column%22:%22characteristic_id%22,%22value%22:%221%22}", "label": "an example"}, {"href": "https://github.com/simonw/datasette/issues/355", "label": "#355"}, {"href": "https://github.com/simonw/datasette/issues/551", "label": "facet by many-to-many"}] |
full_text_search:full-text-search-table-or-view | full_text_search | full-text-search-table-or-view | Configuring full-text search for a table or view | If a table has a corresponding FTS table set up using the content= argument to CREATE VIRTUAL TABLE shown below, Datasette will detect it automatically and add a search interface to the table page for that table. You can also manually configure which table should be used for full-text search using query string parameters or Metadata . You can set the associated FTS table for a specific table and you can also set one for a view - if you do that, the page for that SQL view will offer a search option. Use ?_fts_table=x to over-ride the FTS table for a specific page. If the primary key was something other than rowid you can use ?_fts_pk=col to set that as well. This is particularly useful for views, for example: https://latest.datasette.io/fixtures/searchable_view?_fts_table=searchable_fts&_fts_pk=pk The fts_table metadata property can be used to specify an associated FTS table. If the primary key column in your table which was used to populate the FTS table is something other than rowid , you can specify the column to use with the fts_pk property. The "searchmode": "raw" property can be used to default the table to accepting SQLite advanced search operators, as described in Advanced SQLite search queries . Here is an example which enables full-text search (with SQLite advanced search operators) for a display_ads view which is defined against the ads table and hence needs to run FTS against the ads_fts table, using the id as the primary key: [[[cog from metadata_doc import metadata_example metadata_example(cog, { "databases": { "russian-ads": { "tables": { "display_ads": { "fts_table": "ads_fts", "fts_pk": "id", "searchmode": "raw" } } } } }) ]]] [[[end]]] | ["Full-text search"] | [{"href": "https://latest.datasette.io/fixtures/searchable_view?_fts_table=searchable_fts&_fts_pk=pk", "label": "https://latest.datasette.io/fixtures/searchable_view?_fts_table=searchable_fts&_fts_pk=pk"}] |
authentication:authentication-permissions-table | authentication | authentication-permissions-table | Access to specific tables and views | To limit access to the users table in your bakery.db database: [[[cog config_example(cog, """ databases: bakery: tables: users: allow: id: '*' """) ]]] [[[end]]] This works for SQL views as well - you can list their names in the "tables" block above in the same way as regular tables. Restricting access to tables and views in this way will NOT prevent users from querying them using arbitrary SQL queries, like this for example. If you are restricting access to specific tables you should also use the "allow_sql" block to prevent users from bypassing the limit with their own SQL queries - see Controlling the ability to execute arbitrary SQL . | ["Authentication and permissions", "Access permissions in "] | [{"href": "https://latest.datasette.io/fixtures?sql=select+*+from+facetable", "label": "like this"}] |
authentication:permissions-execute-sql | authentication | permissions-execute-sql | execute-sql | Actor is allowed to run arbitrary SQL queries against a specific database, e.g. https://latest.datasette.io/fixtures?sql=select+100 resource - string The name of the database Default allow . See also the default_allow_sql setting . | ["Authentication and permissions", "Built-in permissions"] | [{"href": "https://latest.datasette.io/fixtures?sql=select+100", "label": "https://latest.datasette.io/fixtures?sql=select+100"}] |
changelog:id13 | changelog | id13 | 0.62 (2022-08-14) | Datasette can now run entirely in your browser using WebAssembly. Try out Datasette Lite , take a look at the code or read more about it in Datasette Lite: a server-side Python web application running in a browser . Datasette now has a Discord community for questions and discussions about Datasette and its ecosystem of projects. | ["Changelog"] | [{"href": "https://lite.datasette.io/", "label": "Datasette Lite"}, {"href": "https://github.com/simonw/datasette-lite", "label": "at the code"}, {"href": "https://simonwillison.net/2022/May/4/datasette-lite/", "label": "Datasette Lite: a server-side Python web application running in a browser"}, {"href": "https://datasette.io/discord", "label": "Discord community"}] |
getting_started:getting-started-datasette-lite | getting_started | getting-started-datasette-lite | Datasette in your browser with Datasette Lite | Datasette Lite is Datasette packaged using WebAssembly so that it runs entirely in your browser, no Python web application server required. You can pass a URL to a CSV, SQLite or raw SQL file directly to Datasette Lite to explore that data in your browser. This example link opens Datasette Lite and loads the SQL Murder Mystery example database from Northwestern University Knight Lab . | ["Getting started"] | [{"href": "https://lite.datasette.io/", "label": "Datasette Lite"}, {"href": "https://lite.datasette.io/?url=https%3A%2F%2Fraw.githubusercontent.com%2FNUKnightLab%2Fsql-mysteries%2Fmaster%2Fsql-murder-mystery.db#/sql-murder-mystery", "label": "example link"}, {"href": "https://github.com/NUKnightLab/sql-mysteries", "label": "Northwestern University Knight Lab"}] |
deploying:nginx-proxy-configuration | deploying | nginx-proxy-configuration | Nginx proxy configuration | Here is an example of an nginx configuration file that will proxy traffic to Datasette: daemon off; events { worker_connections 1024; } http { server { listen 80; location /my-datasette { proxy_pass http://127.0.0.1:8009/my-datasette; proxy_set_header Host $host; } } } You can also use the --uds option to Datasette to listen on a Unix domain socket instead of a port, configuring the nginx upstream proxy like this: daemon off; events { worker_connections 1024; } http { server { listen 80; location /my-datasette { proxy_pass http://datasette/my-datasette; proxy_set_header Host $host; } } upstream datasette { server unix:/tmp/datasette.sock; } } Then run Datasette with datasette --uds /tmp/datasette.sock path/to/database.db --setting base_url /my-datasette/ . | ["Deploying Datasette", "Running Datasette behind a proxy"] | [{"href": "https://nginx.org/", "label": "nginx"}] |
contributing:contributing-formatting-prettier | contributing | contributing-formatting-prettier | Prettier | To install Prettier, install Node.js and then run the following in the root of your datasette repository checkout: npm install This will install Prettier in a node_modules directory. You can then check that your code matches the coding style like so: npm run prettier -- --check > prettier > prettier 'datasette/static/*[!.min].js' "--check" Checking formatting... [warn] datasette/static/plugins.js [warn] Code style issues found in the above file(s). Forgot to run Prettier? You can fix any problems by running: npm run fix | ["Contributing", "Code formatting"] | [{"href": "https://nodejs.org/en/download/package-manager/", "label": "install Node.js"}] |
changelog:id30 | changelog | id30 | 0.57 (2021-06-05) | This release fixes a reflected cross-site scripting security hole with the ?_trace=1 feature. You should upgrade to this version, or to Datasette 0.56.1, as soon as possible. ( #1360 ) In addition to the security fix, this release includes ?_col= and ?_nocol= options for controlling which columns are displayed for a table, ?_facet_size= for increasing the number of facet results returned, re-display of your SQL query should an error occur and numerous bug fixes. | ["Changelog"] | [{"href": "https://owasp.org/www-community/attacks/xss/#reflected-xss-attacks", "label": "reflected cross-site scripting"}, {"href": "https://github.com/simonw/datasette/issues/1360", "label": "#1360"}] |
changelog:id31 | changelog | id31 | 0.56.1 (2021-06-05) | This release fixes a reflected cross-site scripting security hole with the ?_trace=1 feature. You should upgrade to this version, or to Datasette 0.57, as soon as possible. ( #1360 ) | ["Changelog"] | [{"href": "https://owasp.org/www-community/attacks/xss/#reflected-xss-attacks", "label": "reflected cross-site scripting"}, {"href": "https://github.com/simonw/datasette/issues/1360", "label": "#1360"}] |
metadata:specifying-units-for-a-column | metadata | specifying-units-for-a-column | Specifying units for a column | Datasette supports attaching units to a column, which will be used when displaying values from that column. SI prefixes will be used where appropriate. Column units are configured in the metadata like so: [[[cog metadata_example(cog, { "databases": { "database1": { "tables": { "example_table": { "units": { "column1": "metres", "column2": "Hz" } } } } } }) ]]] [[[end]]] Units are interpreted using Pint , and you can see the full list of available units in Pint's unit registry . You can also add custom units to the metadata, which will be registered with Pint: [[[cog metadata_example(cog, { "custom_units": [ "decibel = [] = dB" ] }) ]]] [[[end]]] | ["Metadata"] | [{"href": "https://pint.readthedocs.io/", "label": "Pint"}, {"href": "https://github.com/hgrecco/pint/blob/master/pint/default_en.txt", "label": "unit registry"}, {"href": "http://pint.readthedocs.io/en/latest/defining.html", "label": "custom units"}] |
installation:installation-pipx | installation | installation-pipx | Using pipx | pipx is a tool for installing Python software with all of its dependencies in an isolated environment, to ensure that they will not conflict with any other installed Python software. If you use Homebrew on macOS you can install pipx like this: brew install pipx pipx ensurepath Without Homebrew you can install it like so: python3 -m pip install --user pipx python3 -m pipx ensurepath The pipx ensurepath command configures your shell to ensure it can find commands that have been installed by pipx - generally by making sure ~/.local/bin has been added to your PATH . Once pipx is installed you can use it to install Datasette like this: pipx install datasette Then run datasette --version to confirm that it has been successfully installed. | ["Installation", "Advanced installation options"] | [{"href": "https://pipxproject.github.io/pipx/", "label": "pipx"}, {"href": "https://brew.sh/", "label": "Homebrew"}] |
plugins:id1 | plugins | id1 | Plugins | Datasette's plugin system allows additional features to be implemented as Python code (or front-end JavaScript) which can be wrapped up in a separate Python package. The underlying mechanism uses pluggy . See the Datasette plugins directory for a list of existing plugins, or take a look at the datasette-plugin topic on GitHub. Things you can do with plugins include: Add visualizations to Datasette, for example datasette-cluster-map and datasette-vega . Make new custom SQL functions available for use within Datasette, for example datasette-haversine and datasette-jellyfish . Define custom output formats with custom extensions, for example datasette-atom and datasette-ics . Add template functions that can be called within your Jinja custom templates, for example datasette-render-markdown . Customize how database values are rendered in the Datasette interface, for example datasette-render-binary and datasette-pretty-json . Customize how Datasette's authentication and permissions systems work, for example datasette-auth-passwords and datasette-permissions-sql . | [] | [{"href": "https://pluggy.readthedocs.io/", "label": "pluggy"}, {"href": "https://datasette.io/plugins", "label": "Datasette plugins directory"}, {"href": "https://github.com/topics/datasette-plugin", "label": "datasette-plugin"}, {"href": "https://github.com/simonw/datasette-cluster-map", "label": "datasette-cluster-map"}, {"href": "https://github.com/simonw/datasette-vega", "label": "datasette-vega"}, {"href": "https://github.com/simonw/datasette-haversine", "label": "datasette-haversine"}, {"href": "https://github.com/simonw/datasette-jellyfish", "label": "datasette-jellyfish"}, {"href": "https://github.com/simonw/datasette-atom", "label": "datasette-atom"}, {"href": "https://github.com/simonw/datasette-ics", "label": "datasette-ics"}, {"href": "https://github.com/simonw/datasette-render-markdown#markdown-in-templates", "label": "datasette-render-markdown"}, {"href": "https://github.com/simonw/datasette-render-binary", "label": "datasette-render-binary"}, {"href": "https://github.com/simonw/datasette-pretty-json", "label": "datasette-pretty-json"}, {"href": "https://github.com/simonw/datasette-auth-passwords", "label": "datasette-auth-passwords"}, {"href": "https://github.com/simonw/datasette-permissions-sql", "label": "datasette-permissions-sql"}] |
plugin_hooks:id1 | plugin_hooks | id1 | Plugin hooks | Datasette plugins use plugin hooks to customize Datasette's behavior. These hooks are powered by the pluggy plugin system. Each plugin can implement one or more hooks using the @hookimpl decorator against a function named that matches one of the hooks documented on this page. When you implement a plugin hook you can accept any or all of the parameters that are documented as being passed to that hook. For example, you can implement the render_cell plugin hook like this even though the full documented hook signature is render_cell(row, value, column, table, database, datasette) : @hookimpl def render_cell(value, column): if column == "stars": return "*" * int(value) List of plugin hooks prepare_connection(conn, database, datasette) prepare_jinja2_environment(env, datasette) Page extras extra_template_vars(template, database, table, columns, view_name, request, datasette) extra_css_urls(template, database, table, columns, view_name, request, datasette) extra_js_urls(template, database, table, columns, view_name, request, datasette) extra_body_script(template, database, table, columns, view_name, request, datasette) publish_subcommand(publish) render_cell(row, value, column, table, database, datasette, request) register_output_re… | [] | [{"href": "https://pluggy.readthedocs.io/", "label": "pluggy"}] |
plugin_hooks:plugin-hook-slots | plugin_hooks | plugin-hook-slots | Template slots | The following set of plugin hooks can be used to return extra HTML content that will be inserted into the corresponding page, directly below the <h1> heading. Multiple plugins can contribute content here. The order in which it is displayed can be controlled using Pluggy's call time order options . Each of these plugin hooks can return either a string or an awaitable function that returns a string. | ["Plugin hooks"] | [{"href": "https://pluggy.readthedocs.io/en/stable/#call-time-order", "label": "call time order options"}] |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [sections] ( [id] TEXT PRIMARY KEY, [page] TEXT, [ref] TEXT, [title] TEXT, [content] TEXT, [breadcrumbs] TEXT, [references] TEXT );