id,page,ref,title,content,breadcrumbs,references changelog:alter-table-support-for-create-insert-upsert-and-update,changelog,alter-table-support-for-create-insert-upsert-and-update,"Alter table support for create, insert, upsert and update","The JSON write API can now be used to apply simple alter table schema changes, provided the acting actor has the new alter-table permission. ( #2101 ) The only alter operation supported so far is adding new columns to an existing table. The /db/-/create API now adds new columns during large operations to create a table based on incoming example ""rows"" , in the case where one of the later rows includes columns that were not present in the earlier batches. This requires the create-table but not the alter-table permission. When /db/-/create is called with rows in a situation where the table may have been already created, an ""alter"": true key can be included to indicate that any missing columns from the new rows should be added to the table. This requires the alter-table permission. /db/table/-/insert and /db/table/-/upsert and /db/table/row-pks/-/update all now also accept ""alter"": true , depending on the alter-table permission. Operations that alter a table now fire the new alter-table event .","[""Changelog"", ""1.0a9 (2024-02-16)""]","[{""href"": ""https://github.com/simonw/datasette/issues/2101"", ""label"": ""#2101""}]" changelog:asgi,changelog,asgi,ASGI,"ASGI is the Asynchronous Server Gateway Interface standard. I've been wanting to convert Datasette into an ASGI application for over a year - Port Datasette to ASGI #272 tracks thirteen months of intermittent development - but with Datasette 0.29 the change is finally released. This also means Datasette now runs on top of Uvicorn and no longer depends on Sanic . I wrote about the significance of this change in Porting Datasette to ASGI, and Turtles all the way down . The most exciting consequence of this change is that Datasette plugins can now take advantage of the ASGI standard.","[""Changelog"", ""0.29 (2019-07-07)""]","[{""href"": ""https://asgi.readthedocs.io/"", ""label"": ""ASGI""}, {""href"": ""https://github.com/simonw/datasette/issues/272"", ""label"": ""Port Datasette to ASGI #272""}, {""href"": ""https://www.uvicorn.org/"", ""label"": ""Uvicorn""}, {""href"": ""https://github.com/huge-success/sanic"", ""label"": ""Sanic""}, {""href"": ""https://simonwillison.net/2019/Jun/23/datasette-asgi/"", ""label"": ""Porting Datasette to ASGI, and Turtles all the way down""}]" changelog:authentication,changelog,authentication,Authentication,"Prior to this release the Datasette ecosystem has treated authentication as exclusively the realm of plugins, most notably through datasette-auth-github . 0.44 introduces Authentication and permissions as core Datasette concepts ( #699 ). This enables different plugins to share responsibility for authenticating requests - you might have one plugin that handles user accounts and another one that allows automated access via API keys, for example. You'll need to install plugins if you want full user accounts, but default Datasette can now authenticate a single root user with the new --root command-line option, which outputs a one-time use URL to authenticate as a root actor ( #784 ): datasette fixtures.db --root http://127.0.0.1:8001/-/auth-token?token=5b632f8cd44b868df625f5a6e2185d88eea5b22237fd3cc8773f107cc4fd6477 INFO: Started server process [14973] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit) Plugins can implement new ways of authenticating users using the new actor_from_request(datasette, request) hook.","[""Changelog"", ""0.44 (2020-06-11)""]","[{""href"": ""https://github.com/simonw/datasette-auth-github"", ""label"": ""datasette-auth-github""}, {""href"": ""https://github.com/simonw/datasette/issues/699"", ""label"": ""#699""}, {""href"": ""https://github.com/simonw/datasette/issues/784"", ""label"": ""#784""}]" changelog:better-plugin-documentation,changelog,better-plugin-documentation,Better plugin documentation,"The plugin documentation has been re-arranged into four sections, including a brand new section on testing plugins. ( #687 ) Plugins introduces Datasette's plugin system and describes how to install and configure plugins. Writing plugins describes how to author plugins, from one-off single file plugins to packaged plugins that can be published to PyPI. It also describes how to start a plugin using the new datasette-plugin cookiecutter template. Plugin hooks is a full list of detailed documentation for every Datasette plugin hook. Testing plugins describes how to write tests for Datasette plugins, using pytest and HTTPX .","[""Changelog"", ""0.45 (2020-07-01)""]","[{""href"": ""https://github.com/simonw/datasette/issues/687"", ""label"": ""#687""}, {""href"": ""https://github.com/simonw/datasette-plugin"", ""label"": ""datasette-plugin""}, {""href"": ""https://docs.pytest.org/"", ""label"": ""pytest""}, {""href"": ""https://www.python-httpx.org/"", ""label"": ""HTTPX""}]" changelog:binary-data,changelog,binary-data,Binary data,"SQLite tables can contain binary data in BLOB columns. Datasette now provides links for users to download this data directly from Datasette, and uses those links to make binary data available from CSV exports. See Binary data for more details. ( #1036 and #1034 ).","[""Changelog"", ""0.51 (2020-10-31)""]","[{""href"": ""https://github.com/simonw/datasette/issues/1036"", ""label"": ""#1036""}, {""href"": ""https://github.com/simonw/datasette/issues/1034"", ""label"": ""#1034""}]" changelog:bug-fixes,changelog,bug-fixes,Bug fixes,"Don't show the facet option in the cog menu if faceting is not allowed. ( #1683 ) ?_sort and ?_sort_desc now work if the column that is being sorted has been excluded from the query using ?_col= or ?_nocol= . ( #1773 ) Fixed bug where ?_sort_desc was duplicated in the URL every time the Apply button was clicked. ( #1738 )","[""Changelog"", ""0.62 (2022-08-14)""]","[{""href"": ""https://github.com/simonw/datasette/issues/1683"", ""label"": ""#1683""}, {""href"": ""https://github.com/simonw/datasette/issues/1773"", ""label"": ""#1773""}, {""href"": ""https://github.com/simonw/datasette/issues/1738"", ""label"": ""#1738""}]" changelog:bug-fixes-and-other-improvements,changelog,bug-fixes-and-other-improvements,Bug fixes and other improvements,"Custom pages now work correctly when combined with the base_url setting. ( #1238 ) Fixed intermittent error displaying the index page when the user did not have permission to access one of the tables. Thanks, Guy Freeman. ( #1305 ) Columns with the name ""Link"" are no longer incorrectly displayed in bold. ( #1308 ) Fixed error caused by tables with a single quote in their names. ( #1257 ) Updated dependencies: pytest-asyncio , Black , jinja2 , aiofiles , click , and itsdangerous . The official Datasette Docker image now supports apt-get install . ( #1320 ) The Heroku runtime used by datasette publish heroku is now python-3.8.10 .","[""Changelog"", ""0.57 (2021-06-05)""]","[{""href"": ""https://github.com/simonw/datasette/issues/1238"", ""label"": ""#1238""}, {""href"": ""https://github.com/simonw/datasette/issues/1305"", ""label"": ""#1305""}, {""href"": ""https://github.com/simonw/datasette/issues/1308"", ""label"": ""#1308""}, {""href"": ""https://github.com/simonw/datasette/issues/1257"", ""label"": ""#1257""}, {""href"": ""https://github.com/simonw/datasette/issues/1320"", ""label"": ""#1320""}]" changelog:code-formatting-with-black-and-prettier,changelog,code-formatting-with-black-and-prettier,Code formatting with Black and Prettier,"Datasette adopted Black for opinionated Python code formatting in June 2019. Datasette now also embraces Prettier for JavaScript formatting, which like Black is enforced by tests in continuous integration. Instructions for using these two tools can be found in the new section on Code formatting in the contributors documentation. ( #1167 )","[""Changelog"", ""0.54 (2021-01-25)""]","[{""href"": ""https://github.com/psf/black"", ""label"": ""Black""}, {""href"": ""https://prettier.io/"", ""label"": ""Prettier""}, {""href"": ""https://github.com/simonw/datasette/issues/1167"", ""label"": ""#1167""}]" changelog:configuration,changelog,configuration,Configuration,"Plugin configuration now lives in the datasette.yaml configuration file , passed to Datasette using the -c/--config option. Thanks, Alex Garcia. ( #2093 ) datasette -c datasette.yaml Where datasette.yaml contains configuration that looks like this: plugins: datasette-cluster-map: latitude_column: xlat longitude_column: xlon Previously plugins were configured in metadata.yaml , which was confusing as plugin settings were unrelated to database and table metadata. The -s/--setting option can now be used to set plugin configuration as well. See Configuration via the command-line for details. ( #2252 ) The above YAML configuration example using -s/--setting looks like this: datasette mydatabase.db \ -s plugins.datasette-cluster-map.latitude_column xlat \ -s plugins.datasette-cluster-map.longitude_column xlon The new /-/config page shows the current instance configuration, after redacting keys that could contain sensitive data such as API keys or passwords. ( #2254 ) Existing Datasette installations may already have configuration set in metadata.yaml that should be migrated to datasette.yaml . To avoid breaking these installations, Datasette will silently treat table configuration, plugin configuration and allow blocks in metadata as if they had been specified in configuration instead. ( #2247 ) ( #2248 ) ( #2249 ) Note that the datasette publish command has not yet been updated to accept a datasette.yaml configuration file. This will be addressed in #2195 but for the moment you can include those settings in metadata.yaml instead.","[""Changelog"", ""1.0a8 (2024-02-07)""]","[{""href"": ""https://github.com/simonw/datasette/issues/2093"", ""label"": ""#2093""}, {""href"": ""https://github.com/simonw/datasette/issues/2252"", ""label"": ""#2252""}, {""href"": ""https://github.com/simonw/datasette/issues/2254"", ""label"": ""#2254""}, {""href"": ""https://github.com/simonw/datasette/issues/2247"", ""label"": ""#2247""}, {""href"": ""https://github.com/simonw/datasette/issues/2248"", ""label"": ""#2248""}, {""href"": ""https://github.com/simonw/datasette/issues/2249"", ""label"": ""#2249""}, {""href"": ""https://github.com/simonw/datasette/issues/2195"", ""label"": ""#2195""}]" changelog:control-http-caching-with-ttl,changelog,control-http-caching-with-ttl,Control HTTP caching with ?_ttl=,"You can now customize the HTTP max-age header that is sent on a per-URL basis, using the new ?_ttl= query string parameter. You can set this to any value in seconds, or you can set it to 0 to disable HTTP caching entirely. Consider for example this query which returns a randomly selected member of the Avengers: select * from [avengers/avengers] order by random() limit 1 If you hit the following page repeatedly you will get the same result, due to HTTP caching: /fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1 By adding ?_ttl=0 to the zero you can ensure the page will not be cached and get back a different super hero every time: /fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1&_ttl=0","[""Changelog"", ""0.23 (2018-06-18)""]","[{""href"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1"", ""label"": ""/fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1""}, {""href"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1&_ttl=0"", ""label"": ""/fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1&_ttl=0""}]" changelog:cookie-methods,changelog,cookie-methods,Cookie methods,"Plugins can now use the new response.set_cookie() method to set cookies. A new request.cookies method on the :ref:internals_request` can be used to read incoming cookies.","[""Changelog"", ""0.44 (2020-06-11)""]",[] changelog:csrf-protection,changelog,csrf-protection,CSRF protection,"Since writable canned queries are built using POST forms, Datasette now ships with CSRF protection ( #798 ). This applies automatically to any POST request, which means plugins need to include a csrftoken in any POST forms that they render. They can do that like so: ","[""Changelog"", ""0.44 (2020-06-11)""]","[{""href"": ""https://github.com/simonw/datasette/issues/798"", ""label"": ""#798""}]" changelog:csv-export,changelog,csv-export,CSV export,"Any Datasette table, view or custom SQL query can now be exported as CSV. Check out the CSV export documentation for more details, or try the feature out on https://fivethirtyeight.datasettes.com/fivethirtyeight/bechdel%2Fmovies If your table has more than max_returned_rows (default 1,000) Datasette provides the option to stream all rows . This option takes advantage of async Python and Datasette's efficient pagination to iterate through the entire matching result set and stream it back as a downloadable CSV file.","[""Changelog"", ""0.23 (2018-06-18)""]","[{""href"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight/bechdel%2Fmovies"", ""label"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight/bechdel%2Fmovies""}]" changelog:documentation,changelog,documentation,Documentation,"Documentation describing how to write tests that use signed actor cookies using datasette.client.actor_cookie() . ( #1830 ) Documentation on how to register a plugin for the duration of a test . ( #2234 ) The configuration documentation now shows examples of both YAML and JSON for each setting.","[""Changelog"", ""1.0a8 (2024-02-07)""]","[{""href"": ""https://github.com/simonw/datasette/issues/1830"", ""label"": ""#1830""}, {""href"": ""https://github.com/simonw/datasette/issues/2234"", ""label"": ""#2234""}]" changelog:facet-by-date,changelog,facet-by-date,Facet by date,"If a column contains datetime values, Datasette can now facet that column by date. ( #481 )","[""Changelog"", ""0.29 (2019-07-07)""]","[{""href"": ""https://github.com/simonw/datasette/issues/481"", ""label"": ""#481""}]" changelog:faceting,changelog,faceting,Faceting,"The number of unique values in a facet is now always displayed. Previously it was only displayed if the user specified ?_facet_size=max . ( #1556 ) Facets of type date or array can now be configured in metadata.json , see Facets in metadata . Thanks, David Larlet. ( #1552 ) New ?_nosuggest=1 parameter for table views, which disables facet suggestion. ( #1557 ) Fixed bug where ?_facet_array=tags&_facet=tags would only display one of the two selected facets. ( #625 )","[""Changelog"", ""0.60 (2022-01-13)""]","[{""href"": ""https://github.com/simonw/datasette/issues/1556"", ""label"": ""#1556""}, {""href"": ""https://github.com/simonw/datasette/issues/1552"", ""label"": ""#1552""}, {""href"": ""https://github.com/simonw/datasette/issues/1557"", ""label"": ""#1557""}, {""href"": ""https://github.com/simonw/datasette/issues/625"", ""label"": ""#625""}]" changelog:features,changelog,features,Features,"Now tested against Python 3.11. Docker containers used by datasette publish and datasette package both now use that version of Python. ( #1853 ) --load-extension option now supports entrypoints. Thanks, Alex Garcia. ( #1789 ) Facet size can now be set per-table with the new facet_size table metadata option. ( #1804 ) The truncate_cells_html setting now also affects long URLs in columns. ( #1805 ) The non-JavaScript SQL editor textarea now increases height to fit the SQL query. ( #1786 ) Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. ( #1794 ) The settings.json file used in Configuration directory mode is now validated on startup. ( #1816 ) SQL queries can now include leading SQL comments, using /* ... */ or -- ... syntax. Thanks, Charles Nepote. ( #1860 ) SQL query is now re-displayed when terminated with a time limit error. ( #1819 ) The inspect data mechanism is now used to speed up server startup - thanks, Forest Gregg. ( #1834 ) In Configuration directory mode databases with filenames ending in .sqlite or .sqlite3 are now automatically added to the Datasette instance. ( #1646 ) Breadcrumb navigation display now respects the current user's permissions. ( #1831 )","[""Changelog"", ""0.63 (2022-10-27)""]","[{""href"": ""https://github.com/simonw/datasette/issues/1853"", ""label"": ""#1853""}, {""href"": ""https://github.com/simonw/datasette/pull/1789"", ""label"": ""#1789""}, {""href"": ""https://github.com/simonw/datasette/issues/1804"", ""label"": ""#1804""}, {""href"": ""https://github.com/simonw/datasette/issues/1805"", ""label"": ""#1805""}, {""href"": ""https://github.com/simonw/datasette/issues/1786"", ""label"": ""#1786""}, {""href"": ""https://github.com/simonw/datasette/pull/1794"", ""label"": ""#1794""}, {""href"": ""https://github.com/simonw/datasette/issues/1816"", ""label"": ""#1816""}, {""href"": ""https://github.com/simonw/datasette/issues/1860"", ""label"": ""#1860""}, {""href"": ""https://github.com/simonw/datasette/issues/1819"", ""label"": ""#1819""}, {""href"": ""https://github.com/simonw/datasette/issues/1834"", ""label"": ""#1834""}, {""href"": ""https://github.com/simonw/datasette/issues/1646"", ""label"": ""#1646""}, {""href"": ""https://github.com/simonw/datasette/issues/1831"", ""label"": ""#1831""}]" changelog:flash-messages,changelog,flash-messages,Flash messages,"Writable canned queries needed a mechanism to let the user know that the query has been successfully executed. The new flash messaging system ( #790 ) allows messages to persist in signed cookies which are then displayed to the user on the next page that they visit. Plugins can use this mechanism to display their own messages, see .add_message(request, message, type=datasette.INFO) for details. You can try out the new messages using the /-/messages debug tool, for example at https://latest.datasette.io/-/messages","[""Changelog"", ""0.44 (2020-06-11)""]","[{""href"": ""https://github.com/simonw/datasette/issues/790"", ""label"": ""#790""}, {""href"": ""https://latest.datasette.io/-/messages"", ""label"": ""https://latest.datasette.io/-/messages""}]" changelog:foreign-key-expansions,changelog,foreign-key-expansions,Foreign key expansions,"When Datasette detects a foreign key reference it attempts to resolve a label for that reference (automatically or using the Specifying the label column for a table metadata option) so it can display a link to the associated row. This expansion is now also available for JSON and CSV representations of the table, using the new _labels=on query string option. See Expanding foreign key references for more details.","[""Changelog"", ""0.23 (2018-06-18)""]",[] changelog:id1,changelog,id1,Changelog,,[],[] changelog:id10,changelog,id10,0.64 (2023-01-09),"Datasette now strongly recommends against allowing arbitrary SQL queries if you are using SpatiaLite . SpatiaLite includes SQL functions that could cause the Datasette server to crash. See SpatiaLite for more details. New default_allow_sql setting, providing an easier way to disable all arbitrary SQL execution by end users: datasette --setting default_allow_sql off . See also Controlling the ability to execute arbitrary SQL . ( #1409 ) Building a location to time zone API with SpatiaLite is a new Datasette tutorial showing how to safely use SpatiaLite to create a location to time zone API. New documentation about how to debug problems loading SQLite extensions . The error message shown when an extension cannot be loaded has also been improved. ( #1979 ) Fixed an accessibility issue: the