# sections

id: changelog:new-configuration-settings
page: changelog
ref: new-configuration-settings
title: New configuration settings
breadcrumbs: ["Changelog", "0.23 (2018-06-18)"]
references: [{"href": "https://www.sqlite.org/pragma.html#pragma_cache_size", "label": "per-connection cache"}]
content:
Datasette's  Settings  now also supports boolean settings. A number of new
                    configuration options have been added: 
                 
                     
                         num_sql_threads  - the number of threads used to execute SQLite queries. Defaults to 3. 
                     
                     
                         allow_facet  - enable or disable custom  Facets  using the  _facet=  parameter. Defaults to on. 
                     
                     
                         suggest_facets  - should Datasette suggest facets? Defaults to on. 
                     
                     
                         allow_download  - should users be allowed to download the entire SQLite database? Defaults to on. 
                     
                     
                         allow_sql  - should users be allowed to execute custom SQL queries? Defaults to on. 
                     
                     
                         default_cache_ttl  - Default HTTP caching max-age header in seconds. Defaults to 365 days - caching can be disabled entirely by settings this to 0. 
                     
                     
                         cache_size_kb  - Set the amount of memory SQLite uses for its  per-connection cache , in KB. 
                     
                     
                         allow_csv_stream  - allow users to stream entire result sets as a single CSV file. Defaults to on. 
                     
                     
                         max_csv_mb  - maximum size of a returned CSV file in MB. Defaults to 100MB, set to 0 to disable this limit.


id: changelog:control-http-caching-with-ttl
page: changelog
ref: control-http-caching-with-ttl
title: Control HTTP caching with ?_ttl=
breadcrumbs: ["Changelog", "0.23 (2018-06-18)"]
content:
You can now customize the HTTP max-age header that is sent on a per-URL basis, using the new  ?_ttl=  query string parameter. 
                 You can set this to any value in seconds, or you can set it to 0 to disable HTTP caching entirely. 
                 Consider for example this query which returns a randomly selected member of the Avengers: 
                 select * from [avengers/avengers] order by random() limit 1 
                 If you hit the following page repeatedly you will get the same result, due to HTTP caching: 
                 /fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1 
                 By adding  ?_ttl=0  to the zero you can ensure the page will not be cached and get back a different super hero every time: 
                 /fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1&_ttl=0

references:
[{"href": "https://fivethirtyeight.datasettes.com/fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1", "label": "/fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1"}, {"href": "https://fivethirtyeight.datasettes.com/fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1&_ttl=0", "label": "/fivethirtyeight?sql=select+*+from+%5Bavengers%2Favengers%5D+order+by+random%28%29+limit+1&_ttl=0"}]


id: changelog:improved-support-for-spatialite
page: changelog
ref: improved-support-for-spatialite
title: Improved support for SpatiaLite
breadcrumbs: ["Changelog", "0.23 (2018-06-18)"]
content:
The  SpatiaLite module 
                    for SQLite adds robust geospatial features to the database. 
                 Getting SpatiaLite working can be tricky, especially if you want to use the most
                    recent alpha version (with support for K-nearest neighbor). 
                 Datasette now includes  extensive documentation on SpatiaLite , and thanks to  Ravi Kotecha  our GitHub
                    repo includes a  Dockerfile  that can build
                    the latest SpatiaLite and configure it for use with Datasette. 
                 The  datasette publish  and  datasette package  commands now accept a new
                     --spatialite  argument which causes them to install and configure SpatiaLite
                    as part of the container they deploy.

references:
[{"href": "https://www.gaia-gis.it/fossil/libspatialite/index", "label": "SpatiaLite module"}, {"href": "https://github.com/r4vi", "label": "Ravi Kotecha"}, {"href": "https://github.com/simonw/datasette/blob/master/Dockerfile", "label": "Dockerfile"}]


id: changelog:latest-datasette-io
page: changelog
ref: latest-datasette-io
title: latest.datasette.io
breadcrumbs: ["Changelog", "0.23 (2018-06-18)"]
content:
Every commit to Datasette master is now automatically deployed by Travis CI to
                     https://latest.datasette.io/  - ensuring there is always a live demo of the
                    latest version of the software. 
                 The demo uses  the fixtures  from our
                    unit tests, ensuring it demonstrates the same range of functionality that is
                    covered by the tests. 
                 You can see how the deployment mechanism works in our  .travis.yml  file.

references:
[{"href": "https://latest.datasette.io/", "label": "https://latest.datasette.io/"}, {"href": "https://github.com/simonw/datasette/blob/master/tests/fixtures.py", "label": "the fixtures"}, {"href": "https://github.com/simonw/datasette/blob/master/.travis.yml", "label": ".travis.yml"}]


id: changelog:miscellaneous
page: changelog
ref: miscellaneous
title: Miscellaneous
breadcrumbs: ["Changelog", "0.23 (2018-06-18)"]
content:
Got JSON data in one of your columns? Use the new  ?_json=COLNAME  argument
                            to tell Datasette to return that JSON value directly rather than encoding it
                            as a string. 
                     
                     
                         If you just want an array of the first value of each row, use the new
                             ?_shape=arrayfirst  option -  example .

references:
[{"href": "https://latest.datasette.io/fixtures.json?sql=select+neighborhood+from+facetable+order+by+pk+limit+101&_shape=arrayfirst", "label": "example"}]


id: changelog:id117
page: changelog
ref: id117
title: 0.22.1 (2018-05-23)
breadcrumbs: ["Changelog"]
content:
Bugfix release, plus we now use  versioneer  for our version numbers. 
             
                 
                     Faceting no longer breaks pagination, fixes  #282 
                 
                 
                     Add  __version_info__  derived from  __version__  [Robert Gieseke] 
                     This might be tuple of more than two values (major and minor
                        version) if commits have been made after a release. 
                 
                 
                     Add version number support with Versioneer. [Robert Gieseke] 
                     Versioneer Licence:
                        Public Domain (CC0-1.0) 
                     Closes  #273 
                 
                 
                     Refactor inspect logic [Russ Garrett]

references:
[{"href": "https://github.com/warner/python-versioneer", "label": "versioneer"}, {"href": "https://github.com/simonw/datasette/issues/282", "label": "#282"}, {"href": "https://github.com/simonw/datasette/issues/273", "label": "#273"}]


id: changelog:id120
page: changelog
ref: id120
title: 0.22 (2018-05-20)
breadcrumbs: ["Changelog"]
content:
The big new feature in this release is  Facets . Datasette can now apply faceted browse to any column in any table. It will also suggest possible facets. See the  Datasette Facets  announcement post for more details. 
             In addition to the work on facets: 
             
                 
                     Added  docs for introspection endpoints 
                 
                 
                     New  --config  option, added  --help-config , closes  #274 
                     Removed the  --page_size=  argument to  datasette serve  in favour of: 
                     datasette serve --config default_page_size:50 mydb.db 
                     Added new help section: 
                     datasette --help-config 
                     Config options:
  default_page_size            Default page size for the table view
                               (default=100)
  max_returned_rows            Maximum rows that can be returned from a table
                               or custom query (default=1000)
  sql_time_limit_ms            Time limit for a SQL query in milliseconds
                               (default=1000)
  default_facet_size           Number of values to return for requested facets
                               (default=30)
  facet_time_limit_ms          Time limit for calculating a requested facet
                               (default=200)
  facet_suggest_time_limit_ms  Time limit for calculating a suggested facet
                               (default=50) 
                 
                 
                     Only apply responsive table styles to  .rows-and-column 
                     Otherwise they interfere with tables in the description, e.g. on
                         https://fivethirtyeight.datasettes.com/fivethirtyeight/nba-elo%2Fnbaallelo 
                 
                 
                     Refactored views into new  views/  modules, refs  #256 
                 
                 
                     Documentation for SQLite full-text search  support, closes  #253 
                 
                 
                     /-/versions  now includes SQLite  fts_versions , closes  #252

references:
[{"href": "https://simonwillison.net/2018/May/20/datasette-facets/", "label": "Datasette Facets"}, {"href": "https://docs.datasette.io/en/stable/introspection.html", "label": "docs for introspection endpoints"}, {"href": "https://github.com/simonw/datasette/issues/274", "label": "#274"}, {"href": "https://fivethirtyeight.datasettes.com/fivethirtyeight/nba-elo%2Fnbaallelo", "label": "https://fivethirtyeight.datasettes.com/fivethirtyeight/nba-elo%2Fnbaallelo"}, {"href": "https://github.com/simonw/datasette/issues/256", "label": "#256"}, {"href": "https://docs.datasette.io/en/stable/full_text_search.html", "label": "Documentation for SQLite full-text search"}, {"href": "https://github.com/simonw/datasette/issues/253", "label": "#253"}, {"href": "https://github.com/simonw/datasette/issues/252", "label": "#252"}]


id: changelog:id125
page: changelog
ref: id125
title: 0.21 (2018-05-05)
breadcrumbs: ["Changelog"]
content:
New JSON  _shape=  options, the ability to set table  _size=  and a mechanism for searching within specific columns. 
             
                 
                     Default tests to using a longer timelimit 
                     Every now and then a test will fail in Travis CI on Python 3.5 because it hit
                        the default 20ms SQL time limit. 
                     Test fixtures now default to a 200ms time limit, and we only use the 20ms time
                        limit for the specific test that tests query interruption. This should make
                        our tests on Python 3.5 in Travis much more stable. 
                 
                 
                     Support  _search_COLUMN=text  searches, closes  #237 
                 
                 
                     Show version on  /-/plugins  page, closes  #248 
                 
                 
                     ?_size=max  option, closes  #249 
                 
                 
                     Added  /-/versions  and  /-/versions.json , closes  #244 
                     Sample output: 
                     {
  "python": {
    "version": "3.6.3",
    "full": "3.6.3 (default, Oct  4 2017, 06:09:38) \n[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]"
  },
  "datasette": {
    "version": "0.20"
  },
  "sqlite": {
    "version": "3.23.1",
    "extensions": {
      "json1": null,
      "spatialite": "4.3.0a"
    }
  }
} 
                 
                 
                     Renamed  ?_sql_time_limit_ms=  to  ?_timelimit , closes  #242 
                 
                 
                     New  ?_shape=array  option + tweaks to  _shape , closes  #245 
                     
                         
                             Default is now  ?_shape=arrays  (renamed from  lists ) 
                         
                         
                             New  ?_shape=array  returns an array of objects as the root object 
                         
                         
                             Changed  ?_shape=object  to return the object as the root 
                         
                         
                             Updated docs 
                         
                     
                 
                 
                     FTS tables now detected by  inspect() , closes  #240 
                 
                 
                     New  ?_size=XXX  query string parameter for table view, closes  #229 
                     Also added documentation for all of the  _special  arguments. 
                     Plus deleted some duplicate logic implementing  _group_count . 
                 
                 
                     If  max_returned_rows==page_size , increment  max_returned_rows  - fixes  #230 
                 
                 
                     New  hidden: True  option for table metadata, closes  #239 
                 
                 
                     Hide  idx_*  tables if spatialite detected, closes  #228 
                 
                 
                     Added  class=rows-and-columns  to custom query results table 
                 
                 
                     Added CSS class  rows-and-columns  to main table 
                 
                 
                     label_column  option in  metadata.json  - closes  #234

references:
[{"href": "https://github.com/simonw/datasette/issues/237", "label": "#237"}, {"href": "https://github.com/simonw/datasette/issues/248", "label": "#248"}, {"href": "https://github.com/simonw/datasette/issues/249", "label": "#249"}, {"href": "https://github.com/simonw/datasette/issues/244", "label": "#244"}, {"href": "https://github.com/simonw/datasette/issues/242", "label": "#242"}, {"href": "https://github.com/simonw/datasette/issues/245", "label": "#245"}, {"href": "https://github.com/simonw/datasette/issues/240", "label": "#240"}, {"href": "https://github.com/simonw/datasette/issues/229", "label": "#229"}, {"href": "https://github.com/simonw/datasette/issues/230", "label": "#230"}, {"href": "https://github.com/simonw/datasette/issues/239", "label": "#239"}, {"href": "https://github.com/simonw/datasette/issues/228", "label": "#228"}, {"href": "https://github.com/simonw/datasette/issues/234", "label": "#234"}]


id: changelog:id138
page: changelog
ref: id138
title: 0.20 (2018-04-20)
breadcrumbs: ["Changelog"]
content:
Mostly new work on the  Plugins  mechanism: plugins can now bundle static assets and custom templates, and  datasette publish  has a new  --install=name-of-plugin  option. 
             
                 
                     Add col-X classes to HTML table on custom query page 
                 
                 
                     Fixed out-dated template in documentation 
                 
                 
                     Plugins can now bundle custom templates,  #224 
                 
                 
                     Added /-/metadata /-/plugins /-/inspect,  #225 
                 
                 
                     Documentation for --install option, refs  #223 
                 
                 
                     Datasette publish/package --install option,  #223 
                 
                 
                     Fix for plugins in Python 3.5,  #222 
                 
                 
                     New plugin hooks: extra_css_urls() and extra_js_urls(),  #214 
                 
                 
                     /-/static-plugins/PLUGIN_NAME/ now serves static/ from plugins 
                 
                 
                     <th> now gets class="col-X" - plus added col-X documentation 
                 
                 
                     Use to_css_class for table cell column classes 
                     This ensures that columns with spaces in the name will still
                        generate usable CSS class names. Refs  #209 
                 
                 
                     Add column name classes to <td>s, make PK bold [Russ Garrett] 
                 
                 
                     Don't duplicate simple primary keys in the link column [Russ Garrett] 
                     When there's a simple (single-column) primary key, it looks weird to
                        duplicate it in the link column. 
                     This change removes the second PK column and treats the link column as
                        if it were the PK column from a header/sorting perspective. 
                 
                 
                     Correct escaping for HTML display of row links [Russ Garrett] 
                 
                 
                     Longer time limit for test_paginate_compound_keys 
                     It was failing intermittently in Travis - see  #209 
                 
                 
                     Use application/octet-stream for downloadable databases 
                 
                 
                     Updated PyPI classifiers 
                 
                 
                     Updated PyPI link to pypi.org

references:
[{"href": "https://github.com/simonw/datasette/issues/224", "label": "#224"}, {"href": "https://github.com/simonw/datasette/issues/225", "label": "#225"}, {"href": "https://github.com/simonw/datasette/issues/223", "label": "#223"}, {"href": "https://github.com/simonw/datasette/issues/223", "label": "#223"}, {"href": "https://github.com/simonw/datasette/issues/222", "label": "#222"}, {"href": "https://github.com/simonw/datasette/issues/214", "label": "#214"}, {"href": "https://github.com/simonw/datasette/issues/209", "label": "#209"}, {"href": "https://github.com/simonw/datasette/issues/209", "label": "#209"}]


id: changelog:id147
page: changelog
ref: id147
title: 0.19 (2018-04-16)
breadcrumbs: ["Changelog"]
content:
This is the first preview of the new Datasette plugins mechanism. Only two
                plugin hooks are available so far - for custom SQL functions and custom template
                filters. There's plenty more to come - read  the documentation  and get involved in
                 the tracking ticket  if you
                have feedback on the direction so far. 
             
                 
                     Fix for  _sort_desc=sortable_with_nulls  test, refs  #216 
                 
                 
                     Fixed  #216  - paginate correctly when sorting by nullable column 
                 
                 
                     Initial documentation for plugins, closes  #213 
                     https://docs.datasette.io/en/stable/plugins.html 
                 
                 
                     New  --plugins-dir=plugins/  option ( #212 ) 
                     New option causing Datasette to load and evaluate all of the Python files in
                        the specified directory and register any plugins that are defined in those
                        files. 
                     This new option is available for the following commands: 
                     datasette serve mydb.db --plugins-dir=plugins/
datasette publish now/heroku mydb.db --plugins-dir=plugins/
datasette package mydb.db --plugins-dir=plugins/ 
                 
                 
                     Start of the plugin system, based on pluggy ( #210 ) 
                     Uses  https://pluggy.readthedocs.io/  originally created for the py.test project 
                     We're starting with two plugin hooks: 
                     prepare_connection(conn) 
                     This is called when a new SQLite connection is created. It can be used to register custom SQL functions. 
                     prepare_jinja2_environment(env) 
                     This is called with the Jinja2 environment. It can be used to register custom template tags and filters. 
                     An example plugin which uses these two hooks can be found at  https://github.com/simonw/datasette-plugin-demos  or installed using  pip install datasette-plugin-demos 
                     Refs  #14 
                 
                 
                     Return HTTP 405 on InvalidUsage rather than 500. [Russ Garrett] 
                     This also stops it filling up the logs. This happens for HEAD requests
                        at the moment - which perhaps should be handled better, but that's a
                        different issue.

references:
[{"href": "https://docs.datasette.io/en/stable/plugins.html", "label": "the documentation"}, {"href": "https://github.com/simonw/datasette/issues/14", "label": "the tracking ticket"}, {"href": "https://github.com/simonw/datasette/issues/216", "label": "#216"}, {"href": "https://github.com/simonw/datasette/issues/216", "label": "#216"}, {"href": "https://github.com/simonw/datasette/issues/213", "label": "#213"}, {"href": "https://docs.datasette.io/en/stable/plugins.html", "label": "https://docs.datasette.io/en/stable/plugins.html"}, {"href": "https://github.com/simonw/datasette/issues/212", "label": "#212"}, {"href": "https://github.com/simonw/datasette/issues/14", "label": "#210"}, {"href": "https://pluggy.readthedocs.io/", "label": "https://pluggy.readthedocs.io/"}, {"href": "https://github.com/simonw/datasette-plugin-demos", "label": "https://github.com/simonw/datasette-plugin-demos"}, {"href": "https://github.com/simonw/datasette/issues/14", "label": "#14"}]


id: changelog:id154
page: changelog
ref: id154
title: 0.18 (2018-04-14)
breadcrumbs: ["Changelog"]
content:
This release introduces  support for units ,
                contributed by Russ Garrett ( #203 ).
                You can now optionally specify the units for specific columns using  metadata.json .
                Once specified, units will be displayed in the HTML view of your table. They also become
                available for use in filters - if a column is configured with a unit of distance, you can
                request all rows where that column is less than 50 meters or more than 20 feet for example. 
             
                 
                     Link foreign keys which don't have labels. [Russ Garrett] 
                     This renders unlabeled FKs as simple links. 
                     Also includes bonus fixes for two minor issues: 
                     
                         
                             In foreign key link hrefs the primary key was escaped using HTML
                                escaping rather than URL escaping. This broke some non-integer PKs. 
                         
                         
                             Print tracebacks to console when handling 500 errors. 
                         
                     
                 
                 
                     Fix SQLite error when loading rows with no incoming FKs. [Russ
                        Garrett] 
                     This fixes an error caused by an invalid query when loading incoming FKs. 
                     The error was ignored due to async but it still got printed to the
                        console. 
                 
                 
                     Allow custom units to be registered with Pint. [Russ Garrett] 
                 
                 
                     Support units in filters. [Russ Garrett] 
                 
                 
                     Tidy up units support. [Russ Garrett] 
                     
                         
                             Add units to exported JSON 
                         
                         
                             Units key in metadata skeleton 
                         
                         
                             Docs 
                         
                     
                 
                 
                     Initial units support. [Russ Garrett] 
                     Add support for specifying units for a column in  metadata.json  and
                        rendering them on display using
                         pint

references:
[{"href": "https://docs.datasette.io/en/stable/metadata.html#specifying-units-for-a-column", "label": "support for units"}, {"href": "https://github.com/simonw/datasette/issues/203", "label": "#203"}, {"href": "https://pint.readthedocs.io/en/latest/", "label": "pint"}]


id: changelog:id157
page: changelog
ref: id157
title: 0.16 (2018-04-13)
breadcrumbs: ["Changelog"]
content:
Better mechanism for handling errors; 404s for missing table/database 
                     New error mechanism closes  #193 
                     404s for missing tables/databases closes  #184 
                 
                 
                     long_description in markdown for the new PyPI 
                 
                 
                     Hide SpatiaLite system tables. [Russ Garrett] 
                 
                 
                     Allow  explain select  /  explain query plan select   #201 
                 
                 
                     Datasette inspect now finds primary_keys  #195 
                 
                 
                     Ability to sort using form fields (for mobile portrait mode)  #199 
                     We now display sort options as a select box plus a descending checkbox, which
                        means you can apply sort orders even in portrait mode on a mobile phone where
                        the column headers are hidden.

references:
[{"href": "https://github.com/simonw/datasette/issues/193", "label": "#193"}, {"href": "https://github.com/simonw/datasette/issues/184", "label": "#184"}, {"href": "https://github.com/simonw/datasette/issues/201", "label": "#201"}, {"href": "https://github.com/simonw/datasette/issues/195", "label": "#195"}, {"href": "https://github.com/simonw/datasette/issues/199", "label": "#199"}]


id: changelog:id163
page: changelog
ref: id163
title: 0.15 (2018-04-09)
breadcrumbs: ["Changelog"]
content:
The biggest new feature in this release is the ability to sort by column. On the
                table page the column headers can now be clicked to apply sort (or descending
                sort), or you can specify  ?_sort=column  or  ?_sort_desc=column  directly
                in the URL. 
             
                 
                     table_rows  =>  table_rows_count ,  filtered_table_rows  =>
                         filtered_table_rows_count 
                     Renamed properties. Closes  #194 
                 
                 
                     New  sortable_columns  option in  metadata.json  to control sort options. 
                     You can now explicitly set which columns in a table can be used for sorting
                        using the  _sort  and  _sort_desc  arguments using  metadata.json : 
                     {
    "databases": {
        "database1": {
            "tables": {
                "example_table": {
                    "sortable_columns": [
                        "height",
                        "weight"
                    ]
                }
            }
        }
    }
} 
                     Refs  #189 
                 
                 
                     Column headers now link to sort/desc sort - refs  #189 
                 
                 
                     _sort  and  _sort_desc  parameters for table views 
                     Allows for paginated sorted results based on a specified column. 
                     Refs  #189 
                 
                 
                     Total row count now correct even if  _next  applied 
                 
                 
                     Use .custom_sql() for _group_count implementation (refs  #150 ) 
                 
                 
                     Make HTML title more readable in query template ( #180 ) [Ryan Pitts] 
                 
                 
                     New  ?_shape=objects/object/lists  param for JSON API ( #192 ) 
                     New  _shape=  parameter replacing old  .jsono  extension 
                     Now instead of this: 
                     /database/table.jsono 
                     We use the  _shape  parameter like this: 
                     /database/table.json?_shape=objects 
                     Also introduced a new  _shape  called  object  which looks like this: 
                     /database/table.json?_shape=object 
                     Returning an object for the rows key: 
                     ...
"rows": {
    "pk1": {
        ...
    },
    "pk2": {
        ...
    }
} 
                     Refs  #122 
                 
                 
                     Utility for writing test database fixtures to a .db file 
                     python tests/fixtures.py /tmp/hello.db 
                     This is useful for making a SQLite database of the test fixtures for
                        interactive exploration. 
                 
                 
                     Compound primary key  _next=  now plays well with extra filters 
                     Closes  #190 
                 
                 
                     Fixed bug with keyset pagination over compound primary keys 
                     Refs  #190 
                 
                 
                     Database/Table views inherit  source/license/source_url/license_url 
                        metadata 
                     If you set the  source_url/license_url/source/license  fields in your root
                        metadata those values will now be inherited all the way down to the database
                        and table templates. 
                     The  title/description  are NOT inherited. 
                     Also added unit tests for the HTML generated by the metadata. 
                     Refs  #185 
                 
                 
                     Add metadata, if it exists, to heroku temp dir ( #178 ) [Tony Hirst] 
                 
                 
                     Initial documentation for pagination 
                 
                 
                     Broke up test_app into test_api and test_html 
                 
                 
                     Fixed bug with .json path regular expression 
                     I had a table called  geojson  and it caused an exception because the regex
                        was matching  .json  and not  \.json 
                 
                 
                     Deploy to Heroku with Python 3.6.3

references:
[{"href": "https://github.com/simonw/datasette/issues/194", "label": "#194"}, {"href": "https://github.com/simonw/datasette/issues/189", "label": "#189"}, {"href": "https://github.com/simonw/datasette/issues/189", "label": "#189"}, {"href": "https://github.com/simonw/datasette/issues/189", "label": "#189"}, {"href": "https://github.com/simonw/datasette/issues/150", "label": "#150"}, {"href": "https://github.com/simonw/datasette/issues/180", "label": "#180"}, {"href": "https://github.com/simonw/datasette/issues/192", "label": "#192"}, {"href": "https://github.com/simonw/datasette/issues/122", "label": "#122"}, {"href": "https://github.com/simonw/datasette/issues/190", "label": "#190"}, {"href": "https://github.com/simonw/datasette/issues/190", "label": "#190"}, {"href": "https://github.com/simonw/datasette/issues/185", "label": "#185"}, {"href": "https://github.com/simonw/datasette/issues/178", "label": "#178"}]


id: changelog:id176
page: changelog
ref: id176
title: 0.14 (2017-12-09)
breadcrumbs: ["Changelog"]
content:
The theme of this release is customization: Datasette now allows every aspect
                of its presentation  to be customized 
                either using additional CSS or by providing entirely new templates. 
             Datasette's  metadata.json format 
                has also been expanded, to allow per-database and per-table metadata. A new
                 datasette skeleton  command can be used to generate a skeleton JSON file
                ready to be filled in with per-database and per-table details. 
             The  metadata.json  file can also be used to define
                 canned queries ,
                as a more powerful alternative to SQL views. 
             
                 
                     extra_css_urls / extra_js_urls  in metadata 
                     A mechanism in the  metadata.json  format for adding custom CSS and JS urls. 
                     Create a  metadata.json  file that looks like this: 
                     {
    "extra_css_urls": [
        "https://simonwillison.net/static/css/all.bf8cd891642c.css"
    ],
    "extra_js_urls": [
        "https://code.jquery.com/jquery-3.2.1.slim.min.js"
    ]
} 
                     Then start datasette like this: 
                     datasette mydb.db --metadata=metadata.json 
                     The CSS and JavaScript files will be linked in the  <head>  of every page. 
                     You can also specify a SRI (subresource integrity hash) for these assets: 
                     {
    "extra_css_urls": [
        {
            "url": "https://simonwillison.net/static/css/all.bf8cd891642c.css",
            "sri": "sha384-9qIZekWUyjCyDIf2YK1FRoKiPJq4PHt6tp/ulnuuyRBvazd0hG7pWbE99zvwSznI"
        }
    ],
    "extra_js_urls": [
        {
            "url": "https://code.jquery.com/jquery-3.2.1.slim.min.js",
            "sri": "sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g="
        }
    ]
} 
                     Modern browsers will only execute the stylesheet or JavaScript if the SRI hash
                        matches the content served. You can generate hashes using  https://www.srihash.org/ 
                 
                 
                     Auto-link column values that look like URLs ( #153 ) 
                 
                 
                     CSS styling hooks as classes on the body ( #153 ) 
                     Every template now gets CSS classes in the body designed to support custom
                        styling. 
                     The index template (the top level page at  / ) gets this: 
                     <body class="index"> 
                     The database template ( /dbname/ ) gets this: 
                     <body class="db db-dbname"> 
                     The table template ( /dbname/tablename ) gets: 
                     <body class="table db-dbname table-tablename"> 
                     The row template ( /dbname/tablename/rowid ) gets: 
                     <body class="row db-dbname table-tablename"> 
                     The  db-x  and  table-x  classes use the database or table names themselves IF
                        they are valid CSS identifiers. If they aren't, we strip any invalid
                        characters out and append a 6 character md5 digest of the original name, in
                        order to ensure that multiple tables which resolve to the same stripped
                        character version still have different CSS classes. 
                     Some examples (extracted from the unit tests): 
                     "simple" => "simple"
"MixedCase" => "MixedCase"
"-no-leading-hyphens" => "no-leading-hyphens-65bea6"
"_no-leading-underscores" => "no-leading-underscores-b921bc"
"no spaces" => "no-spaces-7088d7"
"-" => "336d5e"
"no $ characters" => "no--characters-59e024" 
                 
                 
                     datasette --template-dir=mytemplates/  argument 
                     You can now pass an additional argument specifying a directory to look for
                        custom templates in. 
                     Datasette will fall back on the default templates if a template is not
                        found in that directory. 
                 
                 
                     Ability to over-ride templates for individual tables/databases. 
                     It is now possible to over-ride templates on a per-database / per-row or per-
                        table basis. 
                     When you access e.g.  /mydatabase/mytable  Datasette will look for the following: 
                     - table-mydatabase-mytable.html
- table.html 
                     If you provided a  --template-dir  argument to datasette serve it will look in
                        that directory first. 
                     The lookup rules are as follows: 
                     Index page (/):
    index.html

Database page (/mydatabase):
    database-mydatabase.html
    database.html

Table page (/mydatabase/mytable):
    table-mydatabase-mytable.html
    table.html

Row page (/mydatabase/mytable/id):
    row-mydatabase-mytable.html
    row.html 
                     If a table name has spaces or other unexpected characters in it, the template
                        filename will follow the same rules as our custom  <body>  CSS classes
                        - for example, a table called "Food Trucks"
                        will attempt to load the following templates: 
                     table-mydatabase-Food-Trucks-399138.html
table.html 
                     It is possible to extend the default templates using Jinja template
                        inheritance. If you want to customize EVERY row template with some additional
                        content you can do so by creating a row.html template like this: 
                     {% extends "default:row.html" %}

{% block content %}
<h1>EXTRA HTML AT THE TOP OF THE CONTENT BLOCK</h1>
<p>This line renders the original block:</p>
{{ super() }}
{% endblock %} 
                 
                 
                     --static  option for datasette serve ( #160 ) 
                     You can now tell Datasette to serve static files from a specific location at a
                        specific mountpoint. 
                     For example: 
                     datasette serve mydb.db --static extra-css:/tmp/static/css 
                     Now if you visit this URL: 
                     http://localhost:8001/extra-css/blah.css 
                     The following file will be served: 
                     /tmp/static/css/blah.css 
                 
                 
                     Canned query support. 
                     Named canned queries can now be defined in  metadata.json  like this: 
                     {
    "databases": {
        "timezones": {
            "queries": {
                "timezone_for_point": "select tzid from timezones ..."
            }
        }
    }
} 
                     These will be shown in a new "Queries" section beneath "Views" on the database page. 
                 
                 
                     New  datasette skeleton  command for generating  metadata.json  ( #164 ) 
                 
                 
                     metadata.json  support for per-table/per-database metadata ( #165 ) 
                     Also added support for descriptions and HTML descriptions. 
                     Here's an example metadata.json file illustrating custom per-database and per-
                        table metadata: 
                     {
    "title": "Overall datasette title",
    "description_html": "This is a <em>description with HTML</em>.",
    "databases": {
        "db1": {
            "title": "First database",
            "description": "This is a string description & has no HTML",
            "license_url": "http://example.com/",
        "license": "The example license",
            "queries": {
              "canned_query": "select * from table1 limit 3;"
            },
            "tables": {
                "table1": {
                    "title": "Custom title for table1",
                    "description": "Tables can have descriptions too",
                    "source": "This has a custom source",
                    "source_url": "http://example.com/"
                }
            }
        }
    }
} 
                 
                 
                     Renamed  datasette build  command to  datasette inspect  ( #130 ) 
                 
                 
                     Upgrade to Sanic 0.7.0 ( #168 ) 
                     https://github.com/channelcat/sanic/releases/tag/0.7.0 
                 
                 
                     Package and publish commands now accept  --static  and  --template-dir 
                     Example usage: 
                     datasette package --static css:extra-css/ --static js:extra-js/ \
  sf-trees.db --template-dir templates/ --tag sf-trees --branch master 
                     This creates a local Docker image that includes copies of the templates/,
                        extra-css/ and extra-js/ directories. You can then run it like this: 
                     docker run -p 8001:8001 sf-trees 
                     For publishing to Zeit now: 
                     datasette publish now --static css:extra-css/ --static js:extra-js/ \
  sf-trees.db --template-dir templates/ --name sf-trees --branch master 
                 
                 
                     HTML comment showing which templates were considered for a page ( #171 )

references:
[{"href": "https://docs.datasette.io/en/stable/custom_templates.html", "label": "to be customized"}, {"href": "https://docs.datasette.io/en/stable/metadata.html", "label": "metadata.json format"}, {"href": "https://docs.datasette.io/en/stable/sql_queries.html#canned-queries", "label": "canned queries"}, {"href": "https://www.srihash.org/", "label": "https://www.srihash.org/"}, {"href": "https://github.com/simonw/datasette/issues/153", "label": "#153"}, {"href": "https://github.com/simonw/datasette/issues/153", "label": "#153"}, {"href": "https://github.com/simonw/datasette/issues/160", "label": "#160"}, {"href": "https://github.com/simonw/datasette/issues/164", "label": "#164"}, {"href": "https://github.com/simonw/datasette/issues/165", "label": "#165"}, {"href": "https://github.com/simonw/datasette/issues/130", "label": "#130"}, {"href": "https://github.com/simonw/datasette/issues/168", "label": "#168"}, {"href": "https://github.com/channelcat/sanic/releases/tag/0.7.0", "label": "https://github.com/channelcat/sanic/releases/tag/0.7.0"}, {"href": "https://github.com/simonw/datasette/issues/171", "label": "#171"}]


id: changelog:id185
page: changelog
ref: id185
title: 0.13 (2017-11-24)
breadcrumbs: ["Changelog"]
content:
Search now applies to current filters. 
                     Combined search into the same form as filters. 
                     Closes  #133 
                 
                 
                     Much tidier design for table view header. 
                     Closes  #147 
                 
                 
                     Added  ?column__not=blah  filter. 
                     Closes  #148 
                 
                 
                     Row page now resolves foreign keys. 
                     Closes  #132 
                 
                 
                     Further tweaks to select/input filter styling. 
                     Refs  #86  - thanks for the help, @natbat! 
                 
                 
                     Show linked foreign key in table cells. 
                 
                 
                     Added UI for editing table filters. 
                     Refs  #86 
                 
                 
                     Hide FTS-created tables on index pages. 
                     Closes  #129 
                 
                 
                     Add publish to heroku support [Jacob Kaplan-Moss] 
                     datasette publish heroku mydb.db 
                     Pull request  #104 
                 
                 
                     Initial implementation of  ?_group_count=column . 
                     URL shortcut for counting rows grouped by one or more columns. 
                     ?_group_count=column1&_group_count=column2  works as well. 
                     SQL generated looks like this: 
                     select "qSpecies", count(*) as "count"
from Street_Tree_List
group by "qSpecies"
order by "count" desc limit 100 
                     Or for two columns like this: 
                     select "qSpecies", "qSiteInfo", count(*) as "count"
from Street_Tree_List
group by "qSpecies", "qSiteInfo"
order by "count" desc limit 100 
                     Refs  #44 
                 
                 
                     Added  --build=master  option to datasette publish and package. 
                     The  datasette publish  and  datasette package  commands both now accept an
                        optional  --build  argument. If provided, this can be used to specify a branch
                        published to GitHub that should be built into the container. 
                     This makes it easier to test code that has not yet been officially released to
                        PyPI, e.g.: 
                     datasette publish now mydb.db --branch=master 
                 
                 
                     Implemented  ?_search=XXX  + UI if a FTS table is detected. 
                     Closes  #131 
                 
                 
                     Added  datasette --version  support. 
                 
                 
                     Table views now show expanded foreign key references, if possible. 
                     If a table has foreign key columns, and those foreign key tables have
                         label_columns , the TableView will now query those other tables for the
                        corresponding values and display those values as links in the corresponding
                        table cells. 
                     label_columns are currently detected by the  inspect()  function, which looks
                        for any table that has just two columns - an ID column and one other - and
                        sets the  label_column  to be that second non-ID column. 
                 
                 
                     Don't prevent tabbing to "Run SQL" button ( #117 ) [Robert Gieseke] 
                     See comment in  #115 
                 
                 
                     Add keyboard shortcut to execute SQL query ( #115 ) [Robert Gieseke] 
                 
                 
                     Allow  --load-extension  to be set via environment variable. 
                 
                 
                     Add support for  ?field__isnull=1  ( #107 ) [Ray N] 
                 
                 
                     Add spatialite, switch to debian and local build ( #114 ) [Ariel Núñez] 
                 
                 
                     Added  --load-extension  argument to datasette serve. 
                     Allows loading of SQLite extensions. Refs  #110 .

references:
[{"href": "https://github.com/simonw/datasette/issues/133", "label": "#133"}, {"href": "https://github.com/simonw/datasette/issues/147", "label": "#147"}, {"href": "https://github.com/simonw/datasette/issues/148", "label": "#148"}, {"href": "https://github.com/simonw/datasette/issues/132", "label": "#132"}, {"href": "https://github.com/simonw/datasette/issues/86", "label": "#86"}, {"href": "https://github.com/simonw/datasette/issues/86", "label": "#86"}, {"href": "https://github.com/simonw/datasette/issues/129", "label": "#129"}, {"href": "https://github.com/simonw/datasette/issues/104", "label": "#104"}, {"href": "https://github.com/simonw/datasette/issues/44", "label": "#44"}, {"href": "https://github.com/simonw/datasette/issues/131", "label": "#131"}, {"href": "https://github.com/simonw/datasette/issues/117", "label": "#117"}, {"href": "https://github.com/simonw/datasette/issues/115", "label": "#115"}, {"href": "https://github.com/simonw/datasette/issues/115", "label": "#115"}, {"href": "https://github.com/simonw/datasette/issues/107", "label": "#107"}, {"href": "https://github.com/simonw/datasette/issues/114", "label": "#114"}, {"href": "https://github.com/simonw/datasette/issues/110", "label": "#110"}]


id: changelog:id200
page: changelog
ref: id200
title: 0.12 (2017-11-16)
breadcrumbs: ["Changelog"]
content:
Added  __version__ , now displayed as tooltip in page footer ( #108 ). 
                 
                 
                     Added initial docs, including a changelog ( #99 ). 
                 
                 
                     Turned on auto-escaping in Jinja. 
                 
                 
                     Added a UI for editing named parameters ( #96 ). 
                     You can now construct a custom SQL statement using SQLite named
                        parameters (e.g.  :name ) and datasette will display form fields for
                        editing those parameters.  Here’s an example  which lets you see the
                        most popular names for dogs of different species registered through
                        various dog registration schemes in Australia. 
                 
             
             
             
                 
                     Pin to specific Jinja version. ( #100 ). 
                 
                 
                     Default to 127.0.0.1 not 0.0.0.0. ( #98 ). 
                 
                 
                     Added extra metadata options to publish and package commands. ( #92 ). 
                     You can now run these commands like so: 
                     datasette now publish mydb.db \
    --title="My Title" \
    --source="Source" \
    --source_url="http://www.example.com/" \
    --license="CC0" \
    --license_url="https://creativecommons.org/publicdomain/zero/1.0/" 
                     This will write those values into the metadata.json that is packaged with the
                        app. If you also pass  --metadata=metadata.json  that file will be updated with the extra
                        values before being written into the Docker image. 
                 
                 
                     Added production-ready Dockerfile ( #94 ) [Andrew
                        Cutler] 
                 
                 
                     New  ?_sql_time_limit_ms=10  argument to database and table page ( #95 ) 
                 
                 
                     SQL syntax highlighting with Codemirror ( #89 ) [Tom Dyson]

references:
[{"href": "https://github.com/simonw/datasette/issues/108", "label": "#108"}, {"href": "https://github.com/simonw/datasette/issues/99", "label": "#99"}, {"href": "https://github.com/simonw/datasette/issues/96", "label": "#96"}, {"href": "https://australian-dogs.now.sh/australian-dogs-3ba9628?sql=select+name%2C+count%28*%29+as+n+from+%28%0D%0A%0D%0Aselect+upper%28%22Animal+name%22%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2013%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2014%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2015%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22AnimalName%22%29+as+name+from+%5BCity-of-Port-Adelaide-Enfield-Dog_Registrations_2016%5D+where+AnimalBreed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5BMitcham-dog-registrations-2015%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22DOG_NAME%22%29+as+name+from+%5Bburnside-dog-registrations-2015%5D+where+DOG_BREED+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28%22Animal_Name%22%29+as+name+from+%5Bcity-of-playford-2015-dog-registration%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5Bcity-of-prospect-dog-registration-details-2016%5D+where%22Breed+Description%22+like+%3Abreed%0D%0A%0D%0A%29+group+by+name+order+by+n+desc%3B&breed=pug", "label": "Here’s an example"}, {"href": "https://github.com/simonw/datasette/issues/100", "label": "#100"}, {"href": "https://github.com/simonw/datasette/issues/98", "label": "#98"}, {"href": "https://github.com/simonw/datasette/issues/92", "label": "#92"}, {"href": "https://github.com/simonw/datasette/issues/94", "label": "#94"}, {"href": "https://github.com/simonw/datasette/issues/95", "label": "#95"}, {"href": "https://github.com/simonw/datasette/issues/89", "label": "#89"}]


id: changelog:id210
page: changelog
ref: id210
title: 0.11 (2017-11-14)
breadcrumbs: ["Changelog"]
references: []
content:
Added  datasette publish now --force  option. 
                     This calls  now  with  --force  - useful as it means you get a fresh copy of datasette even if Now has already cached that docker layer. 
                 
                 
                     Enable  --cors  by default when running in a container.


id: changelog:id211
page: changelog
ref: id211
title: 0.10 (2017-11-14)
breadcrumbs: ["Changelog"]
references: [{"href": "https://github.com/simonw/datasette/issues/83", "label": "#83"}]
content:
Fixed  #83  - 500 error on individual row pages. 
                 
                 
                     Stop using sqlite WITH RECURSIVE in our tests. 
                     The version of Python 3 running in Travis CI doesn't support this.


id: changelog:id213
page: changelog
ref: id213
title: 0.9 (2017-11-13)
breadcrumbs: ["Changelog"]
references: []
content:
Added  --sql_time_limit_ms  and  --extra-options . 
                     The serve command now accepts  --sql_time_limit_ms  for customizing the SQL time
                        limit. 
                     The publish and package commands now accept  --extra-options  which can be used
                        to specify additional options to be passed to the datasite serve command when
                        it executes inside the resulting Docker containers.


id: changelog:id214
page: changelog
ref: id214
title: 0.8 (2017-11-13)
breadcrumbs: ["Changelog"]
content:
V0.8 - added PyPI metadata, ready to ship. 
                 
                 
                     Implemented offset/limit pagination for views ( #70 ). 
                 
                 
                     Improved pagination. ( #78 ) 
                 
                 
                     Limit on max rows returned, controlled by  --max_returned_rows  option. ( #69 ) 
                     If someone executes 'select * from table' against a table with a million rows
                        in it, we could run into problems: just serializing that much data as JSON is
                        likely to lock up the server. 
                     Solution: we now have a hard limit on the maximum number of rows that can be
                        returned by a query. If that limit is exceeded, the server will return a
                         "truncated": true  field in the JSON. 
                     This limit can be optionally controlled by the new  --max_returned_rows 
                        option. Setting that option to 0 disables the limit entirely.

references:
[{"href": "https://github.com/simonw/datasette/issues/70", "label": "#70"}, {"href": "https://github.com/simonw/datasette/issues/78", "label": "#78"}, {"href": "https://github.com/simonw/datasette/issues/69", "label": "#69"}]


id: csv_export:id1
page: csv_export
ref: id1
title: CSV export
breadcrumbs: []
content:
Any Datasette table, view or custom SQL query can be exported as CSV. 
         To obtain the CSV representation of the table you are looking, click the "this
            data as CSV" link. 
         You can also use the advanced export form for more control over the resulting
            file, which looks like this and has the following options: 
         
         
             
                 download file  - instead of displaying CSV in your browser, this forces
                    your browser to download the CSV to your downloads directory. 
             
             
                 expand labels  - if your table has any foreign key references this option
                    will cause the CSV to gain additional  COLUMN_NAME_label  columns with a
                    label for each foreign key derived from the linked table.  In this example 
                    the  city_id  column is accompanied by a  city_id_label  column. 
             
             
                 stream all rows  - by default CSV files only contain the first
                     max_returned_rows  records. This option will cause Datasette to
                    loop through every matching record and return them as a single CSV file. 
             
         
         You can try that out on  https://latest.datasette.io/fixtures/facetable?_size=4

references:
[{"href": "https://latest.datasette.io/fixtures/facetable.csv?_labels=on&_size=max", "label": "In this example"}, {"href": "https://latest.datasette.io/fixtures/facetable?_size=4", "label": "https://latest.datasette.io/fixtures/facetable?_size=4"}]


id: csv_export:csv-export-url-parameters
page: csv_export
ref: csv-export-url-parameters
title: URL parameters
breadcrumbs: ["CSV export"]
references: []
content:
The following options can be used to customize the CSVs returned by Datasette. 
             
                 
                     ?_header=off 
                     
                         This removes the first row of the CSV file specifying the headings - only the row data will be returned. 
                     
                 
                 
                     ?_stream=on 
                     
                         Stream all matching records, not just the first page of results. See below. 
                     
                 
                 
                     ?_dl=on 
                     
                         Causes Datasette to return a  content-disposition: attachment; filename="filename.csv"  header.


id: csv_export:streaming-all-records
page: csv_export
ref: streaming-all-records
title: Streaming all records
breadcrumbs: ["CSV export"]
references: []
content:
The  stream all rows  option is designed to be as efficient as possible -
                under the hood it takes advantage of Python 3 asyncio capabilities and
                Datasette's efficient  pagination  to stream back the full
                CSV file. 
             Since databases can get pretty large, by default this option is capped at 100MB -
                if a table returns more than 100MB of data the last line of the CSV will be a
                truncation error message. 
             You can increase or remove this limit using the  max_csv_mb  config
                setting. You can also disable the CSV export feature entirely using
                 allow_csv_stream .


