{"rowid": 301, "title": "New plugin hook: extra_template_vars", "content": "The extra_template_vars(template, database, table, columns, view_name, request, datasette) plugin hook allows plugins to inject their own additional variables into the Datasette template context. This can be used in conjunction with custom templates to customize the Datasette interface. datasette-auth-github uses this hook to add custom HTML to the new top navigation bar (which is designed to be modified by plugins, see #540 ).", "sections_fts": 7, "rank": null} {"rowid": 302, "title": "Secret plugin configuration options", "content": "Plugins like datasette-auth-github need a safe way to set secret configuration options. Since the default mechanism for configuring plugins exposes those settings in /-/metadata a new mechanism was needed. Secret configuration values describes how plugins can now specify that their settings should be read from a file or an environment variable: \n {\n \"plugins\": {\n \"datasette-auth-github\": {\n \"client_secret\": {\n \"$env\": \"GITHUB_CLIENT_SECRET\"\n }\n }\n }\n} \n These plugin secrets can be set directly using datasette publish . See Custom metadata and plugins for details. ( #538 and #543 )", "sections_fts": 7, "rank": null} {"rowid": 303, "title": "Facet by date", "content": "If a column contains datetime values, Datasette can now facet that column by date. ( #481 )", "sections_fts": 7, "rank": null} {"rowid": 304, "title": "Easier custom templates for table rows", "content": "If you want to customize the display of individual table rows, you can do so using a _table.html template include that looks something like this: \n {% for row in display_rows %}\n
{{ row[\"description\"] }} Category: {{ row.display(\"category_id\") }}
This line renders the original block:
\n{{ super() }}\n{% endblock %} \n \n \n --static option for datasette serve ( #160 ) \n You can now tell Datasette to serve static files from a specific location at a\n specific mountpoint. \n For example: \n datasette serve mydb.db --static extra-css:/tmp/static/css \n Now if you visit this URL: \n http://localhost:8001/extra-css/blah.css \n The following file will be served: \n /tmp/static/css/blah.css \n \n \n Canned query support. \n Named canned queries can now be defined in metadata.json like this: \n {\n \"databases\": {\n \"timezones\": {\n \"queries\": {\n \"timezone_for_point\": \"select tzid from timezones ...\"\n }\n }\n }\n} \n These will be shown in a new \"Queries\" section beneath \"Views\" on the database page. \n \n \n New datasette skeleton command for generating metadata.json ( #164 ) \n \n \n metadata.json support for per-table/per-database metadata ( #165 ) \n Also added support for descriptions and HTML descriptions. \n Here's an example metadata.json file illustrating custom per-database and per-\n table metadata: \n {\n \"title\": \"Overall datasette title\",\n \"description_html\": \"This is a description with HTML.\",\n \"databases\": {\n \"db1\": {\n \"title\": \"First database\",\n \"description\": \"This is a string description & has no HTML\",\n \"license_url\": \"http://example.com/\",\n \"license\": \"The example license\",\n \"queries\": {\n \"canned_query\": \"select * from table1 limit 3;\"\n },\n \"tables\": {\n \"table1\": {\n \"title\": \"Custom title for table1\",\n \"description\": \"Tables can have descriptions too\",\n \"source\": \"This has a custom source\",\n \"source_url\": \"http://example.com/\"\n }\n }\n }\n }\n} \n \n \n Renamed datasette build command to datasette inspect ( #130 ) \n \n \n Upgrade to Sanic 0.7.0 ( #168 ) \n https://github.com/channelcat/sanic/releases/tag/0.7.0 \n \n \n Package and publish commands now accept --static and --template-dir \n Example usage: \n datasette package --static css:extra-css/ --static js:extra-js/ \\\n sf-trees.db --template-dir templates/ --tag sf-trees --branch master \n This creates a local Docker image that includes copies of the templates/,\n extra-css/ and extra-js/ directories. You can then run it like this: \n docker run -p 8001:8001 sf-trees \n For publishing to Zeit now: \n datasette publish now --static css:extra-css/ --static js:extra-js/ \\\n sf-trees.db --template-dir templates/ --name sf-trees --branch master \n \n \n HTML comment showing which templates were considered for a page ( #171 )", "sections_fts": 7, "rank": null} {"rowid": 342, "title": "0.13 (2017-11-24)", "content": "Search now applies to current filters. \n Combined search into the same form as filters. \n Closes #133 \n \n \n Much tidier design for table view header. \n Closes #147 \n \n \n Added ?column__not=blah filter. \n Closes #148 \n \n \n Row page now resolves foreign keys. \n Closes #132 \n \n \n Further tweaks to select/input filter styling. \n Refs #86 - thanks for the help, @natbat! \n \n \n Show linked foreign key in table cells. \n \n \n Added UI for editing table filters. \n Refs #86 \n \n \n Hide FTS-created tables on index pages. \n Closes #129 \n \n \n Add publish to heroku support [Jacob Kaplan-Moss] \n datasette publish heroku mydb.db \n Pull request #104 \n \n \n Initial implementation of ?_group_count=column . \n URL shortcut for counting rows grouped by one or more columns. \n ?_group_count=column1&_group_count=column2 works as well. \n SQL generated looks like this: \n select \"qSpecies\", count(*) as \"count\"\nfrom Street_Tree_List\ngroup by \"qSpecies\"\norder by \"count\" desc limit 100 \n Or for two columns like this: \n select \"qSpecies\", \"qSiteInfo\", count(*) as \"count\"\nfrom Street_Tree_List\ngroup by \"qSpecies\", \"qSiteInfo\"\norder by \"count\" desc limit 100 \n Refs #44 \n \n \n Added --build=master option to datasette publish and package. \n The datasette publish and datasette package commands both now accept an\n optional --build argument. If provided, this can be used to specify a branch\n published to GitHub that should be built into the container. \n This makes it easier to test code that has not yet been officially released to\n PyPI, e.g.: \n datasette publish now mydb.db --branch=master \n \n \n Implemented ?_search=XXX + UI if a FTS table is detected. \n Closes #131 \n \n \n Added datasette --version support. \n \n \n Table views now show expanded foreign key references, if possible. \n If a table has foreign key columns, and those foreign key tables have\n label_columns , the TableView will now query those other tables for the\n corresponding values and display those values as links in the corresponding\n table cells. \n label_columns are currently detected by the inspect() function, which looks\n for any table that has just two columns - an ID column and one other - and\n sets the label_column to be that second non-ID column. \n \n \n Don't prevent tabbing to \"Run SQL\" button ( #117 ) [Robert Gieseke] \n See comment in #115 \n \n \n Add keyboard shortcut to execute SQL query ( #115 ) [Robert Gieseke] \n \n \n Allow --load-extension to be set via environment variable. \n \n \n Add support for ?field__isnull=1 ( #107 ) [Ray N] \n \n \n Add spatialite, switch to debian and local build ( #114 ) [Ariel N\u00fa\u00f1ez] \n \n \n Added --load-extension argument to datasette serve. \n Allows loading of SQLite extensions. Refs #110 .", "sections_fts": 7, "rank": null} {"rowid": 343, "title": "0.12 (2017-11-16)", "content": "Added __version__ , now displayed as tooltip in page footer ( #108 ). \n \n \n Added initial docs, including a changelog ( #99 ). \n \n \n Turned on auto-escaping in Jinja. \n \n \n Added a UI for editing named parameters ( #96 ). \n You can now construct a custom SQL statement using SQLite named\n parameters (e.g. :name ) and datasette will display form fields for\n editing those parameters. Here\u2019s an example which lets you see the\n most popular names for dogs of different species registered through\n various dog registration schemes in Australia. \n \n \n \n \n \n Pin to specific Jinja version. ( #100 ). \n \n \n Default to 127.0.0.1 not 0.0.0.0. ( #98 ). \n \n \n Added extra metadata options to publish and package commands. ( #92 ). \n You can now run these commands like so: \n datasette now publish mydb.db \\\n --title=\"My Title\" \\\n --source=\"Source\" \\\n --source_url=\"http://www.example.com/\" \\\n --license=\"CC0\" \\\n --license_url=\"https://creativecommons.org/publicdomain/zero/1.0/\" \n This will write those values into the metadata.json that is packaged with the\n app. If you also pass --metadata=metadata.json that file will be updated with the extra\n values before being written into the Docker image. \n \n \n Added production-ready Dockerfile ( #94 ) [Andrew\n Cutler] \n \n \n New ?_sql_time_limit_ms=10 argument to database and table page ( #95 ) \n \n \n SQL syntax highlighting with Codemirror ( #89 ) [Tom Dyson]", "sections_fts": 7, "rank": null} {"rowid": 344, "title": "0.11 (2017-11-14)", "content": "Added datasette publish now --force option. \n This calls now with --force - useful as it means you get a fresh copy of datasette even if Now has already cached that docker layer. \n \n \n Enable --cors by default when running in a container.", "sections_fts": 7, "rank": null} {"rowid": 345, "title": "0.10 (2017-11-14)", "content": "Fixed #83 - 500 error on individual row pages. \n \n \n Stop using sqlite WITH RECURSIVE in our tests. \n The version of Python 3 running in Travis CI doesn't support this.", "sections_fts": 7, "rank": null} {"rowid": 346, "title": "0.9 (2017-11-13)", "content": "Added --sql_time_limit_ms and --extra-options . \n The serve command now accepts --sql_time_limit_ms for customizing the SQL time\n limit. \n The publish and package commands now accept --extra-options which can be used\n to specify additional options to be passed to the datasite serve command when\n it executes inside the resulting Docker containers.", "sections_fts": 7, "rank": null} {"rowid": 347, "title": "0.8 (2017-11-13)", "content": "V0.8 - added PyPI metadata, ready to ship. \n \n \n Implemented offset/limit pagination for views ( #70 ). \n \n \n Improved pagination. ( #78 ) \n \n \n Limit on max rows returned, controlled by --max_returned_rows option. ( #69 ) \n If someone executes 'select * from table' against a table with a million rows\n in it, we could run into problems: just serializing that much data as JSON is\n likely to lock up the server. \n Solution: we now have a hard limit on the maximum number of rows that can be\n returned by a query. If that limit is exceeded, the server will return a\n \"truncated\": true field in the JSON. \n This limit can be optionally controlled by the new --max_returned_rows \n option. Setting that option to 0 disables the limit entirely.", "sections_fts": 7, "rank": null} {"rowid": 348, "title": "Facets", "content": "Datasette facets can be used to add a faceted browse interface to any database table.\n With facets, tables are displayed along with a summary showing the most common values in specified columns.\n These values can be selected to further filter the table. \n Here's an example : \n \n Facets can be specified in two ways: using query string parameters, or in metadata.json configuration for the table.", "sections_fts": 7, "rank": null} {"rowid": 349, "title": "Facets in query strings", "content": "To turn on faceting for specific columns on a Datasette table view, add one or more _facet=COLUMN parameters to the URL.\n For example, if you want to turn on facets for the city_id and state columns, construct a URL that looks like this: \n /dbname/tablename?_facet=state&_facet=city_id \n This works for both the HTML interface and the .json view.\n When enabled, facets will cause a facet_results block to be added to the JSON output, looking something like this: \n {\n \"state\": {\n \"name\": \"state\",\n \"results\": [\n {\n \"value\": \"CA\",\n \"label\": \"CA\",\n \"count\": 10,\n \"toggle_url\": \"http://...?_facet=city_id&_facet=state&state=CA\",\n \"selected\": false\n },\n {\n \"value\": \"MI\",\n \"label\": \"MI\",\n \"count\": 4,\n \"toggle_url\": \"http://...?_facet=city_id&_facet=state&state=MI\",\n \"selected\": false\n },\n {\n \"value\": \"MC\",\n \"label\": \"MC\",\n \"count\": 1,\n \"toggle_url\": \"http://...?_facet=city_id&_facet=state&state=MC\",\n \"selected\": false\n }\n ],\n \"truncated\": false\n }\n \"city_id\": {\n \"name\": \"city_id\",\n \"results\": [\n {\n \"value\": 1,\n \"label\": \"San Francisco\",\n \"count\": 6,\n \"toggle_url\": \"http://...?_facet=city_id&_facet=state&city_id=1\",\n \"selected\": false\n },\n {\n \"value\": 2,\n \"label\": \"Los Angeles\",\n \"count\": 4,\n \"toggle_url\": \"http://...?_facet=city_id&_facet=state&city_id=2\",\n \"selected\": false\n },\n {\n \"value\": 3,\n \"label\": \"Detroit\",\n \"count\": 4,\n \"toggle_url\": \"http://...?_facet=city_id&_facet=state&city_id=3\",\n \"selected\": false\n },\n {\n \"value\": 4,\n \"label\": \"Memnonia\",\n \"count\": 1,\n \"toggle_url\": \"http://...?_facet=city_id&_facet=state&city_id=4\",\n \"selected\": false\n }\n ],\n \"truncated\": false\n }\n} \n If Datasette detects that a column is a foreign key, the \"label\" property will be automatically derived from the detected label column on the referenced table. \n The default number of facet results returned is 30, controlled by the default_facet_size setting.\n You can increase this on an individual page by adding ?_facet_size=100 to the query string, up to a maximum of max_returned_rows (which defaults to 1000).", "sections_fts": 7, "rank": null} {"rowid": 350, "title": "Facets in metadata", "content": "You can turn facets on by default for specific tables by adding them to a \"facets\" key in a Datasette Metadata file. \n Here's an example that turns on faceting by default for the qLegalStatus column in the Street_Tree_List table in the sf-trees database: \n [[[cog\nfrom metadata_doc import metadata_example\nmetadata_example(cog, {\n \"databases\": {\n \"sf-trees\": {\n \"tables\": {\n \"Street_Tree_List\": {\n \"facets\": [\"qLegalStatus\"]\n }\n }\n }\n }\n}) \n ]]] \n [[[end]]] \n Facets defined in this way will always be shown in the interface and returned in the API, regardless of the _facet arguments passed to the view. \n You can specify array or date facets in metadata using JSON objects with a single key of array or date and a value specifying the column, like this: \n [[[cog\nmetadata_example(cog, {\n \"facets\": [\n {\"array\": \"tags\"},\n {\"date\": \"created\"}\n ]\n}) \n ]]] \n [[[end]]] \n You can change the default facet size (the number of results shown for each facet) for a table using facet_size : \n [[[cog\nmetadata_example(cog, {\n \"databases\": {\n \"sf-trees\": {\n \"tables\": {\n \"Street_Tree_List\": {\n \"facets\": [\"qLegalStatus\"],\n \"facet_size\": 10\n }\n }\n }\n }\n}) \n ]]] \n [[[end]]]", "sections_fts": 7, "rank": null} {"rowid": 351, "title": "Suggested facets", "content": "Datasette's table UI will suggest facets for the user to apply, based on the following criteria: \n For the currently filtered data are there any columns which, if applied as a facet... \n \n \n Will return 30 or less unique options \n \n \n Will return more than one unique option \n \n \n Will return less unique options than the total number of filtered rows \n \n \n And the query used to evaluate this criteria can be completed in under 50ms \n \n \n That last point is particularly important: Datasette runs a query for every column that is displayed on a page, which could get expensive - so to avoid slow load times it sets a time limit of just 50ms for each of those queries.\n This means suggested facets are unlikely to appear for tables with millions of records in them.", "sections_fts": 7, "rank": null} {"rowid": 352, "title": "Speeding up facets with indexes", "content": "The performance of facets can be greatly improved by adding indexes on the columns you wish to facet by.\n Adding indexes can be performed using the sqlite3 command-line utility. Here's how to add an index on the state column in a table called Food_Trucks : \n sqlite3 mydatabase.db \n SQLite version 3.19.3 2017-06-27 16:48:08\nEnter \".help\" for usage hints.\nsqlite> CREATE INDEX Food_Trucks_state ON Food_Trucks(\"state\"); \n Or using the sqlite-utils command-line utility: \n sqlite-utils create-index mydatabase.db Food_Trucks state", "sections_fts": 7, "rank": null} {"rowid": 353, "title": "Facet by JSON array", "content": "If your SQLite installation provides the json1 extension (you can check using /-/versions ) Datasette will automatically detect columns that contain JSON arrays of values and offer a faceting interface against those columns. \n This is useful for modelling things like tags without needing to break them out into a new table. \n Example here: latest.datasette.io/fixtures/facetable?_facet_array=tags", "sections_fts": 7, "rank": null} {"rowid": 354, "title": "Facet by date", "content": "If Datasette finds any columns that contain dates in the first 100 values, it will offer a faceting interface against the dates of those values.\n This works especially well against timestamp values such as 2019-03-01 12:44:00 . \n Example here: latest.datasette.io/fixtures/facetable?_facet_date=created", "sections_fts": 7, "rank": null} {"rowid": 355, "title": "JSON API", "content": "Datasette provides a JSON API for your SQLite databases. Anything you can do\n through the Datasette user interface can also be accessed as JSON via the API. \n To access the API for a page, either click on the .json link on that page or\n edit the URL and add a .json extension to it.", "sections_fts": 7, "rank": null} {"rowid": 356, "title": "Default representation", "content": "The default JSON representation of data from a SQLite table or custom query\n looks like this: \n {\n \"ok\": true,\n \"rows\": [\n {\n \"id\": 3,\n \"name\": \"Detroit\"\n },\n {\n \"id\": 2,\n \"name\": \"Los Angeles\"\n },\n {\n \"id\": 4,\n \"name\": \"Memnonia\"\n },\n {\n \"id\": 1,\n \"name\": \"San Francisco\"\n }\n ],\n \"truncated\": false\n} \n \"ok\" is always true if an error did not occur. \n The \"rows\" key is a list of objects, each one representing a row. \n The \"truncated\" key lets you know if the query was truncated. This can happen if a SQL query returns more than 1,000 results (or the max_returned_rows setting). \n For table pages, an additional key \"next\" may be present. This indicates that the next page in the pagination set can be retrieved using ?_next=VALUE .", "sections_fts": 7, "rank": null} {"rowid": 357, "title": "Different shapes", "content": "The _shape parameter can be used to access alternative formats for the\n rows key which may be more convenient for your application. There are three\n options: \n \n \n ?_shape=objects - \"rows\" is a list of JSON key/value objects - the default \n \n \n ?_shape=arrays - \"rows\" is a list of lists, where the order of values in each list matches the order of the columns \n \n \n ?_shape=array - a JSON array of objects - effectively just the \"rows\" key from the default representation \n \n \n ?_shape=array&_nl=on - a newline-separated list of JSON objects \n \n \n ?_shape=arrayfirst - a flat JSON array containing just the first value from each row \n \n \n ?_shape=object - a JSON object keyed using the primary keys of the rows \n \n \n _shape=arrays looks like this: \n {\n \"ok\": true,\n \"next\": null,\n \"rows\": [\n [3, \"Detroit\"],\n [2, \"Los Angeles\"],\n [4, \"Memnonia\"],\n [1, \"San Francisco\"]\n ]\n} \n _shape=array looks like this: \n [\n {\n \"id\": 3,\n \"name\": \"Detroit\"\n },\n {\n \"id\": 2,\n \"name\": \"Los Angeles\"\n },\n {\n \"id\": 4,\n \"name\": \"Memnonia\"\n },\n {\n \"id\": 1,\n \"name\": \"San Francisco\"\n }\n] \n _shape=array&_nl=on looks like this: \n {\"id\": 1, \"value\": \"Myoporum laetum :: Myoporum\"}\n{\"id\": 2, \"value\": \"Metrosideros excelsa :: New Zealand Xmas Tree\"}\n{\"id\": 3, \"value\": \"Pinus radiata :: Monterey Pine\"} \n _shape=arrayfirst looks like this: \n [1, 2, 3] \n _shape=object looks like this: \n {\n \"1\": {\n \"id\": 1,\n \"value\": \"Myoporum laetum :: Myoporum\"\n },\n \"2\": {\n \"id\": 2,\n \"value\": \"Metrosideros excelsa :: New Zealand Xmas Tree\"\n },\n \"3\": {\n \"id\": 3,\n \"value\": \"Pinus radiata :: Monterey Pine\"\n }\n] \n The object shape is only available for queries against tables - custom SQL\n queries and views do not have an obvious primary key so cannot be returned using\n this format. \n The object keys are always strings. If your table has a compound primary\n key, the object keys will be a comma-separated string.", "sections_fts": 7, "rank": null} {"rowid": 358, "title": "Pagination", "content": "The default JSON representation includes a \"next_url\" key which can be used to access the next page of results. If that key is null or missing then it means you have reached the final page of results. \n Other representations include pagination information in the link HTTP header. That header will look something like this: \n link: