Getting Started

The Enigma API allows users to download datasets, query metadata, or perform server side operations on tables in Enigma. All calls to the API are made through a RESTful protocol and require an API key. The Enigma API is served over HTTPS. To ensure data privacy, unencrypted HTTP is not supported.

To receive an API key, please contact your Enigma sales representative, or send an email to support@enigma.io

The general syntax for the api is as follows:
''<version>/<endpoint>/<api key>/<datapath>/<parameters>

API Key & Limits


Datapaths


All of the data in Enigma is addressable through a datapath syntax. Datapaths work like URI addresses and can refer either to specific data tables or to parent nodes under which data tables are organized. Parent nodes contain metadata and contextual information about the children data tables organized under them.

The first two example datapaths (right) refer to specific tables in Enigma. They share three parent datapaths in common – us, us.gov, and us.gov.whitehouse. Parent datapaths are not tables, but rather organizing nodes or database names that refer to the places in the real world where particular datasets come from.

In the example above, both tables are from the United States, from the United States' Federal Government, and from the United States' Federal Government's White House. Each of those parent nodes have metadata associated with them that describe what they are and what children tables they contain. API users can issue a /meta/ query to retrieve metadata for any datapath, while /data/ queries return results for table datapaths only.

Example datapaths
us.gov.whitehouse.visitor-list
us.gov.whitehouse.salaries.2011
us.gov.whitehouse
                


Metadata API

All datapaths can be queried for their metadata. The returned object of the metadata query varies depending on whether or not the datapath is a parent or a table.

Parameters


page/

<number>
The 'children_tables' response attribute (see below) is limited to 50 results per page – use this parameter to specify which page to return.

Valid parameter formats


Slash-separated

<datapath>/key/value/key/value/...

Query string

<datapath>/?key=value&key=value...

JSON

<datapath>/{"key":"value", "key":"value", ...}

Parent node response attributes


datapath:

string, see datapaths

info: {

 

result_type:

string, value is 'parent'

children_tables_limit:

integer

children_tables_total:

integer

current_page:

integer

total_pages:

integer

}

 

result: {

 

path:

array of nodes in path

immediate_nodes:

array of child nodes (not tables)

children_tables:

array of child tables

}

 

Table node response attributes


datapath:

string, see datapaths

info: {

 

result_type:

string, value is 'table'

}

 

result: {

 

path:

array of nodes in path

columns:

array of table column objects

db_boundary_datapath:

string, see datapaths

db_boundary_label:

string

db_boundary_tables:

array of related tables

ancestor_datapaths:

array of ancestor datapath strings

documents:

array of source documents

metadata:

array of metadata objects

}

 



Data API

Table datapaths can be queried for the data they contain. Data queries may be filtered, sorted and paginated using the provided URL parameters.

For large tables and tables with a large number of columns, data API calls may take some time to complete. API users are advised to make use of the "select" and/or "limit" parameters whenever possible to improve performance.

Parameters


limit

<number>
Limit the number of rows returned (max. 500). Defaults to 500.

select

<column1>,<column2>, ...
Comma-delimited list of columns to be returned with each row. Default is to return all columns.

<query>
Filter results by only returning rows that match a search query. Multiple search parameters may be provided.

By default this searches the entire table for matching text. To search particular fields only, use the query format "@fieldname query".

To match multiple queries within a single search parameter, the | (or) operator can be used eg. "query1|query2". See the "Complex Data Search" example on the right for a demonstration.

where

Filter results with a SQL-style "where" clause. Only applies to numerical and date columns – use the "search" parameter for strings. Multiple where parameters may be provided.

<column><operator><value>
Valid operators are >=, >, =, !=, <, and <=.

<column> [not] in (<value>,<value>,...)
Match rows where column matches one of the provided values.

<column> [not] between <value> and <value>
Match rows where column lies within range provided (inclusive).

conjunction

"and" or "or"
Only applicable when more than one "search" or "where" parameter is provided. Defaults to "and".

sort

<column><+|->
Sort rows by a particular column in a given direction. + denotes ascending order, - denotes descending.

page

<number>
Paginate row results and return the nth page of results. Pages are calculated based on the current limit, which defaults to 500.

Valid parameter formats


Slash-separated

<datapath>/key/value/key/value/...

Query string

<datapath>/?key=value&key=value...

JSON

<datapath>/{"key":"value", "key":"value", ...}

Data response attributes


datapath:

string, see datapaths

info: {

 

rows_limit:

integer

current_page:

integer

total_pages:

integer

total_results:

integer

}

 

result:

array of row objects



Stats API

Table datapaths can be queried by column for statistics on the data that it contains. Like data queries, stats queries may be filtered, sorted and paginated using the provided URL parameters.

For large tables and tables with a large number of columns, stats API calls may take some time to complete. API users are advised to make use of the "select" and/or "limit" parameters whenever possible to improve performance.

Parameters


select

<column>
Column to generate statistics for. Required.

operation

<operation>
Operation to run on a given column. For a numerical column, valid operations are sum,avg,stddev,variance,max,min and frequency. For a date column, valid operations are max,min and frequency. For all other columns, the only valid operation is frequency. Defaults to all available operations based on the column's type.

by

<operation>
Compound operation to run on a given pair of columns. Valid compound operations are sum and avg. When running a compound operation query, the "of" parameter is required (see below).

of

<column>
Numerical column to compare against when running a compound operation. Required when using the "by" parameter. Must be a numerical column.

limit

<number>
Limit the number of frequency, compound sum, or compound average results returned (max. 500). Defaults to 500.

search

where

See Data where.

conjunction

sort

<+|->
Sort frequency, compound sum, or compound average results in a given direction. + denotes ascending order, - denotes descending.

page

<number>
Paginate frequency, compound sum, or compound average results and return the nth page of results. Pages are calculated based on the current limit, which defaults to 500.

Valid parameter formats


Slash-separated

<datapath>/key/value/key/value/...

Query string

<datapath>/?key=value&key=value...

JSON

<datapath>/{"key":"value", "key":"value", ...}

Stats response attributes


datapath:

string, see datapaths

info: {

 

column:

object containing column metadata

operations:

array of operations performed

rows_limit:

integer

current_page:

integer

total_pages:

integer

total_results:

integer

}

 

result:

object of operation results



Export API

API users can request exports of table datapaths. Tables are exported as Gzipped CSV files. Linux and Mac users can open .gzip files without additional software. Windows users can download 7-Zip to open .gzip files.

Exports of large tables may take some time, so exports are processed asynchronously. When the export API is called, an export is queued and the API immediately returns a URL pointing to the future location of the exported file. Users should poll the URL until the file becomes available.

Parameters


select

<column1>,<column2>, ...
Comma-delimited list of columns to be export. Default is to export all columns.

search

where

See Data where.

conjunction

sort

See Data sort.

Valid parameter formats


Slash-separated

<datapath>/key/value/key/value/...

Query string

<datapath>/?key=value&key=value...

JSON

<datapath>/{"key":"value", "key":"value", ...}

Export response attributes


datapath:

string, see datapaths

export_url:

string, URL of Gzipped CSV

head_url:

string, URL to poll via HEAD requests



Limits API

Determine how many requests remain in the current time period.

Limits response attributes


period:

Time period after which remaining request counts are reset, e.g. "monthly" means request counts will reset at the end of the month.

data:

Number of data requests remaining

stats:

Number of stats requests remaining

meta:

Number of metadata requests remaining

export:

Number of export requests remaining

seconds_remaining:

Number of seconds until the next time period begins



Community Clients

A number of developers have written clients to interact with the Enigma API. We do not directly support these libraries, but we hope you find them useful! If you have a question about these projects or wish to submit one of your own, please contact us at contact@enigma.io.

R:
https://github.com/ropengov/enigma

Ruby:
https://github.com/scpike/enigma

Go:
https://github.com/mohamedattahri/go-enigma