• Triple Pattern Fragments API
  • Authentication
  • Using OAuth2 authorization
  • RDF exports
  • Opendatasoft RDF data
  • RML Mapping
  • Triple Pattern Fragments API

    Opendatasoft datasets can be queried using Triple Pattern Fragments (TPF). This is an approach for querying linked data described with Resource Description Framework (RDF) without using traditional SPARQL Endpoints. More information on TPF here.

    Concepts

    The TPF API enables triple pattern querying over Opendatasoft datasets.

    A triple representing the fact that Nero was born in Rome

    roman:Nero roman:birth_cty "Rome"^^xsd:string
    

    A triple can be seen as a fact described by a subject, a predicate and an object. Those 3 elements are represented with a Unique Resource Identifier (URI). An object can also be described with a Literal.

    A triple pattern matching all subjects born in Rome

    ?s roman:birth_cty "Rome"^^xsd:string
    

    A triple pattern is a simple query where subject, predicate or object can be variables in order to match triples.

    More complex queries can be run using a TPF client See here.

    Service address

    The service can be reached at the following entry address.

    GET http://documentation-resources.opendatasoft.com/api/tpf/{DATASET_ID}

    For this documentation, the domain http://documentation-resources.opendatasoft.com will be used as an example but it can be replaced with any other custom domain name.

    The TPF API supports GET HTTP methods.

    Parameters

    TPF API call with parameters

    GET https://documentation-resources.opendatasoft.com/api/tpf/roman-emperors/?predicate=http%3A%2F%2Fdocumentation-resources.opendatasoft.com%3A8000%2Fld%2Fontologies%2Froman-emperors%2Fname&object=%22Nero%22%5E%5Ehttp%3A%2F%2Fwww.w3.org%2F2001%2FXMLSchema%23string
    

    When the HTTP GET method is used, the parameters are appended to the URL using a Keyword Value Pair (KVP) encoding. Note that the parameter values have to be URL encoded.

    Here is the list of the 4 parameters supported by the TPF API:

    Parameter Description Possible values Optionality and use
    subject subject of the triple pattern URI (Optional)
    predicate predicate of the triple pattern URI (Optional)
    object object of the triple pattern URI or Literal (Optional)
    page result page to retrieve INTEGER (Optional)

    TPF Client

    A TPF client decomposes a SPARQL query into simple triple patterns. Thus, it can be used to run SPARQL queries over Opendatasoft TPF API.

    Multiple implementations of TPF client can be found here. Online version of the client available here can also be used (Warning, this service does not support HTTPS).

    Parameters

    The following list describes the 2 parameters used by any TPF client:

    Parameter Description Optionality
    datasources TPF API address of the dataset to query (Mandatory)
    query SPARQL query to execute (Mandatory)

    SPARQL query language

    a SPARQL query to retrieve roman emperors whose reign start after AD 14.

    PREFIX roman: <https://documentation-resources.opendatasoft.com/ld/ontologies/roman-emperors/>
    
    SELECT ?name WHERE {
      ?s roman:reign_start ?date .
        FILTER (?date > "0014-12-31T00:00:00+00:00"^^xsd:dateTime)
      ?s  roman:name ?name .
    }
    

    SPARQL Protocol and RDF Query Language (SPARQL) is a SQL-like query language that can be used with a TPF client to access Opendatasoft datasets described in RDF. Such query can be used to run multiple triple patterns, filters and more.

    The following SPARQL query is composed of 2 triple patterns and a filter. See the usage of the PREFIX declaration to shorten predicate URI's. This SPARQL query can be executed here (Warning, this service does not support HTTPS).

    Authentication

    An authenticated user can be granted access to restricted datasets and benefit from extended quotas for API calls. The API features an authentication mechanism for users to be granted their specific authorizations.

    For the platform to authenticate a user, you need to either:

    Finding and generating API keys

    API keys are managed via your user profile page at https://<youropendatasoftportal>.com/account/ or by clicking on your name in the header.

    Link to account settings

    Go to the tab named My API keys to see your existing API keys, revoke them and create new ones.

    Account's API keys page

    Providing API keys within requests

    Unauthenticated request on a private portal

    > GET https://private-portal.opendatasoft.com/api/v2/catalog/datasets/ HTTP/1.1
    
    < HTTP/1.0 401 Unauthorized
    

    Authenticated request using an Authorization: Apikey <API_KEY> header

    > GET https://private-portal.opendatasoft.com/api/v2/catalog/datasets/ HTTP/1.1
    Authorization: Apikey 7511e8cc6d6dbe65f9bc8dae19e08c08a2cab96ef45a86112d303eee
    
    < HTTP/1.0 200 OK
    
    {
        "total_count": 4,
        "links": [{
            "href": "https://private-portal.opendatasoft.com/api/v2/catalog/datasets?include_app_metas=False&limit=10&offset=0",
            "rel": "self"
        }, {
            "href": "https://private-portal.opendatasoft.com/api/v2/catalog/datasets?include_app_metas=False&limit=10&offset=0",
            "rel": "first"
        }, {
            "href": "https://private-portal.opendatasoft.com/api/v2/catalog/datasets?include_app_metas=False&limit=10&offset=0",
            "rel": "last"
        }],
        "datasets": [...]
    }
    

    Authenticated request using an API key as a query parameter

    > GET https://private-portal.opendatasoft.com/api/v2/catalog/datasets/?apikey=7511e8cc6d6dbe65f9bc8dae19e08c08a2cab96ef45a86112d303eee HTTP/1.1
    
    < HTTP/1.0 200 OK
    
    {
        "total_count": 4,
        "links": [{
            "href": "https://private-portal.opendatasoft.com/api/v2/catalog/datasets?include_app_metas=False&limit=10&offset=0",
            "rel": "self"
        }, {
            "href": "https://private-portal.opendatasoft.com/api/v2/catalog/datasets?include_app_metas=False&limit=10&offset=0",
            "rel": "first"
        }, {
            "href": "https://private-portal.opendatasoft.com/api/v2/catalog/datasets?include_app_metas=False&limit=10&offset=0",
            "rel": "last"
        }],
        "datasets": [...]
    }
    

    If you try to access a private portal's catalog without being authenticated, the API returns a 401 Unauthorized error.

    After generating an API key, you can use it to make authenticated requests. Depending on the permissions granted to the user for which the API key has been created, the JSON response contains only data about the datasets this user can access on the portal.

    It is good practice to pass the API key to the Authorization header in the following format:

    Authorization: Apikey <API_KEY>

    Alternatively, you can pass the API key as a query parameter in the following format:

    apikey=<API_KEY>

    Replace <API_KEY>with your API key.

    Using OAuth2 authorization

    Overview

    Opendatasoft implements the OAuth2 authorization flow, allowing third party application makers to access the data hosted on an Opendatasoft platform on behalf of a user while never having to deal with a password, thus avoiding any user credential to be compromised.

    The Opendatasoft OAuth2 authorization flow is compliant with RFC 6749 and makes use of Bearer Tokens in compliance with RFC 6750.

    Application developers who want to use the Opendatasoft APIs with OAuth2 must go through the following steps, which will be explained in this section.

    1. Register their application with the Opendatasoft platform.
    2. Request approval from users via an OAuth2 authorization grant.
    3. Request a bearer token that will allows them to query the Opendatasoft platform APIs for a limited amount of time.
    4. Refresh the Bearer Token when it expires.

    Currently, applications are registered on a specific domain and can only access data on this domain.

    Register an application for OAuth2 authentication

    OAuth2 applications management interface

    1. Go to the My applications tab of your account page on the domain you want to register the application on.
    2. Fill the registration form with the following information:
      • Application name: the name of the application
      • Type:
        • confidential: client password is kept secret from the user and only used from a trusted environment (e.g: a web service, where the client password is stored server-side and never sent to the user)
        • public: client password is embedded in a client-side application, making it potentially available to the world (e.g: a mobile or desktop application)
      • Redirection URL: the URL users will be redirected to after they have granted you permission to access their data
    3. Store the resulting client ID and client secret that will be needed to perform the next steps.

    Getting an authorization grant

    Example call to /oauth2/authorize/

    GET /oauth2/authorize/?
        client_id=123456789&
        redirect_uri=https://example.com&
        response_type=code&
        state=ilovedata&
        scope=all HTTP/1.1
    

    To get an authorization grant from a user:

    1. Redirect them to /oauth2/authorize/ with the appropriate query parameters.
    2. The user will then be authenticated in the platform and redirected to a page identifying your application.
    3. From there, the user will review the information you filled in the form described above and the scope of the requested access, and grant your application the right to access their data.
    4. Once the user has accepted those terms, they will be redirected to your application's redirection URL with query parameters describing your authorization grant.

    The query parameters you need to supply when redirecting the user are the following:

    Redirection following a successful authorization

    HTTP/1.0 302 FOUND
    Location: https://example.com?state=ilovedata&code=gKnAQc2yIfdz2mY25xxgpTY2uyG5Sv
    

    The authorization grant redirect will have these values:

    The 30-character authorization code must now be converted into a bearer token within 1 hour before expiring.

    Converting an authorization grant to a bearer token

    Example call to /oauth2/token/

    POST /oauth2/token/ HTTP/1.1
    
    client_id=cid&
        client_secret=csc&
        grant_type=authorization_code&
        code=GokshWxRFXmW0MaLHkDv5HrG6wieGs&
        scopes=all&
        redirect_uri=https://example.com&
        state=ilovedata
    

    To receive a bearer token, convert the previously obtained authorization grant via a POST request to /oauth2/token/ with the following parameters:

    Alternative call with an Authorization header

    POST /oauth2/token/ HTTP/1.1
    Authorization: Basic Y2lkOmNzYw==
    
    grant_type=authorization_code&
        code=GokshWxRFXmW0MaLHkDv5HrG6wieGs&
        scopes=all&
        redirect_uri=https://example.com&state=ilovedata
    

    Alternatively, you can pass your client ID and client secret through the Authorization header

    Example response for a bearer token request

    HTTP/1.0 200 OK
    Content-Type: application/json
    
    {
        "access_token": "9kxoTUYvSxnAiMpv008NBqRiqk5xWt",
        "expires_in": 3600,
        "token_type": "Bearer",
        "state": "ilovedata",
        "scope": "all",
        "refresh_token": "jFfDUcsK9zzNMs1zwczzJxGrimPtmf"
    }
    

    The response to this request is a JSON representation of a bearer token, which contains the following values:

    Using the bearer token

    Using the token as a query parameter

    GET /api/end/point?access_token=9kxoTUYvSxnAiMpv008NBqRiqk5xWt HTTP/1.1
    

    Using the token in an Authorization header

    GET /api/end/point HTTP/1.1
    Authorization: Bearer 9kxoTUYvSxnAiMpv008NBqRiqk5xWt
    

    Using the token in the request body

    GET /api/end/point HTTP/1.1
    
    access_token=9kxoTUYvSxnAiMpv008NBqRiqk5xWt
    

    The bearer token can be passed along requests for authentication in three different ways:

    Refreshing a bearer token

    Example token refresh call

    POST /oauth2/token/ HTTP/1.1
    
    client_id=cid&
        client_secret=csc&
        grant_type=refresh_token&
        refresh_token=jFfDUcsK9zzNMs1zwczzJxGrimPtmf&
        scopes=all&
        redirect_uri=https://example.com&
        state=ilovedata
    

    To refresh an expired bearer token, send a request to the /oauth2/token/ endpoint, with the following query parameters:

    The response to this request is identical to the bearer token response.

    RDF exports

    Opendatasoft datasets can be exported in multiple RDF formats. More information on RDF here.

    RDF serialization formats

    Opendatasoft RDF datasets are available in 4 serialization formats:

    Format name Description Syntax specification
    n3 Compact and human-friendly line based format Here
    turtle Simplified and RDF-only subset of n3 format Here
    rdfxml XML-based format Here
    jsonld Json-based format Here

    Service address

    Export a dataset using turtle format

    GET http://documentation-resources.opendatasoft.com/api/v2/catalog/datasets/roman-emperors/exports/turtle
    

    RDF exports of datasets are available at this address and use the format name in the previous table.

    GET http://documentation-resources.opendatasoft.com/api/v2/catalog/datasets/{DATASET_ID}/exports/{FORMAT_NAME}

    Opendatasoft RDF data

    Data retrieved with the Opendatasoft TPF API is in triple format described with RDF. Data in such format can use ontologies (vocabularies) to define their meaning.

    Default Structure

    By default, Opendatasoft datasets are not enriched with ontologies. A direct mapping of datasets to triple is used and follows 3 rules:

    Data resulting from this transformation describe dataset structure and contents but not its semantic.

    Edit Structure

    Opendatasoft datasets RDF structure can be described using YARRRML the YAML representation of RML (RDF Mapping Language). Editing the RML mapping of a dataset modifies the structure and resources of the RDF dataset. Thus, it has an impact on RDF features such as TPF API, RDF exports, Classes filter, Properties filter, etc.

    Classes and Properties filters

    The Classes and Properties filters contain all the classes and properties used in the RML mapping of the dataset. They can be used to search datasets representing a specific concept such as Persons, Cars or properties such as age or fuel consumption.

    RML Mapping

    RML (RDF Mapping Language) is used by the Opendatasoft platform to describe how a dataset should be represented in RDF.

    This section explains how to create an RML mapping using YARRRML. YARRRML is a representation of RML using the YAML syntax.

    Running example

    To illustrate how RML mapping works, roman-emperors dataset will be used as an example. It contains information about Roman emperors from 26 BC to 395 AD.

    RML rules

    An RML mapping contains rules that define how facts of an RDF dataset are generated from an Opendatasoft dataset. In other words, it defines how each record is translated into triples (subject of a resource, predicate, object).

    There are 3 types of rules that define:

    Start a YARRRML mapping

    An RML mapping that will describe 3 resources

    mappings:
      person:
      city:
      province:
    

    YARRRML documents start with a mappings element. Child elements of 'mappings' are called mapping keys. They identify all resources that will be generated. Chosen keys have no influence on mapping as long as they are unique.

    Define the IRI of a resource

    A rule defining how the IRI of the resources representing persons will be generated

    mappings:
      person:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/$(name)
    

    An example of 2 IRIs generated by the previous rule

    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Augustus>
    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Nero>
    

    The subject child element of a mapping key is used to define how the IRI of the corresponding resource is generated.

    Value of the subject key is a template. A template is an IRI that can contain references to fields of the dataset.

    A reference is $(field_name) where field_name is the name of a field of the dataset. During the RDF transformation of a record, references are replaced by the value of the referenced field.

    An IRI will be generated for each record of the dataset. If two generated IRIs are equal, resources will be merged as one.

    Use Opendatasoft platform as an IRI provider

    An example of dereferenceable Opendatasoft IRI

    mappings:
      person:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/$(name)
    

    Opendatasoft provides an endpoint that makes the IRIs for resources dereferenceable according to the Principles of Linked Data.

    An IRI is said to be dereferenceable if it can be accessed to get more information about the resource it identifies. In other words, this endpoint enables to generate IRIs that uniquely describe a semantic resource on Opendatasoft, and leads to the corresponding, up-to-date data as long as resource IRIs start with:

    https://{DOMAIN_ID}.opendatasoft.com/ld/resources/{DATASET_ID}/{RESOURCE_CLASS}/...

    Where {DOMAIN_ID} is the id of a domain containing the dataset, {DATASET_ID} is the id of the dataset on the domain {DOMAIN_ID} and {RESOURCE_CLASS} is a word representing the class of the resource.

    Define the classes of a resource

    An example of rules that associate 2 classes to a resource

    mappings:
      person:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/$(name)
        predicateobjects:
        - [a, 'http://schema.org/Person']
        - [a, 'http://xmlns.com/foaf/0.1/Person']
    

    An example of 2 triples generated by the previous rule

    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Nero> a <http://schema.org/Person>,
          <http://xmlns.com/foaf/0.1/Person> ;
    

    The predicateobjects element is used to define predicates and objects of a resource. It is a child element of a mapping key. Value of a predicateobjects child element is a list of arrays [predicate, object].

    To define the class of a resource, predicate should be the IRI of the property type (http://www.w3.org/1999/02/22-rdf-syntax-ns#type) and object should be the IRI of the class.

    Define other predicates and objects

    An example of a reference literal stating that the label identifying an emperor is the value in the field name

    mappings:
      person:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/$(name)
        predicateobjects:
        - ['http://www.w3.org/2000/01/rdf-schema#label', $(name)]
    

    An example of 2 triples generated by the previous rule

    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Augustus>  <http://www.w3.org/2000/01/rdf-schema#label>  "Augustus".
    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Nero>  <http://www.w3.org/2000/01/rdf-schema#label>  "Nero".
    

    An example of a constant literal stating that the position of all person resources is emperor

    mappings:
      person:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/$(name)
        predicateobjects:
        - ['http://purl.org/vocab/bio/0.1/position', 'Emperor']
    

    An example of 2 triples generated by the previous rule

    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Augustus>  <http://purl.org/vocab/bio/0.1/position>  "Emperor" .
    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Nero>  <http://purl.org/vocab/bio/0.1/position>  "Emperor" .
    

    As seen in the previous section, value of a predicateobjects child element is a list of arrays [predicate, object].

    To define a new predicate and object, predicate should be the IRI of any property of a vocabulary and object should be either:

    A literal is a value such as a string, a boolean, a number or a date.

    A reference literal is a string value that contain references to fields of the dataset.

    A reference is a string $(field_name) where field_name is the name of a field in the dataset.

    During the RDF transformation of a record, if literal contains references, each reference is replaced by the value of the field referenced.

    A constant literal is any literal that does not contain reference to fields of the dataset.

    When literal is a constant, the same literal is generated for all resources.

    Define the datatype of a literal value

    An example of a reference literal with a defined datatype

    mappings:
      person:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/$(name)
        predicateobjects:
        - ['http://dbpedia.org/ontology/birthDate', $(birth), 'http://www.w3.org/2001/XMLSchema#dateTime']
    

    An example of triple generated by the previous rule

    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Augustus>  <http://dbpedia.org/ontology/birthDate>  "0062-09-23T00:53:28+00:53"^^xsd:dateTime .
    

    To define the datatype of a literal value, a datatype element is added to the array: [predicate, object (literal), datatype] where datatype is an IRI of a datatype in the XML Schema Definition Language (XSD).

    Define the language of a literal value

    An example of a constant literal with a defined language.

    mappings:
      person:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/$(name)
        predicateobjects:
        - ['http://purl.org/vocab/bio/0.1/position', 'Emperor', en~lang]
    

    An example of triple generated by the previous rule

    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Augustus>  <http://purl.org/vocab/bio/0.1/position>  "Emperor"@en .
    

    To define the datatype of a literal value, a language element is added to the array: [predicate, object (literal), {language}~lang] where {language} is an ISO 639-1 Language Code.

    An example of a rule stating the resource identified by the key 'city' is the birth place of the resource identified by the key 'person'

    mappings:
      person:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/$(name)
        predicateobjects:
        - ['http://dbpedia.org/ontology/birthPlace', city]
      city:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/City/$(birth_cty)
        predicateobjects:
        - [a, 'http://schema.org/City']
    

    An example of 2 triples generated by the previous rule

    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Augustus>  <http://dbpedia.org/ontology/birthPlace>  <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/City/Rome> .
    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/City/Rome>  a  <http://schema.org/City>  .
    

    A link between two resources is an array [predicate, object] where predicate is the IRI of the property that links the resources and object a key of another resource used in the mappings parent element.

    More information

    A complete example of RML mapping in YARRRML syntax

    mappings:
      person:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/$(name)
        predicateobjects:
        - [a, 'http://schema.org/Person']
        - [a, 'http://xmlns.com/foaf/0.1/Person']
        - ['http://www.w3.org/2000/01/rdf-schema#label', $(name)]
        - ['http://purl.org/vocab/bio/0.1/position', 'Emperor', en~lang]
        - ['http://dbpedia.org/ontology/deathDate', $(death), 'http://www.w3.org/2001/XMLSchema#dateTime']
        - ['http://dbpedia.org/ontology/birthDate', $(birth), 'http://www.w3.org/2001/XMLSchema#dateTime']
        - ['http://dbpedia.org/ontology/birthPlace', city]
        - ['http://dbpedia.org/ontology/birthPlace', province]
      city:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/City/$(birth_cty)
        predicateobjects:
        - [a, 'http://schema.org/City']
        - [a, 'http://schema.org/Place']
        - [a, 'http://rdfs.co/juso/SpatialThing']
        - ['http://www.w3.org/2000/01/rdf-schema#label', $(birth_cty)]
      province:
        subject: https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Province/$(birth_prv)
        predicateobjects:
        - [a, 'http://dbpedia.org/ontology/Province']
        - [a, 'http://schema.org/Place']
        - [a, 'http://rdfs.co/juso/SpatialThing']
        - ['http://www.w3.org/2000/01/rdf-schema#label', $(birth_prv)]
    

    An example of a record generated in RDF using the previous mapping

    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Person/Augustus> a <http://schema.org/Person>,
            <http://xmlns.com/foaf/0.1/Person> ;
        rdfs:label "Augustus" ;
        ns1:birthDate "0062-09-23T00:53:28+00:53"^^xsd:dateTime ;
        ns1:birthPlace <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/City/Rome>,
            <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Province/Italia> ;
        ns1:deathDate "0014-08-19T00:53:28+00:53"^^xsd:dateTime ;
        ns2:position "Emperor"@en .
    
    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/City/Rome> a <http://rdfs.co/juso/SpatialThing>,
            <http://schema.org/City>,
            <http://schema.org/Place> ;
        rdfs:label "Rome" .
    
    <https://documentation-resources.opendatasoft.com/ld/resources/roman-emperors/Province/Italia> a ns1:Province,
            <http://rdfs.co/juso/SpatialThing>,
            <http://schema.org/Place> ;
        rdfs:label "Italia" .
    

    This section shows a complete example of an RML mapping for roman-emperors dataset. It also provides more information about YARRRML and useful tools that facilitate the creation of an RML mapping.

    keys shortcuts

    YARRRML allows multiple shortcut versions of keys to be used:

    Tools

    Creating an RML mapping for a dataset is not an easy task. However, multiple tools can be really helpful in that process.

    The Linked Open Vocabulary is a great initiative. It proposes a search engine for vocabularies, properties and classes.

    Opendatasoft Chatbot (beta) is a chatbot that helps users generate an RML mapping for an Opendatasoft dataset.