URI Sources

Concept

URI stands for Uniform Resource Identifier. Sifflet uses URIs to identify any object inside and outside the application with a universal ID.

You can read more about the generic concept of URI starting on the dedicated Wikipedia page: https://en.wikipedia.org/wiki/Uniform_Resource_Identifier

URI in Sifflet

OpenLineage designed a standard URI model to apply to datasets and jobs. Sifflet adapts and extends this standard to its technology requirements.

Every asset in the Sifflet Data Catalog can be identified through a URI that has the following format:

URI = scheme ":" ["//" authority] uniqueName

• The scheme defines the identifier assignment specification. In the data asset domain, it depends on the technology of the asset: for example bigquery for BigQuery, mysql for MySQL, oracle for Oracle, etc…
• The authority defines the instance where the asset is located. Most commonly it is composed of the host and the port of a service, but for some technologies it can simply be a workspace identifier or an account identifier. In the case of a fully decentralized service, you can omit it. A few examples to illustrate:
  • An Oracle asset authority is defined by the address of the host of the Oracle server and its attached port.
  • A Power BI asset authority is defined using the ID of the workspace where the asset is located.
  • BigQuery assets do not have an authority: the BigQuery service is fully decentralized.
• The unique name defines the path to the asset inside an instance. In some cases it is directly the identifier or name of the asset, and in other cases it follows a more complex namespace hierarchy used in the system. Here are some examples:
  • A PostgreSQL asset unique name is defined using the hierarchy database → schema → table. It is written databaseName.schemaName.tableName.
  • A Looker dashboard unique name is defined using the ID of the dashboard inside the Looker instance it belongs to. There is no hierarchy to take into account given a dashboard ID is unique inside a given instance.
  • A BigQuery asset unique name is defined using the hierarchy project → dataset → table. It is written projectName.datasetName.tableName.

For the full URI definition per technology, or to craft a generic URI, see the dedicated pages of the documentation.

Finding Assets' URI

You can get the URI of existing assets from their asset page.

1. Go to the details page of an asset
2. Click the three-dot menu in the top-right corner of the page
3. Click Copy Data Asset URI to copy the asset URI to your clipboard

Finding URIs for Schemas and Other Source Instances

You can get the list of URIs for schemas contained in a source defined in Sifflet using this public API endpoint.

This is useful when working with the public API related to domains.