Generic URI Format

You can add many technologies to Sifflet using a generic Uniform Resource Identifier (URI) format. If the technology you are using is not covered by a dedicated format, you can use the generic format on this page to identify your assets.

Identifier Fragments

• Namespace: {technology name}://{authority} OR {technology name}:
  • Scheme = {technology name}
  • Authority = {authority}
• Unique name: {namespace1}.{namespace2}.{…}.{namespaceN}.{asset identifier}

URI Format

• {technology name}://{authority}/{namespace1}.{namespace2}.{…}.{namespaceN}.{asset identifier}
• OR {technology name}:{namespace1}.{…}.{namespaceN}.{asset identifier}
• OR {technology name}://{authority}/{asset identifier}

Parameter Limitations

• technology name must start with a letter or a number, and include only lowercase letters, uppercase letters, numbers, dot, plus operator and dash (regex: [a-zA-Z0-9][a-zA-Z0-9.+-]*)
• authority must start with a lowercase letter or a number, and include only lowercase letters, numbers, dash, colon and dot (regex: [a-z0-9][a-z0-9-.:]+)
• namespace elements and asset identifier can have two formats, following: quoted format and unquoted format
  • unquoted identifier must include only lowercase and uppercase letters, numbers, underscore, dash, dollar and characters included in the Unicode range U+0080 to U+FFFF
  • quoted identifier must be enclosed in double quotes (") and must include only characters included in the Unicode range U+0001 to U+FFFF. If a double quote character is present in the name, it needs to be escaped with another double quote character in front of it.

Crafting URI Fragments in Practice

1. Scheme ("Technology Name" Identifier)

• For the technology name, use lowercase letters only:
  • MuleSoft → mulesoft
  • Amazon → amazon
  • Salesforce → salesforce
  • MongoDB → mongodb
• If there are multiple words, separate them with a dash:
  • Amazon DynamoDB → amazon-dynamodb
  • Google Ad Manager → google-ad-manager
• Use numbers when required:
  • S3 → s3
  • 360Learning → 360learning

2. Authority

• It is made to identify your repository location.
• In OpenLineage definition it very often uses the host address of the server as an authority, associated with the port of the service, given multiple services of the same type can run on the same host.
• It can also be an account identifier or a workspace identifier depending on the service you are using.
• You can also identify a local instance using a fixed IPv4 address or an IPv6 address.
• Examples:
  • Host only: sifflet-dev-cloud.cfbungc.eu-west-1.rds.com
  • Host + port: sifflet-dev-cloud.cfbungc.eu-west-1.rds.com:5432
  • Account ID with universally unique identifier (UUID) format for a cloud service: e57570e3-43b6-4011-a418-1418bd77d45e
  • Local address: 2001:0db8:85a3:0000:0000:8a2e:0370:7334
• Omit the authority when the service has no required location. For example, the BigQuery URI format does not use an authority because the service is serverless.

3. Unique Name

• The asset identifier parameter is meant to identify the asset in a unique way inside the namespace where it is provided. A unique ID or name for the asset is ideal for that.
• The rest of the unique name {namespace1}.{…}.{namespaceN} is meant to represent the different levels of namespaces you can find in most technologies: for example in a lot of relational databases, inside the same instance you have a database where you can define schemas with unique names and inside schemas you can define tables or views with unique names. In that case the full unique name would be written: databaseName.schemaName.tableName.

Example

• hbase://sifflet-dev-hbase.eg56.eu-west:8080/platform-namespace.test_data
  • Scheme: hbase for Apache HBase
  • Authority: sifflet-dev-hbase.eg56.eu-west:8080 following host:port format
  • Unique Name: platform-namespace.test_data
    • platform-namespace is in place of {namespace1}.{…}.{namespaceN}. There is only one hierarchical level here. In other cases, you can define one or more levels, depending on your hierarchy.
    • test_data is the name of the table identified by this whole URI. It is defined inside the namespace platform-namespace in the server located at sifflet-dev-hbase.eg56.eu-west:8080