What's Custom Metadata?

In Sifflet, metadata is information that describes your data assets, monitors, and other entities. This information can be retrieved from external sources via Sifflet's integrations or created directly in Sifflet. It provides context, making it easier to understand, manage, and monitor your resources.

Custom Metadata expands on the above definition by allowing you to define specific key-value pairs for any entity (including Monitors!). With Custom Metadata, you can define attributes like "Data Owner", "Project ID", "Sensitivity Level", or any other information relevant to your workflows.

Why Is It Useful?

Custom metadata provides significant benefits, enabling you to:

• Enhance Organization: Structure and categorize your assets with much greater detail and specificity than tags alone.
• Improve Discoverability: Find assets quickly and efficiently by filtering on your custom-defined attributes in the catalog and monitor views.
• Add Rich Context: Attach specific business, operational, or governance information directly to assets. Examples include:
  • Data Governance:
    • Enrich the existing Owner metadata with a Functional Owner or Data Steward (using the User type).
    • Specify Data Sensitivity (e.g., "Confidential," "PII," "Public" using the Label type).
    • Track Compliance Status (e.g., "GDPR Compliant," "Needs Review" using the Label type).
  • Project Management:
    • Link assets to a Project Code or Initiative Name (using the String type).
    • Note a Source System ID or Business Unit (using the String type).
  • Operational Clarity:
    • Document Refresh Frequency for a dataset (using the String or Label types).
    • Specify a Contact Person for issues related to an asset (using the User type).
• Streamline Workflows:
  • Quickly identify assets managed by a specific team or user.
  • Filter dashboards or datasets related to a particular project for reporting.
  • Easily locate all assets with a "High" sensitivity level for auditing purposes.

The flexibility of custom metadata means you can adapt Sifflet to the unique terminology and processes of your organization.

How to Create a Custom Metadata Attribute

You can define your organization's custom metadata attributes from the settings page:

1. Navigate to Settings. Click on "Custom Metadata" under the "Platform Settings" section.

2. Click on "New Custom Metadata".

3. Configure your new metadata attribute:

   • Name: This is the name of your metadata field (e.g., "Data Owner," "Department," "Source System"). We recommend keeping it concise and descriptive.
   • Type: You can choose from the following:
     • User: The value will be one or more Sifflet users. The pool of available users for selection will be all users in your Sifflet instance.
     • String: The value can be any text. This type doesn't supports multi-select, meaning you can add only one string value for the same custom metadata on an asset.
     • Label: The value must be chosen from a predefined list of options that you create. This type supports multi-select.
   • Applicable Asset Types: Specify which types of Sifflet entities this metadata can be applied to:
     • Tables and views
     • BI assets (dashboards, reports, etc.)
     • Data pipelines
     • Declarative assets
     • Monitors
   • Configuration Options:
     • Allow multiple values: Check this if you want to be able to assign more than one value for this metadata key to a single asset (e.g., an asset belonging to multiple "Projects"). This is applicable for User and Label types.
     • Show metadata as filter: Check this if you want this custom metadata attribute to be available as a filter in the Sifflet catalog and monitor views.

4. Save your new custom metadata attribute.

🚧

Creating more than 5 custom metadata attributes

By default, organizations are limited to creating 5 custom metadata attributes. If you require more, please reach out to your Customer Success team.

Using Custom Metadata

How to Add Custom Metadata to an Asset

Once a custom metadata attribute is defined, you can assign its values to individual assets:

1. Navigate to the specific asset you want to update (e.g., a particular dataset, BI asset, or monitor).
2. Click on the "Edit" button.
3. For each custom metadata attribute:
   • User Type: Select one or more Sifflet users from the provided list.
   • String Type: Enter your desired text. For convenience, previously used string values may appear in a dropdown (autocomplete) for quick selection.
   • Label Type: Choose one or more labels from the predefined list you configured when creating the metadata attribute. You cannot create new labels directly from this screen; they must be defined in the attribute's settings.
4. Once you click on "Save", the changes will appear automatically. If no value is assigned to a custom metadata field for an asset, it will display as "-".

Permissions for viewing and editing custom metadata are the same as those for other metadata fields like descriptions and tags.

📘

Finding your custom metadata attributes in the asset page

Custom metadata is part of the asset's "Overview" tab. Your defined custom metadata attributes will appear in the "Details" section (typically towards the bottom, ordered by their creation date).

How to Add Custom Metadata at Scale

To add or configure custom metadata at scale, you can leverage the Bulk Actions functionality within the catalog or monitors page.

Using Custom Metadata to Filter Throughout Sifflet

If a custom metadata attribute was configured with "Use as filter" enabled, it becomes a powerful tool for discovery:

• In the Data Catalog: When browsing datasets, BI assets, data pipelines, or declarative assets, you'll find your custom metadata attributes listed among the available filters.
• On the Monitors Page: Similarly, you can filter your lists of monitors based on the custom metadata values you've assigned.

This allows you to quickly narrow down views to find exactly what you're looking for. For example, you could:

• Filter all datasets where "Data Steward" is "Jane Doe."
• Show all BI assets where "Department" is "Sales."
• List all monitors for assets with a "Sensitivity Level" of "High."

By leveraging custom metadata filters, you can significantly streamline your navigation and analysis within Sifflet.

FAQs

1. Who can add/edit custom metadata for a particular asset?

Permissions for viewing and editing custom metadata are the same as those for other metadata fields like descriptions and tags.