Integrations Management

Add your source's credentials

To add your source's credentials, navigate to the "Credentials" page. You can find it in the side panel by selecting "Integrations" and then "Credentials".

The credentials type depends on the source you're integrating. For every source, a credentials example is provided in its dedicated documentation page.

📘

Secret Management

Sifflet uses AWS Secret Manager to securely store credentials.

Add a new source

To add a new source, navigate to the "Sources" page. You can find it in the side panel by selecting "Integrations" and then "Sources". Once you reach the page, click on the "New source" button.

Configure your source

Select your source type and fill out the configuration form with the required information. Press Test Connection to ensure that your connection is working correctly.

📘

Source timezone

We recommend defining the timezone of your source if it is not UTC (Coordinated Universal Time).

By default, the timezone will be UTC.

📘

Source refresh frequency

For every source type, Sifflet has a default refresh frequency defined based on multiple factors:

• Data platforms and transactional databases: A daily refresh to ensure that new assets are visible in Sifflet within 24 hours of creation.
• Data pipelines: An hourly refresh to keep statuses current in Sifflet, enabling prompt detection of pipeline failures.
• BI tools: A weekly refresh to accommodate the slower rate of asset changes compared to other sources, reducing the likelihood of exceeding API quotas.

When creating a new source, we strongly recommend keeping the default refresh frequency to ensure that metadata in Sifflet remains up-to-date. If you need to change the default frequency based on specific needs, you can provide a cron expression.

📘

Copy the parameters of an existing source

The highlighted button allows you to copy the connection parameters of an existing source. This is especially useful when creating multiple sources with nearly identical configurations (like multiple schemas within the same environment).

Manually refresh a source

To manually refresh a source, follow the below steps:

• On the side panel, go to "Integrations" and then "Sources".
• Click on the "Run" button corresponding to the source you want to refresh.

Source sync status

You can see the status of your previous refreshes for a given source by clicking on it to open its dedicated page and then going to the Runs tab.

Delete a source

Deleting a source automatically deletes the assets that are part of the source as well as any monitors and incidents on top of these assets.

FAQ

1. After creating a source, I manually refreshed it by clicking the "Update Source" button but I don't see the new assets in the Catalog, what happened?
   There could be several reasons as to why the new assets do not appear:
   1. You can first check the refresh status (see here), the source refresh might still be running or it may have failed.
   2. If the status shows "Failure", you can check if you have provided Sifflet's account sufficient permissions. For instance, for Snowflake, it is detailed here.
   3. If the issue persists, do not hesitate to reach out to Sifflet's support team.