Navigate to the "Integration" --> submenu "Secrets" page and add your data source secret.
Your secret depends on your data source type.
Sifflet uses AWS Secret Manager to store your secrets securely.
Navigate to the Datasources page and add a new data source connection.
Select your data source type, and provide the required information on the form. Press Test Connection to ensure that your connection is working correctly.
We recommend to define the timezone of your datasource, if it is not UTC (Coordinated Universal Time).
It will be UTC by default.
Time reference and run frequency
The time reference is on UTC.
So a datasource set up with a cron "0 9 * * *" will mean it will synchronize every day at 9 am UTC time.
To manually refresh a data source, follow the below steps:
- On the left panel, go to "Integrations" -> submenu "Sources"
- Tick the data source you want refreshing
- Select the "Run" Button
You can see the status of your previous synchronisations for a given datasource by clicking the arrow on the left.
- After creating a datasource, I manually refreshed it by clicking the "Update Source" but I don't see the new datasets in the Catalog, what happened?
There could be several reasons as to why the new datasets do not appear:
- You can first check the synchronisation status (see here), it might still be running or to confirm it failed.
- If the status shows "Failure", you can check if you have provided Sifflet's account the correct right accesses. For instance, for Snowflake, it is detailed here.
- If the issue persists, do not hesitate to reach out to Sifflet's support
Updated 1 day ago