Concept | Scenarios #

Watch the video

Scenarios are key to automating tasks related to your Dataiku project. Let’s learn about the different types of scenarios and their various components.

Use cases #

Automation scenarios are a set of actions that are scheduled to run when certain conditions are satisfied. They are most useful when automating various kinds of tasks when a project is in production. For example:

  • If new data arrives on a regular basis, a scenario can rebuild the Flow once per day or each time it detects a dataset change.

  • If a metric for a machine learning model falls outside a specified threshold range, a scenario can be triggered to retrain the model.

  • Scenarios can also automate administrative tasks such as cleaning logs or starting and stopping a cluster.

Scenario types #

There are two types of scenarios in Dataiku.

  • Step-based , where scenario steps are configured using the visual interface.

  • Code-based , where the set of actions performed are fully defined by Python code.

Note

Learn more about custom Python script scenarios in our article on custom scenarios .

Dataiku screenshot of the dialog for creating a new scenario.

Scenario components #

Scenarios consist of three main components.

  1. Steps that are actions configured by the user.

  2. Triggers that define when to execute a scenario.

  3. Reporters that send information or alerts about a scenario via a variety of channels.

Slide depicting the three main components of a scenario.

Scenario steps #

Scenario steps let you control what the scenario will do. Common scenario steps include:

  • Building or clearing a dataset.

  • Training a model.

  • Running metrics and checks.

  • Sending messages.

  • Refreshing the cache of charts and dashboards.

Dataiku screenshot of the Add Steps options in a scenario.

Scenario steps run sequentially. However, you can control whether a step runs based on the outcome of a check. Therefore, you have some control over the flow logic of a scenario using metrics and checks .

Note

All available scenarios steps are defined in the reference documentation .

Scenario triggers #

Triggers allow users to define a condition or set of conditions that, if satisfied, start a scenario. Each trigger can be enabled or disabled.

Trigger

Description

Time-based

This will launch the scenario at regular intervals.

Example: Repeat every 30 minutes.

Dataset change

This starts a scenario whenever a change is detected in the dataset. This type of trigger is used for filesystem-based datasets.

SQL query change

This runs a query at a specified interval and starts the scenario when the output of the query changes with respect to the last execution of the query.

Custom (Python)

This will execute a custom Python script that activates a trigger.

Note

Different types of triggers may be available depending on your license.

Dataiku screenshot of different triggers available in a scenario.

Reporters #

Dataiku lets you add reporters to a scenario to inform users about scenario activities through email and other channels. Reporters can be sent when a scenario starts or ends on the condition that it succeeds or fails .

Reporters operate through several channels, including:

  • Mail

  • Slack

  • Microsoft Teams

  • Webhook

  • Twilio

  • Shell command

  • Send to dataset

Dataiku screenshot of different reporters available in a scenario.

What’s next? #

To learn more about scenarios and try hands-on tutorials, please register for the free Academy course on this subject found in the Advanced Designer learning path .

Note

You can also find more information about scenarios in the reference documentation.