You do not yet have any connection for {{dataset.type|datasetTypeToName}}. You need to create a new connection before creating datasets on this connection.

Connection

Manage connections

BigQuery project Database Database Catalog

Leave blank to use the Project ID associated with the connection Leave blank to use the Snowflake database associated with the connection Leave blank to use the MS SQL Server database associated with the connection Leave blank to use the Databricks catalog associated with the connection Leave blank to use the catalog associated with the connection

BigQuery dataset Schema

Table

Datetime mapping

Use DATETIME instead of TIMESTAMP to store "datetime with tz" values

Normalize floats & doubles

Normalize floating point values (force '42' to '42.0')

Teradata settings

Primary index strategy

Index columns

Greenplum settings

Distribution strategy

Distribution key

For partitioned datasets, the partitioning dimensions are always prepended to the sort keys

Sort key

Use an hash based distribution

(Azure documentation on distributed tables)

Distribution column

Forbids writing to the BigQuery table if the dataset is partitioned and there is a mismatch with the BigQuery table partitioning configuration.

Partitioning configuration mismatch between the dataset and the BigQuery native partitioning.

Partitioning configuration mismatch between the dataset and the BigQuery native partitioning. Writing to the table will fail as it is forbidden. Configure BigQuery native partitioning or disable Partitioning consistency.

Partitioning

Create BigQuery partitioned table

(Google BigQuery documentation on partitioned tables)

Partition by column

Partition range

Start (inclusive)

End (exclusive)

Interval (must be a positive integer)

Require partition filter

Requires all queries to include a filter on the partition column (Google BigQuery documentation)

Cluster by column(s)

If the BigQuery table(s) requires a partition filter, and errors occur when listing partitions, enable this. This may increase costs.

Prevent PREVIEW->SELECT fallback

Dataiku normally tries to use the "PREVIEW TABLE" feature of BigQuery to avoid SELECT on the table. In case of failure using PREVIEW, Dataiku automatically falls back to SELECT. Enable this to prevent this fallback.

Table creation mode

Note: The $DKU_CREATE_TABLE_FIELDS will be replaced by the list of fields and types from the schema

Pre-write statements

These statements will be executed before Data Science Studio writes data to the table. You might want to remove temporarily some indices for example

Post-write statements

These statements will be executed after Data Science Studio writes data to the table. You might want to create some indices for example

Teradata settings

Greenplum settings

SQL Spark integration