Enable direct access

Experimental: let Spark access the table in the SQL database directly. Requires setup on the SQL connection.

Partition column

Spark will split the read into several spans of the values of this column. Only numeric {{uiState.sparkJdbcStatus.sparkVersion.startsWith("3.") ? 'and date' : ''}} columns can be used.

Lower bound

Estimate needed to parallelize reads. Use last known metric value: {{getMin()}} Can be computed in the metrics

Upper bound

Estimate needed to parallelize reads. Use last known metric value: {{getMax()}} Can be computed in the metrics

Number of partitions

Number of partitions Spark writes in parallel; if a partition column is set, number of partitions read in parallel. Leave -1 to use the number of executors