Settings Input / Output Advanced History
Remove files from the target folder if they are not present in any source
Download files from the source even if they appear to be up-to-date in the target folder

No source for this download recipe

Sources point to files to retrieve
on various external systems (FTP, S3, ...)

 Add a first source
  • HEAD requests are used to probe the URL existence, size and modification time
    Disable validation of SSL certificate
    for partition
    Insert partitioning dimensions in the path or the URL with the '$DKU_DST_[dimension]' syntax
    Failed to enumerate files
    {{source.$check.totalFileCount}} files, {{source.$check.totalFileSize|filesize}}
    {{path.path}} {{path.size|filesize}}
    ... {{source.$check.paths.length - 100}} more
    Nothing found at the path

Dynamic recipe repeat

The Download recipe repeats using variables from a secondary parameters dataset. When enabled, it will run multiple times, using variable values from rows in the parameters dataset at each iteration. The results of each iteration are downloaded in the output managed folder. This is typically used to download files from several URLs