Limitations and attention points

Spark is a fairly complex execution engine; tuning and troubleshooting Spark jobs require some experience.

Spark’s additional possibilities come with a few limitations:

Warning

Spark’s overhead is non-negligible and its support has some limitations (see above and Usage of Spark in DSS ). If your data fits in the memory of a single machine, other execution engines might be faster and easier to tune. It is recommended that you only use Spark for data that does not fit in the memory of a single machine.