Spark Out Of Memory Error

Spark Out Of Memory Error

OOM error Spark driver level:

1. Spark driver is the main control of spark application .if its configured with less memory to collect all data of files then it throws error.

2. If table size which is to be broadcasted is huge then also driver faces OOM error.

OOM error Spark executor level:

1. Spark job is executed though one or more number of stages and each stages consists of multiple task. No of task at executor level depends on…