Verify the following Spark configuration properties:
- spark.sql.adaptive.skewJoin.enabled (default: true):
Enables or disables adaptive skew join handling.
- spark.sql.adaptive.skewJoin.skewedPartitionFactor (default: 5):
A multiplier used to determine if a partition is skewed.
A partition is considered skewed if its size is more than 5 times the median partition size.
- spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes (default: 256MB):
A minimum size threshold for a partition to be considered skewed.
A partition is identified as skewed when:
Its size exceeds the product of the skewedPartitionFactor and the median partition size.
Its size is greater than the skewedPartitionThresholdInBytes.