79186859

Date: 2024-11-13 22:29:28
Score: 0.5
Natty:
Report link

The issue is resolved. When running in spark, adding the following in the spark-submit command

--conf spark.hadoop.io.compression.codecs=org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.SnappyCodec 
--packages org.apache.hadoop:hadoop-aws:3.2.0
Reasons:
  • Has code block (-0.5):
  • Self-answer (0.5):
  • Low reputation (0.5):
Posted by: TripleH