79241684

Date: 2024-12-01 13:46:04
Score: 0.5
Natty:
Report link

Given that one wants to use the Spark configuration to do this check - and use secrets kept there one can use the following code :

val fs = FileSystem.get(new URI(<url to blob storage>), spark.sparkContext.hadoopConfiguration)
val path = new Path(url + s"/$directory")
fs.exists(path)
Reasons:
  • Low length (0.5):
  • Has code block (-0.5):
  • Low reputation (0.5):
Posted by: Ohad Bitton