my error on writing too: go environment path setting:
HADOOP_HOME
C:\hadoop\hadoop-3
make sure you have the winuntils.exe can be find
create a .env in your project, contain:
HADOOP_HOME=C:\hadoop\hadoop-3
PYSPARK_PYTHON=.venv\Scripts\python.exe
PYSPARK_DRIVER_PYTHON=.venv\Scripts\python.exe
my error as below solved:
write not work: 25/07/20 18:18:12 ERROR FileFormatWriter: Aborting job a32653e8-dddf-4a0d-8aa2-d1fc9731050f.
java.lang.UnsatisfiedLinkError: 'boolean org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(java.lang.String, int)'
at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:793)