For the error that you got, it seems you reload the BigQuery connector twice, as stated here the connectors for Spark BigQuery are pre-installed already in Dataproc 2.1 and later image. It is automatically added when you flag it:

(--metadata spark-bigquery-connector-version=0.42.X)

You can try removing those JARs (--jars=gs://BUCKET/jars/spark-3.5-bigquery-0.42.2.jar) that you are loading manually, since the BigQuery connector you need is already defined in your configuration. This should help resolve the error you are encountering. -as my answer also in Google forum