Databricks has advanced their built-in support for working with spatial data.

Instead of storing and processing WKT (well-known text) formatted vector data, you should use built-in functions "ST_GeomFromWKT / ST_GeomFromText" to convert the data to GEOMETRY columns. All spatial data processing should be done with these data types. The geo data types will store spatial statistics (bounding boxes) that allow for efficient query execution, by leveraging stats to perform file skipping.

Also, Databricks support for ST_INTERSECTS and ST_CONTAINS are now accelerated by an in-memory spatial index. You can experiment with BROADCAST hint if working with larger scales, and know that the Databricks team is working on further improvements that will remove the need to use any hinting.