I finaly manage to use tensorflow transformation to finish the job.

first turn it to numpy array:

feature = np.array(df.select("vector_features_scaled").rdd.map(lambda row: row[0].toArray()).collect())

and turn it to a tensor and voila!

feature = tf.convert_to_tensor(feature, dtype="float32")