It turns out I was using the wrong argument: it should have been newdata, not newx. So:

predict(svm_fit, newdata=dfm_val, type="class")

Gives the expected result of predictions on the validation data.