You shouldn't use the test set since its no longer unseen anymore, it is supposed to only be used once at the end to see the true test of how well it generalizes.

Instead validation sets are used to tune k, which will help the model improve, however afterwards you only use the test set once at the end. Hope this helped!