As demonstrated in There is no difference in search speed between CPU and GPU on a million-level dataset GPU has two additional steps:
target vector copied from CPU memory to GPU memory
result copied from GPU memory to CPU memory
"There are additional time costs associated with transferring data between CPU memory and GPU memory.
The primary benefit of using a GPU for searching is its ability to handle large NQ searches effectively due to its powerful parallel computing capabilities.
For smaller datasets and NQ searches, there is not much difference between CPU indexing and GPU indexing."