from concurrent.futures import ThreadPoolExecutor
...
model = torch.jit.script(model)
...
pool = ThreadPoolExecutor()
...
y_pred = torch.cat(list(pool.map(model, imgs)))
...

but this is not that faster as I expested.