I used this code and it worked.
llm = HuggingFacePipeline( pipeline=local_pipe, model_kwargs={"temperature": 0.5, 'device':0}, )
I set device to cuda.
device