the learning rate could be quite low to converge, try something between 0.001 and 0.0001. Also you can try Biriectional LSTM layers which use to work better in financial time series.

Anyway the error is extremely high. I dont know what currency are you using but the RMSE is around 62000 which can be a solid 100% of error in the prediction. Such a discrepancy it is not likely due to a hyperparameter fixing. Are you also normalizing the output? you can also do it with the MinMaxScaler() as you did with the inputs.

Hope it helps!