I've had this happen to me in the past and it was due to an incompatibility between the CUDA driver and Keras. Make sure the versions you are using are compatible.