The model is too large to fit into memory - either GPU or RAM.

Here is a few things you can try to do.

If the model loads into memory without issue (whether it be RAM or onto the GPU), start with the smallest batch size of 1 and see if your model is able to perform a propagation step; forwards and backwards, i.e. updating the gradients.

If you are able to do this, then incrementally increase the batch size and repeat the above step before saturating all our available memory and causing this error message, then step down the batch size by one and used this to fit your model. This is making sure you're utilizing all your resources properly.

If its the case that whilst you're loading the model into memory you get this error message, then the model itself is too large, let alone the additional parameters generated by whatever optimizer were to perform the fitting routine.