I can't comment so I am submitting my thoughts as an answer.

first, i don't think anything is wrong with your code.

image generation model uses alot of memory. From my personal experience, when using SD2.0 I recall that the memory went up around 23GB. The memory consumption increases as the model gets larger but also from the image size. I recommend you try reduce the image size. Try starting with either 256/512/768 and increase it gradually if you dont get any memory allocation error.

hope this helps.