I'm going to provide an answer to my own question, certainly found a way to workaround this issue for now, I will however not mark this as the best answer because I know this might not always be the ideal solution. (and because this answer might not include detailed information). However it can be used in this use case.

So the solution now is to go to cloudflare and edit both records and disable the proxy option.

After that visiting my domain loads my website correctly with https without any issues.