The "Require linear history" ruleset rule might also be blocking simple Merging as a way of merging pull requests:
Disabling "Require linear history" fixed it for me.