I'm a little late to the party but I had just this issue where ticking or unticking the many-many entities box did nothing.

My problem was me getting the Foreign-Primary relationship the wrong way around, stupid mistake that fooled me for a bit.