This issue was in fact the same as this one.

Redirects are generated by Wagtail, after capturing 404 responses. Here the 404.html template extends base.html which uses variables (here page) that are not defined before the redirect. Hence 404 responses are not generated, and redirect cannot occur.

The issue is solved by making sure that the 404.html template can be rendered even when page is not defined. In my case, following suggestion in this issue, I simply wrapped template section that require the page object in a {% if page %} block.