I would recommend using the mediawiki api which is supported by fandom to do this task. Perhaps wikitext is easier to parse than html for this usecase you're talking about. I can't really tell what opperation you're trying to do but whatever it is it's probably easier to do it on the markdown format of wikitext instead of plain html.
I got the api endpoint above from reading this page (method 3) on the mediawiki api documentation https://www.mediawiki.org/wiki/API:Get_the_contents_of_a_page