For anyone still struggling with similar issues in Azure's authentication process, I made a small repo with the scripts and a step-by-step guide that helped me solve this issue and successfully make requests.
You can use the scripts as a base to write your own version in your preferred language.
This covers the proccess needed to generate a JWT for OAuth 2.0 requests.
Thanks to sk2andy for pointing me to the documentation I used to solve this.