It should be possible with the correct API permissions for the Graph API. You need one of the following Permissions to do your request:
The API permissions need to be added to your App Registration that you created for your GitHub Workflow. In the end it should look like the image at the bottom.