I am going to post the answer here for anyone that stumbles upon this looking for an explicit answer, I know this question is a few years old at this point.

A good endpoint to begin with is:
https://learn.microsoft.com/en-us/graph/api/signin-list?view=graph-rest-1.0&tabs=http

Another one:
https://learn.microsoft.com/en-us/graph/api/user-list?view=graph-rest-beta&tabs=go#example-3-get-users-including-their-last-sign-in-time

Storing this in a DB as was mentioned in the comments is a good idea to keep track of historical data if needed.

Depending on the scale of your data, you may want to use Go or another language to grab the data but if it is less data, PowerShell should be fine.

Be sure you know the auth flow, if the user isn't re-logging in and just uses cookies/tokens, all of their logins may not show.