Solved!
If anyone else has this issue,
Go to your third party services connections
and delete your app from there.
Only the first time will it ask you for a refresh token.