Well,
I found the issue.
The event was different because it was created on a different calendar id.
The 3rd party system that created the event defined it on there own calendar, which is different from regular events that are sent to invitees.

so the only thing was to use the calendar id that is on the event:

event.getOriginalCalendarId()

and use it when calling Calendar.Events.get(calendarId, eventId)