TL;DR - I implement the GetHashCode method wrong for ValueObject.
After creating a different project to test out the User and ChangePasswordCode, I find out that the problem lies in the UserId value object as it works fine when I use Guid as the primary key. At that time, I suspect that the change tracker may compare the 2 UserId in User and ChangePasswordCode entries and somehow doesn't match the 2. So the problem could be in the Equals and GetHashCode method that I implement for ValueObject.
By placing breakpoints in debug mode, I find out that the change tracker indeed calls the GetHashCode first before the Equals and it returns 2 different hash codes for the same UserId. I fix the GetHashCode method and it works now.