The logging answer above helped. Turns out I needed to comment out this in my sshd config:
#Match Group administrators # AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys