In LocalConfiguration.php, set keepFEGroups to 1.
This option allows you to keep locally assigned frontend groups.
'ig_ldap_sso_auth' => [ 'keepFEGroups' => '1' ],