You can use style overrides to hide the icon

@include mat.chips-overrides(
    (
      with-avatar-avatar-size: 0px,
    )
  );

You can check all overrides here https://material.angular.dev/components/chips/styling