Based on the yaml file you shared you're using shell /bin/sh but in the command you run you use /bin/bash. Try to run the :
kubectl alpha debug <biz_pod> -i -t --image=busybox -- /bin/sh
For additional workaround confirm that the ephemeral container was successfully injected. Look for the injected ephemeral container if it’s running or falling :
kubectl describe pod <biz_pod>
You can also check the logs that will help you understand if the debug container is running or failing.
kubectl logs <biz_pod> -c debugger-<name>
Is there a way to delete the ephemeral container without affecting the pod?
You cannot able to delete the ephemeral container after you have added it to a pod.
For additional information see the documentation below :