I was able to figure this out, I had created EC2 instances from a custom AMI which had some other configurations and packages pre-installed, In that custom AMI delete on terminate was set to false.
delete on terminate