Best way to do this is as follows:

df['col'] = df['col'].apply(pd.to_numeric,errors='coerce').astype(pd.Int32Dtype())

So it will first convert any invalid integer value to NaN first & then to NA