Calling fig.canvas.toolbar.push_current() before xlim() is key to doing this:

import matplotlib.pyplot as plt
import numpy as np

fig,ax=plt.subplots()
t = np.linspace(-10,10,1_000)
ax.plot(t,np.sinc(t))
ax.axhline(0,linestyle=':',linewidth=0.4)
fig.canvas.toolbar.push_current()
ax.set_xlim(-2,+2)
plt.show()