Using matplotlib.ticker can provide a solution indeed.

formatter = ticker.ScalarFormatter(useOffset=True, useMathText=True)
formatter.set_scientific(True)
formatter.set_powerlimits((-3, 4))  # define when to apply scientific notation

axs[i].xaxis.set_major_formatter(formatter)
axs[i].yaxis.set_major_formatter(formatter)

powerlimits defines which numbers are shown in scientific and which in decimal notation, referring to the exponents of 10. Outside the interval it will be scientific, inside decimal.

Resulting in: