With ax.set_ylim() I have to always also use ax.set_yticks() in order for the plots to come out correct.

Try:

my_min = flux_min - constant
my_max = flux_max + constant

step   = 0.1  # some reasonable step size for the y-axis

ax.set_ylim(my_min, my_max)
ax.set_yticks(np.arange(my_min, my_max, step))