After trying three variations on the second approach that wasn't working, I was looking at this tutorial and discovered that by setting:

scene = mi.load_file("my_scene.xml", integrator=prb_projective')

instead of

scene = mi.load_file("my_scene.xml", integrator=prb')

my optimizer was again able to find a gradient with respect to the size of the light (in the case that worked for me, I was editing a latent variable that was used to control a scale matrix that positioned the vertices of the area light).

Although this works, there are a couple things that should be noted:

1. It's much slower to use a projective integrator
2. It's extremely unclear how Mitsuba was able to figure out how to adjust the size of the light in the first case when using the non-projective prb integrator, but why it wasn't able to in the second case until I switched to the prb_projective integrator.

Any thoughts on this would be greatly appreciated.