Turns out the issue was with the model I was using. Even though it supposedly supports tools, it wasn't getting the job done. I switched to 4o-mini and everything now works as expected.