I confirm the following stackoverflow post has solved my problem:

https://stackoverflow.com/a/69378029/13633797

After asking chatGpt for more clarity on the delay flag, besides what is in the docs, it mentioned that sometimes it is beneficial to delay the opening of the file to avoid inadvertent file locks; which I believe is what is happening to me. In my application it is possible that a logger is assigned to a variable, opening the log file, before the FileHandler has a chance to rename it and prepare for the new logs to come in for the day.

I believe this is the best description as to how this works without following the granularity of log creation through debugging.