After fighting this for a while, I found a solution to my problem. My website has many SVGs, and these SVGs use a custom font that I serve. When I serve these SVGs in an <img> tag, they can't access my custom font (unless I include it in the nested style, which would increase the file size too much for my liking). In order to get around this, I redownloaded each SVG again on DOMContentLoaded, and I replaced the <img> tags with inline versions of the SVGs:

function fetchSVG(svgUrl, resultFunction = null) {
    return fetch(svgUrl)
        .then(response => response.ok ? response.text() : null)
        .then(svgContent => {
            if (svgContent === null) {
                return;
            }

            const svgElement = new DOMParser().parseFromString(svgContent, "image/svg+xml").documentElement;
            svgElement.classList.add("swapped-svg");

            if (resultFunction !== null) {
                resultFunction(svgElement);
            }
        })
        .catch(error => console.error("Error loading SVG:", error));
}

document.addEventListener("DOMContentLoaded", function() {
    function replaceImageWithSVG(imgElement) {
        fetchSVG(imgElement.src, function(svgElement) {
            imgElement.replaceWith(svgElement);
        });
    }

    document.querySelectorAll(".svg-swap").forEach(svgSwap => {
        svgSwap.addEventListener("load", function() {
            replaceImageWithSVG(svgSwap);
        });

        if (svgSwap.complete) {
            replaceImageWithSVG(svgSwap);
        }
    });
});

After disabling this functionality, I saw that caching was working. I don't know the precise reason why, but fetching via JavaScript right away after the images loading via the <img> tag somehow messed with the cache. I found two solutions:

1. If I didn't use <img> tags for the SVGs initially and I just downloaded them via JavaScript on DOMContentLoaded, the cache would work across refreshes of the website.
2. If I use the attribute loading="lazy" in the <img> tags, it somehow didn't cause this strange behavior and the cache would still work across refreshes of the website. For those that have the same issue, keep in mind you may not want this lazy loading behavior, so this might not work for you.

As I mentioned, I don't know why "eager" downloading of the SVGs immediately with <img> tags without the attribute and then pulling the SVGs via JavaScript messed with the cache, but lazy loading resolved the issue, so I went with option #2.

Either way, both of my solutions end up doing some sort of lazy loading. If I didn't want this, I think I would have been forced to include font styling inside the SVGs, or I would have had to switch the format of the images from SVG to something else.

Lastly, I would like to confirm that I only originally experienced this issue in Safari on MacOS and all browsers on iOS, and the above two solutions solved the issue.