I found that setting the font-size of the icons to inherit works well.

.material-symbols-outlined {
    font-size: inherit !important;
}

This will override the default value with the font size of each icon's parent element.