Thanks to S2B for pointing out that <div> wrappers were needed. Below is the shortest code that I was able to get working...

FluentAccordionTest.razor

@page "/dashboard"
<FluentAccordion>
<div>
    <FluentAccordionItem Heading="My Heading" Expanded="true">
            my content
    </FluentAccordionItem>
</div>
</FluentAccordion>

FluentAccordionTest.razor.css

::deep > fluent-accordion-item::part(heading) {
    background-color: yellow;
}