I figured this out. If you are using the Umbraco Delivery API, you should pretty much be able to use this out of the box just providing your own RichTextFieldBlockItem component if you are embedding block components in your rich text. For others using a similar JSON rich text or other hierarchical json structure, this might be a helpful pattern.

There are two key elements in the following block of code:

1. The import RenderRichTextComponent is a reference to itself. Thus it becomes a recursive component.
2. const GenericTag = genericNode?.tag || 'div' allows for creating custom tags. Note the GenericTag variable name must be capitalized otherwise astro would just write out the variable name instead of the variable value.

code:

---
// filepath: /src/components/richText/RenderRichText.astro
import RichTextFieldBlockItem from './RichTextFieldBlockItem.astro';
import type { ApiBlockItemModel, RichTextGenericElementModel, RichTextRootElementModel, RichTextTextElementModel } from "@/api/umbraco";
import RenderRichTextComponent from './RenderRichText.astro';

interface Props {
  node: RichTextGenericElementModel | RichTextRootElementModel | RichTextTextElementModel | null | undefined;
  blocks: ApiBlockItemModel[] | null | undefined;
}

const { node, blocks } = Astro.props;

if (!node) return null;

const isText = node.tag === '#text';
const textNode = isText ? node as RichTextTextElementModel : null;
const isRoot = node.tag === '#root';
const rootNode = isRoot ? node as RichTextRootElementModel : null;
const isBlock = node.tag === 'umb-rte-block';
const blockNode = isBlock ? node as RichTextGenericElementModel : null;
const block = isBlock ? blocks?.find((b) => b.content && b.content.id === blockNode?.attributes['content-id']) : null;
const isGeneric = !isText && !isRoot && !isBlock;
const genericNode = isGeneric ? node as RichTextGenericElementModel : null;
const GenericTag = genericNode?.tag || 'div';
---

{isText && textNode?.text}

{isRoot && rootNode?.elements.map((child, i) => 
  <RenderRichTextComponent node={child} blocks={blocks} />)}

{isBlock && <RichTextFieldBlockItem block={block} />}

{isGeneric && (
  <GenericTag {...genericNode?.attributes}>
    {genericNode?.elements.map((child, i) => 
    <RenderRichTextComponent node={child} blocks={blocks} />)}
  </GenericTag>
)}