The issue was with the versions of containerd and runc.
I updated the two to resolve the issue.
More details: https://github.com/moby/buildkit/issues/5117