Not sure how to solve this in perlbrew specifically, but in case somebody stumbles upon this StackOverflow thread because they're looking up the clang error thrown by perl Configure, here's what is likely going on. After -arch, with Apple Clang, x86_64 or arm64 is supposed to be passed. However, on ARM, it seems like the perl darwin Configure hint is not recognizing the CPU architecture when 64-bit addressing is enabled (the -Duse64bitall flag is passed to Configure), so it just passes an empty string after -arch. I've opened a Perl PR to address this: https://github.com/Perl/perl5/pull/22672.
The person in this GitHub issue: https://github.com/gugod/App-perlbrew/issues/768 found out that in fact perlbrew is calling Configure with the respective 64-bit addressing flag.
From my perspective, either the perl source used for perlbrew should be modified or a newer perl that contains the hint fix should be used. I'm not sure what perlbrew needs 64-bit addressing for, but in the perlbrew GitHub issue the people found out that if omitted, the perl tests fail, so I guess that's not an option.