This behaviour seems to occur because of two different integer stringification methods within the interpreter. The slowdown might be fixed in a future perl release: https://github.com/Perl/perl5/pull/22927