My issue was that I misunderstood the public schema as a single schema across all databases in the cluster, rather than as a schema named public created separately and by default for all new databases. When I was running GRANT ALL ON SCHEMA public as the admin user, I was doing it in the postgres database and not test_db. Thus the solution I was looking for was to run as admin:

postgres=# \c test_db;
test_db=# GRANT ALL ON SCHEMA public TO testuser;