It appears like I've found a solution.

Somewhere, I found a list of queries to run to track this down. One of these was SELECT * FROM pg_largeobject_metadata WHERE lomowner = OID;

So if I loop-through all large objects and reassign ownership it sort this for me!

DO $$ 
DECLARE 
    lo_id OID;
BEGIN
    FOR lo_id IN 
        SELECT oid FROM pg_largeobject_metadata WHERE lomowner = 21338
    LOOP
        EXECUTE format('ALTER LARGE OBJECT %s OWNER TO postgres;', lo_id);
    END LOOP;
END $$;