This article helped me https://ubuntu.com/server/docs/about-debuginfod
For my case it worked after I set the cache directory in .bashrc to something custom
export DEBUGINFOD_CACHE_PATH="/some/path/to/.gdbcache"
And increased by 1000 the cache clean interval in /some/path/to/.gdbcache/cache_clean_interval_s