The good reason to cleanup libraries is so that libraries may be unloaded without leaking system resources: files, memory (memory mappings), and any other unneeded kernel based process resources.

Compiled library modules are a high level of modularization that modern operating systems provide. It can be a powerful tool to build larger tools, on the fly (as the program runs). It's a kind of software plug-n-play. It's operator-in-the-loop interactive programming with binary code groups. It requires libraries to be robust, and not leaky; in order to be able to unplug stuff in such a dynamic scenario. Most libraries can't be unloaded, if at all, or without leaking. So it's currently hard to do this type of dynamic programming. I can dream. Dynamic linker/loader systems are crazy powerful.

I'm currently trying to fix Cairo (related to FontConfig). So I can work on my dream. Wow, 10 years.