I completely forgot about how Python's memory management system works, I'm used to JIT compiling. The real reason was just that Python had loaded in all of the data. Thanks to @Barmar.