I think you could use this in that case. It is described in the documentation as the way to add non parameter obj to a module

self.register_buffer(name, tensor)