I don't think there is currently any way to do this without a copy unless you use a sketchy technique like the one you talked about. It's probably best to ask this on Github as a new feature/performance idea.