I came to this old question out of frustration after having done optimisation of the rails assets pipeline the whole day. It turns out it doesn't matter what rails version or any other gem you use - if you have activated the throttle function in the browser.
I'm using chrome and for some reason I managed to activate the throttling function used to simulate slow web connections. Check that as one debugging step in your browser.