I have experienced a lot of issues working with virtualized TreeView, mainly:

• scrollbar jumping
• scrollbar thumb randomly resizing
• scrolling over the end (when using wheel or ctrl+end)
• not immediately displaying all items at the bottom after applying filter

Googling just confirmed people have a lot of similar and unresolved problems. And googling put this case as a first hit, so, I'll say here what helped me the most (besides setting VirtualizingStackPanel.ScrollUnit="Item", of course):

Even though all my TreeViewItems have equal height (I've checked), I needed to explicitly set Height to something on my top element under the TreeViewItem. After doing so, the first two problems were reduced to minimum and other two problems disappeared completely.

Maybe it will help someone...

P.S. I have seen this advice in one comment that hasn't received much attention in one of the threads I've encountered while googling...