Came back to this question years later to offer an update.

Laravel 12 has a new feature called Automatic Eager Loading, which fixed this issue of eager loading in recursive relationships for me.

https://laravel.com/docs/12.x/eloquent-relationships#automatic-eager-loading