Actually, the \Barryvdh\Debugbar\Middleware\InjectDebugbar::class, part needs to be after \App\Http\Middleware\DebugMiddleware::class,.
In your middleware you conditionally enable the Debugbar, and this is InjectDebugbar::class which checks if Debugbar is enabled and injects it into your response.