Had struggled with this today myself, so in case anyone will have the same problem:

You need to set mediaItem property with current active MediaItem.

As option, inside your AudioHandler add this:

in constructor add:

_listenForDurationChanges

void _listenForDurationChanges() {
    _player.durationStream.listen((duration) {
      final index = _player.currentIndex;
      final newQueue = queue.value;
      if (index == null || newQueue.isEmpty) return;
      final oldMediaItem = newQueue[index];
      final newMediaItem = oldMediaItem.copyWith(duration: duration);
      newQueue[index] = newMediaItem;
      queue.add(newQueue);
      mediaItem.add(newMediaItem) //without this notification will be empty; 
    });
  }