Remove the using around _ffmpegProcess.StandardInput.BaseStream in AppendAudioBuffer. Keep the stream open for continuous input.

like :

var ffmpegInputStream = _ffmpegProcess.StandardInput.BaseStream;
await ffmpegInputStream.WriteAsync(buffer, 0, buffer.Length);
await ffmpegInputStream.FlushAsync();