I ran into this problem, too. In my case, until I sort out what is the problem, I'm avoiding calling isolates in debug mode like this:

     final decryptedBytes = (kReleaseMode || kProfileMode)
            ? await compute(customEncrypter.decryptTheseBytes, encryptedBytes)
            : customEncrypter.decryptTheseBytes(encryptedBytes);

Although I cannot provide an answer to the question, I found this information usefull to better understand the problem:

Multiple Isolates vs one Isolate

https://martin-robert-fink.medium.com/dart-is-indeed-multi-threaded-94e75f66aa1e