So after more debugging, the problem was in my backend delete request endpoint I never sent a status 200. So in the network tab the pending requests piled up and timed out at its limit which is apparently 7. Yes sorry for vague snippet.

app.delete("/DeleteAudioUrl", async (req, res) => {
  const url = req.body.filePath;
  const filePath = path.join(__dirname, "/audio", url);
  if (fs.existsSync(filePath)) {
    fs.unlinkSync(filePath);
  }
});

was fixed by adding

res.status(200).send("deleted succesfully");