This error occurs because Dremio caches query results and logs metadata by default when executing queries via API. If too many queries are fired in succession:

1. Cache & Logs Fill Up Storage

   • Dremio stores query results in its distributed cache (accelerations) and maintains internal indexes (IndexWriter) for metadata.

   • If storage (disk/memory) gets full, the IndexWriter fails with this error, and metadata (databases/views) may disappear.

2. APIs Stop Responding

   • When storage is exhausted, Dremio’s internal services (e.g., catalog management) may freeze, causing API failures.

How to Fix It

1. Clear Cache Manually (Temporary Fix) - Restart Dremio to force cache cleanup.

2. Adjust Cache Settings (Permanent Fix) - Reduce cache expiration time

3. Increase Storage Monitoring

4. Clear Cache Manually (Temporary Fix) - Restart Dremio to force cache cleanup.

5. Adjust Cache Settings (Permanent Fix) - Reduce cache expiration time

6. Increase Storage Monitoring