Fluent Bit uses inodes to track files in the filesystem. If the log file is deleted and recreated, the new file has a different inode, even if it has the same name. Fluent Bit sees this as a new file and starts reading from the beginning — which causes duplicate log forwarding.

Solutions to avoid re-sending full files:

1. Use Refresh_Interval with Ignore_Older and Skip_Long_Lines:
Helps Fluent Bit ignore old files and reduce re-reading risk.
2. Use Rotate_Wait:
Tells Fluent Bit to wait for a rotated file to finish writing before processing the new.
3. Try Path_Key or Key_Include:
You can tag each log line with the file path or other metadata for downstream de-duplication.
4. Log rotation policy change:
Instead of recreating files, append to existing logs to maintain the inode.
5. Use Mem_Buf_Limit and Storage.type filesystem:
Helps maintain state across restarts so Fluent Bit doesn't reprocess everything.