Backup, reindex and deep scan jobs consistently fail with in Comet Backup during the retention pass: unexpected EOF

Symptoms

• Backup, reindex and deep scan jobs consistently fail and terminate with errors. The backup logs explicitly display the following error message:

  unexpected EOF (End of File) warnings and errors during StatList operations (e.g., StatList(backup-data/snapshots): unexpected EOF and List packs: StatList(backup-data/data): unexpected EOF).

• The retention pass fails to complete, throwing an exit status 1 and "The retention pass encountered an error"

Cause

The unexpected EOF error indicates that the network connection between the Comet Backup client and the storage provider is being abruptly terminated or timed out mid-operation.

This happens during backups, reindexes, and deep scans because these specific processes force Comet to scan and list the entire index structure of the Storage Vault to calculate changes and enforce retention. As a bucket accumulates a large number of files and historical snapshots (often due to an unconfigured or default "keep all backups forever" retention policy), the resulting object list request becomes too large. The storage provider's API or intermediate network infrastructure times out or drops the connection under the heavy payload weight.

Resolution

To prevent the issue from recurring, reduce the metadata load on the Storage Vault:

• Adjust Retention Policy: Change the retention policy from keeping all backups forever to a "Grandfather-Father-Son" structure (e.g., keeping 7 daily backups, 4 weekly backups, and 12 monthly backups). This consolidates and drastically reduces the number of snapshot files the client has to scan.
• Split Protected Items: Avoid pointing too many distinct or heavy Protected Items into a single Storage Vault. Distribute them across separate, smaller Storage Vaults to reduce the size of individual indexes.

Here is the section in Comet Backup documentation that will give a better idea of how to properly maintain and configure the backups:

• Configuring retention | Best practices | Comet Documentation
• Configuring users | Best practices | Comet Documentation