Backup job fails with the error in Comet Backup: Cannot proceed - another task needs to finish using the Storage Vault first - Locked by device

Symptoms

Backup jobs fail usually during the retention phase with messages similar to:

CONFIG_TEXT: Cannot proceed - another task needs to finish using the Storage Vault first
Locked by device '<device-id/device-name>' (PID #xxxx)
The retention encountered a problem: exit status 1

The error usually happens after the backup has finished, at the start of post-backup cleanup. It might uncommonly happen at the very start of the backup.

The lock duration may show many hours or days, indicating a retention/reindex job in progress, stale, or inactive. This is preventing other retention passes from running because they cannot run concurrently.

Cause

The Storage Vault is currently locked by another backup-related process, or Comet believes a previous process is still active. This commonly happens when:

• A previous retention/reindex task did not terminate cleanly. Any jobs that require exclusive lock of the Storage Vault from executing another job.
• The device was rebooted or disconnected during retention cleanup, causing the backup job to get stalled.
• Another backup task is genuinely still running against the same Storage Vault

Resolution

Warning: It is cannot be assumed that a device that appears as “offline” has stopped and can be ignored. A sleeping laptop or paused VM might resume in the middle of a delete operation that is no longer safe. Care must be taken when manually unlocking Storage Vaults. Data loss is possible if Storage Vaults are manually unlocked.

Solution 1:

The preferred solution is to wait for the conflicting job to finish.

1. If the time given is within a few minutes, the other job is running. Wait for it to finish.

2. If the time is within a few hours, the other job may have stalled due to another device going to sleep. Bring the other device back online to finish the job.

3. If the time is longer and you know the process cannot be running (rebooted or decommissioned) then manually unlocking the Storage Vault will resolve the lock.

Solution 2:

If the device cannot be brought back online, e.g. it has been decommissioned, it is possible to unlock the Vault manually. 

Warning: You MUST ensure that no backup jobs are running in that Storage Vault.

Management Consoles running Comet 25.3.0 or newer will attempt to rotate the Storage Vault's credentials during an unlock to prevent unsafe operations. Supported on Wasabi, AWS, and Self-hosted Storage Gateway with an attached Storage Template.

1. Click admin account name (top right) -> toggle-on Advanced Options

2. Accounts > Users > name of the user of the Storage Vault > Devices tab > click Actions > Storage Vault tab > Unlock. This will clear the lock files.

3. When finished, check the server and job logs for confirmation of the unlock job’s success.

Solution 3:

If Locked by device errors are a common occurrence, consider separating some jobs into their own Storage Vaults. Locks are isolated by Storage Vault. A Vault that is less busy will have fewer jobs to conflict.