How to recover deleted users on Comet Self-Hosted with a system self-backup?

Question

How to recover deleted users on Comet Self-Hosted?

Answer

User profiles contain the Storage Vaults' randomised encryption keys. If a user or Storage Vault is deleted, they must be recovered from a past backup of the Comet Core server. Recreating the users with the same details will generate new encryption keys that do not match the stored data.

When a user or Storage Vault is deleted, its Storage Vault bucket remains on the storage as an orphaned bucket, but it cannot be reattached to a new user without the original encryption key. Comet Backup does not keep a log of deleted users or devices. To see which users differ, compare the current comet-users.db with the copy in a system self-backup. The database files use BoltDB, so a tool that can read BoltDB files, such as qbolt, is required.

Comet Core's configuration files are kept in C:\ProgramData\Comet on Windows, and in /var/lib/cometd/ on Linux. They are captured by the system self-backup.

Warning: Replacing comet-users.db rolls back all changes to all users and policies to the date the self-backup was taken. This is safe only if no new Storage Vaults, devices or Protected Items were added or changed since then. If changes were made, use the method for restoring only the Storage Vault below.

To restore the removed users:

1. Retrieve a system self-backup of the server from before the users were removed. If the configuration files were backed up through another method, retrieve the database files from that backup.
2. Stop the cometd service:
   • on Windows, in services.msc or in the Comet Core Service Manager.

   • on Linux:

     # systemctl stop cometd

3. Move or rename the existing comet-users.db file.
4. Copy the comet-users.db file from the backup into the config directory.
5. Start the cometd service.
6. If the service does not start, check the logs in C:\ProgramData\Comet\logs (Windows) or /var/log/cometd (Linux).

Roll back only the users database, not the whole configuration

To roll back changes to users, replace only comet-users.db. Make sure the cometd service is stopped while you replace the file. Restoring the complete set of configuration files from a self-backup, for example after an outdated server that was a replication master was turned on again and pushed outdated settings, can prevent the server from starting. If the server does not start after replacing files, check that the license was relaxed from the account portal and review the system logs for the reason.

Restore only the deleted Storage Vault when other changes were made since the self-backup

If new Storage Vaults, devices or Protected Items were added after the self-backup was taken, extract only the Storage Vault (the Destinations object) from the self-backup and add it to the user's profile in the current database.

Warning: Editing the JSON directly is dangerous. If it is not done correctly, stop the service, put back the copy of comet-users.db that you made, and try again.

1. Stop the cometd service:
   • on Windows, in services.msc or in the Comet Core Service Manager.

   • on Linux:

     # systemctl stop cometd

2. Open the config directory:
   • Windows - C:\ProgramData\Comet
   • Linux - /var/lib/cometd/
3. Make a copy of the current comet-users.db file in case anything goes wrong.
4. Extract the comet-users.db file from the self-backup taken before the deletion into any other location. Do not replace the existing file.
5. Open the extracted users database file in qbolt, a free tool for reading bolt database files.
6. Find the user that the Storage Vault was deleted from, and copy the user's data in JSON format into a text editor.
7. Close that database and open the current comet-users.db in qbolt.
8. Find the object under Destinations in the copied JSON that is missing from the current database, and paste it into the Destinations list of the current user, keeping valid JSON format. Alternatively, to roll back that single user entirely, replace the whole JSON data of the user with the copied data.
9. Save the changes.
10. Start the cometd service.
11. Check that the Storage Vault has been restored. If the server does not start, check the logs.