As described in the previous page there are multiple ways to back up the Builder Vault. This page shows how this can be used to create the required backup coverage to prevent data loss. During this page, the four methods from Backup methods will be referenced. The page uses italics when talking about the process that are described on the backup methods page.

Base Backup

The basis of any computer system is backing up all data in the system. For an instance of the Builder Vault, consisting of a number of MPC nodes, this includes various information:

Database backup:
- On an MPC server node this is a regular database backup (e.g., of a PostgreSQL or MariaDB database).
- On an MPC node embedded on a mobile device where database backup is not possible, then key share backup can be used.
Configuration files for the MPC nodes and any files referenced from the configuration, like certificate or key files.
Any service launch scripts, like Docker or Kubernetes files needed to get the service up and running again.
(if using encryptor plugin/external encryptor) Any information needed to recover these encryption engines.
Any other information outside of the MPC nodes that is needed to launch the system. This could be AWS Vault or other like items.

This should ensure the data needed to recover the MPC node (whether it is a server node or an embedded mobile MPC node in a mobile app).

Recovery

To recover the system with the above information the following steps should be followed:

Restore the database.
Restore any external services and/or external encryption services.
Set up any machines for running the MPC nodes.
Correct docker, kubernetes or configuration files in case IP address (DB or MPC node) or database credentials have changed.
Start the MPC node.
Using the process described in database backup section, remove all the pre-signatures.

Scenario Specific Backup

The above describes basic database backup that should provide a good base coverage. This section will try and go into detail about how to get better coverage for specific scenarios.

Full Backup Before Key Use

When using the Builder Vault to create signatures on items where the key can be replaced without much fuss, it might not be needed to make sure the key is backed up before use as it can always be replaced.

However when keys are used for cryptocurrencies, then once the key have been generated, an address may be created and funds be set into the account belonging to that key. Before funds are sent to the address, the key must be backed up so it can be recovered. This may conflict with the database backup which is only done at certain points in time.

There are different ways this can be handled:

Do not allow the key to be used until a database backup have been created. This may require additional care regarding user interaction, because the user is ready to use the key, but has to wait.
Use the threshold security provided by the Builder to give redundancy. If there is a gap between the number of MPC nodes ( n) holding a share of the key, and the number of MPC nodes needed to generate a signature (t+1), then those nodes could potentially be lost without affecting the ability to create signatures. Note that this may conflict with the need for t+1 = n to give the user full control over the key by having control over the share on a mobile phone.
If an ERS backup is created, as described in the section below, in order to be able to recover the key in an external system, then this backup could potentially be used to recover into the original system, too, in case of a failure before the first database backup is created.
As an alternative to an ERS backup, the regular methods for export/Import of key shares can also be used to create a short-lived backup of the key. (Note, however, that a corrupted MPC node could potentially export a bad key share which would prevent import later.) The key would be exported with the wrapping key of the MPC node exporting the key, meaning the key can be imported back into the same node after it is restored. For this to work, the first backup must be made after the wrapping key have been obtained, to force creation of this key.

Either of these, or a combination, can ensure that the key can be recovered and is thus ready to be used.

External Recovery (ERS)

Sometimes a company will fail, and if the company responsible for the Builder Vault MPC nodes disappear, the Builder Vault may be gone for good. This means that the keys are lost to the user and the company may not be able to start the service again, e.g. because being bankrupt prevents them from paying for new servers. In this case, it may be required that there is an option for the user to recover the private key and import it into a different wallet and continue to access the funds that are associated with that key.

This can be achieved using the Builder Vault Emergency Backup recovery feature. This lets users create an ERS backup of a key. The backup consists of shares of the key, encrypted under an RSA key, along with by zero-knowledge proofs that the encrypted key shares are correct. The zero-knowledge proofs can be used to validate that the encrypted key shares are valid shares of the private key that corresponds to the public key.

In case of an emergency, the private key can be recovered by decrypting the ERS backup using the private RSA key. This means that the ERS backup and the private RSA key must be stored in two places that will be available even when the Builder Vault MPC nodes are gone. Where to store the ERS backup, and the ERS decryption key, will be very solution dependent.

Because the ERS decryption key and the ERS backup used to recover the key together give access to the private key, these two pieces of information MUST never be in the same place, except when an actual recovery or the key is necessary.

You can read more about the Builder Vault Emergency Recovery feature here.

Establishing A Fail-Over Instance of the Builder Vault

When preparing for a disaster it may be relevant to have a redundant fail-over instance of the Builder Vault, that all traffic can be switched to in case the primary Builder Vault instance is hit by disaster or is disconnected from the internet for some reason.

The main goal here is to keep the fail-over instance up to date with the primary instance. The Wrapped Export/Import can be used to continuously keeping the instances in sync. The main way to do this is to bind the two instances together with the wrapping keys of the fail-over instance and transferring keys after each key generation, so the fail-over instance is ready to take over right away.

Note that this can pose some problems when combined with mobile nodes.

To setup a fail-over instance of the Builder Vault, the following steps are needed:

Create configuration for the fail-over instance. The "Allow Export" can be set to false, since the fail-over instance will only import key shares.
Create and start the fail-over Builder Vault instance.
For each MPC node:
1. Run the GetWrappingKey method to get the encryption key.
2. Modify the configuration of the node in the original Builder Vault instance that corresponds to this node (so, for example, MPC Node 1 in the fail-over instance is paired with MPC node 1 of the original instance, etc.):
  1. Setup Export to true in the MPC node configuration file
  2. Setup Export whitelist to the key exported from the fail-over node (base64 encoded).
Start/Restart the original Builder Vault instance in order for the configuration changes to take effect.

When creating the key, the following steps are needed to keep the fail-over Builder Vault instance up to date:

Create the key on the original Builder Vault instance.
Export the key from the original Builder Vault instance. This export is bound to the fail-over instance, because of the export key binding.
Import the key on the fail-over Builder Vault instance.