Message Broker Communication

An alternative to direct node-to-node communication is to use a single message broker for all node to node communication. The advantage of this method is that nodes do not need to accept inbound connections from other nodes which makes certain deployment scenarios easier to implement, for example the case where more than one of the MPC nodes run on mobile devices.

When using a message broker, the MPC nodes secure the communication using encrypt the communication using the private and public keys specified in the configuration files, so the AMQP server is not a single point of trust, even if TLS connections are terminated at the message broker.

Currently, the supported message brokers are AMQP (such as RabbitMQ) and Redis.

AMQP

To configure a node to use AMQP you need the following section in the configuration file of each node:

# MPC server using an AMQP message broker to communicate with other players. Cannot be used with other MPC servers.
MPCAMQPServer]
  
  # URL for the AMQP message broker
  # For a local test instance with the default vhost and guest user the following URL can be used:
  #
  # amqp://guest:guest@localhost:5672/
  ServerURL = ""
  
  # When connection to the broker drops or sending of a message fails, how long should we wait before retrying
  RetryDelay = "5s"
  
  # Specify how many channels should be used when sending messages to the broker. You might want to increase this value
  # if you enable PublisherConfirms below.
  ChannelPoolSize = 2
  
  # Use the RabbitMQ specific publisher confirms feature. This configures the RabbitMQ server to confirm each message
  # before that message is considered delivered by the client. This makes the communication with the broker more
  # resillient when the broker restarts or the network connection fails during a session, but it also has a performance
  # impact.
  PublisherConfirms = false
  
  # Instruct the message broker to persist messages. If using durable queues this ensures that messages will survice
  # a restart of the broker.
  PersistMessages = false
  
  # If DynamicQueues is true then a new queue is created on the broker for each session, instead of using a fixed queue
  # for each player. Dynamic queues are less reliable in case of network failures, but allows for multiple nodes behind
  # a load balancer. The parameters PublisherConfirms, PersistMessages and SkipSetup have no effect on dynamic queues.
  # All players must use the same type of queues.
  DynamicQueues = false
  
  # If SkipSetup is false the client will automatically create exchange and queues on the broker. However, if you need
  # more control over who can send and receive messages set this to true and configure the broker like this:
  #
  # First you need to know how to get the player ID of a player. The player ID is computed by first using SHA-256 to
  # hash the public key and then base64 URL encode (without padding) the output of the hash function.
  #
  # 1. Create a direct exchange with the name tsm.direct
  # 2. Create a queue for each player with the name tsm.playerID and an  x-message-ttl of session timeout + connection timeout
  # 3. Bind the queues above to the exchange with the queue name as the binding key
  # 4. Grant all users write access to the exchange
  # 5. Grant all users read access to their own queue
  SkipSetup = false

Redis

To configure a node to use the Redis broker, you need the following section in the configuration file of each node:

# MPC server using Redis to communicate with other players. Cannot be used with other MPC servers.
#
# It's recommended that you restrict what users can do on the Redis server. To create a user 'player0' with
# password 'pw0' for use by a TSM node, you need the following ACL:
#
# ACL SETUSER player0 on >pw0 ~tsm:* +ping +blmpop +rpush +expire
#
# If you have ExpireKeys set to true, you also need to add +expire to the list above.
[MPCRedisServer]

  # URL for the Redis server
  # For a local instance with no access control the following URL can be used:
  #
  # redis://localhost:6379/0
  ServerURL = ""
  
  # Redis pipelining is used when sending messages. This is the maximum number of messages that goes into one pipeline
  # before being sent to the Redis server.
  SendBatchSize = 20
  
  # When calling BLMPOP to retrive messages from Redis, this is the maximum number of lists to query in one call.
  ReceiveBatchSize = 20
  
  # This controls how many Redis connections are used for fetching messages from the server.
  MaxMessageReceivers = 30
  
  # Number of sessions that can be running at the same time on this TSM node. If you don't need that many sessions,
  # you can lower this number and save a little memory.
  MaxSessions = 10000
  
  # Choose whether keys will automatically expire or not. Under normal operation keys will be deleted once the MPC
  # session finishes, but in case of MPC session failure some data might remain.
  # Automatically expiring keys are disabled by default since it's normally handled by the servers eviction policy.
  ExpireKeys = false
  
  # Set this to true if you are running a Redis cluster. Only use this if you really need a Redis cluster. In most cases
  # you will get much better performance without a cluster.
  ClusterMode = false