Amazon S3 plugin

Introduction

Amazon S3 support two different ways for their users to do data-at-rest encryption for the data they store in S3:

1. Use an Amazon KMS-managed customer master key
2. Use a client-side master key
   (See AWS documentation).

The solution we support is (naturally) option 2.

📘

Client-side encryption vs. data-at-rest encryption

The architecture for encryption described here is called "client-side encryption" by Amazon Web Services, because it happens at the client relative to the S3 service.
Blockdaemon denotes this as data-at-rest encryption.

Using the plugin

📘

Some of these steps may be redundant if you have already used our Java demo.

Requirements

To get started using the Blockdaemon Amazon S3 plugin you first need to complete 4 simple steps:

1. Sign up for our free trial
2. Login to our portal and deploy your KeyServers or use or hosted KeyServers
3. Download Amazon S3 plugin
4. Register your S3-application from the portal

Configuring the plugin

For some platforms we will need the following configuration files. The information in them is obtained from the Blockdaemon KMaas.

blockdaemon.config

{
    "userId": "demo-user",
    "password": "af2788e8665be7adcfccf1c94a1e97f7",
    "keyServerUris": [
      "	demo-ks-1.sepport.sepior.net",
      "	demo-ks-2.sepport.sepior.net",
      "	demo-ks-3.sepport.sepior.net"
    ],
    "applicationId": "4582fe78-b4fe-4f90-a54a-879582bb94d8"
}

aws.config

accessKey='accessKeyId'
secretKey='secretAccessKey'

Then you can configure the plugin as follows (directly or with config files)

Java

SepiorServicesClientConfiguration sepiorConfig = SepiorUtils.getConfigurationFromFile(Paths.get(sepiorServicesClientConfigurationFile));
SepiorServicesClient sepiorClient = SepiorUtils.getSepiorServicesClient(sepiorConfig);
AWSCredentials awsCredentials = new PropertiesFileCredentialsProvider(amazonS3ConfigurationFile).getCredentials();

/* Create Sepior S3 encryption object with the given configuration and get an AWS client. */
s3Enc = new SepiorS3Encryption(sepiorClient, awsCredentials);

JavaScript (node.js)

const AWS = require('aws-sdk')
const {SepiorServicesClient} = require('@sepior/sdk')
const {SepiorServicesS3} = require('@sepior/s3')
const awsS3Client = new AWS.S3()
 
awsS3Client.config.update({accessKeyId: 'accessKeyId', secretAccessKey: 'secretAccessKey'})
const sepiorClient = SepiorServicesClient.createPasswordBasedClient(['ks1.example.com', 'ks2.example.com', 'ks3.example.com'], 'someAppId', {'username': 'username', 'password': 'password'})
const sepiorS3 = SepiorServicesS3.fetchClient(sepiorClient, awsS3Client)

Swift

let sepiorConfig = try! SepiorUtils.getConfigurationFromFile(Bundle.main.path(forResource: "sepior", ofType: "config")!)
sepiorS3 = try! getSepiorServicesS3(sepiorConfig)

Upload and download encrypted data using the interface of the standard AWS S3 SDK

Java

private static void uploadUsingAmazonSDK(String s3Bucket, String s3Key, String filename) throws SepiorServiceException, SepiorUserException {
  AmazonS3 awsClient = s3Enc.getAmazonS3();
  PutObjectRequest put = s3Enc.getPutObjectRequest(s3Bucket, s3Key, new File(filename));
  awsClient.putObject(put);
}

private static void downloadUsingAmazonSDK(String s3Bucket, String s3Key, String filename) throws IOException {
  AmazonS3 awsClient = s3Enc.getAmazonS3();
  GetObjectRequest get = new GetObjectRequest(s3Bucket, s3Key);
  S3Object s3Object = awsClient.getObject(get);
  try (InputStream in = s3Object.getObjectContent()) {
    Files.copy(in, Paths.get(filename));
  }
}

Upload using Blockdaemon S3 plugin directly

Java

private static void uploadUsingSepiorSDK(String s3Bucket, String s3Key, String filename) throws SepiorServiceException, SepiorUserException {
  s3Enc.uploadData(s3Bucket, s3Key, new File(filename));
}

JavaScript (node.js)

const fs = require('fs')
const inStream = fs.createReadStream('./foo.txt')
seps3.uploadData(s3Bucket, s3Key, inStream)
    .then(data => {
        console.log('Done')
    })

Swift

/* 
We assume that you provide the implementation of a new datatype called 
`S3Writer`. This new writer must be a subclass of the Writer protocol, and 
implement how you would like to upload data to S3.
*/
let writer = S3Writer()

let inputStream = InputStream(fileAtPath: "path/to/filename")
s3.getS3Writer(nil, writer, { (result: SepiorResult<(MetaData, Writer)>) in 
    if result.isSuccess {
        let (metadata, encryptionWriter) = result.value!
        // As an example lets upload the entire file to S3
        try encryptionWriter.open()
        try encryptionWriter.consume(InputStreamReader(inputStream))
        try encryptionWriter.close()
    } else {
        // ... do error handling. The error is in result.error!
    }
})

Download using Blockdaemon S3 plugin directly

Java

private static void downloadUsingSepiorSDK(String s3Bucket, String s3Key, String filename) throws IOException {
  InputStream in = s3Enc.downloadData(s3Bucket, s3Key);
  File file = new File(filename);
  	try (OutputStream out = new FileOutputStream(file)) {
    	IOUtils.copy(in, out);
   	}
 }

JavaScript (node.js)

seps3.downloadData(s3Bucket, s3Key)
    .then(stream => {
        const outStream = fs.createWriteStream('./foo.txt')
        stream.pipe(outStream)
    })

Swift

/*
In this example we will decrypt an entire file which we assume you have 
downloaded from S3 and stored in memory already. The decrypted data will
be stored in memory. 

This case is usually suited for small files or for testing.

We assume you have downloaded some file from S3 and stored it in a 
Data value.

The decryptData method is also overloaded to take an instance of the Reader protocol.
*/

let metadata: [String: Any] // = ... the metadata of the file you have downloaded
let encryptedData:Data // = ... the encrypted data you downloaded
s3.decryptData(metadata, encryptedData, { (result: SepiorResult<Data>) in
    if result.isSuccess {
        let decryptedData = result.value!
    } else {
        // ... do error handling. The error is in result.error!
    }
})

Download Amazon S3 plugin

Based on the platforms we support, here is a list of repositories containing our plugins:

Platform	Versions	Repository
Java	7+	https://nexus.sepior.net/#browse/browse/components:maven-public
Node.js	6+	https://www.npmjs.com/package/@sepior/s3
Swift	3	Send us an email

These are available programmatically as follows:

Maven (Java)

<repositories>
  <repository>
    <id>maven-public</id>
    <url>https://nexus.sepior.net/repository/maven-public</url>
  </repository>
</repositories>

npm (node.js)

npm install @sepior/s3