Setup Account

You can use your own S3 bucket with Illumina Connected Analytics Genomic Data Store (GDS) for data storage. Your data will reside on the external storage account. ICA indexes the data records to use in analysis.

This section describes how to configure your AWS account to connect to GDS.

Make sure you have installed the AWS CLI and verified your installation using the documentation provided on the Amazon Web Services Website.

Create the GDS S3 bucket

If you have already created an S3 bucket to use with GDS, continue to Create and Configure the IAM user.

Make sure you consider the following before creating a bucket.

• S3 buckets share a single global namespace across all of AWS. Each bucket requires a unique name and the name cannot be used for a bucket in another account.

• S3 buckets are regional. Make sure you create the S3 bucket in the same region as the ICA environment you're using. In this document we'll be using us-east-1 but this may be different for you.

1. Use the following command to create the S3 bucket. Replace BUCKET_NAME with the name of your bucket.

BUCKET_NAME=gds-bucket-name
aws s3 mb s3://${BUCKET_NAME} --region us-east-1

1. If one of the following errors occurs, resolve as follows:

   • BucketAlreadyExists— Another account has a bucket that uses the same name. Enter a unique name.

   • InvalidLocationConstraint—The region was not specified in the command. Enter the correct region.

2. Verify that the bucket exists using the following command: aws s3 ls

Block Public Access for the S3 bucket

Public access is enabled by default. To increase security on the bucket, block public access with the following command:

aws s3api put-public-access-block --bucket ${BUCKET_NAME} --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

If you would like to block public access to S3 buckets on an account level, use the AWS Console on the Amazon Web Services website.

Create the IAM User and User Policy

Creating an IAM user allows the GDS service access to manage resources in your account. IAM users are global resources and do not require a region to be specified.

1. To create the IAM user, enter the following command:

    aws iam create-user --user-name illumina_ica_admin

1. To create the IAM user policy, create a local file named illumina-ica-admin-policy.json containing the following content. Replace the ${BUCKET_NAME} with the actual name of your bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutBucketNotification",
                "s3:ListBucket",
                "s3:GetBucketNotification",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::${BUCKET_NAME}"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:RestoreObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::${BUCKET_NAME}/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "sts:GetFederationToken"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Note that this provides access to any key within the bucket. Optionally, this can be limited to a specific prefix. If doing so, also provide the chosen prefix into the Volume Configuration when creating it in a subsequent step.

1. Use the following command to attach the policy to the illumina_ica_admin user.

The policy is identified by the AWS Resource Number, which includes your AWS account number. The command creates the policy, retrieves your AWS account number, and then attaches the policy to the user.

ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
aws iam create-policy --policy-name illumina-ica-admin-policy --policy-document file://illumina-ica-admin-policy.json
aws iam attach-user-policy --policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/illumina-ica-admin-policy --user-name illumina_ica_admin

1. To create the access key for the illumina_ica_admin IAM user, enter the following command:

The command creates an access key ID and secret access key for the illumina_ica_admin IAM user.

aws iam create-access-key --user-name illumina_ica_admin

    "AccessKey": {
        "UserName": "illumina_ica_admin",
        "AccessKeyId": "<access key id>",
        "Status": "Active",
        "SecretAccessKey": "<secret access key>",
        "CreateDate": "2020-10-22 09:42:24+00:00"
    }
}

The secret access key is sensitive and should be stored securely. The access key is only displayed when this command is executed and cannot be recovered. A new access key must be created if the secret access key is lost.

Create ICA Secret for an AWS IAM User

To connect your S3 account to GDS, you need to add a secret credential in ICA, and then add a key to that named secret. Only a domain administrator can connect and S3 account to GDS.

1. Log into the domain administrator account using the following command:

ica login <email>

1. Create a aws_ica_user.json JSON file in the following format containing the access key and secret access key.

{
  "aws_access_key_id": "<access key id>",
  "aws_secret_access_key": "<secret access key>"
}

1. To create an ICA secret for your AWS IAM user, enter the following command. For the following command, the type value aws-user indicates that it's a secret for AWS IAM user credentials. Run this command to create a secret named aws_illumina_ica_admin.

ica secrets create aws_illumina_ica_admin --type aws-user --file aws_ica_user.json

Secret aws_illumina_ica_admin (type aws-user) created.

Manage ICA Secrets

Use the following commands to manage secrets.

Get a Secret

To view additional data about the secret, use the secret get command. The command does not return the secret. After created, the secret cannot be retrieved or viewed.

ica secrets get aws_illumina_ica_admin
NAME                      TYPE         TIMECREATED
aws_illumina_ica_admin     aws-user     2020-12-02T01:38:26Z

Update a Secret

To update a secret, enter the following command.

ica secrets update aws_illumina_ica_admin --file aws_ica_user.json

Delete a Secret

To delete a secret, enter the following command.

ica secrets delete aws_illumina_ica_admin
Secret aws_illumina_ica_admin deleted.