Skip to main content

Amazon Web Services

ElemBio Cloud integrates with your AWS as a cloud service provider. The AWS provider enables the following tasks:

Integration Requirements​

To integrate with your AWS account, the following is required:

  • An Authorizing IAM credential: An IAM credential is necessary to authorize ElemBio Cloud to perform actions within your AWS account. Set up one of the following credential options.
  • An IAM policy that grants access: The policy defines the permissions granted to ElemBio Cloud to complete actions. You must associate the IAM policy with your IAM credential. Some permissions are optional depending on the connected services.
  • An AWS S3 bucket: The bucket stores output files from runs on AVITI Systems and analysis executions in ElemBio Cloud.

Optionally, you may also connect to Amazon HealthOmics for FASTQ generation in your AWS account. To set up a Bases2Fastq flow with your account:

  1. Request the shared workflow from Element
  2. Set up your AWS HealthOmics Execution Role
  3. Ensure the IAM policy associated with the IAM role or user for the provider includes all optional policy permissions for Amazon HealthOmics.

Authorizing ElemBio Cloud to Access Your Account​

When ElemBio Cloud completes actions on your behalf, AWS generates temporary credentials through the IAM credential to authorize ElemBio Cloud to complete actions. The IAM credential enables ElemBio Cloud to operate on your AWS account as limited by the permissions granted in the associated IAM policy.

Choose one of the following two options to authorize ElemBio Cloud to access your accounts:

  • Option 1: Create an IAM role. An IAM role is an identity in your account with specific permissions assigned and is associated to a specific user. Roles do not have long-term credentials.
  • Option 2: Create an IAM user. An IAM user is an identity in your account that enables the creation of access key and secret access key credentials with specific permissions assigned. Access keys are long-term credentials.
Tip:

For stronger security, use an IAM role as your AWS credential, as the IAM role does not use long-term credentials. If you use an IAM user, regularly rotate the access keys.

Creating an IAM Role Credential​

To create an IAM role for ElemBio Cloud, see Creating an IAM Role in the AWS Documentation and apply the following requirements:

  • Set up the IAM role as a Custom Trust Policy.
    • Associate the role with the following Trust Relationship policy.
    • Replace the required External-ID-Example with the external ID of your choice. The external ID can include alphanumeric characters and the special characters @:,=-./_. Spaces are not permitted.
Platform Role Credential Trust Policy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": ["arn:aws:iam::588258415937:root"]
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "External-ID-Example"
        }
      }
    }
  ]
}
  • Give the IAM role a name that clearly indicates the access of Element instruments, such as Element-ServiceUser.
  • After creating the role, edit the role and create an inline policy using the IAM Role policy template. Update the template with your bucket name and restrictions.
  • After creating the role, use the AWS Console to set a maximum session duration of 12 hours (43,200 seconds).

Creating an IAM User Credential​

To create an IAM role for ElemBio Cloud, see Creating IAM Users in the AWS Documentation and apply the following requirements:

  • Give the IAM user a name that is clearly associated with Element instruments, such as Element-ServiceUser.
  • Leave the option for AWS management console access unselected.
  • After creating the user, edit the user and create an inline policy using the IAM User policy template. Update the template with your bucket name and restrictions.
  • Create an access key through the AWS console.
    • Select third-party service for the access key use case. Copy the access and secret access keys to use for adding the storage connection.
    • Download the .csv file that is generated, or save your keys to a secure location.

Granting Access through IAM Policies​

AWS grants access to actions through IAM policies, which determine what actions are allowed or denied AWS resources. You must associate the policy you create with the IAM credential that authorizes ElemBio Cloud access.

For more information on JSON policies in AWS, see Creating IAM Policies in the AWS Documentation.

JSON Policy Templates​

The following JSON policy templates can be used to create inline policies for IAM roles or users during configuration. The templates include both required and optional permissions that the temporary credentials grant to ElemBio Cloud. To limit the permissions of the IAM policy, update the template for your bucket and your planned activities in ElemBio Cloud.

To access the correct JSON policy template, select the tab for your credential type.

Template for IAM Role
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3ObjectWrite",
      "Effect": "Allow",
      "Action": ["s3:PutObject"],
      "Resource": "arn:aws:s3:::BUCKET_NAME/*"
    },
    {
      "Sid": "S3ObjectRead",
      "Effect": "Allow",
      "Action": ["s3:GetObject"],
      "Resource": "arn:aws:s3:::BUCKET_NAME/*"
    },
    {
      "Sid": "S3ObjectListing",
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": "arn:aws:s3:::BUCKET_NAME"
    },
    {
      "Sid": "S3BucketLevelOperations",
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation",
        "s3:GetBucketPolicy",
        "s3:PutBucketPolicy",
        "s3:DeleteBucketPolicy"
      ],
      "Resource": "arn:aws:s3:::BUCKET_NAME"
    },
    {
      "Sid": "STSOperations",
      "Effect": "Allow",
      "Action": ["sts:GetCallerIdentity"],
      "Resource": "*"
    },
    {
      "Sid": "OmicsOperations",
      "Effect": "Allow",
      "Action": [
        "omics:GetWorkflow",
        "omics:StartRun",
        "omics:GetRun",
        "omics:CancelRun"
      ],
      "Resource": "*"
    },
    {
      "Sid": "OmicsPassRole",
      "Effect": "Allow",
      "Action": ["iam:PassRole"],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "iam:PassedToService": "omics.amazonaws.com"
        }
      }
    }
  ]
}

Update the JSON Policy Template​

  1. Copy the template policy into the JSON section when creating an IAM policy.
  2. In all "Resource" sections, replace BUCKET_NAME with your bucket name. Make sure to keep /* after the bucket names for the S3ObjectWrite and S3ObjectRead permissions.
  3. If your bucket uses a prefix, make the following additional updates to the policy:
    1. For the S3ObjectWrite and S3ObjectRead permissions, add the prefix to the bucket name in both "Resource" sections, as in the following example:
        "Resource": "arn:aws:s3:::BUCKET_NAME/OPTIONAL_PREFIX/*"
    1. After the "Resource" for the S3ObjectListing permission, add a comma and the following "Condition". Replace OPTIONAL_PREFIX with the prefix.
        "Condition": {
                "StringLike": {
                    "s3:prefix": [
                        "OPTIONAL_PREFIX/*"
                    ]
                }
            }
  4. If you do not want to include optional permissions, remove them from the template.

Policy Permissions​

PermissionRequirementPurposeRequired For
S3:GetBucketLocationRequiredDetermines the region where a bucket residesData uploads from an instrument
S3:ListBucketRequiredLists objects in the bucket as needed by the upload mechanismData uploads from an instrument and data browsing in ElemBio Cloud
S3:PutObjectRequiredPerforms single and multipart uploadsData uploads from an instrument
STS:GetCallerIdentityRequiredVerifies credentials using the current user or role nameData uploads from an instrument
STS:GetFederationTokenRequired for IAM User onlyAllows for the creation of limited temporary credentials for an IAM userData uploads from an instrument and the creation of temporary credentials for the AWS CLI
omics:GetWorkflowOptionalRetrieves the details of a HealthOmics Ready2Run workflowCreation and use of an AWS HealthOmics verified Bases2Fastq flow
omics:StartRunOptionalEnables the start a HealthOmics Ready2Run workflowCreation and use of an AWS HealthOmics verified Bases2Fastq flow
iam:PassRoleOptionalPasses the execution role to the HealthOmics Ready2Run workflow commands for execution permissionsCreation and use of an AWS HealthOmics verified Bases2Fastq flow
omics:GetRunOptionalRetrieves details of a HealthOmics runCreation and use of an AWS HealthOmics verified Bases2Fastq flow
omics:CancelRunOptionalCancels execution of a HealthOmics runCreation and use of an AWS HealthOmics verified Bases2Fastq flow
S3:GetBucketPolicyOptionalRetrieves the bucket policy of an S3 bucketData sharing through a bucket-level policy
S3:PutBucketPolicyOptionalApplies a bucket policy to an S3 bucket.Data sharing through a bucket-level policy
S3:DeleteBucketPolicyOptionalDeletes the bucket policy of an S3 bucketData sharing through a bucket-level policy
S3:GetObjectOptionalAllows for the retrieval of objects from the data browser in AWSUse of presigned URLs for file downloads from data browsing in ElemBio Cloud
Note:

Using an IAM user requires the STS:GetFederationToken permission as a security measure.

Setting Up an AWS S3 Bucket​

An AWS S3 bucket serves as the input and output for run and analysis activities and enables you to stream run data off the instrument. If you do not already have an AWS S3 bucket, you must create one. Element recommends the following settings for your bucket:

  • ACLs disabled
  • Public access blocked
  • Default encryption enabled

Consult your IT representative to confirm the appropriate settings for your lab and determine appropriate encryption. Default encryption in transit and at rest protects the run, which includes genomic data. Bucket versioning and tags are not necessary for uploading runs. You cannot rename buckets. Selecting a region close to you increases the data transfer speed.

For more information on setting up an AWS S3 bucket, see Creating a Bucket in the AWS Documentation.

Setting Up Data Analysis with Amazon HealthOmics​

FASTQ generation in your Amazon account is enabled through a Bases2Fastq flow that integrates with an Amazon HealthOmics private workflow shard by Element. To set up Bases2Fastq in your Amazon account, complete the following steps:

  1. Request the shared workflow from Element
  2. Set up your AWS HealthOmics Execution Role
  3. Ensure the IAM policy associated with the IAM role or user for the provider includes all optional policy permissions for Amazon HealthOmics.

Request a Private Bases2Fastq Workflow Shared by Element​

The private Bases2Fastq workflow that Element shares with you has the following characteristics:

  • Only supports Bases2Fastq v2.0.0.
  • Does not support projects. The workflow always applies the --no-projects optional argument.
  • Is only available in the same region where the share originates.

To obtain access to the private workflow:

  1. Contact Element Technical Support and ask for Element to share the Bases2Fastq private workflow. In the email, make sure to include your AWS Account ID and your AWS region of operation.

    After Element receives and processes this information, Element shares the private Bases2Fastq workflow with your AWS account.

  2. Accept the workflow share in the AWS console.
  3. Copy the Resource ID value from the shared workflow, and add it to your Bases2Fastq flow in ElemBio Cloud as the Shared Workflow ID.

Creating an Execution Role​

Amazon HealthOmics requires a service role, also known as an execution role. The execution role uses a different IAM policy than the policy for the provider so you must set up a separate IAM role from your provider. The execution role requires a trust policy and permisssions policy.

First, associate the execution role with the following Trust Relationship policy.

Execution Role Trust Policy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "omics.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Then, associate the execution role with the following inline IAM policy, replacing BUCKET_NAME with your bucket where data must be accessed.

Execution Role IAM Policy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "LogsAndECR",
      "Effect": "Allow",
      "Action": [
        "cloudwatch:*",
        "ecr:GetDownloadUrlForLayer",
        "logs:*",
        "ecr:BatchGetImage"
      ],
      "Resource": "*"
    },
    {
      "Sid": "S3",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObjectAcl",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:PutObjectAcl"
      ],
      "Resource": ["arn:aws:s3:::BUCKET_NAME/*", "arn:aws:s3:::BUCKET_NAME"]
    }
  ]
}

Configuring ElemBio Cloud​

After fulfilling the provider requirements, complete the following tasks to set up your AWS provider. The available actions depend on your user permissions.

Add an AWS Provider​

  1. Review the requirements for an AWS provider.
  2. On the Providers page, select Amazon Web Services.
  3. Enter a unique name.
  4. Select the default region.

    If you do not see the region you need, contact Element Technical Support.

  5. Select the applicable credential type.
    • For Role, enter the role ARN and external ID.
    • For Access Keys, enter the access key and secret key.
  6. Select Save.

Add an AWS Storage Connection​

A storage connection enables you to transfer data to the AWS bucket you own. Before you create a storage connection, you must set up the provider, including credentials and have or create an AWS s3 bucket.

  1. On the Storage page, select Add Storage.
  2. Select the AWS provider associated to the AWS account where the bucket exists.
  3. Enter a unique name.
  4. Select the region associated with the bucket.

    If you do not see the region you need, contact Element Technical Support.

  5. Enter the Bucket Name.

    Do not include the s3:// in the bucket name.

  6. If applicable, enter a Prefix that indicates the folder structure for run data.

    A prefix is recommended if you want to separate data in your bucket.

  7. Select the Use for Run Upload setting.
    • If you enable the setting, the storage connection is available on instruments at run setup time.
    • If you disable the setting, the storage connection is unavailable on instruments. You can only use the storage connection for ElemBio Cloud activities, such as verified flows.
  8. Select the Use for Data Exploration setting.
    • If you enable the setting, the bucket is available for exploraion in the Data Explorer.
    • If you disable the setting, the storage connection is unavailable for data browsing. Run output files will not display in ElemBio Cloud.
  9. Select Save.

Add an AWS Compute Connection​

  1. On the Compute Connections page, select Add Compute.
  2. Select the AWS provider associated to the AWS account where flows should execute.
  3. Enter a unique name.
  4. Enter the execution role ARN.
  5. Select the region.

    You must select a compatible AWS HealthOmics region. If you do not see the region you need, contact Element Technical Support.

  6. Select Save.

Add an AWS Bases2Fastq Flow​

To create an AWS Bases2Fastq flow, follow the Add Assignment Flow instructions. Enter the Resource ID from the shared private workflow in AWS.

Caution:

To use Amazon HealthOmics, you must first request a shared private Bases2fastq workflow shared by Element.

Sharing Data Through Bucket Policies​

An AWS S3 bucket policy is a JSON-based document that defines permissions for an S3 bucket and its objects. The bucket policy allows you to specify who can access the bucket, what actions they can perform, and under what conditions. Policies are highly customizable, enabling fine-grained control over your data. The policies work in conjunction with other AWS security features, such as IAM roles and IAM users, to ensure secure and efficient management of your S3 resources.

Using ElemBio Cloud, you can grant limited, read-only access to specific types of AWS principals, including IAM roles, IAM users, or AWS accounts. Read-only bucket policies simplify access management for many scenarios where modification of data is not required, such as:

  • Sharing data within your own internal accounts
  • Sharing data to research partners for direct data access to run or analysis data
  • Sharing your FASTQ files directly with third-party analysis platforms for secondary analysis

Grant Access through an AWS Principal​

  1. On the storage connection card on the Storage page, select More, and then select Manage Data Access.
  2. Select Add Principal ARN.
  3. Enter the AWS Principal ARN for the IAM role, IAM user, or account.
  4. Enter an optional description that represents the account you are granting access.
  5. Select Save.

    After you add a principal from ElemBio Cloud, the following policy statement is appended to the existing bucket policy.

Example Bucket Policy Statement for Added Principal
    "Statement": [
        {
            "Sid": "ElemBio_BucketReadAccess_arn:aws:iam::999939710102:user/ExampleUser",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::999939710102:user/ExampleUser"
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKETNAME",
                "arn:aws:s3:::BUCKETNAME/*"
            ]
        },
Last updated on