Skip to main content

Amazon Web Services

ElemBio Cloud integrates with your AWS account as a cloud service provider. Integration with AWS enables the following tasks:

Integration Requirements​

To integrate with your AWS account, the following is required:

  • An Authorizing IAM credential - An IAM credential is necessary to authorize ElemBio Cloud to perform actions within your AWS account. Set up either an IAM role credential (preferred) or an IAM user credential.
  • An IAM policy that grants access - The IAM policy defines the permissions that are granted to ElemBio Cloud to complete actions. You must associate the IAM policy with your IAM credential. Based on the connected services, some permissions are optional.
  • An Amazon S3 bucket - An Amazon S3 bucket stores output files from runs on AVITI Systems and analysis executions in ElemBio Cloud.

Optionally, you can connect to Amazon HealthOmics for FASTQ generation in your AWS account. To set up a Bases2Fastq flow with your account:

  1. Request the shared workflow from Element
  2. Set up your AWS HealthOmics execution role
  3. Make sure that the IAM policy that is associated with the IAM role or user for the provider includes all optional policy permissions for Amazon HealthOmics.

Authorizing ElemBio Cloud to Access Your Account​

When ElemBio Cloud completes actions on your behalf, AWS generates temporary credentials through the IAM credential to authorize ElemBio Cloud to complete actions. The IAM credential enables ElemBio Cloud to operate on your AWS account as limited by the permissions granted in the associated IAM policy.

Choose one of the following options to authorize ElemBio Cloud to access your accounts:

  • Create an IAM role - An IAM role is an identity in your account that has specific permissions assigned and is associated to a specific user. Roles do not have long-term credentials.
  • Create an IAM user - An IAM user is an identity in your account that enables the creation of access key and secret access key credentials with specific permissions assigned. Access keys are long-term credentials.
Tip: 

For stronger security, use an IAM role as your AWS credential. IAM roles do not use long-term credentials. If you use an IAM user, regularly rotate the access keys.

Creating an IAM Role Credential​

To create an IAM role for ElemBio Cloud, see Creating an IAM Role in the AWS Documentation and apply the following requirements:

  • Set up the IAM role as a Custom Trust Policy.
    • Associate the role with the following Trust Relationship policy.
    • Replace the required External-ID-Example with the external ID of your choice. The external ID can include alphanumeric characters and the special characters @:,=-./_. Spaces are not permitted.
Platform Role Credential Trust Policy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": ["arn:aws:iam::588258415937:root"]
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "External-ID-Example"
        }
      }
    }
  ]
}
  • Give the IAM role a name that clearly indicates the access of Element instruments, such as Element-ServiceUser.
  • Edit the role.
  • Use the IAM role policy template to create an inline policy. Update the template with your bucket name and restrictions.
  • After you create the role, use the AWS Console to set a maximum session duration of 12 hours (43,200 seconds).

Creating an IAM User Credential​

To create an IAM role for ElemBio Cloud, see Creating IAM Users in the AWS Documentation and apply the following requirements:

  • Provide the IAM user a name that is clearly associated with Element instruments, such as Element-ServiceUser.
  • Leave the option for AWS Management Console access unselected.
  • Edit the user.
  • Use the IAM User policy template to create an inline policy. Update the template with your bucket name and restrictions.
  • Create an access key through the AWS console.
    • Select third-party service for the access key use case. To add the storage connection, use the access and secret access keys.
    • Download the .csv file that is generated, or save your keys to a secure location.

Granting Access through IAM Policies​

AWS grants access to actions through IAM policies, which determine what actions are allowed or denied AWS resources. You must associate the policy that you create with the IAM credential that authorizes ElemBio Cloud access.

For more information on JSON policies in AWS, see Creating IAM Policies in the AWS Documentation.

JSON Policy Templates​

The following JSON policy templates can be used to create inline policies for IAM roles or users during configuration. The templates include required and optional permissions that the temporary credentials grant to ElemBio Cloud. To limit the permissions of the IAM policy, update the template for your bucket and your planned activities in ElemBio Cloud.

To access the correct JSON policy template, select the tab for your credential type.

Template for IAM Role
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3ObjectWrite",
      "Effect": "Allow",
      "Action": ["s3:PutObject"],
      "Resource": "arn:aws:s3:::BUCKET_NAME/*"
    },
    {
      "Sid": "S3ObjectRead",
      "Effect": "Allow",
      "Action": ["s3:GetObject"],
      "Resource": "arn:aws:s3:::BUCKET_NAME/*"
    },
    {
      "Sid": "S3ObjectListing",
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": "arn:aws:s3:::BUCKET_NAME"
    },
    {
      "Sid": "S3BucketLevelOperations",
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation",
        "s3:GetBucketPolicy",
        "s3:PutBucketPolicy",
        "s3:DeleteBucketPolicy"
      ],
      "Resource": "arn:aws:s3:::BUCKET_NAME"
    },
    {
      "Sid": "STSOperations",
      "Effect": "Allow",
      "Action": ["sts:GetCallerIdentity"],
      "Resource": "*"
    },
    {
      "Sid": "OmicsOperations",
      "Effect": "Allow",
      "Action": [
        "omics:GetWorkflow",
        "omics:StartRun",
        "omics:GetRun",
        "omics:CancelRun"
      ],
      "Resource": "*"
    },
    {
      "Sid": "OmicsPassRole",
      "Effect": "Allow",
      "Action": ["iam:PassRole"],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "iam:PassedToService": "omics.amazonaws.com"
        }
      }
    }
  ]
}

Update the JSON Policy Template​

  1. When you create an IAM policy, copy the template policy into the JSON section.
  2. In all of the "Resource" sections, replace BUCKET_NAME with your bucket name. Make sure that you keep /* after the bucket names for the S3ObjectWrite and S3ObjectRead permissions.
  3. If your bucket uses a prefix, then make the following updates to the policy:
    • For the S3ObjectWrite and S3ObjectRead permissions, add the prefix to the bucket name in both "Resource" sections. See the following example:
        "Resource": "arn:aws:s3:::BUCKET_NAME/OPTIONAL_PREFIX/*"
    • After the "Resource" for the S3ObjectListing permission, add a comma and the following "Condition". Replace OPTIONAL_PREFIX with the prefix.
        "Condition": {
                "StringLike": {
                    "s3:prefix": [
                        "OPTIONAL_PREFIX/*"
                    ]
                }
            }
  4. If you do not want to include optional permissions, then remove them from the template.

Policy Permissions​

PermissionRequirementPurposeRequired For
S3:GetBucketLocationRequiredDetermines the region that a bucket is locatedData uploads from an instrument
S3:ListBucketRequiredLists objects in the bucket as needed by the upload mechanismData uploads from an instrument and data browsing in ElemBio Cloud
S3:PutObjectRequiredPerforms single and multipart uploadsData uploads from an instrument
STS:GetCallerIdentityRequiredUses the current user or role name to verify credentialsData uploads from an instrument
STS:GetFederationTokenRequired for IAM User onlyAllows limited temporary credentials to be created for an IAM userData uploads from an instrument and the creation of temporary credentials for the AWS CLI
omics:GetWorkflowOptionalRetrieves the details of an AWS HealthOmics workflowCreation and use of an AWS HealthOmics verified Bases2Fastq flow
omics:StartRunOptionalEnables the start of an AWS HealthOmics workflowCreation and use of an AWS HealthOmics verified Bases2Fastq flow
iam:PassRoleOptionalPasses the execution role to the HealthOmics workflow commands for execution permissionsCreation and use of an AWS HealthOmics verified Bases2Fastq flow
omics:GetRunOptionalRetrieves details of an AWS HealthOmics runCreation and use of an AWS HealthOmics verified Bases2Fastq flow
omics:CancelRunOptionalCancels the execution of an AWS HealthOmics runCreation and use of an AWS HealthOmics verified Bases2Fastq flow
S3:GetBucketPolicyOptionalRetrieves the Amazon S3 bucket policyData sharing through a bucket-level policy
S3:PutBucketPolicyOptionalApplies an Amazon S3 bucket policyData sharing through a bucket-level policy
S3:DeleteBucketPolicyOptionalDeletes an Amazon S3 bucket policyData sharing through a bucket-level policy
S3:GetObjectOptionalAllows for the retrieval of objects from the data browser in AWSUse of presigned URLs for file downloads from data browsing in ElemBio Cloud
Note: 

Using an IAM user requires the STS:GetFederationToken permission as a security measure.

Setting Up an AWS S3 Bucket​

An Amazon S3 bucket serves as the input and output for run and analysis activities and enables you to stream run data off the instrument. If you do not already have an Amazon S3 bucket, then you must create one. Element recommends the following settings for your bucket:

  • ACLs disabled
  • Public access blocked
  • Default encryption enabled

To confirm the appropriate settings for your lab and determine appropriate encryption, consult your IT representative. Default encryption in transit and at rest protects the run, which includes genomic data. Bucket versioning and tags are not necessary for uploading runs. You cannot rename buckets. Selecting a region close to you increases the data transfer speed.

For more information on setting up an Amazon S3 bucket, see Creating a Bucket in the AWS Documentation.

Setting Up Data Analysis with Amazon HealthOmics​

FASTQ generation in your Amazon account is enabled through a Bases2Fastq flow that integrates with an AWS HealthOmics private workflow that is shared by Element. To set up Bases2Fastq in your Amazon account, complete the following steps:

  1. Request the shared workflow from Element
  2. Set up your AWS HealthOmics execution role
  3. Make sure that the IAM policy that is associated with the IAM role or user for the provider includes all optional policy permissions for AWS HealthOmics.

Request a Private Bases2Fastq Workflow that is Shared by Element​

The private Bases2Fastq workflow that Element shares with you has the following characteristics:

  • Does not support projects. The workflow always applies the --no-projects optional argument.
  • Is only available in the same region that the share originated from.
  • Is only available in AWS HealthOmics Regions.
  • Only supports running in the same region of the bucket location.

To obtain access to the private workflow, complete the following steps:

  1. Contact Element Technical Support and ask for Element to share the Bases2Fastq private workflow. In the email, make sure that you include your AWS account ID and your AWS region of operation.

    After Element receives and processes this information, Element shares the private Bases2Fastq workflow with your AWS account.

  2. Accept the workflow share in the AWS console.
  3. Copy the Resource ID value from the shared workflow, and then add it to your Bases2Fastq flow in ElemBio Cloud as the Shared Workflow ID.

Creating an Execution Role​

Amazon HealthOmics requires a service role, also known as an execution role. The execution role uses a different IAM policy than the policy for the provider, so you must set up a separate IAM role from your provider. The execution role requires a trust policy and permissions policy.

First, associate the execution role with the following Trust Relationship policy:

Execution Role Trust Policy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "omics.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Then, associate the execution role with the following inline IAM policy and replace BUCKET_NAME with the bucket that data must be accessed from.

Execution Role IAM Policy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "LogsAndECR",
      "Effect": "Allow",
      "Action": [
        "cloudwatch:*",
        "ecr:GetDownloadUrlForLayer",
        "logs:*",
        "ecr:BatchGetImage"
      ],
      "Resource": "*"
    },
    {
      "Sid": "S3",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObjectAcl",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:PutObjectAcl"
      ],
      "Resource": ["arn:aws:s3:::BUCKET_NAME/*", "arn:aws:s3:::BUCKET_NAME"]
    }
  ]
}

Configuring ElemBio Cloud​

After you complete the provider requirements, complete the following tasks to set up your AWS provider. Available actions are based on your user permissions.

Add an AWS Provider​

  1. Review the requirements for an AWS provider.
  2. On the Providers page, select Amazon Web Services.
  3. Enter a unique name.
  4. Select the default region.

    If you do not see the region you need, then contact Element Technical Support.

  5. Select the applicable credential type.
    • For Role, enter the role ARN and external ID.
    • For Access Keys, enter the access key and secret key.
  6. Select Save.

Add an AWS Storage Connection​

A storage connection enables you to transfer data to your Amazon S3 bucket. Before you create a storage connection, set up the provider and include credentials. If you don't already have an Amazon S3 bucket, then create an Amazon S3 bucket.

  1. On the Storage page, select Add Storage.
  2. Select the AWS provider that is associated to the AWS account where the bucket exists.
  3. Enter a unique name.
  4. Select the region that is associated with the bucket.

    If you do not see the region that you need, then contact Element Technical Support.

  5. Enter the bucket name.

    Do not include the s3:// in the bucket name.

  6. If applicable, enter a Prefix that indicates the folder structure for run data.

    A prefix is recommended if you want to separate data in your bucket.

  7. Select the Use for Run Upload setting.
    • If you enable this setting, then the storage connection is available on instruments at run setup time.
    • If you disable this setting, then the storage connection is unavailable on instruments. You can only use the storage connection for ElemBio Cloud activities, such as verified flows.
  8. Select the Use for Data Exploration setting.
    • If you enable this setting, then the bucket is available for exploration in the Data Explorer.
    • If you disable this setting, then the storage connection is unavailable for data browsing. Run output files will not display in ElemBio Cloud.
  9. Select Save.

Add an AWS Compute Connection​

  1. On the Compute Connections page, select Add Compute.
  2. Select the AWS provider that is associated to the AWS account where flows should execute.
  3. Enter a unique name.
  4. Enter the execution role ARN.
  5. Select the region.

    You must select a compatible AWS HealthOmics region. If you do not see the region that you need, then contact Element Technical Support.

  6. Select Save.

Add an AWS Bases2Fastq Flow​

To create an AWS Bases2Fastq flow, follow the Add Assignment Flow instructions. Enter the Resource ID from the shared private workflow in AWS.

Caution: 

To use AWS HealthOmics, request a shared private Bases2fastq workflow that is shared by Element.

Sharing Data Through Bucket Policies​

An Amazon S3 bucket policy is a JSON-based document that defines permissions for an Amazon S3 bucket and its objects. The bucket policy allows you to grant access to the bucket, specify actions that users can perform, and set conditions for the actions. Policies are highly customizable and enable fine-grained control over your data. The policies work in conjunction with other AWS security features, such as IAM roles and IAM users, to ensure secure and efficient management of your Amazon S3 resources.

Using ElemBio Cloud, you can grant limited, read-only access to specific types of AWS principals, such as IAM roles, IAM users, or AWS accounts. Read-only bucket policies simplify access management for many scenarios where modification of data is not required, such as:

  • Sharing data within your own internal accounts
  • Sharing data to research partners for direct data access to run or analysis data
  • Sharing your FASTQ files directly with third-party analysis platforms for secondary analysis

Grant Access through an AWS Principal​

  1. On the storage connection card on the Storage page, select More, and then select Manage Data Access.
  2. Select Add Principal ARN.
  3. Enter the AWS Principal ARN for the IAM role, IAM user, or account.
  4. (Optional) Enter a description that represents the account you are granting access to.
  5. Select Save.

    After you add a principal from ElemBio Cloud, the following policy statement is appended to the existing bucket policy:

Example Bucket Policy Statement for Added Principal
    "Statement": [
        {
            "Sid": "ElemBio_BucketReadAccess_arn:aws:iam::999939710102:user/ExampleUser",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::999939710102:user/ExampleUser"
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKETNAME",
                "arn:aws:s3:::BUCKETNAME/*"
            ]
        },
Last updated on