Skip to main content

Flows

With verified workflows, Element develops the analysis pipeline for you, simplifying analysis setup. When a flow is added in ElemBio Cloud, the configured workflow values (e.g. inputs, outputs, and parameters) can be reused across executions and can automatically or manually initiate analysis when a run completes.

Requirements​

Using an analysis flow in ElemBio Cloud requires a connection to a cloud provider for storage and compute activities. Before you create a flow, connect your account to at least one cloud provider. Depending the the flow tht you are setting up, different providers are compatible. ElemBio Catalyst, a native cloud storage and compute provider in ElemBio Cloud, includes basic flows that Element sets up for you, which you can then edit to customize.

The following providers also allow you to configure flows.

Add a Flow​

Add a flow to reuse a workflow configuration across many analysis executions. When executed, the flow will apply the global parameters set in the configuration automatically, unless you overide them at launch time. Assignment flows may be set to launch automatically on run complete or manually, while all other flows may only be set to manual launch.

Add a flow from the Workflow Library page by selecting Add a Flow from a workflow card or from the Flows page by selecting Add a Flow.

Assignment Workflows (Basic)​

  1. Select Add a Flow from an Assignment Workflow card (e.g. Bases2Fastq or Cells2Stats).
  2. Select the compute provider that will execute the flow.
  3. More form fields will appear based on the selected compute provider. Enter a unique name for the flow.
  4. Optionally enter a description for the flow.
  5. In the Workflow Version drop-down menu, select Always use latest or a specific workflow version.
    • Latest automatically upgrades the flow to the most recent version of the Workflow. Selecting this option can cause executions to use different Workflow versions.
    • Selecting a specific Workflow version allows the flow to always execute with the same version and maintain stability between executions.
  6. Some providers may require another ID:
  7. Select a storage connection for Output Storage.

    The provider will write data to this bucket for this storage connection when analysis completes.

  8. Select a Trigger option:
    • Automatically on completed runs automatically executes the assignment flow when a run completes
    • Manually trigger on runs requires you to manually start executions from the Analysis Execution tab on Run detail pages.
  9. For a flow that is set to automatically trigger, select a storage connection for Input Storage.

    The provider will read data from this bucket for this storage connection when a run completes.

  10. Review the Workflow Terms of Use, and then select the checkbox to agree.
  11. Select Next to move to the Parameters step.
  12. In the Parameters step, apply any optional parameters globally to all executions of the flow. These can be input in the form fields or edited as JSON.
    • Parameters may be overridden at time of execution.
    • Leave a parameter value to keep the default.
    • Find a list of Workflow Parameters in the corresponding Analysis Workflow section section.
  13. Select Save.

Secondary Analysis Workflows (Premium)​

Adding a secondary analysis flow requires early access.

Element Biosciences Premium Workflows are Early Access

  1. On the Workflow Library page, select Add a Flow from a Secondary Analysis Workflow card (e.g. Sentieon Germline DNA or Parabricks Germline DNA).
  2. Select the compute provider that will execute the flow.
  3. More form fields will appear based on the selected compute provider. Enter a unique name for the flow.
  4. Optionally enter a description for the flow.
  5. In the Workflow Version drop-down menu, select Always use latest or a specific workflow version.
    • Latest automatically upgrades the flow to the most recent version of the Workflow. Selecting this option can cause executions to use different Workflow versions.
    • Selecting a specific Workflow version allows the flow to always execute with the same version and maintain stability between executions.
  6. Select a storage connection for Output Storage.
    • The provider will write data to this bucket for this storage connection when analysis completes.
    • Note: This flow is manually triggered and takes FASTQ files as input, chosen at time of execution.
  7. Select Next to move to the Parameters step.
  8. In the Parameters step, apply any optional parameters globally to all executions of the flow. These can be input in the form fields or edited as JSON.
    • Parameters may be overridden at time of execution.
    • Leave a parameter value to keep the default.
    • Find a list of Workflow Parameters in the corresponding Analysis Workflow section.
  9. Select Save.

Flow Statuses​

After adding a flow, a card will show up with a summary of the saved configurations on the Flows page. A badge on flow cards indicates one of the following connection statuses:

  • Connected: ElemBio Cloud successfully verifies the flow.
  • Partially Connected: ElemBio Cloud only successfully completes some of the verifications for the flow, the associated provider, and the providers for any associated storage connections. Review the connection status of the associated providers to identify the issue.
  • Unverified: ElemBio Cloud cannot verify the flow and the associated features. To resolve, verify the associated providers for the flow and the associated storage connections. Review any error messages that appear.

Managing Flows​

The following buttons on the flow cards allow you to manage flows. The available actions depend on your user permissions.

  • Edit: Modifies the flow settings.
    • Changes take effect immediately and only apply to future executions.
    • To apply changes to previous or in-progress executions, relaunch the flow.
  • Delete: Removes the flow permanently from ElemBio Cloud. When prompted, enter the name of the flow to confirm deletion.
  • Verify: Verifies the permissions for the flow.
    • If the flow is successfully connected, a green success message appears.
    • If a red error message appears, review the error message to address the issue.

Launch a Flow​

You can manually launch a flow by selecting the Launch Analysis button on the flow card or from the Analysis Executions tab on a run details page.

For Assignment Workflows (Basic)​

  1. Select Launch Analysis.
  2. In step 1 of the wizard, name the execution and select the assignment flow to launch.
    • A reference of the cloud provider, compute connection, and output location for this flow is displayed in a Flow Details table for quick reference.
  3. Select Next to move to Step 2: Inputs, where you will fill out details about the run directory to use as input.
    • Select a run name (if you are launching from a run detail page, the run name is auto-selected).
    • In the run manifest field, either choose to use the original run manifest from the completed run, or choose to upload a corrected run manifest csv.
  4. Select Next to move to Step 3: Parameters, where you will see a list of workflow parameters to optionally customize. Parameters are available in form or JSON views and default values from the saved flow are automatically applied.
    • When launching a Bases2Fastq flow, you can modify any optional parameters for this execution of the flow.
    • When launching a Cells2Stats flow, you can modify any optional parameters for this execution of the flow.
  5. Select Start Analysis to launch the flow.
  6. A new execution begins and is accessible from the Executions table.

For Secondary Analysis Workflows (Premium)​

  1. Select the Launch Analysis option.
  2. In step 1 of the wizard, name the execution and select the secondary analysis flow to launch.
    • A reference of the cloud provider, compute connection, and output location for this flow is displayed in a Flow Details table for easy reference.
  3. Select Next to move to Step 2: Inputs, where you will curate a list of sample FASTQ files to use as input.
    • Select a Bases2Fastq execution to start from. If you are launching from a run detail page, the latest Bases2Fastq execution is auto-selected for you.
    • You can add more than one Bases2Fastq execution in the auto-complete field.
    • Select Load Samples. A table of FASTQ files will load based on the specified executions.
    • From the Actions column you can edit a sample name by selecting the pencil.
    • From the Actions column you can delete a sample row by selecting the trashcan.
  4. After building your list of samples, select Next to move to Step 3: Parameters, where you will see a list of workflow parameters to optionally customize. Parameters are available in form or JSON views and default values from the saved flow are applied.
    • For example, when launching a Sentieon Germline DNA flow, you can modify any optional parameters for this execution of the flow.
  5. Select Start Analysis to launch the flow.
  6. A new execution starts and is accessible from the Executions table.

Relaunch a Flow​

Relaunching a flow allows you to resolve errors, address quality issues, apply new optional parameters, or use a corrected run manifest. Once an execution is complete, you can relaunch from the Actions column on the All Executions table or from the top of an Execution Details page.

  1. Relaunch from one of two places:
    • The execution overview section at the top of the Executions page will display a Relaunch button once the execution is complete.
    • The All Executions table offers a relaunch option in the the actions column for each row.
  2. Select the Relaunch option.
  3. From the Launch wizard form, follow the steps to fine-tune the inputs and parameters that are autofilled with values from the reference execution.
  4. Select Start Analysis to relaunch.
  5. A new execution starts and is accessible from the Executions table.
Last updated on