Skip to main content

Flows

Set up flows to run verified analysis workflows automatically on new data.

With verified workflows, Element Biosciences develops the analysis pipeline for you and simplifies the analysis setup. When a flow is added in ElemBio Cloud, the configured workflow values such as inputs, outputs, and parameters, can be reused across executions and can automatically or manually initiate analysis when a run completes.

Requirements​

A flow requires a connection to a cloud provider for storage and compute. Before you create a flow, connect your account to at least one provider. ElemBio Catalyst includes customizable basic flows that Element sets up for you.

The following providers allow you to configure flows:

Add a flow​

To reuse a workflow configuration across many analysis executions, add a flow. When you execute a flow, global parameters that are set in the configuration are applied automatically, unless you override them at the start of the launch. Assignment flows can be set to launch manually or automatically when a run completes. All other flows can only launch manually.

To add a flow from the Workflow Library page, select Add a Flow from a workflow card, or select Add a Flow from the Flows page.

Assignment workflows (Basic)​

  1. Select Add a Flow from an Assignment workflow card, such as Bases2Fastq™ or Cells2Stats.
  2. Select the compute provider that will execute the flow. Additional form fields might appear based on the provider that you select.
  3. Enter a unique name for the flow.
  4. (Optional) Enter a description for the flow.
  5. In the Workflow Version drop-down menu, select Always use latest, or select a specific workflow version.
    • Always use latest upgrades the flow to the most recent version of the workflow automatically, so executions can use different versions over time.
    • A specific version always executes with the same version and maintains stability between executions.
  6. (Optional) If your provider requires it, enter a workflow ID. For AWS Compute, enter a Shared Workflow ID.
  7. Select the storage connection to use for Output Storage. The provider writes analysis results to this bucket.
  8. Select one of the following Trigger options:
    • Automatically on completed runs executes the flow when a run completes.
    • Manually trigger on runs requires you to start executions from the Analysis Execution tab on Run Detail pages.
  9. If the flow triggers automatically, select the storage connection to use for Input Storage. The provider reads run data from this bucket.
  10. Review the Workflow Terms of Use, and then select the checkbox to agree.
  11. Select Next.
  12. (Optional) In the Parameters step, apply optional parameters to all executions of the flow.
    • You can override any parameter at the time of execution.
    • Leave a parameter empty to keep its default value.
    • For the available parameters, see the workflow in the Workflow Library.
  13. Select Save.

Secondary analysis workflows (Premium)​

Adding a secondary analysis flow, such as Sentieon Germline DNA or Parabricks Germline DNA, follows the assignment flow steps with these differences:

  • Start from a Secondary Analysis Workflow card on the Workflow Library page.
  • There is no Trigger step. Secondary analysis flows are always triggered manually and take FASTQ files as input, which you choose at the time of execution.
  • Because the flow has no automatic trigger, there is no Input Storage step.
  • In the Parameters step, you can edit parameters in the form fields or as JSON. For the available parameters, see the workflow in the Workflow Library.

Flow statuses​

After you add a flow, a card appears with a summary of the saved configurations on the Flows page. A badge on flow cards indicates one of the following connection statuses:

  • Connected: ElemBio Cloud verified the flow.
  • Partially Connected: ElemBio Cloud completed only some of the verifications for the flow, its provider, and the providers for any associated storage connections. Review the status of those providers to identify the issue.
  • Unverified: ElemBio Cloud cannot verify the flow. Verify the providers for the flow and its storage connections, and review any error messages that appear.

Managing flows​

The following buttons on the flow cards allow you to manage flows. The available actions are based on your user permissions.

  • Edit: Modifies the flow settings. Changes take effect immediately and apply only to future executions. To apply changes to a previous or in-progress execution, relaunch it.
  • Delete: Removes the flow permanently from ElemBio Cloud. When prompted, enter the name of the flow to confirm deletion.
  • Verify: Verifies the permissions for the flow. A green success message confirms the connection. If a red error message appears, review the message to address the issue.

Launch a flow​

To manually launch a flow, select the Launch Analysis button on the flow card, or from the Analysis Executions tab on a Run Details page.

Launch assignment workflows (Basic)​

  1. Select Launch Analysis.
  2. In Step 1, name the execution, and then select the flow to launch. A Flow Details table shows the cloud provider, compute connection, and output location for the flow.
  3. Select Next to move to Step 2: Inputs, and specify the run directory to use as input.
    • Select a run name. If you launch from a Run Details page, the run name is selected for you.
    • In the Run Manifest field, use the original run manifest from the completed run, or upload a corrected run manifest CSV.
  4. Select Next to move to Step 3: Parameters. Default values from the saved flow are applied automatically, and you can edit parameters in form or JSON view. For the parameters you can modify, see Bases2Fastq or Cells2Stats.
  5. Select Start Analysis. A new execution starts and appears in the Executions table.

Launch secondary analysis workflows (Premium)​

Launching a secondary analysis flow follows the assignment flow steps, except that Step 2 curates a list of sample FASTQ files instead of a run directory:

  1. Select a Bases2Fastq execution to start from. If you launch from a Run Details page, the latest Bases2Fastq execution is selected for you. You can add more than one execution in the auto-complete field.
  2. Select Load Samples to load a table of FASTQ files from the specified executions.
  3. In the Actions column, select a Sample Name field to rename a sample, or select the trashcan icon to delete a sample row.
  4. When the sample list is ready, select Next to continue to the Parameters step.

Relaunch a flow​

To resolve errors, address quality issues, apply new optional parameters, or use a corrected run manifest, relaunch a completed execution of the flow. For the steps, see Relaunch an execution.

Last updated on