> ## Documentation Index
> Fetch the complete documentation index at: https://opensre.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# AWS Batch

> Install Tracer on AWS Batch (Multi-Node) under two minutes

<style>
  {`
    .capability-comparison-table td {
      vertical-align: middle !important;
      text-align: left !important;
    }
    .capability-comparison-table th {
      vertical-align: middle !important;
      text-align: left !important;
      color: inherit !important;
    }
    .prerequisites-table table {
      border: none !important;
    }
    .prerequisites-table td,
    .prerequisites-table th {
      border: none !important;
      padding: 0.25rem 1rem 0.25rem 0 !important;
    }
    .prerequisites-table thead {
      display: none !important;
    }
    `}
</style>

Try Tracer in Your AWS Batch Workloads

<Tabs>
  <Tab title="CloudFormation">
    <Info>
      Use CloudFormation to set up AWS Batch infrastructure for bioinformatics and
      HPC pipelines.
    </Info>

    ### 1. Connect AWS Integration

    Open the following link to connect your AWS account automatically to Tracer. This will allow Tracer to monitor your AWS Batch jobs.

    [→ Connect AWS](https://app.tracer.cloud/onboarding/linux-cloud-multi)

    ### 2. Locate the EC2 LaunchTemplate UserData

    In your CloudFormation template, find the `UserData` field within the Launch Template used by your AWS Batch compute environment.

    Reference example: [CloudFormation Template](https://github.com/Tracer-Cloud/nextflow-test-pipelines/blob/f2fea612adfb3cd9a17ea6baedd953d0c7e9ec01/infrastructure/aws_batch/cloudformation.yml#L323)

    ```
    Resources:
      NextflowLaunchTemplate:
        Properties:
          LaunchTemplateData:
            UserData: !Base64 |
              #cloud-config
              runcmd:
                - ...
    ```

    ### 3. Append Tracer install in EC2 Launch Template UserData

    Add these commands to the `runcmd`: section in your UserData:

    ```bash theme={null}
    - curl -sSL https://install.tracer.cloud | sh
    ```

    ```bash theme={null}
    - tracer init --token YOUR_TOKEN_HERE \
         --pipeline-name YOUR_PIPELINE_NAME \
         --pipeline-type YOUR_PIPELINE_TYPE \
         --environment aws_batch \
         --watch-dir="/var/log"
    ```

    <Info>
      {" "}

      Go to our [onboarding](https://app.tracer.cloud/onboarding/linux-cloud-multi)
      to get your own personal token
    </Info>

    ### 4. Validate, apply change set, then verify instance

    ```bash theme={null}
    aws cloudformation validate-template \
      --template-body file://YOUR_TEMPLATE.yaml
    ```

    ```bash theme={null}
    aws cloudformation create-change-set \
      --stack-name YOUR_STACK_NAME \
      --template-body file://YOUR_TEMPLATE.yaml \
      --change-set-name tracer-userdata-update \
      --capabilities CAPABILITY_NAMED_IAM
    ```

    ```bash theme={null}
    aws cloudformation execute-change-set \
      --stack-name YOUR_STACK_NAME \
      --change-set-name tracer-userdata-update
    ```

    ### 5. Configure Nextflow batch.config with Trace ID

    In your Nextflow `batch.config` file, you need to:

    1. Add a `params` section to generate a UUID that will be shared across all jobs in the workflow
    2. Add `containerOptions` in the `process` section to pass the trace ID as an environment variable to each container

    This allows Tracer to correlate all jobs from the same workflow execution.

    ```bash theme={null}
    // 1. Add params section to generate UUID
    params {customUUID = java.util.UUID.randomUUID().toString()}

    process {executor = 'awsbatch'
        queue = 'NextflowCPU'

    // 2. Add containerOptions to pass trace ID to containers
    containerOptions = "--env TRACER_TRACE_ID=${params.customUUID}"

    resourceLabels = ['custom-session-uuid': "${params.customUUID}"]
    }

    aws {region = 'YOUR_REGION'
      batch
      {cliPath = '/usr/local/aws-cli/v2/current/bin/aws'}
    }
    ```

    ### Monitor and Optimize Your Pipeline

    <Card href="https://app.tracer.cloud/">
      <img className="block dark:hidden" src="https://mintcdn.com/tracer/lwvLzCSZlgrNXB_D/images/logo/tracer/Tracer%20Full%20Body%20-%20Black.png?fit=max&auto=format&n=lwvLzCSZlgrNXB_D&q=85&s=3ba40411baa608f2cf09ea4372ad16b2" alt="Tracer Logo" style={{ width: '10%', height: 'auto', marginBottom: '1rem' }} width="311" height="84" data-path="images/logo/tracer/Tracer Full Body - Black.png" />

      <img className="hidden dark:block" src="https://mintcdn.com/tracer/lwvLzCSZlgrNXB_D/images/logo/tracer/Tracer%20Full%20Body%20-%20White.png?fit=max&auto=format&n=lwvLzCSZlgrNXB_D&q=85&s=472f081526d63153691ccbccc1dcd9da" alt="Tracer Logo" style={{ width: '10%', height: 'auto', marginBottom: '1rem' }} width="311" height="84" data-path="images/logo/tracer/Tracer Full Body - White.png" />

      <div
        style={{
fontSize: "1.3rem",
fontWeight: "700",
marginBottom: "1 rem",
color: "inherit",
}}
      >
        Watch your pipeline run in the Tracer dashboard
      </div>

      <div style={{ color: 'inherit' }}>
        View real-time metrics, resource usage, and performance insights for your pipeline runs.
      </div>
    </Card>
  </Tab>

  <Tab title="Terraform">
    <Info>
      Use Terraform to provision infrastructure and run RNA-seq pipeline on AWS
      Batch.
    </Info>

    <Accordion title="Prerequisites">
      Make sure these tools are installed locally before proceeding.

      <table
        style={{
border: "none",
marginTop: "0",
width: "100%",
tableLayout: "fixed",
}}
      >
        <tbody>
          <tr style={{ border: "none" }}>
            <td
              style={{
      border: "none",
      padding: "0.25rem 1rem 0.25rem 0",
      width: "18%",
    }}
            >
              • AWS CLI v2
            </td>

            <td
              style={{
      border: "none",
      padding: "0.25rem 1rem 0.25rem 0",
      width: "18%",
    }}
            >
              • Terraform
            </td>

            <td
              style={{
      border: "none",
      padding: "0.25rem 1rem 0.25rem 0",
      width: "18%",
    }}
            >
              • Nextflow
            </td>

            <td
              style={{
      border: "none",
      padding: "0.25rem 1rem 0.25rem 0",
      width: "18%",
    }}
            >
              • Java
            </td>

            <td
              style={{
      border: "none",
      padding: "0.25rem 1rem 0.25rem 0",
      width: "25%",
    }}
            >
              • Docker (optional)
            </td>
          </tr>
        </tbody>
      </table>
    </Accordion>

    ### 1. Connect AWS Integration

    Open the following link to connect your AWS account automatically to Tracer. This will allow Tracer to monitor your AWS Batch jobs.

    [→ Connect AWS](https://app.tracer.cloud/onboarding/linux-cloud-multi)

    ### 2. Clone the repository

    Clone the nextflow-test-pipelines repository to get access to the AWS Batch RNA-seq pipeline configuration.

    ```bash theme={null}
    git clone https://github.com/Tracer-Cloud/nextflow-test-pipelines.git
    cd nextflow-test-pipelines/pipelines/aws-batch/rnaseq
    ```

    ### 3. Update user\_data\_mime.sh

    Update the user\_data\_mime.sh file to include your Tracer user ID. Navigate to the terraform directory and edit the file:

    ```bash theme={null}
    cd nextflow-test-pipelines/pipelines/aws-batch/rnaseq/terraform
    vim user_data_mime.sh
    ```

    <Info>
      **Important:** Find line 120 in the file and update it with your user ID:

      ```bash theme={null}
      tracer init --token YOUR_TOKEN_HERE
      --pipeline-name aws_batch_rnaseq
      --pipeline-type RNA-SEQ
      --environment aws-batch
      --watch-dir="/var/log"
      ```

      <Info>
        {" "}

        Go to our [onboarding](https://app.tracer.cloud/onboarding/linux-cloud-multi)
        to get your own personal token
      </Info>

      **Note:** If you need to change this later, you'll need to refresh the infrastructure.
    </Info>

    [View the exact line to update](https://github.com/Tracer-Cloud/nextflow-test-pipelines/blob/dab93c3ba17a04c0e6a945264e90e2b69d1fca02/pipelines/aws-batch/rnaseq/terraform/user_data_mime.sh#L120)

    ### 4. Setup infrastructure

    Run the setup script to create the AWS infrastructure (VPC, S3, Batch, etc.):

    ```bash theme={null}
    cd nextflow-test-pipelines/pipelines/aws-batch/rnaseq
    ./run.sh setup
    ```

    <Info>
      **What this does:** Creates VPC with public/private subnets, AWS Batch compute
      environment, S3 buckets for work directory and outputs, IAM roles and
      policies, security groups, and EC2 Instance Connect endpoint.
    </Info>

    <Info>
      **Alternative for Tracer AWS Account Users:** If you have access to Tracer's
      AWS account, you can skip the infrastructure setup and run the pipeline
      directly: `bash cd nextflow-test-pipelines/pipelines/aws-batch/rnaseq
              ./run.sh tracer run `
    </Info>

    ### 5. Run the RNA-seq pipeline

    Execute the RNA-seq pipeline using the run script:

    ```bash theme={null}
    cd nextflow-test-pipelines/pipelines/aws-batch/rnaseq
    ./run.sh run
    ```

    ### Monitor and Optimize Your Pipeline

    <Card href="https://app.tracer.cloud/">
      <img className="block dark:hidden" src="https://mintcdn.com/tracer/lwvLzCSZlgrNXB_D/images/logo/tracer/Tracer%20Full%20Body%20-%20Black.png?fit=max&auto=format&n=lwvLzCSZlgrNXB_D&q=85&s=3ba40411baa608f2cf09ea4372ad16b2" alt="Tracer Logo" style={{ width: '10%', height: 'auto', marginBottom: '1rem' }} width="311" height="84" data-path="images/logo/tracer/Tracer Full Body - Black.png" />

      <img className="hidden dark:block" src="https://mintcdn.com/tracer/lwvLzCSZlgrNXB_D/images/logo/tracer/Tracer%20Full%20Body%20-%20White.png?fit=max&auto=format&n=lwvLzCSZlgrNXB_D&q=85&s=472f081526d63153691ccbccc1dcd9da" alt="Tracer Logo" style={{ width: '10%', height: 'auto', marginBottom: '1rem' }} width="311" height="84" data-path="images/logo/tracer/Tracer Full Body - White.png" />

      <div
        style={{
fontSize: "1.3rem",
fontWeight: "700",
marginBottom: "1 rem",
color: "inherit",
}}
      >
        Watch your pipeline run in the Tracer dashboard
      </div>

      <div style={{ color: 'inherit' }}>
        View real-time metrics, resource usage, and performance insights for your pipeline runs.
      </div>
    </Card>
  </Tab>
</Tabs>
