BaseSpace_Fetch¶

Quick Facts¶

BaseSpace_Fetch_PHB¶

The BaseSpace_Fetch workflow facilitates the transfer of Illumina sequencing data from BaseSpace (a cloud location) to a workspace on the Terra.bio platform. Rather than downloading the files to a local drive and then re-uploading them to another location, we can perform a cloud-to-cloud transfer with the BaseSpace_Fetch workflow.

What you need before you start:

• A BaseSpace account with access to the Run or Project holding your data.
• A BaseSpace access token.
• A Terra data table with one row per sample.

Step 1 — Get your basespace_access_token¶

This workflow requires a BaseSpace access token to authenticate your BaseSpace account and download FASTQ files from your Runs/Projects. Each basespace_access_token is tied to a single BaseSpace account, so you'll need a separate token for every account you want the workflow to access. The simplest way to obtain this token is with the BaseSpace command-line tool (bs), described below.

Already have a command-line environment available? You can skip ahead to 1.2 bs CLI installation.

1.1 Create a command-line environment¶

1.2 Install the BaseSpace command-line tool and retrieve the access token¶

1.3 Store the token as Terra workspace data¶

Step 2 — Find your basespace_collection_id¶

In this guide, a "collection" refers to the name of the BaseSpace Run or Project that contains your FASTQ files. The examples below show where to find the Run or Project name to enter as your basespace_collection_id.

Step 3 — Find your basespace_sample_id¶

basespace_sample_id is matched against the datasets inside your "collection". These datasets are shown under the FastQ Dataset column for a Run, or the Dataset Name column for a Project.

3.1 Starting from a SampleSheet.csv¶

Runs → {run} → Files → SampleSheet.csv

If you have the Run's sample sheet, its Sample_ID column is a good starting point, however keep in mind that they may not always provide an exact match. Per Illumina's BaseSpace data model, Sample_ID becomes the biosample name and Sample_Name becomes the library name. A FastQ Dataset is a separate object produced downstream by demultiplexing, so while the Sample_ID often matches the resulting dataset name, it is not guaranteed to. In practice, if the FastQ Dataset name differs from the Sample_ID, it's usually because a lane suffix was added.

3.2 Without a SampleSheet.csv¶

Runs → {run} → Biosamples → "FastQ Dataset" column

Projects → {project} → FASTQS → "Dataset Name" column

3.3 How your basespace_sample_id name is matched¶

The workflow searches for each basespace_sample_id in the following order, stopping after the first successful match:

1. Exact dataset name. If a dataset name exactly matches the value you provided, that dataset is used to retrieve the associated FASTQ files.

   Duplicate dataset names

   If a collection has more than one dataset with the same name (for example, a requeued sample), the workflow fails with a Duplicate datasets error by default. Set use_latest_dataset to true (default: false) to use the most recently created dataset instead, or remove the extra datasets in BaseSpace.

2. Lane-suffixed datasets. If no dataset name exactly matches the value you provided, and group_by_lane is set to true (default: false), the workflow searches for datasets with that value followed by a lane suffix. FASTQ files from matching datasets are then merged into a single R1/R2 pair.

   Lane-suffixed dataset names

   A dataset is considered a lane-suffixed dataset if removing its lane suffix (for example, _L1 or _L001) leaves a name that exactly matches the value you provided. See examples below.

If neither step finds a match, the workflow fails with No exact dataset match. If only lane-suffixed datasets exist and group_by_lane is false, it fails with Partial dataset match. Set group_by_lane to true to merge them.

3.4 Worked examples using group_by_lane¶

Each example below lists the datasets that exist in a collection, then shows what a given basespace_sample_id resolves to with group_by_lane on and off (the default).

Dataset Name:

Sample-2026-001_L001
Sample-2026-001_L002
Sample-2026-001_L003
Sample-2026-001_L004
Sample-2026-0010_L001

Dataset Name:

Sample-2026-002
Sample-2026-002_L001
Sample-2026-002_L002

Step 4 — Build the Terra data table¶

To build and format a data table, follow Terra's guide: How to make a data table from scratch or a template. Alternatively, feel free to download our BaseSpace_Fetch template. It already has the required columns, so you only need to fill in one row per sample and rename the table in the first column header. See the table below for a description of each column.

Step 5 — Upload the data table to Terra¶

Inputs¶

Outputs¶

The outputs of this workflow will be the fastq files imported from BaseSpace into the data table where the sample ID information had originally been uploaded, along with a basespace_log describing each BaseSpace API interaction.