{"slug":"pipeline-rnaseq-2","title":"pipeline-rnaseq","summary":"Execute ENCODE RNA-seq pipeline from FASTQ to gene quantification and signal tracks. Child of pipeline-guide. Provides Nextflow execution with Docker and cloud deployment. Use when processing RNA-seq data with STAR alignment, RSEM/Kallisto quantification, or generating expression","platform":"Claude","tags":[],"authorName":"LLM Mart","authorSlug":"llm-mart","score":0,"source":"github","price":null,"verified":false,"createdAt":"2026-09-22T13:25:44.840475Z","repo":{"url":"https://github.com/ammawla/encode-toolkit","stars":21,"forks":5,"license":"AGPL-3.0","updatedAt":"2026-09-27T01:22:42Z"},"bodyHtml":"<hr>\n<h2>name: pipeline-rnaseq\ndescription: \"Execute ENCODE RNA-seq pipeline from FASTQ to gene quantification and signal tracks. Child of pipeline-guide. Provides Nextflow execution with Docker and cloud deployment. Use when processing RNA-seq data with STAR alignment, RSEM/Kallisto quantification, or generating expression matrices. Trigger on: RNA-seq pipeline, gene expression, STAR alignment, RSEM quantification, transcript quantification, TPM, FPKM, RNA processing, run RNA-seq.\"</h2>\n<h1>ENCODE RNA-seq Pipeline</h1>\n<h2>When to Use</h2>\n<ul>\n<li>User wants to run an RNA-seq processing pipeline from FASTQ to gene quantification</li>\n<li>User asks about \"RNA-seq pipeline\", \"STAR alignment\", \"RSEM\", \"gene expression quantification\", or \"Kallisto\"</li>\n<li>User needs to process bulk RNA-seq data with ENCODE-standard 2-pass STAR alignment</li>\n<li>Example queries: \"process my RNA-seq FASTQs\", \"quantify gene expression from RNA-seq\", \"run STAR and RSEM on my data\"</li>\n</ul>\n<p>Execute the ENCODE RNA-seq processing pipeline from raw FASTQ files through splice-aware\nalignment, gene/transcript quantification, and strand-specific signal track generation.\nThis skill provides a complete Nextflow DSL2 implementation following ENCODE uniform\nanalysis standards.</p>\n<h2>Overview</h2>\n<p>RNA-seq measures transcriptome-wide gene expression by sequencing cDNA derived from\ncellular RNA. The ENCODE pipeline processes RNA-seq data through quality control,\nsplice-aware alignment with STAR (2-pass mode), gene and transcript quantification\nwith RSEM, optional fast pseudoalignment with Kallisto, and generation of strand-specific\nsignal tracks as bigWig files.</p>\n<p>Key design decisions: STAR 2-pass mode for maximum splice junction sensitivity, RSEM\nfor accurate gene/transcript/isoform quantification including multi-mapped reads,\nstranded library protocol (dUTP/rf-stranded) as the ENCODE standard, and paired-end\nsequencing with a minimum of 30 million uniquely mapped reads per replicate.</p>\n<h2>Key Literature</h2>\n<table>\n<thead>\n<tr>\n<th>Reference</th>\n<th>Journal</th>\n<th>Year</th>\n<th>DOI</th>\n<th>Relevance</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Dobin et al. \"STAR: ultrafast universal RNA-seq aligner\"</td>\n<td>Bioinformatics</td>\n<td>2013</td>\n<td>10.1093/bioinformatics/bts635</td>\n<td>Splice-aware aligner (~12,000 citations)</td>\n</tr>\n<tr>\n<td>Li &amp; Dewey \"RSEM: accurate transcript quantification from RNA-Seq data\"</td>\n<td>BMC Bioinformatics</td>\n<td>2011</td>\n<td>10.1186/1471-2105-12-323</td>\n<td>Gene/transcript quantification (~6,000 citations)</td>\n</tr>\n<tr>\n<td>Bray et al. \"Near-optimal probabilistic RNA-seq quantification\"</td>\n<td>Nature Biotechnology</td>\n<td>2016</td>\n<td>10.1038/nbt.3519</td>\n<td>Fast pseudoalignment (~4,000 citations)</td>\n</tr>\n<tr>\n<td>Wang et al. \"RSeQC: quality control of RNA-seq experiments\"</td>\n<td>Bioinformatics</td>\n<td>2012</td>\n<td>10.1093/bioinformatics/bts356</td>\n<td>RNA-seq QC suite (~3,500 citations)</td>\n</tr>\n<tr>\n<td>ENCODE Project Consortium \"Expanded encyclopaedias\"</td>\n<td>Nature</td>\n<td>2020</td>\n<td>10.1038/s41586-020-2493-4</td>\n<td>ENCODE Phase 3 standards</td>\n</tr>\n<tr>\n<td>Frankish et al. \"GENCODE 2021\"</td>\n<td>Nucleic Acids Research</td>\n<td>2021</td>\n<td>10.1093/nar/gkaa1087</td>\n<td>Gene annotation reference</td>\n</tr>\n</tbody>\n</table>\n<h2>Pipeline Stages</h2>\n<pre><code>FASTQ\n  ├─&gt; FastQC (raw reads)\n  └─&gt; Trim Galore (+ FastQC on the trimmed reads)\n        ├─&gt; Kallisto (optional) ──────────&gt; kallisto/&lt;sample&gt;/abundance.tsv\n        └─&gt; STAR (2-pass)\n              ├─&gt; transcriptome BAM ─&gt; RSEM ─&gt; &lt;sample&gt;.genes.results / .isoforms.results\n              ├─&gt; bedGraph str1/str2 ─&gt; bedGraphToBigWig ─&gt; signal/&lt;sample&gt;_{plus,minus}.bw\n              └─&gt; genome BAM ─&gt; RSeQC (infer_experiment, read_distribution,\n                                       geneBody_coverage, inner_distance)\n\nMultiQC &lt;── FastQC (raw + trimmed), trimming reports, STAR Log.final.out,\n            RSEM .stat/, RSeQC infer_experiment + read_distribution\n  └─&gt; qc/multiqc/multiqc_report.html\n</code></pre>\n<p>The Kallisto abundances, the gene body coverage and the inner distance files are\npublished but are not part of the MultiQC report; read those files directly.</p>\n<h3>Stage Summary</h3>\n<table>\n<thead>\n<tr>\n<th>Stage</th>\n<th>Tool</th>\n<th>Input</th>\n<th>Output</th>\n<th>Reference</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1. QC &amp; Trimming</td>\n<td>FastQC, Trim Galore</td>\n<td>Raw FASTQ</td>\n<td>Trimmed FASTQ + FastQC reports</td>\n<td>references/01-qc-trimming.md</td>\n</tr>\n<tr>\n<td>2. Alignment</td>\n<td>STAR (2-pass)</td>\n<td>Trimmed FASTQ</td>\n<td>Genome BAM + Transcriptome BAM + bedGraph</td>\n<td>references/02-star-alignment.md</td>\n</tr>\n<tr>\n<td>3. Quantification</td>\n<td>RSEM, Kallisto</td>\n<td>Transcriptome BAM / trimmed FASTQ</td>\n<td>Gene/transcript counts, TPM, FPKM</td>\n<td>references/03-quantification.md</td>\n</tr>\n<tr>\n<td>4. Signal Tracks</td>\n<td>bedGraphToBigWig</td>\n<td>STAR bedGraph</td>\n<td>Strand-specific bigWig</td>\n<td>references/04-signal-tracks.md</td>\n</tr>\n<tr>\n<td>5. QC Metrics</td>\n<td>RSeQC, MultiQC</td>\n<td>Genome BAM, logs</td>\n<td>Strandedness, read distribution, gene body coverage</td>\n<td>references/05-qc-metrics.md</td>\n</tr>\n</tbody>\n</table>\n<h2>Input Requirements</h2>\n<h3>Required Files</h3>\n<ul>\n<li><strong>RNA-seq FASTQ</strong>: paired-end reads matched by the <code>--reads</code> glob (ENCODE standard;\nsingle-end with <code>--single_end</code>)</li>\n<li><strong>STAR genome index directory</strong> (<code>--star_index</code>)</li>\n<li><strong>RSEM reference prefix</strong> (<code>--rsem_index</code>) produced by <code>rsem-prepare-reference</code></li>\n<li><strong>BED12 gene model for RSeQC</strong> (<code>--rseqc_bed</code>)</li>\n<li><strong>Kallisto index file</strong> (<code>--kallisto_index</code>) built with kallisto 0.50.1, unless\n<code>--skip_kallisto</code> is set. kallisto 0.50.1 writes index version 13 and rejects an index\nbuilt with 0.48 or earlier</li>\n</ul>\n<p>There is no sample sheet: samples are the pairs that <code>--reads</code> matches, and the sample\nID is the shared prefix of each pair. The gene annotation is not a workflow parameter\nand there is no <code>--gtf</code>; the GTF is consumed when the STAR and RSEM references are built\n(references/02 and references/03), so the annotation is fixed by the index you pass in.</p>\n<h2>Library Strandedness</h2>\n<p><code>--strandedness</code> takes one value for the whole run — <code>reverse</code> (default), <code>forward</code> or\n<code>none</code> — and is validated before the first task. It drives three things at once: the RSEM\n<code>--strandedness</code> flag, the kallisto strand flag, and which STAR bedGraph becomes which\nsignal track.</p>\n<table>\n<thead>\n<tr>\n<th>Protocol</th>\n<th><code>--strandedness</code></th>\n<th>RSEM receives</th>\n<th>Kallisto receives</th>\n<th>Signal tracks</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>dUTP (ENCODE standard), Illumina TruSeq Stranded</td>\n<td><code>reverse</code></td>\n<td><code>--strandedness reverse</code></td>\n<td><code>--rf-stranded</code></td>\n<td><code>&lt;sample&gt;_plus.bw</code>, <code>&lt;sample&gt;_minus.bw</code></td>\n</tr>\n<tr>\n<td>Directional ligation (some legacy protocols)</td>\n<td><code>forward</code></td>\n<td><code>--strandedness forward</code></td>\n<td><code>--fr-stranded</code></td>\n<td><code>&lt;sample&gt;_plus.bw</code>, <code>&lt;sample&gt;_minus.bw</code></td>\n</tr>\n<tr>\n<td>SMARTer / SMART-Seq2 and other unstranded kits</td>\n<td><code>none</code></td>\n<td><code>--strandedness none</code></td>\n<td>no strand flag</td>\n<td><code>&lt;sample&gt;_unstranded.bw</code></td>\n</tr>\n</tbody>\n</table>\n<p>There is no per-sample strandedness and the workflow does not detect it. RSeQC\n<code>infer_experiment.py</code> runs as a post-hoc check and writes\n<code>qc/rseqc/&lt;sample&gt;.infer_experiment.txt</code>. If the library type is unknown, run a first\npass, read that file (references/05 explains the output), and rerun with the correct\n<code>--strandedness</code> — the RSEM counts, the kallisto abundances and the signal tracks all\ndepend on it, so a wrong value has to be corrected by rerunning, not by post-processing.</p>\n<h2>QC Thresholds</h2>\n<table>\n<thead>\n<tr>\n<th>Metric</th>\n<th>Threshold</th>\n<th>Produced by</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Total sequenced reads</td>\n<td>&gt;=30M PE reads</td>\n<td><code>fastqc/</code>, <code>star/&lt;sample&gt;.Log.final.out</code></td>\n</tr>\n<tr>\n<td>Uniquely mapped reads</td>\n<td>&gt;=70% of input reads</td>\n<td><code>star/&lt;sample&gt;.Log.final.out</code></td>\n</tr>\n<tr>\n<td>Multi-mapped reads</td>\n<td>&lt;10%</td>\n<td><code>star/&lt;sample&gt;.Log.final.out</code></td>\n</tr>\n<tr>\n<td>Strandedness agreement</td>\n<td>&gt;90% for a stranded library</td>\n<td><code>qc/rseqc/&lt;sample&gt;.infer_experiment.txt</code></td>\n</tr>\n<tr>\n<td>Exonic rate</td>\n<td>&gt;60%</td>\n<td><code>qc/rseqc/&lt;sample&gt;.read_distribution.txt</code></td>\n</tr>\n<tr>\n<td>Gene body coverage</td>\n<td>Relatively uniform (5'/3' bias &lt;1.5)</td>\n<td><code>qc/rseqc/&lt;sample&gt;.geneBody_coverage.geneBodyCoverage.txt</code></td>\n</tr>\n</tbody>\n</table>\n<p>Not computed by this workflow: rRNA rate, library duplication rate (beyond the\nsequence-level estimate inside the FastQC report), detected-gene counts, and saturation\ncurves. references/05-qc-metrics.md gives the commands to run those by hand on the\npublished BAM and RSEM output.</p>\n<h3>Read Depth Guidelines</h3>\n<table>\n<thead>\n<tr>\n<th>Application</th>\n<th>Minimum Reads (PE)</th>\n<th>Recommended</th>\n<th>Notes</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Gene-level expression</td>\n<td>20M</td>\n<td>30M</td>\n<td>ENCODE minimum</td>\n</tr>\n<tr>\n<td>Transcript-level expression</td>\n<td>40M</td>\n<td>60M</td>\n<td>Isoform resolution requires more depth</td>\n</tr>\n<tr>\n<td>Differential expression</td>\n<td>20M per sample</td>\n<td>30M per sample</td>\n<td>3+ biological replicates per condition</td>\n</tr>\n<tr>\n<td>Novel junction discovery</td>\n<td>60M</td>\n<td>100M+</td>\n<td>STAR 2-pass mode benefits from depth</td>\n</tr>\n<tr>\n<td>Fusion detection</td>\n<td>50M</td>\n<td>80M+</td>\n<td>Chimeric reads are rare; needs a separate STAR run (references/02)</td>\n</tr>\n</tbody>\n</table>\n<h2>Execution</h2>\n<p>The versions the workflow runs are the ones in <code>scripts/Dockerfile</code>: STAR 2.7.11b,\nRSEM 1.3.3, kallisto 0.50.1, samtools 1.19, RSeQC 5.0.3, Trim Galore 0.6.10, cutadapt 4.6,\nMultiQC 1.21 and FastQC 0.12.1. The conda environment in <code>bioinformatics-installer</code>\n(<code>environments/rnaseq-env.yml</code>) is a separate manual route pinned to the same versions of\nSTAR, RSEM, kallisto, samtools, RSeQC, Trim Galore, FastQC and MultiQC; it leaves cutadapt\nto the Trim Galore package, adds salmon and subread, and does not carry\n<code>bedGraphToBigWig</code>, which the signal-track step needs.</p>\n<p>Every index flag is shown in the examples below because the defaults are bare names\nresolved in the launch directory (<code>GRCh38_star_index</code>, <code>GRCh38_rsem_index/GRCh38</code>,\n<code>gencode.v38.kallisto.idx</code>, <code>hg38_RefSeq.bed</code> for <code>--genome GRCh38</code>). The run stops\nbefore the first task if any of them is missing. <code>--kallisto_index</code> is the one exception:\nit is not read when <code>--skip_kallisto</code> is set.</p>\n<h3>Quick Start (local, Docker)</h3>\n<pre><code>nextflow run scripts/main.nf -profile local \\\n    --reads 'fastq/*_R{1,2}.fq.gz' \\\n    --genome GRCh38 \\\n    --star_index /ref/GRCh38_star_index \\\n    --rsem_index /ref/GRCh38_rsem_index/GRCh38 \\\n    --kallisto_index /ref/gencode.v38.kallisto.idx \\\n    --rseqc_bed /ref/hg38_RefSeq.bed \\\n    --strandedness reverse \\\n    --outdir results/\n</code></pre>\n<h3>SLURM HPC</h3>\n<pre><code>nextflow run scripts/main.nf -profile slurm \\\n    --container /path/to/pipeline-rnaseq.sif \\\n    --slurm_queue normal \\\n    --reads 'fastq/*_R{1,2}.fq.gz' \\\n    --genome GRCh38 \\\n    --star_index /ref/GRCh38_star_index \\\n    --rsem_index /ref/GRCh38_rsem_index/GRCh38 \\\n    --kallisto_index /ref/gencode.v38.kallisto.idx \\\n    --rseqc_bed /ref/hg38_RefSeq.bed \\\n    --outdir results/\n</code></pre>\n<h3>Cloud</h3>\n<pre><code># Google Cloud Batch\nnextflow run scripts/main.nf -profile gcp \\\n    --container us-docker.pkg.dev/&lt;project&gt;/&lt;repo&gt;/pipeline-rnaseq:1.0.0 \\\n    --gcp_project &lt;project&gt; \\\n    --gcp_workdir gs://&lt;bucket&gt;/work \\\n    --reads 'gs://&lt;bucket&gt;/fastq/*_R{1,2}.fq.gz' \\\n    --genome GRCh38 \\\n    --star_index gs://&lt;bucket&gt;/ref/GRCh38_star_index \\\n    --rsem_index gs://&lt;bucket&gt;/ref/GRCh38_rsem_index/GRCh38 \\\n    --kallisto_index gs://&lt;bucket&gt;/ref/gencode.v38.kallisto.idx \\\n    --rseqc_bed gs://&lt;bucket&gt;/ref/hg38_RefSeq.bed \\\n    --outdir gs://&lt;bucket&gt;/results\n\n# AWS Batch\nnextflow run scripts/main.nf -profile aws \\\n    --container &lt;account&gt;.dkr.ecr.&lt;region&gt;.amazonaws.com/pipeline-rnaseq:1.0.0 \\\n    --aws_queue &lt;job-queue&gt; \\\n    --aws_workdir s3://&lt;bucket&gt;/work \\\n    --reads 's3://&lt;bucket&gt;/fastq/*_R{1,2}.fq.gz' \\\n    --genome GRCh38 \\\n    --star_index s3://&lt;bucket&gt;/ref/GRCh38_star_index \\\n    --rsem_index s3://&lt;bucket&gt;/ref/GRCh38_rsem_index/GRCh38 \\\n    --kallisto_index s3://&lt;bucket&gt;/ref/gencode.v38.kallisto.idx \\\n    --rseqc_bed s3://&lt;bucket&gt;/ref/hg38_RefSeq.bed \\\n    --outdir s3://&lt;bucket&gt;/results\n</code></pre>\n<p><code>--outdir</code> only sets where results are published; Google Batch and AWS Batch stage every\ntask through the work directory, and the workflow stops with an error if it or the\nproject/queue is missing.</p>\n<h2>Pipeline Parameters</h2>\n<table>\n<thead>\n<tr>\n<th>Parameter</th>\n<th>Default</th>\n<th>Description</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><code>--reads</code></td>\n<td>required</td>\n<td>Glob matching the FASTQ pairs, e.g. <code>'fastq/*_R{1,2}.fq.gz'</code>. Quote it</td>\n</tr>\n<tr>\n<td><code>--genome</code></td>\n<td><code>GRCh38</code></td>\n<td><code>GRCh38</code> or <code>mm10</code>; selects the default index names and nothing else</td>\n</tr>\n<tr>\n<td><code>--outdir</code></td>\n<td><code>./results</code></td>\n<td>Directory results are published to</td>\n</tr>\n<tr>\n<td><code>--single_end</code></td>\n<td><code>false</code></td>\n<td>Treat <code>--reads</code> as single files; kallisto then runs with the fixed <code>--single -l 200 -s 20</code>, and RSeQC <code>inner_distance.py</code> is skipped</td>\n</tr>\n<tr>\n<td><code>--strandedness</code></td>\n<td><code>reverse</code></td>\n<td><code>reverse</code>, <code>forward</code> or <code>none</code>; one value for the whole run</td>\n</tr>\n<tr>\n<td><code>--skip_kallisto</code></td>\n<td><code>false</code></td>\n<td>Skip <code>KALLISTO_QUANT</code>; <code>--kallisto_index</code> is then not read</td>\n</tr>\n<tr>\n<td><code>--star_index</code></td>\n<td><code>GRCh38_star_index</code> (<code>mm10_star_index</code>)</td>\n<td>STAR genome directory</td>\n</tr>\n<tr>\n<td><code>--rsem_index</code></td>\n<td><code>GRCh38_rsem_index/GRCh38</code> (<code>mm10_rsem_index/mm10</code>)</td>\n<td>RSEM reference <strong>prefix</strong>, not a directory; every file starting with it is staged</td>\n</tr>\n<tr>\n<td><code>--kallisto_index</code></td>\n<td><code>gencode.v38.kallisto.idx</code> (<code>gencode.vM27.kallisto.idx</code>)</td>\n<td>kallisto index file; must be built with kallisto 0.50.1 (index version 13), not with 0.48 or earlier</td>\n</tr>\n<tr>\n<td><code>--rseqc_bed</code></td>\n<td><code>hg38_RefSeq.bed</code> (<code>mm10_RefSeq.bed</code>)</td>\n<td>BED12 gene model used by all four RSeQC modules</td>\n</tr>\n<tr>\n<td><code>--chrom_sizes</code></td>\n<td><code>&lt;star_index&gt;/chrNameLength.txt</code></td>\n<td>Chromosome sizes for <code>bedGraphToBigWig</code></td>\n</tr>\n</tbody>\n</table>\n<h3>Infrastructure parameters (<code>nextflow.config</code>)</h3>\n<table>\n<thead>\n<tr>\n<th>Parameter</th>\n<th>Default</th>\n<th>Description</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><code>--container</code></td>\n<td><code>encode-toolkit/pipeline-rnaseq:1.0.0</code></td>\n<td>Image built from <code>scripts/Dockerfile</code>. Pass a registry image for <code>gcp</code>/<code>aws</code>, or a <code>.sif</code> file for <code>slurm</code></td>\n</tr>\n<tr>\n<td><code>--max_cpus</code>, <code>--max_memory</code>, <code>--max_time</code></td>\n<td><code>16</code>, <code>64.GB</code>, <code>24.h</code></td>\n<td>Upper bounds applied to every process</td>\n</tr>\n<tr>\n<td><code>--slurm_queue</code>, <code>--slurm_account</code></td>\n<td><code>normal</code>, none</td>\n<td>SLURM partition and account</td>\n</tr>\n<tr>\n<td><code>--gcp_project</code>, <code>--gcp_workdir</code></td>\n<td>none (both required for <code>-profile gcp</code>)</td>\n<td>Google Cloud project and <code>gs://</code> work directory</td>\n</tr>\n<tr>\n<td><code>--gcp_location</code>, <code>--gcp_disk</code></td>\n<td><code>us-central1</code>, <code>200.GB</code></td>\n<td>Google Batch region and per-task disk</td>\n</tr>\n<tr>\n<td><code>--aws_queue</code>, <code>--aws_workdir</code></td>\n<td>none (both required for <code>-profile aws</code>)</td>\n<td>AWS Batch job queue and <code>s3://</code> work directory</td>\n</tr>\n<tr>\n<td><code>--aws_region</code>, <code>--aws_cli_path</code></td>\n<td><code>us-east-1</code>, <code>/home/ec2-user/miniconda/bin/aws</code></td>\n<td>AWS region, and the AWS CLI path inside the Batch AMI</td>\n</tr>\n</tbody>\n</table>\n<h2>Cloud Cost Estimates</h2>\n<table>\n<thead>\n<tr>\n<th>Platform</th>\n<th>Instance</th>\n<th>Cost/Sample</th>\n<th>Time/Sample</th>\n<th>Notes</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>GCP</td>\n<td>n1-highmem-8</td>\n<td>~$3-6</td>\n<td>2-4 hours</td>\n<td>STAR index loading dominates; the <code>gcp</code> profile uses spot VMs</td>\n</tr>\n<tr>\n<td>AWS</td>\n<td>r5.2xlarge</td>\n<td>~$3-6</td>\n<td>2-4 hours</td>\n<td>r-series for STAR memory; spot recommended</td>\n</tr>\n<tr>\n<td>Local</td>\n<td>8 cores, 36 GB</td>\n<td>$0</td>\n<td>3-6 hours</td>\n<td>Docker required</td>\n</tr>\n<tr>\n<td>SLURM</td>\n<td>8 cores, 36 GB</td>\n<td>Varies</td>\n<td>2-4 hours</td>\n<td>Singularity; pass the <code>.sif</code> with <code>--container</code></td>\n</tr>\n</tbody>\n</table>\n<p><strong>Memory note</strong>: <code>nextflow.config</code> asks for 36 GB for <code>STAR_ALIGN</code> (multiplied by the\nattempt number on a retry, capped by <code>--max_memory</code>). The local executor refuses the task\non a machine with less, so a 32 GB host is not enough. If 36 GB is out of reach, rebuild the STAR index\nwith a larger <code>--genomeSAsparseD</code> (which shrinks the loaded index at some cost in\nmapping speed) or move to a bigger machine. <code>--limitGenomeGenerateRAM</code> is a\n<code>genomeGenerate</code> option and is not a parameter of this workflow.</p>\n<h2>Output Directory Structure</h2>\n<pre><code>results/\n  fastqc/                                        # FastQC HTML/zip for raw and trimmed reads\n  trimmed/\n    &lt;sample&gt;_R1_val_1.fq.gz, &lt;sample&gt;_R2_val_2.fq.gz\n    &lt;sample&gt;_R1.fq.gz_trimming_report.txt        # one per input file\n  star/\n    &lt;sample&gt;.Aligned.sortedByCoord.out.bam\n    &lt;sample&gt;.Aligned.sortedByCoord.out.bam.bai\n    &lt;sample&gt;.Aligned.toTranscriptome.out.bam     # RSEM input\n    &lt;sample&gt;.Log.final.out\n    &lt;sample&gt;.SJ.out.tab\n    &lt;sample&gt;.ReadsPerGene.out.tab                # STAR gene counts\n    &lt;sample&gt;.Signal.UniqueMultiple.str1.out.bg   # plus str2 unless --strandedness none\n    &lt;sample&gt;.Signal.Unique.str1.out.bg           # same signal, unique mappers only\n  rsem/\n    &lt;sample&gt;.genes.results                       # gene_id, TPM, FPKM, expected_count\n    &lt;sample&gt;.isoforms.results                    # transcript_id, TPM, FPKM, IsoPct\n    &lt;sample&gt;.stat/                               # RSEM model statistics, read by MultiQC\n  kallisto/                                      # absent with --skip_kallisto\n    &lt;sample&gt;/abundance.tsv\n    &lt;sample&gt;/abundance.h5                        # only from a kallisto build with HDF5 support\n    &lt;sample&gt;/run_info.json\n  signal/\n    &lt;sample&gt;_plus.bw, &lt;sample&gt;_minus.bw          # or &lt;sample&gt;_unstranded.bw\n  qc/\n    rseqc/\n      &lt;sample&gt;.infer_experiment.txt\n      &lt;sample&gt;.read_distribution.txt\n      &lt;sample&gt;.geneBody_coverage.*\n      &lt;sample&gt;.inner_distance.*                  # paired-end only\n    multiqc/\n      multiqc_report.html\n      multiqc_data/\n  pipeline_info/\n    timeline.html, report.html, trace.txt        # Nextflow execution reports\n</code></pre>\n<h2>Common Pitfalls</h2>\n<h3>1. Insufficient Memory for STAR</h3>\n<p>STAR loads the whole genome index into memory. <code>STAR_ALIGN</code> is configured for 36 GB, so a\n32 GB machine will not schedule the task at all under <code>-profile local</code>. On shared HPC\nsystems, check the per-job memory limit before submitting.</p>\n<h3>2. Wrong Strandedness Setting</h3>\n<p>Using incorrect strandedness results in near-zero gene counts. If you see uniformly low\ncounts, read <code>qc/rseqc/&lt;sample&gt;.infer_experiment.txt</code> and rerun with the matching\n<code>--strandedness</code>. ENCODE dUTP libraries are <code>reverse</code> stranded.</p>\n<h3>3. Using FPKM for Cross-Sample Comparison</h3>\n<p>FPKM values are not comparable across samples because they depend on total library\ncomposition. Use TPM (comparable across samples) or raw counts with DESeq2/edgeR\nnormalization for differential expression.</p>\n<h3>4. Ignoring Multi-Mapped Reads</h3>\n<p>RSEM uses an expectation-maximization algorithm to probabilistically assign multi-mapped\nreads. This is critical for gene families and repetitive elements. Do not pre-filter\nmulti-mappers before RSEM quantification.</p>\n<h3>5. Assuming rRNA Contamination Was Checked</h3>\n<p>High rRNA contamination (&gt;10%) indicates failed rRNA depletion and reduces effective\nsequencing depth. This workflow does not measure it — run the manual check in\nreferences/05-qc-metrics.md against <code>star/&lt;sample&gt;.Aligned.sortedByCoord.out.bam</code> before\ntrusting the quantifications.</p>\n<h3>6. Not Using 2-Pass Mode for Novel Junctions</h3>\n<p>STAR 1-pass mode only uses annotated splice junctions. 2-pass mode first discovers novel\njunctions then re-maps, critical for non-model organisms or samples with extensive\nalternative splicing. This workflow always runs <code>--twopassMode Basic</code>.</p>\n<h2>Pipeline Scripts</h2>\n<table>\n<thead>\n<tr>\n<th>File</th>\n<th>Description</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><code>scripts/main.nf</code></td>\n<td>Nextflow DSL2 pipeline</td>\n</tr>\n<tr>\n<td><code>scripts/nextflow.config</code></td>\n<td>Execution profiles (local/slurm/gcp/aws)</td>\n</tr>\n<tr>\n<td><code>scripts/Dockerfile</code></td>\n<td>Docker build with STAR, RSEM, Kallisto, RSeQC</td>\n</tr>\n</tbody>\n</table>\n<h2>ENCODE Data Integration</h2>\n<p>After running on your own data, compare with ENCODE reference:</p>\n<pre><code># Find matching ENCODE RNA-seq experiments\nencode_search_experiments(\n    assay_title=\"total RNA-seq\",\n    organ=\"pancreas\",\n    biosample_type=\"tissue\"\n)\n\n# Download ENCODE gene quantifications for comparison\nencode_batch_download(\n    download_dir=\"/data/encode_reference/\",\n    output_type=\"gene quantifications\",\n    assay_title=\"total RNA-seq\",\n    organ=\"pancreas\",\n    assembly=\"GRCh38\"\n)\n\n# Download ENCODE signal tracks for browser visualization\nencode_search_files(\n    file_format=\"bigWig\",\n    assay_title=\"total RNA-seq\",\n    organ=\"pancreas\",\n    output_type=\"signal of unique reads\"\n)\n</code></pre>\n<h2>Pitfalls &amp; Edge Cases</h2>\n<ul>\n<li><strong>Strandedness must match library prep</strong>: <code>--strandedness</code> feeds RSEM, kallisto and the\nsignal tracks. Using the wrong value can halve gene counts, assign reads to antisense\ngenes, and swap the plus/minus bigWigs.</li>\n<li><strong>rRNA contamination</strong>: rRNA &gt;10% wastes sequencing depth. Ribosomal depletion libraries\nshould have &lt;5%, poly-A selection libraries &lt;1%. The workflow does not measure it; see\nreferences/05-qc-metrics.md for a manual count, or run Picard CollectRnaSeqMetrics\noutside the container.</li>\n<li><strong>STAR 2-pass mode is required</strong>: The first pass discovers novel splice junctions; the\nsecond pass uses them. Single-pass STAR misses tissue-specific or rare splicing events,\nreducing sensitivity for differential exon usage.</li>\n<li><strong>Gene-level vs transcript-level quantification</strong>: RSEM provides transcript-level estimates but gene-level aggregation is more robust for differential expression. Transcript-level analysis requires many more replicates (≥6).</li>\n<li><strong>TPM normalization is not for cross-sample comparison</strong>: TPM normalizes within a sample but is NOT appropriate for comparing expression across conditions. Use DESeq2 size factors or TMM normalization for differential expression.</li>\n<li><strong>Batch effects in multi-lab data</strong>: RNA-seq is highly sensitive to library prep method, sequencer, and lab. Always check for batch effects with PCA before combining datasets from different sources.</li>\n</ul>\n<h2>Walkthrough: Processing ENCODE RNA-seq from FASTQ to Gene Quantification</h2>\n<p><strong>Goal</strong>: Process raw RNA-seq FASTQ files through the ENCODE pipeline to generate gene expression quantifications (TPM/FPKM) and signal tracks.\n<strong>Context</strong>: The ENCODE RNA-seq pipeline uses STAR 2-pass alignment and RSEM quantification, producing both gene-level and transcript-level expression estimates.</p>\n<h3>Step 1: Find RNA-seq experiment</h3>\n<pre><code>encode_get_experiment(accession=\"ENCSR000CPR\")\n</code></pre>\n<p>Expected output:</p>\n<pre><code>{\n  \"accession\": \"ENCSR000CPR\",\n  \"assay_title\": \"total RNA-seq\",\n  \"biosample_summary\": \"K562\",\n  \"assembly\": [\"GRCh38\"],\n  \"bio_replicate_count\": 2,\n  \"status\": \"released\"\n}\n</code></pre>\n<h3>Step 2: List FASTQ files</h3>\n<pre><code>encode_list_files(experiment_accession=\"ENCSR000CPR\", file_format=\"fastq\")\n</code></pre>\n<p>Expected output (a JSON array of file records; fields abridged):</p>\n<pre><code>[\n  {\"accession\": \"ENCFF200RN1\", \"file_format\": \"fastq\", \"output_type\": \"reads\", \"file_size_human\": \"3.2 GB\", \"biological_replicates\": [1], \"status\": \"released\"},\n  {\"accession\": \"ENCFF201RN2\", \"file_format\": \"fastq\", \"output_type\": \"reads\", \"file_size_human\": \"3.3 GB\", \"biological_replicates\": [1], \"status\": \"released\"}\n]\n</code></pre>\n<h3>Step 3: Download and name the FASTQs so a read-pair glob can find them</h3>\n<pre><code>encode_download_files(file_accessions=[\"ENCFF200RN1\", \"ENCFF201RN2\"], download_dir=\"/data/rnaseq/fastq\")\n</code></pre>\n<p>ENCODE names every FASTQ after its accession (<code>ENCFF200RN1.fastq.gz</code>), with no <code>_R1</code>/<code>_R2</code>\nin the name, so the two files of a pair share no prefix and the <code>--reads</code> glob cannot pair\nthem. Link them into the shape the glob expects. Which mate an accession is comes from the\nENCODE file record on encodeproject.org, which carries <code>paired_end</code> (1 or 2) and\n<code>paired_with</code>; the MCP file tools do not return those two fields:</p>\n<pre><code>cd /data/rnaseq/fastq\nln -s ENCFF200RN1.fastq.gz k562_rep1_R1.fq.gz\nln -s ENCFF201RN2.fastq.gz k562_rep1_R2.fq.gz\n</code></pre>\n<h3>Step 4: Run the RNA-seq pipeline</h3>\n<pre><code>nextflow run scripts/main.nf -profile local \\\n    --reads '/data/rnaseq/fastq/k562_*_R{1,2}.fq.gz' \\\n    --genome GRCh38 \\\n    --star_index /ref/GRCh38_star_index \\\n    --rsem_index /ref/GRCh38_rsem_index/GRCh38 \\\n    --kallisto_index /ref/gencode.v38.kallisto.idx \\\n    --rseqc_bed /ref/hg38_RefSeq.bed \\\n    --strandedness reverse \\\n    --outdir results/\n</code></pre>\n<p>Key pipeline steps:</p>\n<ol>\n<li>FastQC on the raw reads, quality and adapter trimming with Trim Galore (which also runs FastQC on the trimmed reads)</li>\n<li>STAR 2-pass alignment (splice-aware), writing the genome BAM, the transcriptome BAM, gene counts and bedGraphs</li>\n<li>RSEM gene and isoform quantification (TPM, FPKM, expected counts)</li>\n<li>Kallisto transcript quantification (optional, skipped with <code>--skip_kallisto</code>)</li>\n<li>Signal track generation (bedGraph to bigWig)</li>\n<li>RSeQC (infer_experiment, read_distribution, geneBody_coverage, inner_distance) and MultiQC</li>\n</ol>\n<h3>Step 5: Validate output quality</h3>\n<table>\n<thead>\n<tr>\n<th>Metric</th>\n<th>Threshold</th>\n<th>Where to read it</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Uniquely mapped rate</td>\n<td>&gt;= 70%</td>\n<td><code>star/&lt;sample&gt;.Log.final.out</code></td>\n</tr>\n<tr>\n<td>Strandedness agreement</td>\n<td>&gt; 90% and matching <code>--strandedness</code></td>\n<td><code>qc/rseqc/&lt;sample&gt;.infer_experiment.txt</code></td>\n</tr>\n<tr>\n<td>Exonic rate</td>\n<td>&gt; 60%</td>\n<td><code>qc/rseqc/&lt;sample&gt;.read_distribution.txt</code></td>\n</tr>\n<tr>\n<td>Replicate correlation</td>\n<td>&gt;= 0.9</td>\n<td>compute yourself from the TPM column of <code>rsem/&lt;sample&gt;.genes.results</code></td>\n</tr>\n</tbody>\n</table>\n<h3>Step 6: Use expression data with ENCODE epigenomic data</h3>\n<p>Compare gene expression with enhancer marks:</p>\n<pre><code>encode_search_experiments(assay_title=\"Histone ChIP-seq\", biosample_term_name=\"K562\", target=\"H3K27ac\", organism=\"Homo sapiens\")\n</code></pre>\n<p><strong>Interpretation</strong>: Genes with high TPM AND nearby H3K27ac peaks have validated enhancer-gene connections. Low expression despite nearby enhancer marks suggests poised or tissue-specific regulation.</p>\n<h3>Integration with downstream skills</h3>\n<ul>\n<li>Gene quantifications feed into -&gt; <strong>peak-annotation</strong> for expression-validated peak targets</li>\n<li>Expression data connects to -&gt; <strong>gtex-expression</strong> for tissue comparison</li>\n<li>Processed data feeds into -&gt; <strong>compare-biosamples</strong> for differential expression analysis</li>\n<li>Pipeline provenance logged by -&gt; <strong>data-provenance</strong></li>\n</ul>\n<h2>Code Examples</h2>\n<h3>1. Find RNA-seq experiments for a tissue</h3>\n<pre><code>encode_search_experiments(assay_title=\"total RNA-seq\", organ=\"liver\", organism=\"Homo sapiens\")\n</code></pre>\n<p>Expected output:</p>\n<pre><code>{\n  \"results\": [\n    {\"accession\": \"ENCSR300RNA\", \"assay_title\": \"total RNA-seq\", \"biosample_summary\": \"liver tissue male adult (54 years)\", \"status\": \"released\"}\n  ],\n  \"total\": 35,\n  \"limit\": 25,\n  \"offset\": 0,\n  \"has_more\": true,\n  \"next_offset\": 25\n}\n</code></pre>\n<h3>2. Check for existing gene quantifications</h3>\n<pre><code>encode_list_files(experiment_accession=\"ENCSR300RNA\", file_format=\"tsv\", output_type=\"gene quantifications\", assembly=\"GRCh38\")\n</code></pre>\n<p>Expected output (a JSON array of file records; fields abridged):</p>\n<pre><code>[\n  {\"accession\": \"ENCFF400GEQ\", \"file_format\": \"tsv\", \"output_type\": \"gene quantifications\", \"assembly\": \"GRCh38\", \"file_size_human\": \"5.2 MB\", \"status\": \"released\"}\n]\n</code></pre>\n<h3>3. Download expression data</h3>\n<pre><code>encode_download_files(file_accessions=[\"ENCFF400GEQ\"], download_dir=\"/data/rnaseq/quantification\")\n</code></pre>\n<p>Expected output (fields abridged):</p>\n<pre><code>{\n  \"downloaded\": [\n    {\"accession\": \"ENCFF400GEQ\", \"file_path\": \"/data/rnaseq/quantification/ENCFF400GEQ.tsv\", \"file_size_human\": \"5.2 MB\", \"success\": true, \"md5_verified\": true}\n  ],\n  \"errors\": [],\n  \"summary\": {\"total_requested\": 1, \"successful\": 1, \"failed\": 0, \"total_size_human\": \"5.2 MB\"}\n}\n</code></pre>\n<h2>Integration</h2>\n<table>\n<thead>\n<tr>\n<th>This skill produces...</th>\n<th>Feed into...</th>\n<th>Purpose</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Gene expression (TPM/FPKM)</td>\n<td><strong>peak-annotation</strong></td>\n<td>Validate enhancer targets with expression data</td>\n</tr>\n<tr>\n<td>Expression matrix</td>\n<td><strong>gtex-expression</strong></td>\n<td>Compare cell-line vs. tissue expression</td>\n</tr>\n<tr>\n<td>Differential expression results</td>\n<td><strong>compare-biosamples</strong></td>\n<td>Identify tissue-specific gene regulation</td>\n</tr>\n<tr>\n<td>Signal tracks (bigWig)</td>\n<td><strong>visualization-workflow</strong></td>\n<td>Display expression signal in genome browser</td>\n</tr>\n<tr>\n<td>Expression quantifications</td>\n<td><strong>disease-research</strong></td>\n<td>Connect gene expression to disease phenotypes</td>\n</tr>\n<tr>\n<td>Pipeline run parameters</td>\n<td><strong>data-provenance</strong></td>\n<td>Record STAR/RSEM versions and settings</td>\n</tr>\n<tr>\n<td>QC metrics</td>\n<td><strong>quality-assessment</strong></td>\n<td>Validate against ENCODE RNA-seq standards</td>\n</tr>\n</tbody>\n</table>\n<h2>Related Skills</h2>\n<ul>\n<li><strong>pipeline-guide</strong> (parent): General pipeline selection and resource assessment</li>\n<li><strong>quality-assessment</strong>: Deep-dive QC analysis beyond basic metrics</li>\n<li><strong>integrative-analysis</strong>: Combine RNA-seq with ChIP-seq/ATAC-seq for regulatory inference</li>\n<li><strong>compare-biosamples</strong>: Compare expression profiles across cell types</li>\n<li><strong>single-cell-encode</strong>: For scRNA-seq data processing (different pipeline)</li>\n<li><strong>pipeline-chipseq</strong>: Sibling pipeline for ChIP-seq data</li>\n<li><strong>pipeline-atacseq</strong>: Sibling pipeline for ATAC-seq data</li>\n<li><strong>publication-trust</strong>: Verify literature claims backing analytical decisions</li>\n</ul>\n<h2>Presenting Results</h2>\n<p>When reporting RNA-seq pipeline results:</p>\n<ul>\n<li><strong>Mapping rate</strong>: Report the STAR uniquely mapped rate (&gt;70% expected), multi-mapped rate (&lt;10%), and unmapped rate from <code>star/&lt;sample&gt;.Log.final.out</code></li>\n<li><strong>Quantification paths</strong>: Provide paths to <code>rsem/&lt;sample&gt;.genes.results</code> (TPM, FPKM, expected_count), <code>rsem/&lt;sample&gt;.isoforms.results</code>, and <code>kallisto/&lt;sample&gt;/abundance.tsv</code> when Kallisto ran</li>\n<li><strong>Strandedness</strong>: Confirm the orientation reported in <code>qc/rseqc/&lt;sample&gt;.infer_experiment.txt</code> matches the <code>--strandedness</code> value the run used; if it does not, the run has to be repeated</li>\n<li><strong>Key QC metrics</strong>: Present the exonic rate from <code>read_distribution.txt</code> and the gene body coverage uniformity from <code>geneBody_coverage.geneBodyCoverage.txt</code> in a summary table, alongside the MultiQC report at <code>qc/multiqc/multiqc_report.html</code></li>\n<li><strong>Derived metrics</strong>: Detected-gene counts (TPM&gt;1), rRNA rate, library duplication rate and saturation are not produced by this workflow. Compute them separately if they are needed, and say so when reporting</li>\n<li><strong>Signal tracks</strong>: Provide paths to <code>signal/&lt;sample&gt;_plus.bw</code> and <code>signal/&lt;sample&gt;_minus.bw</code> (or <code>signal/&lt;sample&gt;_unstranded.bw</code> for an unstranded run)</li>\n<li><strong>Next steps</strong>: Suggest <code>integrative-analysis</code> to combine RNA-seq with ChIP-seq/ATAC-seq for regulatory inference, or <code>compare-biosamples</code> for cross-tissue expression comparison</li>\n</ul>\n<h2>For the request: \"$ARGUMENTS\"</h2>\n","files":[{"path":"references/01-qc-trimming.md","sizeBytes":3445,"isText":true},{"path":"references/02-star-alignment.md","sizeBytes":6043,"isText":true},{"path":"references/03-quantification.md","sizeBytes":5780,"isText":true},{"path":"references/04-signal-tracks.md","sizeBytes":5331,"isText":true},{"path":"references/05-qc-metrics.md","sizeBytes":6884,"isText":true},{"path":"references/literature.md","sizeBytes":12050,"isText":true},{"path":"scripts/Dockerfile","sizeBytes":2723,"isText":false},{"path":"scripts/main.nf","sizeBytes":11914,"isText":false},{"path":"scripts/nextflow.config","sizeBytes":4815,"isText":false},{"path":"SKILL.md","sizeBytes":28885,"isText":true}],"reviewScore":null,"reviewSummary":null,"trust":{"provenance":"trusted-source-unreviewed","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow.","bodySource":null},"bodyLocked":false,"purchaseUrl":null,"sourceUrl":null,"report":{"provenance":"trusted-source-unreviewed","screen":{"ran":true,"outcome":"clean","suspicious":0,"notes":0,"hiddenCharacters":false},"virusScan":{"engine":"clamav","status":"clean","scannedAt":"2026-09-22T13:27:40.232241Z","sha256":"FC3ADDBA9F4547357728095975D0A73EF287BE1D54A53910EE5356C69DCC884E","sizeBytes":34308},"review":null,"source":{"repositoryUrl":"https://github.com/ammawla/encode-toolkit","path":"skills/pipeline-rnaseq","license":"AGPL-3.0","commit":"36836c8725fd4d20d9c851ce314f5151aea5f57c","subtreeSha":"2BC1490CADC85B49FA68EC9F66C80107D921258024D1FE349D4133911D011A0F","lastSyncedAt":"2026-09-29T20:56:54.045383Z"},"reviewedAt":"2026-09-22T13:31:09.958362Z","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow."},"install":[{"target":"skills-cli","command":"npx skills add https://github.com/ammawla/encode-toolkit/tree/main/skills/pipeline-rnaseq"},{"target":"claude-code","command":"claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install ammawla-encode-toolkit@llmmart"},{"target":"git","command":"git clone https://github.com/ammawla/encode-toolkit.git"}]}