Claude Cursor Skill

quality-assessment

Evaluate ENCODE experiment quality using standard metrics and audit flags. Use when the user asks about data quality, wants to filter for high-quality experiments, needs to interpret quality metrics (FRiP, NSC, RSC, NRF, IDR, TSS enrichment, fragment size), wants to understand EN

LLM Mart · 0 points · 0 0 0

Virus-scanned Reviewed automatically before listing.

Full trust report

Download ammawla-encode-toolkit-plugin_skills_quality-assessment-36836c8.zip · 19 KB

ammawla/encode-toolkit 20 5 forks AGPL-3.0 Updated 4d ago

Part of ammawla/encode-toolkit — 90 skills

Install

skills CLI npx skills add https://github.com/ammawla/encode-toolkit/tree/main/plugin/skills/quality-assessment

Claude Code

claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install ammawla-encode-toolkit@llmmart

Git git clone https://github.com/ammawla/encode-toolkit.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole ammawla/encode-toolkit collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Assess ENCODE Data Quality

When to Use

User asks about data quality, QC metrics, or whether an experiment is reliable
User wants to filter experiments by quality (FRiP, NSC, RSC, NRF, IDR, TSS enrichment)
User asks "is this experiment good enough?" or "should I use this data?"
User needs to interpret ENCODE audit flags (ERROR, NOT_COMPLIANT, WARNING)
User wants to compare quality across multiple experiments
User is selecting high-quality experiments for a meta-analysis or aggregation

Help the user evaluate whether ENCODE experiments meet quality standards for their analysis. Quality assessment is not a single-metric exercise — it requires integrating multiple orthogonal measures in the context of the specific assay, biological system, and analytical goals.

Literature Foundation

#	Reference	Key Contribution
1	Landt et al. 2012, Genome Res, DOI:10.1101/gr.136184.111 (~3,500 cit)	ENCODE/modENCODE ChIP-seq guidelines; defined NSC, RSC, NRF, FRiP thresholds
2	ENCODE Project Consortium 2020, Nature, DOI:10.1038/s41586-020-2493-4 (~1,656 cit)	ENCODE Phase 3; expanded quality standards to new assays, defined cCRE registry
3	Buenrostro et al. 2013, Nat Methods, DOI:10.1038/nmeth.2688 (~7,000 cit)	Introduced ATAC-seq; established fragment size and TSS enrichment as key QC
4	Ou et al. 2018, BMC Genomics, DOI:10.1186/s12864-018-4559-3	ATACseqQC R package; systematic quality metrics for ATAC-seq
5	Conesa et al. 2016, Genome Biol, DOI:10.1186/s13059-016-0881-8 (~2,363 cit)	RNA-seq best practices survey; defined mapping rate, rRNA, gene body coverage
6	Foox et al. 2021, Genome Biol, DOI:10.1186/s13059-021-02529-2	SEQC2 EpiQC consortium; multi-platform WGBS benchmarking
7	Yardimci et al. 2019, Genome Biol, DOI:10.1186/s13059-019-1658-7	Hi-C quality measures; cis/trans ratio, distance-dependent decay, resolution
8	Skene & Henikoff 2017, eLife, DOI:10.7554/eLife.21856 (~1,800 cit)	CUT&RUN method; established spike-in normalization and low-background QC
9	Kaya-Okur et al. 2019, Nat Commun, DOI:10.1038/s41467-019-09982-5 (~1,200 cit)	CUT&Tag method; tagmentation-based profiling with distinct QC profile
10	Li et al. 2011, Ann Appl Stat, DOI:10.1214/11-AOAS466 (~1,500 cit)	Irreproducible Discovery Rate (IDR); principled replicate concordance
11	Hitz et al. 2023, Nucleic Acids Res, DOI:10.1093/nar/gkad243	ENCODE uniform processing pipelines; standardized QC across all assays
12	Nordin et al. 2023, Genome Biol, DOI:10.1186/s13059-023-03027-3	CUT&RUN suspect list; identified artifact-prone regions specific to CUT&RUN/CUT&Tag
13	Amemiya et al. 2019, Sci Rep, DOI:10.1038/s41598-019-45839-z (~1,372 cit)	ENCODE Blacklist v2; artifact regions to exclude from all analyses

Step 1: Retrieve Experiment Details and Audit Status

Use encode_get_experiment with the accession to get full metadata including:

Audit counts — audit_error_count, audit_not_compliant_count, audit_warning_count, audit_internal_action_count (four ints, one per ENCODE severity level)
Replicate information — bio_replicate_count, tech_replicate_count, replication_type
The file list (files) with format, output type, assembly, size and md5sum per file

Two things this tool does not return: the ENCODE pipeline record (use encode_track_experiment, which returns pipelines[]) and per-file quality metrics such as FRiP or NSC (read those from the file's page on encodeproject.org).

encode_get_experiment(accession="ENCSR...")

For batch assessment across multiple experiments:

encode_search_experiments(assay_title="...", organ="...", limit=50)
# Then iterate through results checking audit flags

Step 2: Interpret ENCODE Audit Flags

ENCODE audits are generated by automated validators during the ENCODE uniform processing pipeline (Hitz et al. 2023). They flag experiments by severity:

Level	Meaning	Action
ERROR	Critical issues — data may be unreliable	Avoid using unless no alternative exists. Document thoroughly if used.
NOT_COMPLIANT	Does not meet current ENCODE standards	Usable with caveats. Check which specific standard is violated.
WARNING	Minor issues detected	Generally safe. Document the specific warning.
INTERNAL_ACTION	DCC processing notes	Usually not a concern for external users.

Common audit categories and what they mean:

Audit Category	What It Checks
`replicate concordance`	IDR or correlation between biological replicates
`library complexity`	NRF, PBC1, PBC2 — whether library is saturated
`read depth`	Whether minimum depth thresholds are met
`control quality`	Whether input/IgG control is adequate
`mapping quality`	Alignment rate and uniquely mapped fraction
`peak calling`	Whether peaks were called successfully, FRiP
`antibody validation`	Whether antibody meets ENCODE standards

Present every audit flag to the user and explain each one. A single ERROR audit does not automatically disqualify an experiment — context matters.

Step 3: Evaluate ChIP-seq Quality (Landt et al. 2012)

The ENCODE ChIP-seq guidelines (Landt et al. 2012) established the foundational metrics still used today. These were developed from analysis of hundreds of ChIP-seq experiments and reflect empirically-derived thresholds.

Core Metrics

Metric	Threshold	Concern	What It Measures	Why It Matters
FRiP	≥1% (TF), ≥5% (histone)	Below threshold	Fraction of reads in peaks	Signal enrichment. Very low FRiP means most reads are background. TF ChIP typically has lower FRiP than broad histone marks.
NSC	>1.05	≤1.05	Normalized strand cross-correlation	Signal-to-noise ratio. Computed from strand shift analysis. Values near 1.0 indicate no enrichment.
RSC	>0.8	≤0.8	Relative strand cross-correlation	Signal relative to phantom peak. More robust than NSC for shallow libraries.
NRF	≥0.8	<0.8	Non-redundant fraction (unique/total)	Library complexity. Low NRF = excessive PCR duplication = wasted sequencing.
PBC1	≥0.8	<0.5	PCR bottleneck coefficient 1	N1/Nd: fraction of locations with exactly 1 read. More sensitive than NRF at high depth.
PBC2	≥3	<1	PCR bottleneck coefficient 2	N1/N2: ratio of 1-read to 2-read locations. <1 indicates severe bottleneck.

Read Depth Requirements

Target Type	Minimum per Replicate	Recommended	Notes
Transcription factor	10M uniquely mapped	20M	Narrow peaks, need depth for detection
Broad histone mark (H3K27me3, H3K9me3, H3K36me3)	20M uniquely mapped	45M	Broad domains require more reads
Narrow histone mark (H3K4me3, H3K27ac)	20M uniquely mapped	20M	Sharp peaks, similar to TF
Input/IgG control	10M uniquely mapped	Match IP depth	Should match or exceed IP library depth

IDR Analysis (Li et al. 2011)

The Irreproducible Discovery Rate provides principled assessment of replicate concordance:

IDR Comparison	Expected	Concern	Interpretation
Nt (true replicates)	≥50% of Np	<50% Np	Low concordance between biological replicates
Np (pooled pseudoreplicates)	Reference set	—	Represents total discoverable peaks
Self-consistency (Ns)	≥50% of Np	<50% Np	Individual replicate quality
Rescue ratio (Np/max(Nt,Ns))	<2	>2	High ratio = one replicate much weaker

Key insight: IDR thresholded peaks represent peaks passing replicate concordance analysis. Pseudoreplicated peaks = single-replicate fallback (lower confidence). Optimal IDR peaks from pooled data = most complete peak set.

Antibody Validation

ENCODE requires characterization for every antibody:

Primary: IP followed by mass spectrometry or immunoprecipitation-western
Secondary: At least one of: knockdown/knockout, motif enrichment, genomic annotation enrichment
No encode_* tool returns antibody data: read antibody_lot_reviews on the experiment's page on encodeproject.org

Step 4: Evaluate ATAC-seq Quality (Buenrostro et al. 2013; Ou et al. 2018)

ATAC-seq has a distinct quality profile driven by the transposase insertion mechanism.

Core Metrics

Metric	Good	Concern	What It Measures
TSS enrichment	≥5 GRCh38 / ≥6 hg19 / ≥10 mm10 (ENCODE data standards)	<4	Signal enrichment at transcription start sites. The single most informative ATAC-seq QC metric.
FRiP	≥20%	<10%	Higher expected FRiP than ChIP-seq because accessible chromatin = true signal
Fragment size distribution	Clear nucleosomal ladder	Monotonic decay	Should show peaks at <150bp (NFR), ~200bp (mono-nuc), ~400bp (di-nuc), ~600bp (tri-nuc)
NFR ratio	>2× mono-nucleosomal	<1×	Ratio of sub-nucleosomal to mono-nucleosomal fragments
Mitochondrial reads	<20% (after filtering)	>50%	Mitochondrial DNA is highly accessible; excessive = poor nuclear enrichment
Duplicate rate	<30%	>50%	PCR duplication. Omni-ATAC protocol reduces this.
NRF	≥0.7	<0.5	Library complexity, same concept as ChIP-seq

Read Depth Requirements

Sample Type	Minimum	Recommended
Bulk ATAC-seq	25M uniquely mapped (post-dedup, post-mito filter)	50M
Single-cell ATAC-seq	25K unique fragments per cell	50K per cell

Fragment Size Interpretation (Buenrostro et al. 2013)

The fragment size distribution is the signature QC plot for ATAC-seq:

Sub-nucleosomal (<150bp): Nucleosome-free regions — these are the "open chromatin" signal
Mono-nucleosomal (~200bp): Single nucleosome wrapped
Di-nucleosomal (~400bp): Two nucleosomes
Tri-nucleosomal (~600bp): Three nucleosomes

A clean ATAC-seq library shows a clear nucleosomal ladder. Monotonic decay (no peaks) suggests either dead cells, over-transposition, or excessive DNA damage.

Step 5: Evaluate RNA-seq Quality (Conesa et al. 2016)

Core Metrics

Metric	Good	Concern	What It Measures
Mapping rate	70-90% uniquely mapped	<70%	Alignment success. Low rate = contamination, adapter issues, or wrong reference
rRNA contamination	<10%	>20%	Ribosomal RNA depletion efficiency. High = failed ribo-depletion
Gene body coverage	Uniform 5'→3'	Strong 3' bias	Even coverage across gene bodies. 3' bias = degraded RNA or poly-A capture bias
Duplication rate	<50%	>70%	PCR amplification artifacts
Replicate correlation	Spearman ≥0.9 (same condition)	<0.8	Concordance between replicates
Exonic reads	>60% of mapped	<40%	Reads mapping to annotated exons vs intergenic
Intergenic reads	<10%	>20%	Reads mapping between genes — may indicate genomic DNA contamination

Read Depth Requirements (Conesa et al. 2016)

Application	Minimum	Recommended	Notes
Gene-level quantification	10M mapped	30M mapped	Standard bulk RNA-seq
Transcript-level quantification	30M mapped	60M mapped	Isoform detection requires more depth
Differential expression	10M per sample, ≥3 bio reps	20M per sample	Statistical power depends more on replicates than depth
Rare transcript detection	50M+ mapped	100M mapped	Long-tail of expression distribution
total RNA-seq	50M+ mapped	100M mapped	Includes non-coding RNA, intergenic transcripts

Strand Specificity

ENCODE RNA-seq data may be stranded or unstranded:

Stranded: Can distinguish sense vs antisense transcription. Required for accurate quantification of overlapping genes.
Unstranded: Cannot resolve strand of origin. No encode_* tool reports this — check run_type and the library's strand specificity on the experiment's page on encodeproject.org.

Step 6: Evaluate WGBS Quality (ENCODE data standards)

Core Metrics

Metric	Good	Concern	What It Measures
Bisulfite conversion rate	≥98%	<98%	Efficiency of C→U conversion of unmethylated cytosines. Measured from spike-in controls (lambda phage DNA).
CpG coverage	>80% of CpGs at ≥1×	<50%	Fraction of CpG sites covered by at least one read
Mean CpG coverage	≥10× for DMR analysis	<5×	Average sequencing depth at CpG sites. 10× needed for reliable methylation calls.
Mapping rate	>60% unique	<40%	Lower than standard WGS due to reduced complexity after bisulfite conversion
Duplication rate	<30%	>50%	PCR duplicates
CpG methylation distribution	Bimodal (near 0% and near 100%)	Unimodal	Healthy cells show bimodal: most CpGs are either fully methylated or unmethylated
Lambda/pUC19 conversion	≥98% conversion rate	<98%	Spike-in controls for bisulfite conversion efficiency

Platform Considerations (Foox et al. 2021)

The SEQC2 EpiQC benchmark found significant platform effects:

Different sequencing platforms (Illumina HiSeq, NovaSeq, MGI) can produce systematically different methylation calls
Cross-platform comparisons require careful normalization
RRBS (Reduced Representation Bisulfite Sequencing) covers only ~10% of CpGs — do NOT combine RRBS with WGBS directly

Coverage Requirements

Application	Minimum CpG Coverage	Recommended
Methylation landscape	1×	5×
Differentially methylated regions	5× per sample	10× per sample
Allele-specific methylation	15×	30×
Single CpG resolution	10×	30×

Step 7: Evaluate Hi-C Quality (Yardimci et al. 2019)

Core Metrics

Metric	Good	Concern	What It Measures
Cis/trans ratio	>60% cis	<40% cis	Fraction of contacts within same chromosome. Low cis = random ligation = poor quality
Long-range cis (>20kb)	>40% of cis	<15%	True 3D interactions vs random proximity. Short-range contacts are noise-enriched
Unique valid pairs	>50% of total	<25%	Pairs surviving all filters (mapping, dedup, chimera removal)
Duplicate rate	<40%	>60%	PCR duplicates in Hi-C are especially problematic because they inflate contact frequencies
Contact distance decay	Smooth P(s)∝s^-1 curve	Irregular/plateau	Expected power-law decay with genomic distance

Read Depth and Resolution

Resolution Target	Minimum Valid Pairs	Recommended
Compartment-level (100kb)	50M	100M
TAD-level (40kb)	200M	500M
Loop-level (5-10kb)	500M	1B+
Sub-TAD (1kb)	2B+	5B+

Note: Hi-C resolution is not just about read depth — it also depends on restriction enzyme site density, ligation efficiency, and fragment size distribution. In situ Hi-C (Rao et al. 2014) generally produces cleaner data than dilution Hi-C.

Step 8: Evaluate CUT&RUN / CUT&Tag Quality (Skene & Henikoff 2017; Kaya-Okur et al. 2019)

These newer profiling methods have distinct quality profiles from ChIP-seq.

Key Differences from ChIP-seq

Feature	ChIP-seq	CUT&RUN / CUT&Tag
Background	High (requires input control)	Low (targeted cleavage)
Required depth	10-45M	3-8M sufficient
FRiP	>1-5%	>20% typical
Input control	Required	IgG control recommended but lower priority
Fragment size	Size-selected ~200-600bp	Variable; CUT&RUN releases <120bp fragments
Spike-in	Not standard	Recommended (E. coli carry-over or added spike-in)

CUT&RUN Quality Metrics

Metric	Good	Concern
FRiP	>20%	<5%
Fragment size	Peak at <120bp (released fragments)	Only large fragments
Read depth	3-8M unique mapped	<1M
Spike-in ratio	Consistent across conditions	>5× variation
Duplicate rate	<30%	>60%

CUT&RUN Suspect List (Nordin et al. 2023)

CUT&RUN and CUT&Tag generate artifacts at specific genomic regions (distinct from the ENCODE Blacklist). These are regions with apparent enrichment that is not target-specific:

Use the CUT&RUN suspect list (Nordin et al. 2023) IN ADDITION to the ENCODE Blacklist
Available at: https://github.com/Boyle-Lab/CUT-RUN_suspect_list
Particularly important for H3K4me3 and H3K27me3 CUT&RUN data

Step 9: Evaluate Single-Cell Quality (scRNA-seq and scATAC-seq)

ENCODE includes scRNA-seq and scATAC-seq experiments (primarily 10X Chromium platform). Single-cell data has distinct quality metrics from bulk assays, focused on per-cell quality rather than per-experiment signal-to-noise.

scRNA-seq Quality Metrics

Metric	Acceptable Range	Red Flag	Notes
Genes per cell (median)	1,500–4,000 (10X) / 4,000–8,000 (Smart-seq2)	<500	Tissue-dependent; immune cells typically lower than epithelial
UMIs per cell (median)	3,000–15,000 (10X)	<1,000	N/A for Smart-seq2 (no UMIs)
Mitochondrial % (median)	<10–15%	>25%	High mito% indicates cell stress or lysis; tissue-dependent thresholds
Doublet rate (estimated)	2–8% (10X, cell-count dependent) / <2% (plate-based)	>10%	Increases with cell loading density; use Scrublet or DoubletFinder
Mapping rate	>80%	<60%	Low mapping suggests contamination or mismapping
Sequencing saturation	>40%	<20%	Low saturation may miss rare transcripts
Cell count vs expected	Within 50–150% of expected	<30% or >200%	Very low = failed capture; very high = doublets or debris

scATAC-seq Quality Metrics

Metric	Acceptable Range	Red Flag	Notes
Unique fragments per cell	>3,000	<1,000	Sparse data below threshold makes peak calling unreliable
TSS enrichment per cell	>5	<2	Low TSS enrichment indicates failed Tn5 insertion bias
Fraction in peaks (FRiP)	>20%	<10%	Measures signal-to-noise at single-cell level
Fraction of mitochondrial reads	<5%	>10%	Dead/dying cells captured
Duplicate rate	<40%	>60%	High duplication indicates low library complexity

Single-Cell-Specific Quality Pitfalls

Ambient RNA contamination (scRNA-seq): Cell-free RNA from lysed cells during droplet capture inflates apparent expression of highly-expressed genes across all cells. Use CellBender (best-in-class), SoupX, or DecontX to estimate and remove ambient contamination BEFORE downstream analysis.
Barcode multiplets (scATAC-seq): Multiple cells per droplet inflate fragment counts and blur cell-type signals. ArchR and SnapATAC2 include doublet detection modules.
Cell-type composition bias: Quality metrics vary by cell type. A "low-quality" cell may be a small immune cell, not a damaged cell. Apply adaptive QC thresholds per cluster (e.g., miQC) rather than global cutoffs.
Batch effects across donors: For ENCODE tissue scRNA-seq from multiple donors, batch correction (Harmony, scVI) is typically needed before integration. The single-cell-encode skill covers integration workflows.

Where to Find Single-Cell Quality in ENCODE

Single-cell experiments in ENCODE include cell-level quality summaries in their metadata. Check:

encode_get_experiment(accession="ENCSR...")

Look at the four audit_*_count fields — ENCODE applies automated QC checks including minimum cell counts, minimum genes per cell, and maximum doublet rates. Also check bio_replicate_count and replication_type; library preparation details are on the experiment's ENCODE page, not in the tool output.

Step 10: Assess Replication

ENCODE requires minimum 2 independent biological replicates for released data.

Replicate Types and Their Meaning

Replicate Type	Definition	Use Case
Biological	Independent biological samples	Gold standard — captures biological variation
Technical	Same sample, different library prep	Assesses technical reproducibility
Isogenic	Same genotype, different growth/collection	Common for cell lines (e.g., K562, GM12878)
Anisogenic	Different genotypes/donors	Common for tissue samples

Concordance Assessment

Use encode_list_files to check for replicated peak files:

File Output Type	What It Means	Confidence
IDR thresholded peaks	Passed replicate concordance analysis (Li et al. 2011)	Highest
Optimal IDR peaks	Peaks from pooled data, thresholded by IDR	Complete set
Conservative IDR peaks	Stricter IDR threshold	Most conservative
Pseudoreplicated peaks	IDR on pseudoreplicates from pooled data	Single-replicate fallback
Replicated peaks	Found in multiple replicates (non-IDR method)	Moderate

Cross-Replicate Correlation

For quantitative data (RNA-seq, signal tracks):

Spearman correlation ≥0.9: Excellent concordance
0.8-0.9: Acceptable, check for outliers
<0.8: Investigate — batch effect, sample mix-up, or biological variation

Step 11: Apply the ENCODE Blacklist (Amemiya et al. 2019)

Before interpreting any peak-based quality metric, confirm that the ENCODE Blacklist has been applied:

Blacklist v2 regions: High-signal artifacts (satellite repeats, centromeric regions, high-copy sequences)
Available at: https://github.com/Boyle-Lab/Blacklist/
hg38: hg38-blacklist.v2.bed.gz (910 regions)
mm10: mm10-blacklist.v2.bed.gz

Failure to remove blacklisted regions will inflate FRiP, create false peaks, and confound enrichment analyses. If analyzing CUT&RUN/CUT&Tag, also apply the CUT&RUN suspect list (Nordin et al. 2023).

Step 12: File Quality Tiers and Selection

When listing files with encode_list_files, use quality-informed selection:

# Get preferred default files (ENCODE's recommendation)
encode_list_files(experiment_accession="ENCSR...", preferred_default=True)

# Get IDR thresholded peaks (gold standard for ChIP-seq)
encode_list_files(experiment_accession="ENCSR...", output_type="IDR thresholded peaks", assembly="GRCh38")

# Get signal tracks for visualization
encode_list_files(experiment_accession="ENCSR...", output_type="fold change over control", assembly="GRCh38")

File Quality Hierarchy

Priority	File Type	When to Use
1	`preferred_default=True`	ENCODE's recommended files — start here
2	IDR thresholded peaks	Gold standard for ChIP-seq peak calls
3	Fold change over control	Normalized signal for visualization and quantitative comparison
4	Signal of unique reads	Clean signal tracks (unnormalized)
5	Pseudoreplicated peaks	Fallback when IDR fails or only 1 replicate available
6	Unfiltered alignments	Only for custom re-analysis

Step 13: Summarize Quality Verdict

Provide a structured quality assessment:

Quality Tiers

Tier	Criteria	Recommendation
High quality	No ERROR/NOT_COMPLIANT audits, all metrics above thresholds, ≥2 biological replicates, IDR peaks available	Use confidently. Ideal for primary analysis.
Usable with caveats	WARNING-level audits or borderline metrics (within 20% of threshold), good replication	Usable. Document specific limitations in methods.
Use with caution	NOT_COMPLIANT flags, one metric below threshold, or single-replicate	Use only if no better alternative. Document all issues. Flag in results.
Not recommended	ERROR flags, multiple metrics below threshold, poor replication, no IDR peaks	Avoid. Seek alternative experiments or datasets.

Quality Summary Template

For each experiment assessed, provide:

Accession: ENCSR...
Assay: type and target
Audit status: list all flags with explanations
Key metrics: table with values and pass/fail
Replication: number and type of replicates, concordance
Pipeline: which ENCODE uniform pipeline version was used
Verdict: tier assignment with justification
Caveats: any specific limitations to note

Pitfalls and Common Mistakes

Single-metric decisions: No single metric captures quality. FRiP alone can be misleading — some TF ChIP-seq with biological signal has low FRiP due to focal binding patterns. Always evaluate collectively.
Comparing across assays: Do NOT compare ChIP-seq metrics to ATAC-seq metrics to CUT&RUN metrics. Each assay has its own quality profile and thresholds.
Ignoring batch effects: Experiments from different labs, dates, or platforms may have systematic quality differences. When combining data, check for batch-correlated quality variation.
Assembly mismatch: Quality metrics computed on different assemblies (hg19 vs GRCh38) may differ slightly. Always verify the assembly of quality metrics matches your analysis assembly.
Antibody lot variation: The same antibody target can show different enrichment across lots. Check antibody_lot_reviews on the experiment's page on encodeproject.org (it is not part of any tool response).
Read depth ≠ quality: A deeply sequenced bad library is still a bad library. Check NRF/PBC first — if complexity is exhausted, more sequencing wastes resources.
Control quality matters: An IP library is only as good as its control. Poor input/IgG control undermines all downstream peak-based metrics.
Newer assays, different rules: CUT&RUN and CUT&Tag have inherently different quality profiles from ChIP-seq. Applying ChIP-seq thresholds to CUT&RUN will incorrectly flag high-quality data.

Walkthrough: Quality Assessment of ENCODE ChIP-seq Before Analysis

Goal: Evaluate the quality of ENCODE ChIP-seq experiments against ENCODE consortium standards before including them in downstream analysis. Context: Not all ENCODE experiments meet the highest quality standards. Quality assessment prevents garbage-in-garbage-out in aggregation and integration analyses.

Step 1: Get experiment details and audit status

encode_get_experiment(accession="ENCSR000AKA")

Expected output (fields abridged; the full response also carries files and the rest of the experiment metadata):

{
  "accession": "ENCSR000AKA",
  "assay_title": "Histone ChIP-seq",
  "target": "H3K27ac",
  "biosample_summary": "GM12878",
  "bio_replicate_count": 2,
  "tech_replicate_count": 2,
  "replication_type": "isogenic",
  "status": "released",
  "audit_error_count": 0,
  "audit_not_compliant_count": 0,
  "audit_warning_count": 1,
  "audit_internal_action_count": 0
}

Interpretation: audit_error_count 0 and audit_not_compliant_count 0 = experiment meets ENCODE standards. One warning is acceptable. The counts say how many flags there are, not which ones — open the experiment on encodeproject.org for the individual audit messages.

Step 2: Check file-level quality

encode_list_files(experiment_accession="ENCSR000AKA", file_format="bed", output_type="IDR thresholded peaks", assembly="GRCh38")

Step 3: Review quality metrics

Key ChIP-seq quality thresholds (Landt et al. 2012):

Metric	Threshold	Meaning
FRiP	>= 1%	Signal enrichment over background
NSC	> 1.05	Strand cross-correlation signal
RSC	> 0.8	Relative strand correlation
NRF	>= 0.8	Library complexity
IDR	< 0.05	Reproducibility between replicates

Step 4: Track quality-verified experiments

encode_track_experiment(accession="ENCSR000AKA", notes="QC PASSED: FRiP=3.2%, NSC=1.12, RSC=0.95, 0 audit errors")

Integration with downstream skills

Quality-filtered experiments feed into histone-aggregation and other aggregation skills
QC metrics inform pipeline-chipseq parameter tuning
Audit status guides search-encode experiment selection
QC documentation supports data-provenance and scientific-writing

Code Examples

1. Check experiment audit status

encode_get_experiment(accession="ENCSR000AKA")

Expected output (audit fields only; the response carries the full experiment record):

{
  "accession": "ENCSR000AKA",
  "audit_error_count": 0,
  "audit_not_compliant_count": 0,
  "audit_warning_count": 1,
  "audit_internal_action_count": 0
}

2. List the analysis files for an experiment

encode_list_files(experiment_accession="ENCSR000AKA", file_format="bed", assembly="GRCh38")

Expected output (a JSON array of files; fields abridged):

[
  {"accession": "ENCFF001ABC", "output_type": "IDR thresholded peaks", "file_format": "bed", "file_type": "bed narrowPeak", "assembly": "GRCh38", "file_size": 1153434, "file_size_human": "1.1 MB", "preferred_default": true}
]

3. Get detailed file info

encode_get_file_info(accession="ENCFF001ABC")

Expected output (no QC metrics here — FileSummary has no FRiP/NSC/RSC fields; get those from the file's page on encodeproject.org):

{
  "accession": "ENCFF001ABC",
  "file_format": "bed",
  "file_type": "bed narrowPeak",
  "output_type": "IDR thresholded peaks",
  "output_category": "annotation",
  "assembly": "GRCh38",
  "file_size": 1153434,
  "file_size_human": "1.1 MB",
  "biological_replicates": [1, 2],
  "status": "released",
  "md5sum": "5d41402abc4b2a76b9719d911017c592",
  "experiment_accession": "ENCSR000AKA",
  "preferred_default": true
}

Integration

This skill produces...	Feed into...	Purpose
Quality-verified experiments	histone-aggregation	Only aggregate high-quality data
QC pass/fail decisions	search-encode	Filter search results by quality
Quality metric reports	data-provenance	Document QC criteria used
Audit interpretation	pipeline-guide	Guide reprocessing decisions
QC documentation	scientific-writing	Methods section QC reporting
Quality thresholds	publication-trust	Verify QC threshold citations
Validated experiment lists	batch-analysis	Process only quality-approved experiments
QC-filtered peaks	regulatory-elements	High-confidence regulatory element maps

Related Skills

publication-trust: Assess scientific integrity of publications before relying on their methods or findings — complements experiment quality with publication quality
histone-aggregation: Uses quality filtering before aggregating peaks
accessibility-aggregation: ATAC-seq quality directly impacts aggregation
data-provenance: Log quality decisions and thresholds used
compare-biosamples: Quality must be comparable across samples being compared
integrative-analysis: All data sources need quality assessment before integration
single-cell-encode: Quality metrics for scRNA-seq and scATAC-seq (genes/cell, fragments/cell, TSS enrichment)
epigenome-profiling: Quality assessment is a prerequisite for epigenomic profile assembly
variant-annotation: Quality of ENCODE experiments determines reliability of variant annotation
pipeline-guide: Pipeline version affects quality metric computation
batch-analysis: Batch QC screening across multiple experiments for systematic quality filtering

Presenting Results

Present QC metrics as a traffic-light table: metric | value | threshold | status (PASS/WARN/FAIL). Always include the ENCODE audit level. Suggest: "Would you like to filter to only experiments meeting all QC thresholds?"

For the request: "$ARGUMENTS"

Files (encode-toolkit)

references

literature.md 18.7 KB

# Quality Assessment — Literature References

**Last updated:** 2026-03-07
**Purpose:** Reference catalog for the quality-assessment skill — key papers defining assay-specific
quality control metrics, sequencing depth requirements, blacklist filtering, and multi-assay QC
frameworks for ChIP-seq, ATAC-seq, RNA-seq, WGBS, Hi-C, and CUT&RUN/CUT&Tag.

The quality-assessment skill evaluates ENCODE experiments against established quality thresholds,
interpreting audit flags (ERROR, NOT_COMPLIANT, WARNING, INTERNAL_ACTION) and providing
assay-specific QC guidance. Quality is never determined by a single metric — it requires collective
interpretation of library complexity, signal-to-noise ratios, replicate concordance, and
assay-specific benchmarks.

These 12 papers are organized by assay type, covering the quality standards for each major
functional genomics method represented in ENCODE. The final section covers universal tools and
filtering requirements that apply across all assay types.

---

## ChIP-seq Quality Standards

ChIP-seq is the most established ENCODE assay with the most mature quality framework. The three
papers below define the core metrics (FRiP, NSC, RSC, NRF), sequencing depth requirements, and
the interaction between artifact removal and quality metric calculation. Understanding these metrics
is essential because the ENCODE audit system derives its ChIP-seq quality flags directly from them.

---

### Landt et al. 2012 — ENCODE/modENCODE ChIP-seq quality guidelines

- **Citation:** Landt SG, Marinov GK, Kundaje A, Kheradpour P, Pauli F, Batzoglou S,
  Bernstein BE, Bickel P, Brown JB, Cayting P, Chen Y, DeSalvo G, Epstein C,
  Fisher-Aylor KI, Euskirchen G, Gerstein M, Gertz J, Hartemink AJ, Hoffman MM,
  Iyer VR, Jung YL, Karmakar S, Kellis M, Kharchenko PV, Li Q, Liu T, Liu XS,
  Ma L, Milosavljevic A, Myers RM, Park PJ, Pazin MJ, Perry MD, Raha D,
  Reddy TE, Rozowsky J, Shoresh N, Sidow A, Slattery M, Stamatoyannopoulos JA,
  Tolstorukov MY, White KP, Xi S, Farnham PJ, Lieb JD, Wold BJ, Snyder M.
  ChIP-seq guidelines and practices of the ENCODE and modENCODE consortia.
  *Genome Research*, 22(9), 1813-1831, 2012.
- **DOI:** [10.1101/gr.136184.111](https://doi.org/10.1101/gr.136184.111)
- **PMID:** 22955991 | **PMC:** PMC3431496
- **Citations:** ~3,400
- **Key findings:** The definitive quality framework for ChIP-seq, establishing four core metrics:
  FRiP (fraction of reads in peaks, >= 1% for broad marks, typically 5-20% for TFs),
  NSC (normalized strand coefficient, >1.05),
  RSC (relative strand coefficient, >0.8), and
  NRF (non-redundant fraction, >= 0.8).
  Mandated biological replicates with IDR (Irreproducible Discovery Rate) analysis for peak
  reproducibility, requiring IDR < 0.05 for optimal peaks. The paper distinguished quality
  expectations between sharp-peak factors (transcription factors, H3K4me3) and broad-domain marks
  (H3K27me3, H3K36me3), noting that broad marks have lower FRiP and strand correlation scores by
  nature. The quality-assessment skill uses these thresholds directly when evaluating ChIP-seq
  experiments, applying different expectations for sharp vs. broad targets.

---

### Marinov et al. 2014 — Sequencing depth requirements for ChIP-seq saturation

- **Citation:** Marinov GK, Kundaje A, Park PJ, Wold BJ. Large-scale quality analysis
  of published ChIP-seq data. *G3: Genes, Genomes, Genetics*, 4(2), 209-223, 2014.
- **DOI:** [10.1534/g3.113.008680](https://doi.org/10.1534/g3.113.008680)
- **PMID:** 24347632 | **PMC:** PMC3931556
- **Citations:** ~800
- **Key findings:** Systematically evaluated quality metrics across >400 published ChIP-seq
  datasets, establishing practical depth requirements: 20-40 million unique reads for sharp TF
  peaks and 40-60 million for broad histone marks. Demonstrated that sequencing depth below these
  thresholds leads to incomplete peak detection and poor IDR concordance between replicates.
  Found that ~30% of published ChIP-seq datasets had quality issues detectable by strand
  correlation metrics alone, indicating widespread quality problems in the published literature.
  The quality-assessment skill uses these depth thresholds to flag under-sequenced experiments
  and recommend whether additional sequencing is needed before analysis.

---

### Carroll et al. 2014 — Impact of artifact removal on ChIP quality metrics

- **Citation:** Carroll TS, Liang Z, Salama R, Stark R, de Santiago I. Impact of artifact
  removal on ChIP quality metrics in ChIP-seq and ChIP-exo data. *Frontiers in Genetics*,
  5, 75, 2014.
- **DOI:** [10.3389/fgene.2014.00075](https://doi.org/10.3389/fgene.2014.00075)
- **PMID:** 24782889 | **PMC:** PMC3988386
- **Citations:** ~600
- **Key findings:** Investigated how artifact removal steps (duplicate filtering, blacklist
  filtering, mapability masking) affect downstream ChIP-seq quality metrics. Demonstrated that
  applying the ENCODE blacklist prior to quality metric calculation substantially improves NSC
  and RSC scores, revealing that many apparently "low-quality" datasets become acceptable after
  proper filtering. Established the correct ordering of QC steps: adapter trimming, alignment,
  duplicate marking, blacklist filtering, then quality metric calculation. The quality-assessment
  skill follows this pipeline order and warns users when quality metrics were calculated before
  blacklist filtering, which can produce misleadingly low scores for otherwise acceptable data.

---

## ATAC-seq Quality Standards

ATAC-seq has distinct quality signatures from ChIP-seq, including the nucleosomal fragment size
ladder and TSS enrichment score. The two papers below define the method's intrinsic quality
indicators and provide systematic benchmarks for quantitative evaluation.

---

### Buenrostro et al. 2013 — ATAC-seq method and intrinsic quality signatures

- **Citation:** Buenrostro JD, Giresi PG, Zaba LC, Chang HY, Greenleaf WJ. Transposition of
  native chromatin for fast and sensitive epigenomic profiling of open chromatin, DNA-binding
  proteins, and nucleosome position. *Nature Methods*, 10(12), 1213-1218, 2013.
- **DOI:** [10.1038/nmeth.2688](https://doi.org/10.1038/nmeth.2688)
- **PMID:** 24097267 | **PMC:** PMC3959825
- **Citations:** ~5,000
- **Key findings:** Introduced ATAC-seq (Assay for Transposase-Accessible Chromatin using
  sequencing) and defined its characteristic quality signatures: the nucleosomal ladder pattern
  in fragment size distribution (sub-nucleosomal <147bp, mono-nucleosomal 147-294bp,
  di-nucleosomal 294-441bp), enrichment of signal at transcription start sites (TSS enrichment),
  and high signal-to-noise ratio at known open chromatin regions. ATAC-seq requires only
  500-50,000 cells (vs. millions for DNase-seq), making it the preferred accessibility assay
  for limited-material samples. The quality-assessment skill checks for the nucleosomal ladder
  and TSS enrichment score (>= 5 GRCh38 / >= 6 hg19 / >= 10 mm10 per ENCODE data standards) as primary quality indicators.

---

### Yan et al. 2020 — Systematic ATAC-seq quality metrics and benchmarks

- **Citation:** Yan F, Powell DR, Curtis DJ, Wong NC. From reads to insight: a hitchhiker's
  guide to ATAC-seq data analysis. *Genome Biology*, 21(1), 22, 2020.
- **DOI:** [10.1186/s13059-020-1929-3](https://doi.org/10.1186/s13059-020-1929-3)
- **PMID:** 32014034 | **PMC:** PMC6996192
- **Citations:** ~400
- **Key findings:** Provided a comprehensive benchmarking framework for ATAC-seq QC, establishing
  quantitative thresholds:
  - TSS enrichment score >= 5 GRCh38 / >= 6 hg19 / >= 10 mm10 (ENCODE data standards)
  - Mitochondrial read fraction < 20% (ideally < 5%)
  - Duplicate rate < 30%
  - Unique alignment rate > 80%
  - FRiP >= 20% for accessibility peaks
  - NFR/mono-nucleosomal ratio > 2 indicates successful Tn5 insertion

  The quality-assessment skill uses these thresholds in its ATAC-seq quality evaluation,
  reporting each metric alongside the expected range and flagging outliers. High mitochondrial
  read fractions indicate nuclear isolation problems, while absent nucleosomal periodicity
  suggests over-transposition or insufficient crosslinking.

---

## RNA-seq Quality Standards

RNA-seq quality assessment differs fundamentally from ChIP-seq because the signal is gene-level
expression rather than genomic binding. Quality metrics focus on mapping rates, library composition,
gene body coverage uniformity, and replicate concordance rather than peak-based measures.

---

### Conesa et al. 2016 — RNA-seq best practices and quality assessment framework

- **Citation:** Conesa A, Madrigal P, Tarazona S, Gomez-Cabrero D, Cervera A, McPherson A,
  Szczesniak MW, Gaffney DJ, Elo LL, Zhang X, Mortazavi A. A survey of best practices for
  RNA-seq data analysis. *Genome Biology*, 17, 13, 2016.
- **DOI:** [10.1186/s13059-016-0881-8](https://doi.org/10.1186/s13059-016-0881-8)
- **PMID:** 26813401 | **PMC:** PMC4728800
- **Citations:** ~4,500
- **Key findings:** Established the consensus RNA-seq quality framework:
  - Mapping rate 70-90% to reference genome
  - Ribosomal RNA fraction < 10% (< 5% ideal for poly-A libraries)
  - Gene body coverage uniformity (5'-to-3' bias ratio < 2)
  - Replicate Spearman correlation >= 0.9
  - Library complexity > 80% unique reads
  - >= 30 million reads for differential expression, >= 100 million for transcript assembly

  Defined the distinction between quality metrics for poly-A RNA-seq (gene-level quantification)
  vs. total RNA-seq (transcript-level, including non-coding RNAs and pre-mRNAs). The
  quality-assessment skill evaluates these metrics when assessing RNA-seq experiments and
  distinguishes between poly-A and total RNA protocols, applying different expectations for
  rRNA fraction (poly-A should have < 5%, total RNA may have up to 10% after rRNA depletion).

---

## Bisulfite Sequencing Quality Standards

Whole-genome bisulfite sequencing (WGBS) has unique quality considerations arising from the chemical
conversion of unmethylated cytosines to uracils. Conversion efficiency is the single most critical
quality metric — incomplete conversion produces systematic false positives that cannot be
computationally distinguished from true methylation.

---

### Foox et al. 2021 — WGBS benchmarking and bisulfite conversion quality

- **Citation:** Foox J, Nordlund J, Lalancette C, Gong T, Samber M, Gaggiotti K, Bhatt V,
  Buber DCE, Mundt F, Butler DJ, Mozsary C, Fehlmann T, Holley J, Prill RJ, Keller A,
  Mason CE. The SEQC2 epigenomics quality control (EpiQC) study. *Genome Biology*,
  22, 307, 2021.
- **DOI:** [10.1186/s13059-021-02529-2](https://doi.org/10.1186/s13059-021-02529-2)
- **PMID:** 34872606 | **PMC:** PMC8650396
- **Citations:** ~300
- **Key findings:** Benchmarked WGBS performance across 18 laboratories, establishing minimum
  quality thresholds:
  - Bisulfite conversion rate ≥ 98% (measured by lambda spike-in or non-CpG methylation)
  - CpG coverage >= 10x for reliable methylation calling (>= 30x for DMR detection)
  - Mapping rate > 60% (lower than standard due to reduced sequence complexity)
  - Concordance > 0.95 between technical replicates at covered CpGs

  Demonstrated that conversion rates below 98% introduce systematic false positives that
  cannot be computationally corrected. The quality-assessment skill flags WGBS experiments
  with conversion rates below this threshold and recommends minimum coverage depths based
  on intended analysis type (global profiling vs. DMR detection vs. allele-specific methylation).

---

### Schwartzman & Tanay 2015 — Statistical frameworks for bisulfite sequencing QC

- **Citation:** Schwartzman O, Tanay A. Single-cell epigenomics: techniques and emerging
  applications. *Nature Reviews Genetics*, 16, 716-726, 2015.
- **DOI:** [10.1038/nrg3980](https://doi.org/10.1038/nrg3980)
- **PMID:** 26460349
- **Citations:** ~500
- **Key findings:** Reviewed the analytical challenges of bisulfite sequencing data including
  incomplete conversion artifacts, alignment biases in repetitive regions, and the statistical
  frameworks needed for methylation calling at different coverage depths. Established that
  per-CpG methylation estimates require binomial modeling with a minimum of 5x coverage for
  reliable calls, and that smoothing approaches (BSmooth, methylKit) can improve estimates
  in low-coverage regions by borrowing information from neighboring CpGs. The quality-assessment
  skill uses these statistical principles when evaluating whether WGBS coverage is sufficient,
  distinguishing between global methylation profiling (5-10x sufficient), DMR calling
  (30x+ recommended), and single-CpG resolution analysis (50x+ recommended).

---

## Hi-C and 3D Genome Quality Standards

Hi-C quality metrics are distinct from linear assays because the signal of interest is the
frequency of chromatin contacts between pairs of genomic loci. Quality depends on crosslinking
efficiency, ligation specificity, and sequencing depth relative to the resolution desired.

---

### Yardimci et al. 2019 — Hi-C data quality metrics and normalization assessment

- **Citation:** Yardimci GG, Ozadam H, Saber ME, Aslankurt A, Yang X, Barutcu AR, Singer O,
  Boone D, Skalska L, Osman F, Razavi M, Libbrecht MW, Lajoie BR, Sanyal A, Dekker J,
  Noble WS, Mirny LA. Measuring the reproducibility and quality of Hi-C data.
  *Genome Biology*, 20(1), 57, 2019.
- **DOI:** [10.1186/s13059-019-1658-7](https://doi.org/10.1186/s13059-019-1658-7)
- **PMID:** 30890172 | **PMC:** PMC6423771
- **Citations:** ~200
- **Key findings:** Established quantitative quality metrics for Hi-C data:
  - Cis/trans ratio > 60% (higher indicates better ligation specificity)
  - Long-range cis interactions > 40% of total cis (>10kb distance)
  - Valid pair fraction > 40% of total read pairs
  - Duplicate rate assessment for library complexity

  Demonstrated that Hi-C quality is highly sensitive to crosslinking conditions and ligation
  efficiency, with poor-quality libraries showing elevated trans interactions and enrichment
  for short-range cis contacts (an indicator of random ligation rather than true chromatin
  contacts). Resolution achievable depends on sequencing depth: 500M valid pairs for 5kb
  resolution, 1B for 1kb resolution. The quality-assessment skill uses the cis/trans ratio
  as the single most informative initial quality indicator for Hi-C experiments.

---

## CUT&RUN/CUT&Tag Quality Standards

CUT&RUN and CUT&Tag use antibody-tethered nucleases (pA-MNase or pA-Tn5) instead of
immunoprecipitation, producing fundamentally different signal characteristics from ChIP-seq.
Standard ChIP-seq quality metrics (NSC, RSC) have different expected distributions, and a
distinct set of artifact regions must be filtered.

---

### Nordin et al. 2023 — CUT&RUN artifact regions (suspect list) for quality filtering

- **Citation:** Nordin A, Zambanini G, Pagella P, Cantu C. The CUT&RUN suspect list of
  problematic regions of the genome. *Genome Biology*, 24(1), 185, 2023.
- **DOI:** [10.1186/s13059-023-03027-3](https://doi.org/10.1186/s13059-023-03027-3)
- **PMID:** 37563719 | **PMC:** PMC10416431
- **Citations:** ~150
- **Key findings:** Identified a set of genomic regions (~1.5% of the genome) that produce
  systematic artifact signals specifically in CUT&RUN and CUT&Tag experiments, distinct from
  the standard ENCODE ChIP-seq blacklist. These "suspect regions" arise from MNase cleavage
  biases at highly accessible chromatin and pA-MNase diffusion artifacts, producing
  false-positive peaks that pass conventional quality filters designed for ChIP-seq.
  The quality-assessment skill applies the CUT&RUN suspect list in addition to the standard
  ENCODE blacklist when evaluating these assay types, and warns users that ChIP-seq quality
  metrics (NSC, RSC) have different expected distributions — CUT&RUN typically produces
  higher signal-to-noise but at fewer total peaks than ChIP-seq for the same target.

---

## Blacklist and Universal Quality Filtering

Regardless of assay type, all functional genomics data must be filtered against the ENCODE
blacklist — a set of genomic regions that produce artifact signal due to anomalous sequence
composition and assembly errors. The blacklist is the single most impactful quality filter.

---

### Amemiya et al. 2019 — ENCODE Blacklist v2: universal artifact regions

- **Citation:** Amemiya HM, Kundaje A, Boyle AP. The ENCODE Blacklist: identification of
  problematic regions of the genome. *Scientific Reports*, 9(1), 9354, 2019.
- **DOI:** [10.1038/s41598-019-45839-z](https://doi.org/10.1038/s41598-019-45839-z)
- **PMID:** 31249361 | **PMC:** PMC6597582
- **Citations:** ~1,400
- **Key findings:** Defined the ENCODE Blacklist v2 — genomic regions producing artifact signal
  across all functional genomics assays due to anomalous mappability from collapsed repeats,
  satellite sequences, and assembly gaps. The blacklist covers 910 regions in GRCh38 (0.5%
  of the genome, ~15 Mb) but can contain >50% of the signal in input/control samples.
  Blacklist filtering should be applied before all quality metric calculations and peak
  calling — failing to do so inflates peak counts and distorts quality scores. The
  quality-assessment skill checks whether experiments have been filtered against the blacklist
  (ENCFF356LFX for GRCh38, ENCFF547MET for mm10) and flags those that have not, as
  unfiltered data will produce unreliable quality assessments.

---

## Multi-Assay QC Framework

When evaluating ENCODE experiments across multiple assay types, a unified QC reporting framework
enables systematic comparison and identification of batch effects or quality outliers. MultiQC
provides this framework by aggregating metrics from diverse bioinformatics tools into standardized
reports.

---

### Ewels et al. 2016 — MultiQC for unified quality reporting across assay types

- **Citation:** Ewels P, Magnusson M, Lundin S, Kaller M. MultiQC: summarize analysis
  results for multiple tools and samples. *Bioinformatics*, 32(19), 3047-3048, 2016.
- **DOI:** [10.1093/bioinformatics/btw354](https://doi.org/10.1093/bioinformatics/btw354)
- **PMID:** 27312411 | **PMC:** PMC5039924
- **Citations:** ~2,500
- **Key findings:** Introduced MultiQC, the standard tool for aggregating quality metrics from
  diverse bioinformatics tools (FastQC, Picard, STAR, featureCounts, sambamba, samtools, etc.)
  into unified HTML reports with interactive visualizations. MultiQC parses output from >120
  tools and presents cross-sample quality comparisons including alignment statistics, duplication
  rates, fragment size distributions, insert sizes, GC content distributions, and assay-specific
  metrics. For ENCODE data, MultiQC aggregates metrics across all experiments in a project,
  enabling batch-level quality assessment that identifies outlier experiments.
  The quality-assessment skill recommends MultiQC as the primary reporting tool for
  multi-experiment quality evaluation and uses its metric definitions as reference standards.

---

SKILL.md 32.1 KB

---
name: quality-assessment
description: Evaluate ENCODE experiment quality using standard metrics and audit flags. Use when the user asks about data quality, wants to filter for high-quality experiments, needs to interpret quality metrics (FRiP, NSC, RSC, NRF, IDR, TSS enrichment, fragment size), wants to understand ENCODE audit warnings, needs to compare quality across experiments, or is deciding whether data is usable for their analysis. Also use when the user mentions QC, quality control, or data filtering.
---

# Assess ENCODE Data Quality

## When to Use

- User asks about data quality, QC metrics, or whether an experiment is reliable
- User wants to filter experiments by quality (FRiP, NSC, RSC, NRF, IDR, TSS enrichment)
- User asks "is this experiment good enough?" or "should I use this data?"
- User needs to interpret ENCODE audit flags (ERROR, NOT_COMPLIANT, WARNING)
- User wants to compare quality across multiple experiments
- User is selecting high-quality experiments for a meta-analysis or aggregation

Help the user evaluate whether ENCODE experiments meet quality standards for their analysis. Quality assessment is not a single-metric exercise — it requires integrating multiple orthogonal measures in the context of the specific assay, biological system, and analytical goals.

## Literature Foundation

| # | Reference | Key Contribution |
|---|-----------|-----------------|
| 1 | Landt et al. 2012, Genome Res, DOI:10.1101/gr.136184.111 (~3,500 cit) | ENCODE/modENCODE ChIP-seq guidelines; defined NSC, RSC, NRF, FRiP thresholds |
| 2 | ENCODE Project Consortium 2020, Nature, DOI:10.1038/s41586-020-2493-4 (~1,656 cit) | ENCODE Phase 3; expanded quality standards to new assays, defined cCRE registry |
| 3 | Buenrostro et al. 2013, Nat Methods, DOI:10.1038/nmeth.2688 (~7,000 cit) | Introduced ATAC-seq; established fragment size and TSS enrichment as key QC |
| 4 | Ou et al. 2018, BMC Genomics, DOI:10.1186/s12864-018-4559-3 | ATACseqQC R package; systematic quality metrics for ATAC-seq |
| 5 | Conesa et al. 2016, Genome Biol, DOI:10.1186/s13059-016-0881-8 (~2,363 cit) | RNA-seq best practices survey; defined mapping rate, rRNA, gene body coverage |
| 6 | Foox et al. 2021, Genome Biol, DOI:10.1186/s13059-021-02529-2 | SEQC2 EpiQC consortium; multi-platform WGBS benchmarking |
| 7 | Yardimci et al. 2019, Genome Biol, DOI:10.1186/s13059-019-1658-7 | Hi-C quality measures; cis/trans ratio, distance-dependent decay, resolution |
| 8 | Skene & Henikoff 2017, eLife, DOI:10.7554/eLife.21856 (~1,800 cit) | CUT&RUN method; established spike-in normalization and low-background QC |
| 9 | Kaya-Okur et al. 2019, Nat Commun, DOI:10.1038/s41467-019-09982-5 (~1,200 cit) | CUT&Tag method; tagmentation-based profiling with distinct QC profile |
| 10 | Li et al. 2011, Ann Appl Stat, DOI:10.1214/11-AOAS466 (~1,500 cit) | Irreproducible Discovery Rate (IDR); principled replicate concordance |
| 11 | Hitz et al. 2023, Nucleic Acids Res, DOI:10.1093/nar/gkad243 | ENCODE uniform processing pipelines; standardized QC across all assays |
| 12 | Nordin et al. 2023, Genome Biol, DOI:10.1186/s13059-023-03027-3 | CUT&RUN suspect list; identified artifact-prone regions specific to CUT&RUN/CUT&Tag |
| 13 | Amemiya et al. 2019, Sci Rep, DOI:10.1038/s41598-019-45839-z (~1,372 cit) | ENCODE Blacklist v2; artifact regions to exclude from all analyses |

## Step 1: Retrieve Experiment Details and Audit Status

Use `encode_get_experiment` with the accession to get full metadata including:
- **Audit counts** — `audit_error_count`, `audit_not_compliant_count`, `audit_warning_count`, `audit_internal_action_count` (four ints, one per ENCODE severity level)
- **Replicate information** — `bio_replicate_count`, `tech_replicate_count`, `replication_type`
- **The file list** (`files`) with format, output type, assembly, size and md5sum per file

Two things this tool does not return: the ENCODE pipeline record (use `encode_track_experiment`, which returns `pipelines[]`) and per-file quality metrics such as FRiP or NSC (read those from the file's page on encodeproject.org).

```
encode_get_experiment(accession="ENCSR...")
```

For batch assessment across multiple experiments:
```
encode_search_experiments(assay_title="...", organ="...", limit=50)
# Then iterate through results checking audit flags
```

## Step 2: Interpret ENCODE Audit Flags

ENCODE audits are generated by automated validators during the ENCODE uniform processing pipeline (Hitz et al. 2023). They flag experiments by severity:

| Level | Meaning | Action |
|-------|---------|--------|
| **ERROR** | Critical issues — data may be unreliable | Avoid using unless no alternative exists. Document thoroughly if used. |
| **NOT_COMPLIANT** | Does not meet current ENCODE standards | Usable with caveats. Check which specific standard is violated. |
| **WARNING** | Minor issues detected | Generally safe. Document the specific warning. |
| **INTERNAL_ACTION** | DCC processing notes | Usually not a concern for external users. |

**Common audit categories and what they mean:**

| Audit Category | What It Checks |
|---------------|---------------|
| `replicate concordance` | IDR or correlation between biological replicates |
| `library complexity` | NRF, PBC1, PBC2 — whether library is saturated |
| `read depth` | Whether minimum depth thresholds are met |
| `control quality` | Whether input/IgG control is adequate |
| `mapping quality` | Alignment rate and uniquely mapped fraction |
| `peak calling` | Whether peaks were called successfully, FRiP |
| `antibody validation` | Whether antibody meets ENCODE standards |

Present every audit flag to the user and explain each one. A single ERROR audit does not automatically disqualify an experiment — context matters.

## Step 3: Evaluate ChIP-seq Quality (Landt et al. 2012)

The ENCODE ChIP-seq guidelines (Landt et al. 2012) established the foundational metrics still used today. These were developed from analysis of hundreds of ChIP-seq experiments and reflect empirically-derived thresholds.

### Core Metrics

| Metric | Threshold | Concern | What It Measures | Why It Matters |
|--------|-----------|---------|-----------------|----------------|
| **FRiP** | ≥1% (TF), ≥5% (histone) | Below threshold | Fraction of reads in peaks | Signal enrichment. Very low FRiP means most reads are background. TF ChIP typically has lower FRiP than broad histone marks. |
| **NSC** | >1.05 | ≤1.05 | Normalized strand cross-correlation | Signal-to-noise ratio. Computed from strand shift analysis. Values near 1.0 indicate no enrichment. |
| **RSC** | >0.8 | ≤0.8 | Relative strand cross-correlation | Signal relative to phantom peak. More robust than NSC for shallow libraries. |
| **NRF** | ≥0.8 | <0.8 | Non-redundant fraction (unique/total) | Library complexity. Low NRF = excessive PCR duplication = wasted sequencing. |
| **PBC1** | ≥0.8 | <0.5 | PCR bottleneck coefficient 1 | N1/Nd: fraction of locations with exactly 1 read. More sensitive than NRF at high depth. |
| **PBC2** | ≥3 | <1 | PCR bottleneck coefficient 2 | N1/N2: ratio of 1-read to 2-read locations. <1 indicates severe bottleneck. |

### Read Depth Requirements

| Target Type | Minimum per Replicate | Recommended | Notes |
|------------|----------------------|-------------|-------|
| Transcription factor | 10M uniquely mapped | 20M | Narrow peaks, need depth for detection |
| Broad histone mark (H3K27me3, H3K9me3, H3K36me3) | 20M uniquely mapped | 45M | Broad domains require more reads |
| Narrow histone mark (H3K4me3, H3K27ac) | 20M uniquely mapped | 20M | Sharp peaks, similar to TF |
| Input/IgG control | 10M uniquely mapped | Match IP depth | Should match or exceed IP library depth |

### IDR Analysis (Li et al. 2011)

The Irreproducible Discovery Rate provides principled assessment of replicate concordance:

| IDR Comparison | Expected | Concern | Interpretation |
|---------------|----------|---------|---------------|
| Nt (true replicates) | ≥50% of Np | <50% Np | Low concordance between biological replicates |
| Np (pooled pseudoreplicates) | Reference set | — | Represents total discoverable peaks |
| Self-consistency (Ns) | ≥50% of Np | <50% Np | Individual replicate quality |
| Rescue ratio (Np/max(Nt,Ns)) | <2 | >2 | High ratio = one replicate much weaker |

**Key insight**: IDR thresholded peaks represent peaks passing replicate concordance analysis. Pseudoreplicated peaks = single-replicate fallback (lower confidence). Optimal IDR peaks from pooled data = most complete peak set.

### Antibody Validation

ENCODE requires characterization for every antibody:
- **Primary**: IP followed by mass spectrometry or immunoprecipitation-western
- **Secondary**: At least one of: knockdown/knockout, motif enrichment, genomic annotation enrichment
- No `encode_*` tool returns antibody data: read `antibody_lot_reviews` on the experiment's page on encodeproject.org

## Step 4: Evaluate ATAC-seq Quality (Buenrostro et al. 2013; Ou et al. 2018)

ATAC-seq has a distinct quality profile driven by the transposase insertion mechanism.

### Core Metrics

| Metric | Good | Concern | What It Measures |
|--------|------|---------|-----------------|
| **TSS enrichment** | ≥5 GRCh38 / ≥6 hg19 / ≥10 mm10 (ENCODE data standards) | <4 | Signal enrichment at transcription start sites. The single most informative ATAC-seq QC metric. |
| **FRiP** | ≥20% | <10% | Higher expected FRiP than ChIP-seq because accessible chromatin = true signal |
| **Fragment size distribution** | Clear nucleosomal ladder | Monotonic decay | Should show peaks at <150bp (NFR), ~200bp (mono-nuc), ~400bp (di-nuc), ~600bp (tri-nuc) |
| **NFR ratio** | >2× mono-nucleosomal | <1× | Ratio of sub-nucleosomal to mono-nucleosomal fragments |
| **Mitochondrial reads** | <20% (after filtering) | >50% | Mitochondrial DNA is highly accessible; excessive = poor nuclear enrichment |
| **Duplicate rate** | <30% | >50% | PCR duplication. Omni-ATAC protocol reduces this. |
| **NRF** | ≥0.7 | <0.5 | Library complexity, same concept as ChIP-seq |

### Read Depth Requirements

| Sample Type | Minimum | Recommended |
|------------|---------|-------------|
| Bulk ATAC-seq | 25M uniquely mapped (post-dedup, post-mito filter) | 50M |
| Single-cell ATAC-seq | 25K unique fragments per cell | 50K per cell |

### Fragment Size Interpretation (Buenrostro et al. 2013)

The fragment size distribution is the signature QC plot for ATAC-seq:
- **Sub-nucleosomal (<150bp)**: Nucleosome-free regions — these are the "open chromatin" signal
- **Mono-nucleosomal (~200bp)**: Single nucleosome wrapped
- **Di-nucleosomal (~400bp)**: Two nucleosomes
- **Tri-nucleosomal (~600bp)**: Three nucleosomes

A clean ATAC-seq library shows a clear nucleosomal ladder. Monotonic decay (no peaks) suggests either dead cells, over-transposition, or excessive DNA damage.

## Step 5: Evaluate RNA-seq Quality (Conesa et al. 2016)

### Core Metrics

| Metric | Good | Concern | What It Measures |
|--------|------|---------|-----------------|
| **Mapping rate** | 70-90% uniquely mapped | <70% | Alignment success. Low rate = contamination, adapter issues, or wrong reference |
| **rRNA contamination** | <10% | >20% | Ribosomal RNA depletion efficiency. High = failed ribo-depletion |
| **Gene body coverage** | Uniform 5'→3' | Strong 3' bias | Even coverage across gene bodies. 3' bias = degraded RNA or poly-A capture bias |
| **Duplication rate** | <50% | >70% | PCR amplification artifacts |
| **Replicate correlation** | Spearman ≥0.9 (same condition) | <0.8 | Concordance between replicates |
| **Exonic reads** | >60% of mapped | <40% | Reads mapping to annotated exons vs intergenic |
| **Intergenic reads** | <10% | >20% | Reads mapping between genes — may indicate genomic DNA contamination |

### Read Depth Requirements (Conesa et al. 2016)

| Application | Minimum | Recommended | Notes |
|------------|---------|-------------|-------|
| Gene-level quantification | 10M mapped | 30M mapped | Standard bulk RNA-seq |
| Transcript-level quantification | 30M mapped | 60M mapped | Isoform detection requires more depth |
| Differential expression | 10M per sample, ≥3 bio reps | 20M per sample | Statistical power depends more on replicates than depth |
| Rare transcript detection | 50M+ mapped | 100M mapped | Long-tail of expression distribution |
| total RNA-seq | 50M+ mapped | 100M mapped | Includes non-coding RNA, intergenic transcripts |

### Strand Specificity

ENCODE RNA-seq data may be stranded or unstranded:
- **Stranded**: Can distinguish sense vs antisense transcription. Required for accurate quantification of overlapping genes.
- **Unstranded**: Cannot resolve strand of origin. No `encode_*` tool reports this — check `run_type` and the library's strand specificity on the experiment's page on encodeproject.org.

## Step 6: Evaluate WGBS Quality (ENCODE data standards)

### Core Metrics

| Metric | Good | Concern | What It Measures |
|--------|------|---------|-----------------|
| **Bisulfite conversion rate** | ≥98% | <98% | Efficiency of C→U conversion of unmethylated cytosines. Measured from spike-in controls (lambda phage DNA). |
| **CpG coverage** | >80% of CpGs at ≥1× | <50% | Fraction of CpG sites covered by at least one read |
| **Mean CpG coverage** | ≥10× for DMR analysis | <5× | Average sequencing depth at CpG sites. 10× needed for reliable methylation calls. |
| **Mapping rate** | >60% unique | <40% | Lower than standard WGS due to reduced complexity after bisulfite conversion |
| **Duplication rate** | <30% | >50% | PCR duplicates |
| **CpG methylation distribution** | Bimodal (near 0% and near 100%) | Unimodal | Healthy cells show bimodal: most CpGs are either fully methylated or unmethylated |
| **Lambda/pUC19 conversion** | ≥98% conversion rate | <98% | Spike-in controls for bisulfite conversion efficiency |

### Platform Considerations (Foox et al. 2021)

The SEQC2 EpiQC benchmark found significant platform effects:
- Different sequencing platforms (Illumina HiSeq, NovaSeq, MGI) can produce systematically different methylation calls
- Cross-platform comparisons require careful normalization
- RRBS (Reduced Representation Bisulfite Sequencing) covers only ~10% of CpGs — do NOT combine RRBS with WGBS directly

### Coverage Requirements

| Application | Minimum CpG Coverage | Recommended |
|------------|---------------------|-------------|
| Methylation landscape | 1× | 5× |
| Differentially methylated regions | 5× per sample | 10× per sample |
| Allele-specific methylation | 15× | 30× |
| Single CpG resolution | 10× | 30× |

## Step 7: Evaluate Hi-C Quality (Yardimci et al. 2019)

### Core Metrics

| Metric | Good | Concern | What It Measures |
|--------|------|---------|-----------------|
| **Cis/trans ratio** | >60% cis | <40% cis | Fraction of contacts within same chromosome. Low cis = random ligation = poor quality |
| **Long-range cis (>20kb)** | >40% of cis | <15% | True 3D interactions vs random proximity. Short-range contacts are noise-enriched |
| **Unique valid pairs** | >50% of total | <25% | Pairs surviving all filters (mapping, dedup, chimera removal) |
| **Duplicate rate** | <40% | >60% | PCR duplicates in Hi-C are especially problematic because they inflate contact frequencies |
| **Contact distance decay** | Smooth P(s)∝s^-1 curve | Irregular/plateau | Expected power-law decay with genomic distance |

### Read Depth and Resolution

| Resolution Target | Minimum Valid Pairs | Recommended |
|-------------------|-------------------|-------------|
| Compartment-level (100kb) | 50M | 100M |
| TAD-level (40kb) | 200M | 500M |
| Loop-level (5-10kb) | 500M | 1B+ |
| Sub-TAD (1kb) | 2B+ | 5B+ |

**Note**: Hi-C resolution is not just about read depth — it also depends on restriction enzyme site density, ligation efficiency, and fragment size distribution. In situ Hi-C (Rao et al. 2014) generally produces cleaner data than dilution Hi-C.

## Step 8: Evaluate CUT&RUN / CUT&Tag Quality (Skene & Henikoff 2017; Kaya-Okur et al. 2019)

These newer profiling methods have distinct quality profiles from ChIP-seq.

### Key Differences from ChIP-seq

| Feature | ChIP-seq | CUT&RUN / CUT&Tag |
|---------|---------|-------------------|
| Background | High (requires input control) | Low (targeted cleavage) |
| Required depth | 10-45M | 3-8M sufficient |
| FRiP | >1-5% | >20% typical |
| Input control | Required | IgG control recommended but lower priority |
| Fragment size | Size-selected ~200-600bp | Variable; CUT&RUN releases <120bp fragments |
| Spike-in | Not standard | Recommended (E. coli carry-over or added spike-in) |

### CUT&RUN Quality Metrics

| Metric | Good | Concern |
|--------|------|---------|
| FRiP | >20% | <5% |
| Fragment size | Peak at <120bp (released fragments) | Only large fragments |
| Read depth | 3-8M unique mapped | <1M |
| Spike-in ratio | Consistent across conditions | >5× variation |
| Duplicate rate | <30% | >60% |

### CUT&RUN Suspect List (Nordin et al. 2023)

CUT&RUN and CUT&Tag generate artifacts at specific genomic regions (distinct from the ENCODE Blacklist). These are regions with apparent enrichment that is not target-specific:
- Use the CUT&RUN suspect list (Nordin et al. 2023) IN ADDITION to the ENCODE Blacklist
- Available at: https://github.com/Boyle-Lab/CUT-RUN_suspect_list
- Particularly important for H3K4me3 and H3K27me3 CUT&RUN data

## Step 9: Evaluate Single-Cell Quality (scRNA-seq and scATAC-seq)

ENCODE includes scRNA-seq and scATAC-seq experiments (primarily 10X Chromium platform). Single-cell data has distinct quality metrics from bulk assays, focused on per-cell quality rather than per-experiment signal-to-noise.

### scRNA-seq Quality Metrics

| Metric | Acceptable Range | Red Flag | Notes |
|--------|-----------------|----------|-------|
| **Genes per cell** (median) | 1,500–4,000 (10X) / 4,000–8,000 (Smart-seq2) | <500 | Tissue-dependent; immune cells typically lower than epithelial |
| **UMIs per cell** (median) | 3,000–15,000 (10X) | <1,000 | N/A for Smart-seq2 (no UMIs) |
| **Mitochondrial %** (median) | <10–15% | >25% | High mito% indicates cell stress or lysis; tissue-dependent thresholds |
| **Doublet rate** (estimated) | 2–8% (10X, cell-count dependent) / <2% (plate-based) | >10% | Increases with cell loading density; use Scrublet or DoubletFinder |
| **Mapping rate** | >80% | <60% | Low mapping suggests contamination or mismapping |
| **Sequencing saturation** | >40% | <20% | Low saturation may miss rare transcripts |
| **Cell count vs expected** | Within 50–150% of expected | <30% or >200% | Very low = failed capture; very high = doublets or debris |

### scATAC-seq Quality Metrics

| Metric | Acceptable Range | Red Flag | Notes |
|--------|-----------------|----------|-------|
| **Unique fragments per cell** | >3,000 | <1,000 | Sparse data below threshold makes peak calling unreliable |
| **TSS enrichment per cell** | >5 | <2 | Low TSS enrichment indicates failed Tn5 insertion bias |
| **Fraction in peaks (FRiP)** | >20% | <10% | Measures signal-to-noise at single-cell level |
| **Fraction of mitochondrial reads** | <5% | >10% | Dead/dying cells captured |
| **Duplicate rate** | <40% | >60% | High duplication indicates low library complexity |

### Single-Cell-Specific Quality Pitfalls

- **Ambient RNA contamination** (scRNA-seq): Cell-free RNA from lysed cells during droplet capture inflates apparent expression of highly-expressed genes across all cells. Use CellBender (best-in-class), SoupX, or DecontX to estimate and remove ambient contamination BEFORE downstream analysis.
- **Barcode multiplets** (scATAC-seq): Multiple cells per droplet inflate fragment counts and blur cell-type signals. ArchR and SnapATAC2 include doublet detection modules.
- **Cell-type composition bias**: Quality metrics vary by cell type. A "low-quality" cell may be a small immune cell, not a damaged cell. Apply adaptive QC thresholds per cluster (e.g., miQC) rather than global cutoffs.
- **Batch effects across donors**: For ENCODE tissue scRNA-seq from multiple donors, batch correction (Harmony, scVI) is typically needed before integration. The single-cell-encode skill covers integration workflows.

### Where to Find Single-Cell Quality in ENCODE

Single-cell experiments in ENCODE include cell-level quality summaries in their metadata. Check:
```
encode_get_experiment(accession="ENCSR...")
```
Look at the four `audit_*_count` fields — ENCODE applies automated QC checks including minimum cell counts, minimum genes per cell, and maximum doublet rates. Also check `bio_replicate_count` and `replication_type`; library preparation details are on the experiment's ENCODE page, not in the tool output.

## Step 10: Assess Replication

ENCODE requires minimum 2 independent biological replicates for released data.

### Replicate Types and Their Meaning

| Replicate Type | Definition | Use Case |
|---------------|-----------|----------|
| **Biological** | Independent biological samples | Gold standard — captures biological variation |
| **Technical** | Same sample, different library prep | Assesses technical reproducibility |
| **Isogenic** | Same genotype, different growth/collection | Common for cell lines (e.g., K562, GM12878) |
| **Anisogenic** | Different genotypes/donors | Common for tissue samples |

### Concordance Assessment

Use `encode_list_files` to check for replicated peak files:

| File Output Type | What It Means | Confidence |
|-----------------|---------------|------------|
| **IDR thresholded peaks** | Passed replicate concordance analysis (Li et al. 2011) | Highest |
| **Optimal IDR peaks** | Peaks from pooled data, thresholded by IDR | Complete set |
| **Conservative IDR peaks** | Stricter IDR threshold | Most conservative |
| **Pseudoreplicated peaks** | IDR on pseudoreplicates from pooled data | Single-replicate fallback |
| **Replicated peaks** | Found in multiple replicates (non-IDR method) | Moderate |

### Cross-Replicate Correlation

For quantitative data (RNA-seq, signal tracks):
- **Spearman correlation ≥0.9**: Excellent concordance
- **0.8-0.9**: Acceptable, check for outliers
- **<0.8**: Investigate — batch effect, sample mix-up, or biological variation

## Step 11: Apply the ENCODE Blacklist (Amemiya et al. 2019)

Before interpreting any peak-based quality metric, confirm that the ENCODE Blacklist has been applied:
- **Blacklist v2 regions**: High-signal artifacts (satellite repeats, centromeric regions, high-copy sequences)
- **Available at**: https://github.com/Boyle-Lab/Blacklist/
- **hg38**: `hg38-blacklist.v2.bed.gz` (910 regions)
- **mm10**: `mm10-blacklist.v2.bed.gz`

Failure to remove blacklisted regions will inflate FRiP, create false peaks, and confound enrichment analyses. If analyzing CUT&RUN/CUT&Tag, also apply the CUT&RUN suspect list (Nordin et al. 2023).

## Step 12: File Quality Tiers and Selection

When listing files with `encode_list_files`, use quality-informed selection:

```
# Get preferred default files (ENCODE's recommendation)
encode_list_files(experiment_accession="ENCSR...", preferred_default=True)

# Get IDR thresholded peaks (gold standard for ChIP-seq)
encode_list_files(experiment_accession="ENCSR...", output_type="IDR thresholded peaks", assembly="GRCh38")

# Get signal tracks for visualization
encode_list_files(experiment_accession="ENCSR...", output_type="fold change over control", assembly="GRCh38")
```

### File Quality Hierarchy

| Priority | File Type | When to Use |
|----------|----------|-------------|
| 1 | `preferred_default=True` | ENCODE's recommended files — start here |
| 2 | IDR thresholded peaks | Gold standard for ChIP-seq peak calls |
| 3 | Fold change over control | Normalized signal for visualization and quantitative comparison |
| 4 | Signal of unique reads | Clean signal tracks (unnormalized) |
| 5 | Pseudoreplicated peaks | Fallback when IDR fails or only 1 replicate available |
| 6 | Unfiltered alignments | Only for custom re-analysis |

## Step 13: Summarize Quality Verdict

Provide a structured quality assessment:

### Quality Tiers

| Tier | Criteria | Recommendation |
|------|----------|---------------|
| **High quality** | No ERROR/NOT_COMPLIANT audits, all metrics above thresholds, ≥2 biological replicates, IDR peaks available | Use confidently. Ideal for primary analysis. |
| **Usable with caveats** | WARNING-level audits or borderline metrics (within 20% of threshold), good replication | Usable. Document specific limitations in methods. |
| **Use with caution** | NOT_COMPLIANT flags, one metric below threshold, or single-replicate | Use only if no better alternative. Document all issues. Flag in results. |
| **Not recommended** | ERROR flags, multiple metrics below threshold, poor replication, no IDR peaks | Avoid. Seek alternative experiments or datasets. |

### Quality Summary Template

For each experiment assessed, provide:
1. **Accession**: ENCSR...
2. **Assay**: type and target
3. **Audit status**: list all flags with explanations
4. **Key metrics**: table with values and pass/fail
5. **Replication**: number and type of replicates, concordance
6. **Pipeline**: which ENCODE uniform pipeline version was used
7. **Verdict**: tier assignment with justification
8. **Caveats**: any specific limitations to note

## Pitfalls and Common Mistakes

1. **Single-metric decisions**: No single metric captures quality. FRiP alone can be misleading — some TF ChIP-seq with biological signal has low FRiP due to focal binding patterns. Always evaluate collectively.

2. **Comparing across assays**: Do NOT compare ChIP-seq metrics to ATAC-seq metrics to CUT&RUN metrics. Each assay has its own quality profile and thresholds.

3. **Ignoring batch effects**: Experiments from different labs, dates, or platforms may have systematic quality differences. When combining data, check for batch-correlated quality variation.

4. **Assembly mismatch**: Quality metrics computed on different assemblies (hg19 vs GRCh38) may differ slightly. Always verify the assembly of quality metrics matches your analysis assembly.

5. **Antibody lot variation**: The same antibody target can show different enrichment across lots. Check `antibody_lot_reviews` on the experiment's page on encodeproject.org (it is not part of any tool response).

6. **Read depth ≠ quality**: A deeply sequenced bad library is still a bad library. Check NRF/PBC first — if complexity is exhausted, more sequencing wastes resources.

7. **Control quality matters**: An IP library is only as good as its control. Poor input/IgG control undermines all downstream peak-based metrics.

8. **Newer assays, different rules**: CUT&RUN and CUT&Tag have inherently different quality profiles from ChIP-seq. Applying ChIP-seq thresholds to CUT&RUN will incorrectly flag high-quality data.

## Walkthrough: Quality Assessment of ENCODE ChIP-seq Before Analysis

**Goal**: Evaluate the quality of ENCODE ChIP-seq experiments against ENCODE consortium standards before including them in downstream analysis.
**Context**: Not all ENCODE experiments meet the highest quality standards. Quality assessment prevents garbage-in-garbage-out in aggregation and integration analyses.

### Step 1: Get experiment details and audit status

```
encode_get_experiment(accession="ENCSR000AKA")
```

Expected output (fields abridged; the full response also carries `files` and the rest of the experiment metadata):
```json
{
  "accession": "ENCSR000AKA",
  "assay_title": "Histone ChIP-seq",
  "target": "H3K27ac",
  "biosample_summary": "GM12878",
  "bio_replicate_count": 2,
  "tech_replicate_count": 2,
  "replication_type": "isogenic",
  "status": "released",
  "audit_error_count": 0,
  "audit_not_compliant_count": 0,
  "audit_warning_count": 1,
  "audit_internal_action_count": 0
}
```

**Interpretation**: `audit_error_count` 0 and `audit_not_compliant_count` 0 = experiment meets ENCODE standards. One warning is acceptable. The counts say how many flags there are, not which ones — open the experiment on encodeproject.org for the individual audit messages.

### Step 2: Check file-level quality

```
encode_list_files(experiment_accession="ENCSR000AKA", file_format="bed", output_type="IDR thresholded peaks", assembly="GRCh38")
```

### Step 3: Review quality metrics

Key ChIP-seq quality thresholds (Landt et al. 2012):
| Metric | Threshold | Meaning |
|---|---|---|
| FRiP | >= 1% | Signal enrichment over background |
| NSC | > 1.05 | Strand cross-correlation signal |
| RSC | > 0.8 | Relative strand correlation |
| NRF | >= 0.8 | Library complexity |
| IDR | < 0.05 | Reproducibility between replicates |

### Step 4: Track quality-verified experiments

```
encode_track_experiment(accession="ENCSR000AKA", notes="QC PASSED: FRiP=3.2%, NSC=1.12, RSC=0.95, 0 audit errors")
```

### Integration with downstream skills
- Quality-filtered experiments feed into **histone-aggregation** and other aggregation skills
- QC metrics inform **pipeline-chipseq** parameter tuning
- Audit status guides **search-encode** experiment selection
- QC documentation supports **data-provenance** and **scientific-writing**

## Code Examples

### 1. Check experiment audit status
```
encode_get_experiment(accession="ENCSR000AKA")
```

Expected output (audit fields only; the response carries the full experiment record):
```json
{
  "accession": "ENCSR000AKA",
  "audit_error_count": 0,
  "audit_not_compliant_count": 0,
  "audit_warning_count": 1,
  "audit_internal_action_count": 0
}
```

### 2. List the analysis files for an experiment
```
encode_list_files(experiment_accession="ENCSR000AKA", file_format="bed", assembly="GRCh38")
```

Expected output (a JSON array of files; fields abridged):
```json
[
  {"accession": "ENCFF001ABC", "output_type": "IDR thresholded peaks", "file_format": "bed", "file_type": "bed narrowPeak", "assembly": "GRCh38", "file_size": 1153434, "file_size_human": "1.1 MB", "preferred_default": true}
]
```

### 3. Get detailed file info
```
encode_get_file_info(accession="ENCFF001ABC")
```

Expected output (no QC metrics here — `FileSummary` has no FRiP/NSC/RSC fields; get those from the file's page on encodeproject.org):
```json
{
  "accession": "ENCFF001ABC",
  "file_format": "bed",
  "file_type": "bed narrowPeak",
  "output_type": "IDR thresholded peaks",
  "output_category": "annotation",
  "assembly": "GRCh38",
  "file_size": 1153434,
  "file_size_human": "1.1 MB",
  "biological_replicates": [1, 2],
  "status": "released",
  "md5sum": "5d41402abc4b2a76b9719d911017c592",
  "experiment_accession": "ENCSR000AKA",
  "preferred_default": true
}
```

## Integration

| This skill produces... | Feed into... | Purpose |
|---|---|---|
| Quality-verified experiments | **histone-aggregation** | Only aggregate high-quality data |
| QC pass/fail decisions | **search-encode** | Filter search results by quality |
| Quality metric reports | **data-provenance** | Document QC criteria used |
| Audit interpretation | **pipeline-guide** | Guide reprocessing decisions |
| QC documentation | **scientific-writing** | Methods section QC reporting |
| Quality thresholds | **publication-trust** | Verify QC threshold citations |
| Validated experiment lists | **batch-analysis** | Process only quality-approved experiments |
| QC-filtered peaks | **regulatory-elements** | High-confidence regulatory element maps |

## Related Skills

- **publication-trust**: Assess scientific integrity of publications before relying on their methods or findings — complements experiment quality with publication quality
- **histone-aggregation**: Uses quality filtering before aggregating peaks
- **accessibility-aggregation**: ATAC-seq quality directly impacts aggregation
- **data-provenance**: Log quality decisions and thresholds used
- **compare-biosamples**: Quality must be comparable across samples being compared
- **integrative-analysis**: All data sources need quality assessment before integration
- **single-cell-encode**: Quality metrics for scRNA-seq and scATAC-seq (genes/cell, fragments/cell, TSS enrichment)
- **epigenome-profiling**: Quality assessment is a prerequisite for epigenomic profile assembly
- **variant-annotation**: Quality of ENCODE experiments determines reliability of variant annotation
- **pipeline-guide**: Pipeline version affects quality metric computation
- **batch-analysis**: Batch QC screening across multiple experiments for systematic quality filtering

## Presenting Results

- Present QC metrics as a traffic-light table: metric | value | threshold | status (PASS/WARN/FAIL). Always include the ENCODE audit level. Suggest: "Would you like to filter to only experiments meeting all QC thresholds?"

## For the request: "$ARGUMENTS"

Comments (0)

No comments yet.

Reviews (0)

No reviews yet.

Claude Prompt

Tighten-my-prose editing prompt

Claude Skill

Adversarial code-review skill for Claude

Claude Workflow

Deep-research workflow (fan-out + verify)

Claude Skill

Conventional-commit writer for Claude