{
  "skill": "alterlab-cosmic",
  "evals": [
    {
      "id": "cancer-gene-census",
      "prompt": "I want the expert-curated Cancer Gene Census from COSMIC so I can filter my variant list down to known cancer genes and split them into oncogenes vs tumor suppressors.",
      "expected_output": "Invokes alterlab-cosmic: downloads the Cancer Gene Census archive (grch38/cosmic/v104/Cosmic_CancerGeneCensus_Tsv_v104_GRCh38.tar, via get_common_file_path('gene_census') and download_cosmic_file with COSMIC credentials), extracts the TSV, and uses the role-in-cancer field to separate oncogenes from TSGs among the ~700+ curated cancer genes. Does not use the retired cancer_gene_census.csv name.",
      "assertions": [
        { "type": "should_trigger", "value": true },
        { "type": "output_contains", "value": "Cancer Gene Census" }
      ]
    },
    {
      "id": "somatic-mutations-by-gene",
      "prompt": "Download the COSMIC coding mutations file (GRCh38) and pull all the recurrent somatic TP53 mutations for my cancer cohort analysis.",
      "expected_output": "Invokes alterlab-cosmic: downloads the GRCh38 coding-mutation archive (Cosmic_GenomeScreensMutant_Tsv_v104_GRCh38.tar, optionally plus the targeted-screens file) via download_cosmic_file, reads the extracted TSV with pandas, and filters GENE_SYMBOL == 'TP53' (or uses the web Filtered download for one gene) to extract somatic mutations. Does not rely on the retired CosmicMutantExport.tsv.gz.",
      "assertions": [
        { "type": "should_trigger", "value": true },
        { "type": "behavior", "value": "Retrieves somatic coding mutations from a downloaded COSMIC export (GRCh38) and filters to the named gene, with authentication." }
      ]
    },
    {
      "id": "mutational-signatures",
      "prompt": "I need the COSMIC SBS mutational signature definitions to run signature decomposition on my whole-genome tumor samples.",
      "expected_output": "Invokes alterlab-cosmic: points to the COSMIC mutational signatures download site (https://cancer.sanger.ac.uk/signatures/downloads/, reference set v3.6) for the SBS/DBS/ID signature matrices matching the genome build, suitable for feeding into signature-analysis tools like SigProfiler or deconstructSigs.",
      "assertions": [
        { "type": "should_trigger", "value": true },
        { "type": "output_contains", "value": "signature" }
      ]
    },
    {
      "id": "resistance-mutations",
      "prompt": "Get me COSMIC's drug resistance mutation data so I can annotate which variants in my samples confer known resistance.",
      "expected_output": "Invokes alterlab-cosmic: downloads the resistance-mutations archive (Cosmic_ResistanceMutations_Tsv_v104_GRCh38.tar) via the resistance_mutations shortcut and surfaces the clinically annotated drug-resistance variants.",
      "assertions": [
        { "type": "should_trigger", "value": true },
        { "type": "behavior", "value": "Downloads the COSMIC resistance mutations file specifically, not generic mutation data." }
      ]
    },
    {
      "id": "near-miss-clinvar",
      "prompt": "Is the germline BRCA1 c.68_69delAG variant classified as pathogenic? I need the clinical significance and review status.",
      "expected_output": "Does NOT invoke this skill; defers to alterlab-clinvar. The user wants germline variant clinical-significance classification, whereas COSMIC catalogues somatic mutations in cancer, not germline pathogenicity calls.",
      "assertions": [
        { "type": "should_not_trigger", "value": true },
        { "type": "output_contains", "value": "alterlab-clinvar" }
      ]
    },
    {
      "id": "near-miss-cbioportal",
      "prompt": "Show me the mutation frequency of EGFR across the TCGA lung adenocarcinoma study with an OncoPrint and survival breakdown.",
      "expected_output": "Does NOT invoke this skill; defers to alterlab-cbioportal. The user wants interactive cohort-level visualization (OncoPrint, survival) from cBioPortal study data, not raw COSMIC catalogue file downloads.",
      "assertions": [
        { "type": "should_not_trigger", "value": true },
        { "type": "output_contains", "value": "alterlab-cbioportal" }
      ]
    }
  ]
}

# COSMIC Database Reference

## Overview

COSMIC (Catalogue of Somatic Mutations in Cancer) is the world's largest and most comprehensive resource for exploring the impact of somatic mutations in human cancer. Maintained by the Wellcome Sanger Institute, it catalogs millions of mutations across thousands of cancer types.

**Website**: https://cancer.sanger.ac.uk/cosmic
**Releases**: Two per year (May and November). Current: **v104**, released 2026-05-19 (v103: 2025-11-18). Check the [release notes](https://cancer.sanger.ac.uk/cosmic/release_notes) for newer versions.

## Data Access

### Authentication
- **Academic users**: Free access (registration required)
- **Commercial users**: Commercial licence required (commercial R&D, products/services,
  patient services/clinical reporting) — https://www.cosmickb.org/licensing
- **Registration**: https://cancer.sanger.ac.uk/cosmic/register

### Download Methods
From https://cancer.sanger.ac.uk/cosmic/download/cosmic, pick a release and product and
click the file name; three options appear:
1. **Download in browser** — the whole `.tar` (gzipped TSV/VCF + README of all columns)
2. **Scripted download** — two-step API: `GET https://cancer.sanger.ac.uk/api/mono/products/v1/downloads/scripted?path=<archive path>&bucket=downloads`
   with HTTP Basic auth (email:password) → JSON `{"url": <signed URL>}` → fetch that URL
   without auth. `path` and `bucket` are both required (HTTP 400 otherwise); bad
   credentials give 401. `scripts/download_cosmic.py` implements this.
3. **Filtered download** — a subset by gene symbol, primary site, or sample name

## Available Data Types

### 1. Core Mutation Data
**Main archives** (`grch38/cosmic/v104/…`; legacy names in brackets):
- `Cosmic_GenomeScreensMutant_Tsv_v104_GRCh38.tar` - Coding mutations from genome-wide screens (WGS/WES)
- `Cosmic_CompleteTargetedScreensMutant_Tsv_v104_GRCh38.tar` - Coding mutations from targeted screens
  (together these replace `CosmicMutantExport.tsv.gz`)
- `VCF/Cosmic_GenomeScreensMutant_Vcf_v104_GRCh38.tar` (also `…_VcfNormal_…` normalized) - VCF [`CosmicCodingMuts.vcf.gz`]
- `VCF/Cosmic_NonCodingVariants_Vcf_v104_GRCh38.tar` - Non-coding variants [`CosmicNonCodingVariants.vcf.gz`]
- `Cosmic_MutantCensus_Tsv_v104_GRCh38.tar` - Coding mutations in Cancer Gene Census genes [`CosmicMutantExportCensus.tsv.gz`]

**Content**:
- Point mutations (SNVs)
- Small insertions and deletions (indels)
- Genomic coordinates
- Variant annotations
- Sample information
- Tumor type associations

### 2. Cancer Gene Census
**File**: `Cosmic_CancerGeneCensus_Tsv_v104_GRCh38.tar` [legacy `cancer_gene_census.csv`];
hallmarks in `Cosmic_CancerGeneCensusHallmarksOfCancer_Tsv_v104_GRCh38.tar`

**Content**:
- Expert-curated list of cancer genes
- ~700+ genes with substantial evidence of involvement in cancer
- Gene roles (oncogene, tumor suppressor, fusion)
- Mutation types
- Tissue associations
- Molecular genetics information

### 3. Mutational Signatures
**Where**: https://cancer.sanger.ac.uk/signatures/downloads/ (a separate site; not part of
the product archives above)
- Single Base Substitution (SBS), Doublet Base Substitution (DBS), Insertion/Deletion (ID),
  Copy Number (CN), and Structural Variant (SV) reference signatures
- Matrices per reference genome (GRCh37, GRCh38, mouse builds)

**Current Version**: v3.6 (May 2026)

**Content**:
- Signature profiles (96-channel, 78-channel, 83-channel)
- Etiology annotations
- Reference signatures for signature analysis

### 4. Structural Variants
**Files**: `Cosmic_StructuralVariants_Tsv_v104_GRCh38.tar`, `Cosmic_Breakpoints_Tsv_v104_GRCh38.tar`,
fusions in `Cosmic_Fusion_Tsv_v104_GRCh38.tar` [legacy `CosmicStructExport.tsv.gz`, `CosmicFusionExport.tsv.gz`]

**Content**:
- Gene fusions
- Structural breakpoints
- Translocation events
- Large deletions/insertions
- Complex rearrangements

### 5. Copy Number Variations
**File**: `Cosmic_CompleteCNA_Tsv_v104_GRCh38.tar` [legacy `CosmicCompleteCNA.tsv.gz`]

**Content**:
- Copy number gains and losses
- Amplifications and deletions
- Segment-level data
- Gene-level annotations

### 6. Gene Expression
**File**: `Cosmic_CompleteGeneExpression_Tsv_v104_GRCh38.tar` [legacy `CosmicCompleteGeneExpression.tsv.gz`]

**Content**:
- Over/under-expression data
- Gene expression Z-scores
- Tissue-specific expression patterns

### 7. Resistance Mutations
**File**: `Cosmic_ResistanceMutations_Tsv_v104_GRCh38.tar` [legacy `CosmicResistanceMutations.tsv.gz`]

**Content**:
- Drug resistance mutations
- Treatment associations
- Clinical relevance

### 8. Cell Lines Project
**Files**: Various cell line-specific files

**Content**:
- Mutations in cancer cell lines
- Copy number data for cell lines
- Fusion genes in cell lines
- Microsatellite instability status

### 9. Sample Information
**Files**: `Cosmic_Sample_Tsv_v104_GRCh38.tar`, tumour classification in
`Cosmic_Classification_Tsv_v104_GRCh38.tar` [legacy `CosmicSample.tsv.gz`]

**Content**:
- Sample metadata
- Tumor site/histology
- Sample sources
- Study references

## Genome Assemblies

All genomic data is available for two reference genomes:
- **GRCh37** (hg19) - Legacy assembly
- **GRCh38** (hg38) - Current assembly (recommended)

Archive paths use the pattern: `{assembly lower-case}/cosmic/{version}/[VCF/]Cosmic_{Product}_{Tsv|Vcf|VcfNormal}_{version}_{Assembly}.tar`

## File Formats

### TSV Format
- Delivered inside a `.tar` together with a README that documents every column
- Tab-separated, gzip compressed (.gz), column headers included
- Can be read with pandas, awk, or standard tools

### VCF Format
- Standard Variant Call Format
- Version 4.x specification
- Includes INFO fields with COSMIC annotations
- Gzip compressed and indexed (.vcf.gz, .vcf.gz.tbi)

## Common File Paths

Release v104 (the download service lists only explicit-version paths):

```
# Coding mutations (TSV): genome-wide and targeted screens
grch38/cosmic/v104/Cosmic_GenomeScreensMutant_Tsv_v104_GRCh38.tar
grch38/cosmic/v104/Cosmic_CompleteTargetedScreensMutant_Tsv_v104_GRCh38.tar

# Coding mutations (VCF)
grch38/cosmic/v104/VCF/Cosmic_GenomeScreensMutant_Vcf_v104_GRCh38.tar

# Cancer Gene Census
grch38/cosmic/v104/Cosmic_CancerGeneCensus_Tsv_v104_GRCh38.tar

# Structural variants / fusions / copy number / expression
grch38/cosmic/v104/Cosmic_StructuralVariants_Tsv_v104_GRCh38.tar
grch38/cosmic/v104/Cosmic_Fusion_Tsv_v104_GRCh38.tar
grch38/cosmic/v104/Cosmic_CompleteCNA_Tsv_v104_GRCh38.tar
grch38/cosmic/v104/Cosmic_CompleteGeneExpression_Tsv_v104_GRCh38.tar

# Resistance mutations
grch38/cosmic/v104/Cosmic_ResistanceMutations_Tsv_v104_GRCh38.tar

# Samples and tumour classification
grch38/cosmic/v104/Cosmic_Sample_Tsv_v104_GRCh38.tar
grch38/cosmic/v104/Cosmic_Classification_Tsv_v104_GRCh38.tar
```

## Key Data Fields

> The fields below are described by their meaning. The current TSVs use upper-case
> column names (e.g. `GENE_SYMBOL`, `SAMPLE_NAME`) that differ from the legacy exports
> (e.g. `Gene name`, `Primary site`); read the README bundled with each archive for the
> exact names before writing filters.

### Mutation Data Fields
- **Gene name** - HGNC gene symbol
- **Accession Number** - Transcript identifier
- **COSMIC ID** - Unique mutation identifier
- **CDS mutation** - Coding sequence change
- **AA mutation** - Amino acid change
- **Primary site** - Anatomical tumor location
- **Primary histology** - Tumor type classification
- **Genomic coordinates** - Chromosome, position, strand
- **Mutation type** - Substitution, insertion, deletion, etc.
- **Zygosity** - Heterozygous/homozygous status
- **Pubmed ID** - Literature references

### Cancer Gene Census Fields
- **Gene Symbol** - Official gene name
- **Entrez GeneId** - NCBI gene identifier
- **Role in Cancer** - Oncogene, TSG, fusion
- **Mutation Types** - Types of alterations observed
- **Translocation Partner** - For fusion genes
- **Tier** - Evidence classification (1 or 2)
- **Hallmark** - Cancer hallmark associations
- **Somatic** - Whether somatic mutations are documented
- **Germline** - Whether germline mutations are documented

## Data Updates

COSMIC publishes two releases a year (May and November). Each release includes:
- New mutation data from literature and databases
- Updated Cancer Gene Census annotations
- Revised mutational signatures if applicable
- Enhanced sample annotations

## Citation

When using COSMIC data, cite the current database paper:
Sondka Z, Dhir NB, Carvalho-Silva D, et al. COSMIC: a curated database of somatic variants and clinical data for cancer. Nucleic Acids Research. 2024;52(D1):D1210-D1217. doi:10.1093/nar/gkad986

## Additional Resources

- **Documentation**: https://cancer.sanger.ac.uk/cosmic/help
- **Release Notes**: https://cancer.sanger.ac.uk/cosmic/release_notes
- **Contact**: cosmic@sanger.ac.uk
- **Licensing**: https://www.cosmickb.org/licensing

#!/usr/bin/env python3
"""
COSMIC Data Download Utility

This script provides functions to download data from the COSMIC database
(Catalogue of Somatic Mutations in Cancer).

Usage:
    from download_cosmic import download_cosmic_file, get_common_file_path

    # Download a specific product archive
    download_cosmic_file(
        email="user@example.com",
        password="password",
        filepath=get_common_file_path("gene_census"),
        # -> "grch38/cosmic/v104/Cosmic_CancerGeneCensus_Tsv_v104_GRCh38.tar"
    )

Requirements:
    - requests library: uv pip install requests
    - Valid COSMIC account credentials (register at cancer.sanger.ac.uk/cosmic)

How COSMIC scripted downloads work (current download service):
    1. GET https://cancer.sanger.ac.uk/api/mono/products/v1/downloads/scripted
       ?path=<archive path>&bucket=downloads with HTTP Basic auth
       ("Authorization: Basic <base64(email:password)>"; requests' auth=(email,
       password) tuple produces exactly that header). Both query parameters are
       required — omitting either returns HTTP 400; bad credentials return 401.
    2. The response is JSON with a time-limited signed "url"; fetch the file
       from that url WITHOUT the auth header.

    Files are per-product .tar archives (the gzipped TSV/VCF plus a README that
    describes every column), with explicit release versions in the path, e.g.
    grch38/cosmic/v104/Cosmic_GenomeScreensMutant_Tsv_v104_GRCh38.tar.
    The legacy /cosmic/file_download/ endpoint and legacy names such as
    CosmicMutantExport.tsv.gz no longer work for scripts (the endpoint now
    redirects to the login page).

    If COSMIC changes the endpoint, copy the command shown under "Scripted
    download" on https://cancer.sanger.ac.uk/cosmic/download/cosmic and set
    COSMIC_SCRIPTED_URL accordingly.
"""

import os
import sys
from typing import Optional

import requests

SCRIPTED_URL = os.environ.get(
    "COSMIC_SCRIPTED_URL",
    "https://cancer.sanger.ac.uk/api/mono/products/v1/downloads/scripted",
)

# Current COSMIC release as of 2026-09 (v104, released 2026-05-19). COSMIC
# ships two releases a year (May and November); check
# https://cancer.sanger.ac.uk/cosmic/release_notes and pass --version to override.
CURRENT_RELEASE = "v104"

# data_type shortcut -> product archive stem (VCF products live in a VCF/ subfolder)
PRODUCTS = {
    'mutations': 'Cosmic_GenomeScreensMutant_Tsv',           # genome-wide screens (WGS/WES)
    'targeted_mutations': 'Cosmic_CompleteTargetedScreensMutant_Tsv',
    'mutations_vcf': 'VCF/Cosmic_GenomeScreensMutant_Vcf',
    'non_coding_vcf': 'VCF/Cosmic_NonCodingVariants_Vcf',
    'mutation_census': 'Cosmic_MutantCensus_Tsv',             # coding mutations in CGC genes
    'gene_census': 'Cosmic_CancerGeneCensus_Tsv',
    'census_hallmarks': 'Cosmic_CancerGeneCensusHallmarksOfCancer_Tsv',
    'resistance_mutations': 'Cosmic_ResistanceMutations_Tsv',
    'structural_variants': 'Cosmic_StructuralVariants_Tsv',
    'breakpoints': 'Cosmic_Breakpoints_Tsv',
    'fusion_genes': 'Cosmic_Fusion_Tsv',
    'copy_number': 'Cosmic_CompleteCNA_Tsv',
    'gene_expression': 'Cosmic_CompleteGeneExpression_Tsv',
    'methylation': 'Cosmic_CompleteDifferentialMethylation_Tsv',
    'sample_info': 'Cosmic_Sample_Tsv',
    'classification': 'Cosmic_Classification_Tsv',
    'genes': 'Cosmic_Genes_Tsv',
    'transcripts': 'Cosmic_Transcripts_Tsv',
}

SIGNATURES_URL = "https://cancer.sanger.ac.uk/signatures/downloads/"


def download_cosmic_file(
    email: str,
    password: str,
    filepath: str,
    output_filename: Optional[str] = None,
    bucket: str = "downloads",
) -> bool:
    """
    Download a file from COSMIC database.

    The genome assembly and release are encoded in `filepath` (e.g.
    "grch38/cosmic/v104/..."). Use get_common_file_path() to build one.

    Args:
        email: COSMIC account email
        password: COSMIC account password
        filepath: Archive path, e.g. "grch38/cosmic/v104/Cosmic_Genes_Tsv_v104_GRCh38.tar"
        output_filename: Optional custom output filename (default: last part of filepath)
        bucket: Download bucket name expected by the scripted API (default "downloads")

    Returns:
        True if download successful, False otherwise
    """
    # Determine output filename
    if output_filename is None:
        output_filename = os.path.basename(filepath)

    try:
        # Step 1: Get the signed download URL
        print(f"Requesting download URL for: {filepath}")
        r = requests.get(
            SCRIPTED_URL,
            params={"path": filepath, "bucket": bucket},
            auth=(email, password),
            timeout=30
        )

        if r.status_code == 401:
            print("ERROR: Authentication failed. Check email and password.")
            return False
        elif r.status_code == 400:
            print("ERROR: Bad request — check the archive path (release version, "
                  "assembly, product name) against the COSMIC download page.")
            print(f"Response: {r.text}")
            return False
        elif r.status_code == 404:
            print(f"ERROR: File not found: {filepath}")
            return False
        elif r.status_code != 200:
            print(f"ERROR: Request failed with status code {r.status_code}")
            print(f"Response: {r.text}")
            return False

        # Parse response to get download URL
        response_data = r.json()
        download_url = response_data.get("url")

        if not download_url:
            print("ERROR: No download URL in response")
            return False

        # Step 2: Download the file (no auth header on the signed URL)
        print("Downloading file from signed URL")
        file_response = requests.get(download_url, stream=True, timeout=300)

        if file_response.status_code != 200:
            print(f"ERROR: Download failed with status code {file_response.status_code}")
            return False

        # Step 3: Write to disk
        print(f"Saving to: {output_filename}")
        total_size = int(file_response.headers.get('content-length', 0))

        with open(output_filename, 'wb') as f:
            if total_size == 0:
                f.write(file_response.content)
            else:
                downloaded = 0
                for chunk in file_response.iter_content(chunk_size=8192):
                    if chunk:
                        f.write(chunk)
                        downloaded += len(chunk)
                        # Show progress
                        progress = (downloaded / total_size) * 100
                        print(f"\rProgress: {progress:.1f}%", end='', flush=True)
                print()  # New line after progress

        print(f"Successfully downloaded: {output_filename}")
        return True

    except requests.exceptions.Timeout:
        print("ERROR: Request timed out")
        return False
    except requests.exceptions.RequestException as e:
        print(f"ERROR: Request failed: {e}")
        return False
    except Exception as e:
        print(f"ERROR: Unexpected error: {e}")
        return False


def get_common_file_path(
    data_type: str,
    genome_assembly: str = "GRCh38",
    version: str = CURRENT_RELEASE
) -> Optional[str]:
    """
    Get the archive path for common COSMIC data products.

    Args:
        data_type: Shortcut from PRODUCTS (e.g. 'mutations', 'gene_census')
        genome_assembly: GRCh37 or GRCh38
        version: COSMIC release, e.g. "v104". Paths carry explicit versions;
            "latest" is mapped to CURRENT_RELEASE.

    Returns:
        Archive path string, or None if the type is unknown or not served
        by the scripted download API (e.g. 'signatures').
    """
    if data_type == 'signatures':
        # Mutational signatures (COSMIC v3.x) are downloaded from the separate
        # signatures site, not the COSMIC product archives.
        return None
    stem = PRODUCTS.get(data_type)
    if stem is None:
        return None
    if version == "latest":
        version = CURRENT_RELEASE
    subdir, _, name = stem.rpartition("/")
    prefix = f"{genome_assembly.lower()}/cosmic/{version}/"
    if subdir:
        prefix += f"{subdir}/"
    return f"{prefix}{name}_{version}_{genome_assembly}.tar"


def main():
    """Command-line interface for downloading COSMIC files."""
    import argparse

    parser = argparse.ArgumentParser(
        description='Download files from COSMIC database',
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
Examples:
  # Download the Cancer Gene Census (current release, GRCh38)
  %(prog)s user@email.com --data-type gene_census

  # Download a specific archive path
  %(prog)s user@email.com --filepath grch38/cosmic/v104/Cosmic_GenomeScreensMutant_Tsv_v104_GRCh38.tar

  # Download for GRCh37 / a pinned release
  %(prog)s user@email.com --data-type mutations --assembly GRCh37 --version v103
        """
    )

    parser.add_argument('email', help='COSMIC account email')
    parser.add_argument('--password', help='COSMIC account password (will prompt if not provided)')
    parser.add_argument('--filepath', help='Full archive path to download')
    parser.add_argument('--data-type',
                       choices=sorted([*PRODUCTS, 'signatures']),
                       help='Common data type shorthand')
    parser.add_argument('--assembly', default='GRCh38',
                       choices=['GRCh37', 'GRCh38'],
                       help='Genome assembly (default: GRCh38)')
    parser.add_argument('--version', default=CURRENT_RELEASE,
                       help=f'COSMIC release (default: {CURRENT_RELEASE})')
    parser.add_argument('-o', '--output', help='Output filename')

    args = parser.parse_args()

    # Determine filepath
    if args.filepath:
        filepath = args.filepath
    elif args.data_type:
        filepath = get_common_file_path(args.data_type, args.assembly, args.version)
        if not filepath:
            if args.data_type == 'signatures':
                print(f"Mutational signatures are downloaded from {SIGNATURES_URL}")
            else:
                print(f"ERROR: Unknown data type: {args.data_type}")
            return 1
    else:
        print("ERROR: Must provide either --filepath or --data-type")
        parser.print_help()
        return 1

    # Get password if not provided
    if not args.password:
        import getpass
        args.password = getpass.getpass('COSMIC password: ')

    # Download the file
    success = download_cosmic_file(
        email=args.email,
        password=args.password,
        filepath=filepath,
        output_filename=args.output,
    )

    return 0 if success else 1


if __name__ == '__main__':
    sys.exit(main())