Claude Skill

compute-env-setup

Set up and validate a reproducible Python or R environment on a Wisp execution context. Use for a selected local, WSL, or direct SSH context when installing scientific packages, configuring caches, recording interpreter activation, or producing an environment smoke test. Do not u

LLM Mart · 0 points · 15 0 0

Virus-scanned Reviewed automatically before listing.

Full trust report

Download xuzhougeng-wisp-science-skills_compute-env-setup-2b7fd45.zip · 4 KB

xuzhougeng/wisp-science 1170 122 forks AGPL-3.0 Updated 9h ago

Part of xuzhougeng/wisp-science — 25 skills

Install

skills CLI npx skills add https://github.com/xuzhougeng/wisp-science/tree/main/skills/compute-env-setup

Claude Code

claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install xuzhougeng-wisp-science@llmmart

Git git clone https://github.com/xuzhougeng/wisp-science.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole xuzhougeng/wisp-science collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Set up a compute environment

Treat the selected and probed ExecutionContext as authoritative. Wisp currently supports local, wsl:<distro>, and direct ssh:<alias> contexts; it does not expose an authenticated provider SDK inside Python.

Plan the environment

Define before installing:

Python or R version;
ordered conda/pip/R package phases with important pins;
required CUDA capability and minimum VRAM;
cache variables and durable weight locations;
import checks, CLI checks, and one seeded representative workload;
the exact activation command later Runs must include.

Use references/envs_reference.md for package-order and cache examples, but replace container-specific paths with paths valid on the selected context.

Direct SSH workflow

Require a selected ssh:<alias> context with a recent Probe result. Respect recorded GPU, privilege, interpreter, conda/mamba, module, and scheduler capabilities.
If a scheduler is detected, stop. Do not install or run long work on a shared login node; Wisp needs a scheduler-aware Run backend first.
Use at most a few bounded read-only shell commands to confirm free space, existing environments, and cache paths.
Write an idempotent project script such as runs/setup-<environment>.sh. It must use user-writable paths, fail fast, activate the environment explicitly, run all smoke checks, and write a small JSON manifest only after validation succeeds.
Submit the setup script through one persisted Run:

{
  "context_id": "ssh:gpu-box",
  "title": "Set up singlecell environment",
  "command": "bash setup-singlecell.sh /home/me/envs/singlecell /home/me/wisp-env-manifests/singlecell.json",
  "timeout_secs": 14400,
  "input_paths": ["runs/setup-singlecell.sh"],
  "output_specs": [
    {
      "glob": "ssh://gpu-box/home/me/wisp-env-manifests/singlecell.json",
      "kind": "environment-manifest",
      "residency": "remote"
    }
  ]
}

Replace all example paths with probed absolute paths. Call monitor_run when waiting is useful (again after wait_interrupted; do not resubmit). Use one get_run snapshot later or cancel_run when requested.
Record the validated activation command, versions, cache paths, GPU witness, date, and known limitations in a normal project file such as environments/<context>/<name>.md. This file is documentation, not a hidden resolver.

Setup-script requirements

Make repeated execution safe: reuse a matching environment or stop with an actionable version mismatch.
Keep pip install phases ordered; a later dependency resolver must not silently replace pinned torch, CUDA, JAX, NumPy, or compiled extensions.
Never use sudo unless the Probe explicitly records suitable privilege and the user authorizes it. Prefer conda packages, modules, or user paths.
Put multi-gigabyte weights in durable remote storage. Populate them with the model's real loader, verify non-empty content and completion markers, then run a representative inference witness.
Write the manifest atomically only after imports, GPU visibility, and the representative workload pass.

Local and WSL boundary

Local and WSL Runs are currently capped at 300 seconds and do not support input_paths. Use local-env-setup for normal interactive setup. Use run_in_context only for a bounded command that finishes within that limit and writes outputs to host-visible project paths.

Unsupported backends

Wisp has no scheduler, Modal, RunPod, cloud Batch, container-service, or managed endpoint execution context today. Do not invent a provider id or hide those lifecycles inside an SSH submission command. Explain the boundary or use a dedicated direct SSH host until a backend implementing submit, poll, cancel, recovery, secrets, and artifact harvest exists.

Files (wisp-science)

references

envs_reference.md 5.4 KB

---
name: compute-envs-reference
description: Worked package recipes for common scientific stacks — install order, system libraries, egress needs, a validation witness, and the traps each stack hides. Consulted from compute-env-setup when assembling a user-space environment on a direct SSH context.
---

# Environment recipes

Nothing below is provisioned automatically — each entry is a tested recipe
to translate into an idempotent setup script for the selected SSH context.
Container base images and resource tiers are the conditions the recipe was
verified under, not requirements: substitute probed, user-writable paths and
the actual hardware. When done, record the activation command and the
validation evidence in the project; do not assume any environment-name
resolver exists.

Quick index (base / GPU / verified tier):

| recipe | base image | GPU | tier |
|---|---|---|---|
| dataml-cpu | python:3.12-slim | — | 4c/16G |
| bio-cpu | python:3.12-slim | — | 4c/16G |
| chem-cpu | python:3.12-slim | — | 4c/16G |
| singlecell-cpu | python:3.12-slim | — | 8c/32G |
| genomics-cpu | python:3.12-slim | — | 8c/64G |
| imaging-cpu | python:3.12-slim | — | 4c/32G |
| torch-geometric-gpu | pytorch:2.7.1-cu126-runtime | sm_90 | 1gpu/32G |

## CPU recipes

All six install in a single pip phase on `python:3.12-slim`, need no model
weights and no network egress at runtime. What varies is only the apt-level
shared libraries the wheels link against, plus any CLI binaries.

### dataml-cpu — general ML / stats

- System libs: `libgomp1 build-essential`
- Python: scikit-learn, xgboost, statsmodels, pymc, arviz, shap,
  umap-learn, networkx, dask[complete], polars, zarr, gcsfs, s3fs, aeon,
  pymoo
- Witness: RF + XGBoost fit on a 200×5 toy set scores (1.0, 1.0); polars
  DataFrame round-trips.
- Traps: on some mirrors the name `aeon` resolves to a 0.0.0 squatter —
  pin `aeon>=1.0`. The xgboost wheel drags in `nvidia-nccl-cu12`, ~200MB of
  dead weight on a CPU box.

### bio-cpu — sequence / omics toolkits

- System libs: `libgomp1 build-essential libgl1 libglib2.0-0` (the glib is
  what pyopenms's `.so` links against)
- Python: biopython, prody, biotite, scikit-bio, pyopenms, ete3, cobra,
  neurokit2, FlowIO, matchms, numpy, scipy, pandas
- Witness: ubiquitin FASTA through ProtParam → 76 aa, MW 8564.7, pI 6.56.
- Traps: none encountered.

### chem-cpu — cheminformatics

- System libs: `build-essential libxrender1 libxext6 libsm6 libgomp1`
  (the X libraries serve rdkit's 2D drawing)
- Python: rdkit, openbabel-wheel, datamol, useful_rdkit_utils, molfeat,
  PyTDC, aizynthfinder
- Weights: aizynthfinder's retrosynthesis data is *not* baked in —
  `download_public_data` stays a runtime step.
- Witness: aspirin SMILES → MolWt 180.16, Morgan fingerprint with 24
  on-bits.
- Traps: PyTDC transitively pulls torch + jupyter + scanpy (~250 packages)
  and forces sklearn to build from source — ~19 min and ~5GB. Drop it
  unless TDC datasets are actually needed.

### singlecell-cpu — scRNA-seq

- System libs: `libgomp1 build-essential`
- Python: scanpy, anndata, leidenalg, igraph, scrublet, cellxgene-census,
  samap
- Witness: scanpy normalize → PCA → neighbors → leiden on a 100×50 random
  AnnData yields one cluster.
- Traps: louvain has no py3.12 wheel (leidenalg covers the need). samap has
  historically pinned scanpy<1.10 — drop samap on conflict.

### genomics-cpu — alignment / variant stacks

- System libs: `samtools bedtools bwa spades wget bzip2 build-essential
  libgomp1 libcurl4-openssl-dev libbz2-dev liblzma-dev`
- Extra step: Debian apt only carries legacy `bwa`; fetch the bwa-mem2
  v2.2.1 static tarball into `/opt` and symlink the dispatcher plus arch
  variants into `/usr/local/bin/`.
- Python: pysam, deeptools, gtars, pydeseq2, anndata, biopython
- Witness: bwa-mem2 indexes an 800bp reference, aligns 2 reads, pysam
  parses the SAM (`2.2.1 2`).
- Traps: bwa-mem2 preallocates a fixed 3.6GB of host RAM regardless of
  reference size — the tier needs `mem_gib≥32`.

### imaging-cpu — medical / slide imaging

- System libs: `libopenslide0 libopenslide-dev libvips42 libgl1
  libglib2.0-0 build-essential`
- Python: pydicom, pylibjpeg, pylibjpeg-libjpeg, openslide-python, pillow,
  scikit-image
- Witness: sobel filter over 128×128 random uint8 → mean 0.2256; pydicom
  imports cleanly.
- Traps: histolab was dropped (pins numpy<1.22). openslide-python needs the
  apt `libopenslide0`; the wheel alone is not enough.

## GPU recipe

### torch-geometric-gpu

Isolated in its own environment for one reason: PyG's compiled wheels lag
each torch release by weeks, and their URL encodes the torch minor version
plus CUDA variant.

- Base: `pytorch/pytorch:2.7.1-cuda12.6-cudnn9-runtime`
- System libs: `git build-essential`
- Install in three ordered phases:
  1. `pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv`
     with `find_links=https://data.pyg.org/whl/torch-2.7.0+cu126.html` —
     that page is flat HTML, not PEP-503, so it must be `find_links`, never
     `extra_index`.
  2. `torch_geometric` — pure Python, no version coupling.
  3. `lightning>=2.2` — Trainer workflows are the usual PyG consumer;
     including it keeps the env self-contained.
- Egress: `github.com raw.githubusercontent.com codeload.github.com
  data.pyg.org` (`torch_geometric.datasets.*` downloads benchmark data).
- Witness: `GCNConv(8→4)` forward returns a `(4,4)` CUDA tensor; a 2-layer
  KarateClub forward+backward shows decreasing loss.

SKILL.md 4.2 KB

---
name: compute-env-setup
description: Set up and validate a reproducible Python or R environment on a Wisp execution context. Use for a selected local, WSL, or direct SSH context when installing scientific packages, configuring caches, recording interpreter activation, or producing an environment smoke test. Do not use for scheduler clusters or managed cloud providers that Wisp cannot track yet.
license: Apache-2.0
---

# Set up a compute environment

Treat the selected and probed `ExecutionContext` as authoritative. Wisp
currently supports `local`, `wsl:<distro>`, and direct `ssh:<alias>` contexts;
it does not expose an authenticated provider SDK inside Python.

## Plan the environment

Define before installing:

- Python or R version;
- ordered conda/pip/R package phases with important pins;
- required CUDA capability and minimum VRAM;
- cache variables and durable weight locations;
- import checks, CLI checks, and one seeded representative workload;
- the exact activation command later Runs must include.

Use `references/envs_reference.md` for package-order and cache examples, but
replace container-specific paths with paths valid on the selected context.

## Direct SSH workflow

1. Require a selected `ssh:<alias>` context with a recent Probe result. Respect
   recorded GPU, privilege, interpreter, conda/mamba, module, and scheduler
   capabilities.
2. If a scheduler is detected, stop. Do not install or run long work on a
   shared login node; Wisp needs a scheduler-aware Run backend first.
3. Use at most a few bounded read-only `shell` commands to confirm free space,
   existing environments, and cache paths.
4. Write an idempotent project script such as
   `runs/setup-<environment>.sh`. It must use user-writable paths, fail fast,
   activate the environment explicitly, run all smoke checks, and write a
   small JSON manifest only after validation succeeds.
5. Submit the setup script through one persisted Run:

```json
{
  "context_id": "ssh:gpu-box",
  "title": "Set up singlecell environment",
  "command": "bash setup-singlecell.sh /home/me/envs/singlecell /home/me/wisp-env-manifests/singlecell.json",
  "timeout_secs": 14400,
  "input_paths": ["runs/setup-singlecell.sh"],
  "output_specs": [
    {
      "glob": "ssh://gpu-box/home/me/wisp-env-manifests/singlecell.json",
      "kind": "environment-manifest",
      "residency": "remote"
    }
  ]
}
```

6. Replace all example paths with probed absolute paths. Call `monitor_run`
   when waiting is useful (again after `wait_interrupted`; do not resubmit).
   Use one `get_run` snapshot later or `cancel_run` when requested.
7. Record the validated activation command, versions, cache paths, GPU witness,
   date, and known limitations in a normal project file such as
   `environments/<context>/<name>.md`. This file is documentation, not a hidden
   resolver.

## Setup-script requirements

- Make repeated execution safe: reuse a matching environment or stop with an
  actionable version mismatch.
- Keep pip install phases ordered; a later dependency resolver must not silently
  replace pinned torch, CUDA, JAX, NumPy, or compiled extensions.
- Never use `sudo` unless the Probe explicitly records suitable privilege and
  the user authorizes it. Prefer conda packages, modules, or user paths.
- Put multi-gigabyte weights in durable remote storage. Populate them with the
  model's real loader, verify non-empty content and completion markers, then run
  a representative inference witness.
- Write the manifest atomically only after imports, GPU visibility, and the
  representative workload pass.

## Local and WSL boundary

Local and WSL Runs are currently capped at 300 seconds and do not support
`input_paths`. Use `local-env-setup` for normal interactive setup. Use
`run_in_context` only for a bounded command that finishes within that limit and
writes outputs to host-visible project paths.

## Unsupported backends

Wisp has no scheduler, Modal, RunPod, cloud Batch, container-service, or managed
endpoint execution context today. Do not invent a provider id or hide those
lifecycles inside an SSH submission command. Explain the boundary or use a
dedicated direct SSH host until a backend implementing submit, poll, cancel,
recovery, secrets, and artifact harvest exists.