Claude
Skill
data-processing
Process JSON with jq and YAML/TOML with yq. Filter, transform, query structured data efficiently. Triggers on: parse JSON, extract from YAML, query config, Docker Compose, K8s manifests, GitHub Actions workflows, package.json, filter data.
Virus-scanned
Reviewed automatically before listing.
Download
0xdarkmatter-claude-mods-skills_data-processing-3dfaf0b.zip · 4 KB
Install
skills CLI
npx skills add https://github.com/0xDarkMatter/claude-mods/tree/main/skills/data-processing
Claude Code
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install 0xdarkmatter-claude-mods@llmmart
Git
git clone https://github.com/0xDarkMatter/claude-mods.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole 0xdarkmatter/claude-mods collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Data Processing
Query, filter, and transform structured data (JSON, YAML, TOML) efficiently from the command line.
Tools
| Tool | Command | Use For |
|---|---|---|
| jq | jq '.key' file.json |
JSON processing |
| yq | yq '.key' file.yaml |
YAML/TOML processing |
jq Essentials
# Extract single field
jq '.name' package.json
# Extract nested field
jq '.scripts.build' package.json
# Extract from array
jq '.dependencies[0]' package.json
# Extract multiple fields
jq '{name, version}' package.json
# Navigate deeply nested
jq '.data.users[0].profile.email' response.json
# Filter by condition
jq '.users[] | select(.active == true)' data.json
# Transform each element
jq '.users | map({id, name})' data.json
# Count elements
jq '.users | length' data.json
# Raw string output
jq -r '.name' package.json
yq Essentials
# Extract field
yq '.name' config.yaml
# Extract nested
yq '.services.web.image' docker-compose.yml
# List all keys
yq 'keys' config.yaml
# List all service names (Docker Compose)
yq '.services | keys' docker-compose.yml
# Get container images (K8s)
yq '.spec.template.spec.containers[].image' deployment.yaml
# Update value (in-place)
yq -i '.version = "2.0.0"' config.yaml
# TOML to JSON
yq -p toml -o json '.' config.toml
Quick Reference
| Task | jq | yq |
|---|---|---|
| Get field | jq '.key' |
yq '.key' |
| Array element | jq '.[0]' |
yq '.[0]' |
| Filter array | jq '.[] \| select(.x)' |
yq '.[] \| select(.x)' |
| Transform | jq 'map(.x)' |
yq 'map(.x)' |
| Count | jq 'length' |
yq 'length' |
| Keys | jq 'keys' |
yq 'keys' |
| Pretty print | jq '.' |
yq '.' |
| Compact | jq -c |
yq -o json -I0 |
| Raw output | jq -r |
yq -r |
| In-place edit | - | yq -i |
When to Use
- Reading package.json dependencies
- Parsing Docker Compose configurations
- Analyzing Kubernetes manifests
- Processing GitHub Actions workflows
- Extracting data from API responses
- Filtering large JSON datasets
- Config file manipulation
- Data format conversion
Additional Resources
For complete pattern libraries, load:
./references/jq-patterns.md- Arrays, filtering, transformation, aggregation, output formatting./references/yq-patterns.md- Docker Compose, K8s, GitHub Actions, TOML, YAML modification./references/config-files.md- package.json, tsconfig, eslint/prettier patterns
Files (claude-mods)
-
assets
-
.gitkeep 0 B · in bundle
-
-
references
-
config-files.md 794 B
# Config File Patterns Common patterns for processing configuration files. ## package.json ```bash # List all dependencies jq '.dependencies | keys' package.json # Get all scripts jq '.scripts' package.json # Find outdated patterns jq '.dependencies | to_entries | map(select(.value | startswith("^")))' package.json # Extract dev dependencies jq '.devDependencies | keys | .[]' package.json ``` ## tsconfig.json ```bash # Get compiler options jq '.compilerOptions' tsconfig.json # Check strict mode jq '.compilerOptions.strict' tsconfig.json # List paths aliases jq '.compilerOptions.paths' tsconfig.json ``` ## ESLint/Prettier ```bash # Get enabled rules jq '.rules | to_entries | map(select(.value != "off"))' .eslintrc.json # Check prettier options jq '.' .prettierrc.json ``` -
jq-patterns.md 3.2 KB
# jq Patterns Reference Complete jq patterns for JSON processing. ## Array Operations ```bash # Get all array elements jq '.users[]' data.json # Get specific index jq '.users[0]' data.json # Slice array jq '.users[0:3]' data.json # First 3 elements jq '.users[-2:]' data.json # Last 2 elements # Array length jq '.users | length' data.json # Get array of specific field jq '.users[].name' data.json # Wrap results in array jq '[.users[].name]' data.json ``` ## Filtering with select ```bash # Filter by condition jq '.users[] | select(.active == true)' data.json # Multiple conditions jq '.users[] | select(.age > 21 and .status == "active")' data.json # String contains jq '.users[] | select(.email | contains("@gmail"))' data.json # Regex match jq '.users[] | select(.email | test("@(gmail|yahoo)"))' data.json # Not null check jq '.users[] | select(.profile != null)' data.json ``` ## Transformation with map ```bash # Transform each element jq '.users | map({id, name})' data.json # Add computed field jq '.users | map(. + {full_name: (.first + " " + .last)})' data.json # Filter and transform jq '.users | map(select(.active)) | map(.email)' data.json # map_values for objects jq '.config | map_values(. * 2)' data.json ``` ## Object Manipulation ```bash # Add/update field jq '.version = "2.0.0"' package.json # Delete field jq 'del(.devDependencies)' package.json # Rename key jq '.dependencies | to_entries | map(.key |= gsub("@"; ""))' package.json # Merge objects jq '. + {newField: "value"}' data.json # Update nested field jq '.scripts.test = "jest --coverage"' package.json # Conditional update jq 'if .version == "1.0.0" then .version = "1.0.1" else . end' package.json ``` ## Aggregation ```bash # Count jq '.users | length' data.json # Sum jq '[.items[].price] | add' data.json # Min/Max jq '[.scores[]] | min' data.json jq '[.scores[]] | max' data.json # Average jq '[.scores[]] | add / length' data.json # Group by jq 'group_by(.category) | map({category: .[0].category, count: length})' data.json # Unique values jq '[.users[].role] | unique' data.json # Sort jq '.users | sort_by(.created_at)' data.json jq '.users | sort_by(.name) | reverse' data.json ``` ## Output Formatting ```bash # Pretty print jq '.' response.json # Compact output (single line) jq -c '.results[]' data.json # Raw strings (no quotes) jq -r '.name' package.json # Tab-separated output jq -r '.users[] | [.id, .name, .email] | @tsv' data.json # CSV output jq -r '.users[] | [.id, .name, .email] | @csv' data.json # URI encoding jq -r '.query | @uri' data.json ``` ## Advanced Patterns ```bash # Process multiple files for f in *.json; do jq '.name' "$f"; done # Pipeline with other tools curl -s https://api.github.com/users/octocat | jq '.login' # Assign to variable VERSION=$(jq -r '.version' package.json) # Conditional logic jq -e '.errors | length == 0' response.json && echo "Success" # Flatten nested structure jq '[.categories[].items[]] | flatten' data.json # Reshape data jq '.users | map({(.id | tostring): .name}) | add' data.json # Pivot data jq 'group_by(.date) | map({date: .[0].date, values: map(.value)})' data.json # Join arrays jq -s '.[0] + .[1]' file1.json file2.json ``` -
yq-patterns.md 1.9 KB
# yq Patterns Reference Complete yq patterns for YAML/TOML processing. ## Basic YAML Operations ```bash # Extract field yq '.name' config.yaml # Extract nested yq '.services.web.image' docker-compose.yml # List all keys yq 'keys' config.yaml # Get array element yq '.volumes[0]' docker-compose.yml ``` ## Docker Compose Queries ```bash # List all service names yq '.services | keys' docker-compose.yml # Get all images yq '.services[].image' docker-compose.yml # Get environment variables for a service yq '.services.web.environment' docker-compose.yml # Find services with specific image yq '.services | to_entries | map(select(.value.image | contains("nginx")))' docker-compose.yml ``` ## Kubernetes Manifests ```bash # Get resource name yq '.metadata.name' deployment.yaml # Get container images yq '.spec.template.spec.containers[].image' deployment.yaml # Get all labels yq '.metadata.labels' deployment.yaml # Multi-document YAML (---) yq eval-all '.metadata.name' manifests.yaml ``` ## GitHub Actions Workflows ```bash # List all jobs yq '.jobs | keys' .github/workflows/ci.yml # Get steps for a job yq '.jobs.build.steps[].name' .github/workflows/ci.yml # Find jobs using specific action yq '.jobs[].steps[] | select(.uses | contains("actions/checkout"))' .github/workflows/ci.yml # Get all environment variables yq '.env' .github/workflows/ci.yml ``` ## TOML Processing ```bash # Read TOML file yq -p toml '.dependencies' Cargo.toml # Convert TOML to JSON yq -p toml -o json '.' config.toml # Extract pyproject.toml dependencies yq -p toml '.project.dependencies[]' pyproject.toml ``` ## YAML Modification ```bash # Update value (in-place) yq -i '.version = "2.0.0"' config.yaml # Add new field yq -i '.new_field = "value"' config.yaml # Delete field yq -i 'del(.old_field)' config.yaml # Add to array yq -i '.tags += ["new-tag"]' config.yaml # Merge YAML files yq eval-all 'select(fileIndex == 0) * select(fileIndex == 1)' base.yaml override.yaml ```
-
-
scripts
-
.gitkeep 0 B · in bundle
-
-
SKILL.md 2.8 KB
--- name: data-processing description: "Process JSON with jq and YAML/TOML with yq. Filter, transform, query structured data efficiently. Triggers on: parse JSON, extract from YAML, query config, Docker Compose, K8s manifests, GitHub Actions workflows, package.json, filter data." license: MIT compatibility: "Requires jq and yq CLI tools. Install: brew install jq yq (macOS)." allowed-tools: "Bash Read" metadata: author: claude-mods --- # Data Processing Query, filter, and transform structured data (JSON, YAML, TOML) efficiently from the command line. ## Tools | Tool | Command | Use For | |------|---------|---------| | jq | `jq '.key' file.json` | JSON processing | | yq | `yq '.key' file.yaml` | YAML/TOML processing | ## jq Essentials ```bash # Extract single field jq '.name' package.json # Extract nested field jq '.scripts.build' package.json # Extract from array jq '.dependencies[0]' package.json # Extract multiple fields jq '{name, version}' package.json # Navigate deeply nested jq '.data.users[0].profile.email' response.json # Filter by condition jq '.users[] | select(.active == true)' data.json # Transform each element jq '.users | map({id, name})' data.json # Count elements jq '.users | length' data.json # Raw string output jq -r '.name' package.json ``` ## yq Essentials ```bash # Extract field yq '.name' config.yaml # Extract nested yq '.services.web.image' docker-compose.yml # List all keys yq 'keys' config.yaml # List all service names (Docker Compose) yq '.services | keys' docker-compose.yml # Get container images (K8s) yq '.spec.template.spec.containers[].image' deployment.yaml # Update value (in-place) yq -i '.version = "2.0.0"' config.yaml # TOML to JSON yq -p toml -o json '.' config.toml ``` ## Quick Reference | Task | jq | yq | |------|----|----| | Get field | `jq '.key'` | `yq '.key'` | | Array element | `jq '.[0]'` | `yq '.[0]'` | | Filter array | `jq '.[] \| select(.x)'` | `yq '.[] \| select(.x)'` | | Transform | `jq 'map(.x)'` | `yq 'map(.x)'` | | Count | `jq 'length'` | `yq 'length'` | | Keys | `jq 'keys'` | `yq 'keys'` | | Pretty print | `jq '.'` | `yq '.'` | | Compact | `jq -c` | `yq -o json -I0` | | Raw output | `jq -r` | `yq -r` | | In-place edit | - | `yq -i` | ## When to Use - Reading package.json dependencies - Parsing Docker Compose configurations - Analyzing Kubernetes manifests - Processing GitHub Actions workflows - Extracting data from API responses - Filtering large JSON datasets - Config file manipulation - Data format conversion ## Additional Resources For complete pattern libraries, load: - `./references/jq-patterns.md` - Arrays, filtering, transformation, aggregation, output formatting - `./references/yq-patterns.md` - Docker Compose, K8s, GitHub Actions, TOML, YAML modification - `./references/config-files.md` - package.json, tsconfig, eslint/prettier patterns
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.