Claude Skill

logsql

VictoriaLogs LogsQL query language: filters (word, phrase, prefix, regexp, range, IPv4 range, exact, stream, time), pipe operators (stats, sort, fields, format, extract, unpack_json, unpack_logfmt, math, limit, top, uniq), stats functions, log stream filtering with Prometheus-sty

LLM Mart · 0 points · 0 views 0 listing impressions 0 install-command copies
Download xobotyi-cc-foundry-plugins_grafana_skills_logsql-278c16c.zip · 20 KB

Install

skills CLI npx skills add https://github.com/xobotyi/cc-foundry/tree/master/plugins/grafana/skills/logsql
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install xobotyi-cc-foundry@llmmart
Git git clone https://github.com/xobotyi/cc-foundry.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole xobotyi/cc-foundry collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

LogsQL

VictoriaLogs query language. Three stages: filters select logs, pipes transform, stats aggregate. Master the shape; rest is detail.

References

  • Filters — [${CLAUDE_SKILL_DIR}/references/filters.md] All filter types (word, phrase, prefix, substring, regexp, range, IPv4/IPv6/string range, length range, exact, multi-exact, contains_all/any, subquery, case-insensitive, sequence, value_type, field comparison); _time, _stream, _stream_id; AND/OR/NOT precedence.
  • Pipes — [${CLAUDE_SKILL_DIR}/references/pipes.md] All pipe operators: field manipulation, sorting/limiting, transformation, structured parsing, exploration, stream context, joining.
  • Stats — [${CLAUDE_SKILL_DIR}/references/stats.md] Stats pipe shape, grouping (fields, time buckets, timezone-aware, field buckets, IPv4 buckets, additional filters); all stats functions; running_stats/total_stats; math pipe.
  • Patterns — [${CLAUDE_SKILL_DIR}/references/patterns.md] Recipes (top-N, error counting, latency percentiles, field extraction, rate over time), performance tips, troubleshooting workflows.

Read the relevant reference before proceeding.

The Three-Stage Shape

<filters> | <pipe1> | <pipe2> | ... | <pipeN>
  • Filters select logs. At least one required. Run first; downstream sees only matched logs.
  • Pipes transform. Each reads previous-stage output, emits new rows. Reshape (fields, rename), parse (unpack_json, extract), aggregate (stats), limit (limit, sort).
  • No final stage marker. Last pipe's output is the response.

Simplest valid query — one word, searched in _msg:

error

Returns every log containing error, in arbitrary order, across all time.

The _time Filter — Almost Always Required

Without _time, the query scans all stored logs.

Common forms:

  • _time:5m — last 5 minutes.
  • _time:1h — last hour.
  • _time:1d — last day.
  • _time:2.5d15m42.345s — composite duration.
  • _time:>=2024-01-01 — since.
  • _time:<=2024-01-01 — before.
  • _time:[2024-01-01, 2024-01-02) — closed-open range.
  • _time:2024-03-15T10:00:00+02:00 — exact RFC3339 with timezone.

Grafana plugin and web UI inject the dashboard time range. Don't add _time: in dashboard queries — disables time picker. Do add it for ad-hoc API queries.

Stream Filter — The Performance Lever

Logs belong to streams — Prometheus-style label sets identifying the source. Use {...} braces with PromQL selector syntax:

{app="nginx"}
{app="nginx", instance=~"web-.*"}
{app in ("nginx", "envoy")}
{app not_in ("buggy", "deprecated")}

Optional _stream: prefix: _stream:{app="nginx"}.

Performance: most aggressive pruning available. Restricts which on-disk blocks need reading. A precise stream filter

  • _time range may read 0.1% of what free-text alone would.

Combine _time + {stream} at the top of every production query.

Filter Types

Full catalog: [${CLAUDE_SKILL_DIR}/references/filters.md].

Word Filter

error
log.level:error

Bare word matches _msg. field:word matches named field. Case-sensitive by default; log.level:i(error) for case-insensitive.

Phrase Filter

"error: cannot open file"
"event.original":"cannot open file"

Quoted phrases match literally, whitespace and punctuation included.

Prefix Filter

error*
log.level:error*
"warn"*

* matches anything starting with the prefix.

Exact / Multi-Exact

log.level:="error"
log.level:in("error", "fatal", "panic")

= requires whole-field equality. in(...) matches any of multiple exact values — much faster than regex.

Regexp

~"(?i)error|warn"
log.level:~"^err"
log.level:!~"^debug"

RE2 syntax. !~ negates. Slower than word/exact — use sparingly.

Range Filter

duration_ms:>1000
duration_ms:[100, 1000]
duration_ms:>=500

Numeric. Supports >, >=, <, <=, and [low, high] / (low, high) intervals.

IPv4 Range

client_ip:ipv4_range("10.0.0.0", "10.255.255.255")
client_ip:ipv4_range("192.168.0.0/16")

CIDR or low/high. IPv6: ipv6_range.

Length Range

_msg:len_range(100, 1000)

Byte length of field value. For suspiciously short or long lines.

Logical Composition

error AND _time:5m AND {app="nginx"}
error _time:5m {app="nginx"}              # AND can be omitted
error AND NOT buggy_app
error -buggy_app                          # - is shorthand for NOT
(error OR warn) AND _time:1h
error -(buggy_app OR foobar)

Precedence: NOT > AND > OR. Parenthesize when in doubt — error -buggy_app OR foobar parses as (error AND NOT buggy_app) OR foobar.

The * Wildcard for Fields

Multi-field search by name prefix:

*:error          # any field containing 'error' word
kubernetes.*:nginx   # any field starting with 'kubernetes.' containing 'nginx'

Slows queries — scans more field data. Prefer explicit field names.

Pipe Composition

Pipes chain with |.

fields — Select Output Fields

error _time:5m | fields _time, _stream, _msg

Reduces to named subset — faster transit, easier reading.

sort — Order Results

error _time:5m | sort by (_time)
error _time:5m | sort by (_time desc)

Unordered by default. Expensive — only sort small sets (< few million).

limit / first / last

error _time:5m | sort by (_time) desc | limit 10
error _time:5m | first 10 by (_time desc)
error _time:5m | last 10 by (_time)

limit truncates after sort. first/last combine sort+limit. Aliases: head = limit, skip = offset.

stats — Aggregate

_time:5m | stats count() as total_errors
_time:5m | stats by (_stream) count() as errors_per_stream
_time:5m | stats by (_time:1m) count() as per_minute_errors

stats and as keywords are optional: _time:5m | count() total_errors works.

Most powerful pipe. Grouping by fields, time buckets, field buckets, IPv4 buckets, per-row filters. Full mechanics: [${CLAUDE_SKILL_DIR}/references/stats.md].

unpack_json / unpack_logfmt — Parse Structured Logs

{app="my-app"} | unpack_json from _msg
{app="my-app"} | unpack_logfmt from _msg
{app="my-app"} | unpack_json from _msg | stats by (level) count()

Parses field as JSON/logfmt, flattens into top-level fields. Downstream pipes filter and group by extracted fields.

extract / extract_regexp — Pattern-Based Field Extraction

_msg:"GET" | extract "GET <path> HTTP" from _msg
_msg:"client" | extract_regexp "client (?P<client_ip>[0-9.]+)" from _msg

extract uses <field_name> placeholders. extract_regexp uses RE2 named groups.

math / eval — Numeric Calculations

| math duration_ms / 1000 as duration_sec
| eval (errors / total) * 100 as error_percent

Arithmetic, bitwise, defaults, max/min, abs/ceil/floor/round/exp/ln on numeric field values.

top — N Largest Groups

_time:5m | top 10 (_stream)
_time:5m | top 10 (host, path)

Shortcut for stats by (group) count() | sort desc | limit N.

uniq — Distinct Rows

_time:5m | uniq by (host, path)

Like SQL SELECT DISTINCT. One row per unique field combination.

Common Query Recipes

# Recent error logs from one service
_time:5m {app="api"} error | sort by (_time) desc | limit 100

# Error count per service over last hour
_time:1h error | stats by (_stream) count() as errors | sort by (errors) desc

# Per-minute error rate
_time:1h error | stats by (_time:1m) count() as errors_per_min

# Top 10 noisiest streams
_time:5m | stats by (_stream) count() as logs | sort by (logs) desc | limit 10
# Equivalent:
_time:5m | top 10 (_stream)

# Latency percentiles per endpoint
_time:1h {app="api"} | unpack_json from _msg
  | stats by (path) quantile(0.5, duration_ms) p50, quantile(0.95, duration_ms) p95

# Errors from JSON logs, parsed and filtered
{app="api"} _time:5m | unpack_json from _msg
  | filter level:error
  | stats by (component) count()

Stats Functions — Quick Reference

Most common; full catalog: [${CLAUDE_SKILL_DIR}/references/stats.md].

  • count() — row count.
  • count(field) — rows with non-empty field.
  • count_empty(field) — rows with empty/missing field.
  • count_uniq(field, ...) — distinct combinations. Memory scales with cardinality.
  • count_uniq_hash(field, ...) — cheaper approximate distinct via hashing.
  • sum, avg, min, max, median of field.
  • quantile(phi, field) — phi in [0..1].
  • uniq_values(field) — distinct values as JSON array.
  • values(field) — all values including duplicates.
  • field_min(target, sort) / field_max(target, sort) — target value for smallest/largest sort row.
  • row_min(sort) / row_max(sort) — full row at min/max.
  • row_any() — arbitrary row per group.
  • rate() — per-second rate of matching rows.
  • histogram(field) — histogram of numeric values.
  • stddev(field).

Grouping in Stats

By Fields

| stats by (host, path) count() as logs

by keyword optional.

By Time Buckets

| stats by (_time:1m) count() as per_minute
| stats by (_time:5m, host) count() as per_5m_per_host

Bucket = any duration. Named buckets: nanosecond, microsecond, millisecond, second, minute, hour, day, week, month, year (month/year account for variable lengths).

Time Buckets with Timezone

| stats by (_time:1d offset 'America/New_York') count() as per_day_ny

Aligned to the named timezone, not UTC. Needed for daily/weekly stats in non-UTC offices.

By Field Buckets

| stats by (duration_ms:100) count() as histogram

Buckets numeric values into ranges. duration_ms:100 → [0,100), [100,200), etc.

With Additional Filters

| stats
    count() as total,
    count() if (level:error) as errors,
    count() if (level:warn) as warns

Per-aggregate filtering. One pass beats N separate queries.

Performance Tips

  • Always include _time. Otherwise every block is candidate for scan.
  • Use {...} stream filters when possible. Block-level pruning — orders of magnitude faster than field-value.
  • Exact over regex. log.level:="error" is much faster than log.level:~"^err".
  • Filter before aggregating. Push filters left.
  • Avoid sorting > 10M rows. Combine with limit or use first N by (...).
  • count_uniq_hash over count_uniq for high cardinality when ±1% is acceptable.
  • fields early to drop unused columns.
  • Stream filter at top level. Multiple stream filters work but are less optimized.

Query Options

_time:5m error | options concurrency=4, parallel_readers=8
  • concurrency=N — CPU cores.
  • parallel_readers=N — parallel block readers.
  • ignore_global_time_filter — bypass dashboard time (mostly absolute-time alert rules).
  • allow_partial_response — partial results on timeout instead of erroring.
  • time_offset=duration — shift query time perception.
  • global_filter — applies a filter to every subquery.

Application

When writing:

  • Start with _time + {stream} + content filter. Avoid free-text-only in production.
  • unpack_json once near top; downstream references are cheap.
  • Group by _stream for per-service rollups.
  • Parenthesize OR inside negation: error -(a OR b), not error -a OR b.

When debugging:

  • Drop pipes, run filter alone, sample 10 results.
  • Add pipes one at a time with | limit 10 to keep intermediate small.
  • For unexpected stats: | limit 5 before stats to inspect input.
  • Use field_names and field_values to enumerate what's available.

When reviewing:

  • Flag missing _time in production dashboards.
  • Flag bare regex without exact pre-filter.
  • Flag sort without preceding limit or stream filter.

Integration

metricsql and promql — Prometheus metric queries. Cross-correlate at the stream level: log streams use the same label selectors.

observability — general logging discipline. This skill — LogsQL syntax and performance.

Files (cc-foundry)
  • .dev
    • reference-inventory.json 1.7 KB
      Loading…
  • references
    • filters.md 9.9 KB
      Loading…
    • patterns.md 11 KB
      Loading…
    • pipes.md 9.4 KB
      Loading…
    • stats.md 8.3 KB
      Loading…
  • SKILL.md 12.6 KB
    ---
    name: logsql
    description: >-
      VictoriaLogs LogsQL query language: filters (word, phrase, prefix, regexp, range, IPv4 range,
      exact, stream, time), pipe operators (stats, sort, fields, format, extract, unpack_json,
      unpack_logfmt, math, limit, top, uniq), stats functions, log stream filtering with
      Prometheus-style label selectors, field extraction, and performance patterns. Invoke whenever
      task involves any interaction with LogsQL — writing, debugging, optimizing, or reviewing
      VictoriaLogs queries.
    ---
    
    # LogsQL
    
    VictoriaLogs query language. Three stages: **filters** select logs, **pipes** transform, **stats** aggregate. Master the
    shape; rest is detail.
    
    ## References
    
    - **Filters** — [`${CLAUDE_SKILL_DIR}/references/filters.md`] All filter types (word, phrase, prefix, substring, regexp,
      range, IPv4/IPv6/string range, length range, exact, multi-exact, contains_all/any, subquery, case-insensitive,
      sequence, value_type, field comparison); `_time`, `_stream`, `_stream_id`; AND/OR/NOT precedence.
    - **Pipes** — [`${CLAUDE_SKILL_DIR}/references/pipes.md`] All pipe operators: field manipulation, sorting/limiting,
      transformation, structured parsing, exploration, stream context, joining.
    - **Stats** — [`${CLAUDE_SKILL_DIR}/references/stats.md`] Stats pipe shape, grouping (fields, time buckets,
      timezone-aware, field buckets, IPv4 buckets, additional filters); all stats functions; `running_stats`/`total_stats`;
      math pipe.
    - **Patterns** — [`${CLAUDE_SKILL_DIR}/references/patterns.md`] Recipes (top-N, error counting, latency percentiles,
      field extraction, rate over time), performance tips, troubleshooting workflows.
    
    Read the relevant reference before proceeding.
    
    ## The Three-Stage Shape
    
    ```text
    <filters> | <pipe1> | <pipe2> | ... | <pipeN>
    ```
    
    - **Filters select logs.** At least one required. Run first; downstream sees only matched logs.
    - **Pipes transform.** Each reads previous-stage output, emits new rows. Reshape (`fields`, `rename`), parse
      (`unpack_json`, `extract`), aggregate (`stats`), limit (`limit`, `sort`).
    - **No final stage marker.** Last pipe's output is the response.
    
    Simplest valid query — one word, searched in `_msg`:
    
    ```text
    error
    ```
    
    Returns every log containing `error`, in arbitrary order, across all time.
    
    ## The `_time` Filter — Almost Always Required
    
    Without `_time`, the query scans all stored logs.
    
    Common forms:
    
    - `_time:5m` — last 5 minutes.
    - `_time:1h` — last hour.
    - `_time:1d` — last day.
    - `_time:2.5d15m42.345s` — composite duration.
    - `_time:>=2024-01-01` — since.
    - `_time:<=2024-01-01` — before.
    - `_time:[2024-01-01, 2024-01-02)` — closed-open range.
    - `_time:2024-03-15T10:00:00+02:00` — exact RFC3339 with timezone.
    
    Grafana plugin and web UI inject the dashboard time range. **Don't** add `_time:` in dashboard queries — disables time
    picker. **Do** add it for ad-hoc API queries.
    
    ## Stream Filter — The Performance Lever
    
    Logs belong to **streams** — Prometheus-style label sets identifying the source. Use `{...}` braces with PromQL selector
    syntax:
    
    ```text
    {app="nginx"}
    {app="nginx", instance=~"web-.*"}
    {app in ("nginx", "envoy")}
    {app not_in ("buggy", "deprecated")}
    ```
    
    Optional `_stream:` prefix: `_stream:{app="nginx"}`.
    
    **Performance:** most aggressive pruning available. Restricts which on-disk blocks need reading. A precise stream filter
    
    - `_time` range may read 0.1% of what free-text alone would.
    
    Combine `_time` + `{stream}` at the top of every production query.
    
    ## Filter Types
    
    Full catalog: [`${CLAUDE_SKILL_DIR}/references/filters.md`].
    
    ### Word Filter
    
    ```text
    error
    log.level:error
    ```
    
    Bare word matches `_msg`. `field:word` matches named field. Case-sensitive by default; `log.level:i(error)` for
    case-insensitive.
    
    ### Phrase Filter
    
    ```text
    "error: cannot open file"
    "event.original":"cannot open file"
    ```
    
    Quoted phrases match literally, whitespace and punctuation included.
    
    ### Prefix Filter
    
    ```text
    error*
    log.level:error*
    "warn"*
    ```
    
    `*` matches anything starting with the prefix.
    
    ### Exact / Multi-Exact
    
    ```text
    log.level:="error"
    log.level:in("error", "fatal", "panic")
    ```
    
    `=` requires whole-field equality. `in(...)` matches any of multiple exact values — much faster than regex.
    
    ### Regexp
    
    ```text
    ~"(?i)error|warn"
    log.level:~"^err"
    log.level:!~"^debug"
    ```
    
    RE2 syntax. `!~` negates. Slower than word/exact — use sparingly.
    
    ### Range Filter
    
    ```text
    duration_ms:>1000
    duration_ms:[100, 1000]
    duration_ms:>=500
    ```
    
    Numeric. Supports `>`, `>=`, `<`, `<=`, and `[low, high]` / `(low, high)` intervals.
    
    ### IPv4 Range
    
    ```text
    client_ip:ipv4_range("10.0.0.0", "10.255.255.255")
    client_ip:ipv4_range("192.168.0.0/16")
    ```
    
    CIDR or low/high. IPv6: `ipv6_range`.
    
    ### Length Range
    
    ```text
    _msg:len_range(100, 1000)
    ```
    
    Byte length of field value. For suspiciously short or long lines.
    
    ### Logical Composition
    
    ```text
    error AND _time:5m AND {app="nginx"}
    error _time:5m {app="nginx"}              # AND can be omitted
    error AND NOT buggy_app
    error -buggy_app                          # - is shorthand for NOT
    (error OR warn) AND _time:1h
    error -(buggy_app OR foobar)
    ```
    
    Precedence: NOT > AND > OR. Parenthesize when in doubt — `error -buggy_app OR foobar` parses as
    `(error AND NOT buggy_app) OR foobar`.
    
    ## The `*` Wildcard for Fields
    
    Multi-field search by name prefix:
    
    ```text
    *:error          # any field containing 'error' word
    kubernetes.*:nginx   # any field starting with 'kubernetes.' containing 'nginx'
    ```
    
    Slows queries — scans more field data. Prefer explicit field names.
    
    ## Pipe Composition
    
    Pipes chain with `|`.
    
    ### `fields` — Select Output Fields
    
    ```text
    error _time:5m | fields _time, _stream, _msg
    ```
    
    Reduces to named subset — faster transit, easier reading.
    
    ### `sort` — Order Results
    
    ```text
    error _time:5m | sort by (_time)
    error _time:5m | sort by (_time desc)
    ```
    
    Unordered by default. Expensive — only sort small sets (< few million).
    
    ### `limit` / `first` / `last`
    
    ```text
    error _time:5m | sort by (_time) desc | limit 10
    error _time:5m | first 10 by (_time desc)
    error _time:5m | last 10 by (_time)
    ```
    
    `limit` truncates after sort. `first`/`last` combine sort+limit. Aliases: `head` = `limit`, `skip` = `offset`.
    
    ### `stats` — Aggregate
    
    ```text
    _time:5m | stats count() as total_errors
    _time:5m | stats by (_stream) count() as errors_per_stream
    _time:5m | stats by (_time:1m) count() as per_minute_errors
    ```
    
    `stats` and `as` keywords are optional: `_time:5m | count() total_errors` works.
    
    Most powerful pipe. Grouping by fields, time buckets, field buckets, IPv4 buckets, per-row filters. Full mechanics:
    [`${CLAUDE_SKILL_DIR}/references/stats.md`].
    
    ### `unpack_json` / `unpack_logfmt` — Parse Structured Logs
    
    ```text
    {app="my-app"} | unpack_json from _msg
    {app="my-app"} | unpack_logfmt from _msg
    {app="my-app"} | unpack_json from _msg | stats by (level) count()
    ```
    
    Parses field as JSON/logfmt, flattens into top-level fields. Downstream pipes filter and group by extracted fields.
    
    ### `extract` / `extract_regexp` — Pattern-Based Field Extraction
    
    ```text
    _msg:"GET" | extract "GET <path> HTTP" from _msg
    _msg:"client" | extract_regexp "client (?P<client_ip>[0-9.]+)" from _msg
    ```
    
    `extract` uses `<field_name>` placeholders. `extract_regexp` uses RE2 named groups.
    
    ### `math` / `eval` — Numeric Calculations
    
    ```text
    | math duration_ms / 1000 as duration_sec
    | eval (errors / total) * 100 as error_percent
    ```
    
    Arithmetic, bitwise, defaults, max/min, abs/ceil/floor/round/exp/ln on numeric field values.
    
    ### `top` — N Largest Groups
    
    ```text
    _time:5m | top 10 (_stream)
    _time:5m | top 10 (host, path)
    ```
    
    Shortcut for `stats by (group) count() | sort desc | limit N`.
    
    ### `uniq` — Distinct Rows
    
    ```text
    _time:5m | uniq by (host, path)
    ```
    
    Like SQL `SELECT DISTINCT`. One row per unique field combination.
    
    ## Common Query Recipes
    
    ```text
    # Recent error logs from one service
    _time:5m {app="api"} error | sort by (_time) desc | limit 100
    
    # Error count per service over last hour
    _time:1h error | stats by (_stream) count() as errors | sort by (errors) desc
    
    # Per-minute error rate
    _time:1h error | stats by (_time:1m) count() as errors_per_min
    
    # Top 10 noisiest streams
    _time:5m | stats by (_stream) count() as logs | sort by (logs) desc | limit 10
    # Equivalent:
    _time:5m | top 10 (_stream)
    
    # Latency percentiles per endpoint
    _time:1h {app="api"} | unpack_json from _msg
      | stats by (path) quantile(0.5, duration_ms) p50, quantile(0.95, duration_ms) p95
    
    # Errors from JSON logs, parsed and filtered
    {app="api"} _time:5m | unpack_json from _msg
      | filter level:error
      | stats by (component) count()
    ```
    
    ## Stats Functions — Quick Reference
    
    Most common; full catalog: [`${CLAUDE_SKILL_DIR}/references/stats.md`].
    
    - **`count()`** — row count.
    - **`count(field)`** — rows with non-empty `field`.
    - **`count_empty(field)`** — rows with empty/missing `field`.
    - **`count_uniq(field, ...)`** — distinct combinations. Memory scales with cardinality.
    - **`count_uniq_hash(field, ...)`** — cheaper approximate distinct via hashing.
    - **`sum`**, **`avg`**, **`min`**, **`max`**, **`median`** of field.
    - **`quantile(phi, field)`** — phi in `[0..1]`.
    - **`uniq_values(field)`** — distinct values as JSON array.
    - **`values(field)`** — all values including duplicates.
    - **`field_min(target, sort)`** / **`field_max(target, sort)`** — `target` value for smallest/largest `sort` row.
    - **`row_min(sort)`** / **`row_max(sort)`** — full row at min/max.
    - **`row_any()`** — arbitrary row per group.
    - **`rate()`** — per-second rate of matching rows.
    - **`histogram(field)`** — histogram of numeric values.
    - **`stddev(field)`**.
    
    ## Grouping in Stats
    
    ### By Fields
    
    ```text
    | stats by (host, path) count() as logs
    ```
    
    `by` keyword optional.
    
    ### By Time Buckets
    
    ```text
    | stats by (_time:1m) count() as per_minute
    | stats by (_time:5m, host) count() as per_5m_per_host
    ```
    
    Bucket = any duration. Named buckets: `nanosecond`, `microsecond`, `millisecond`, `second`, `minute`, `hour`, `day`,
    `week`, `month`, `year` (month/year account for variable lengths).
    
    ### Time Buckets with Timezone
    
    ```text
    | stats by (_time:1d offset 'America/New_York') count() as per_day_ny
    ```
    
    Aligned to the named timezone, not UTC. Needed for daily/weekly stats in non-UTC offices.
    
    ### By Field Buckets
    
    ```text
    | stats by (duration_ms:100) count() as histogram
    ```
    
    Buckets numeric values into ranges. `duration_ms:100` → `[0,100)`, `[100,200)`, etc.
    
    ### With Additional Filters
    
    ```text
    | stats
        count() as total,
        count() if (level:error) as errors,
        count() if (level:warn) as warns
    ```
    
    Per-aggregate filtering. One pass beats N separate queries.
    
    ## Performance Tips
    
    - **Always include `_time`.** Otherwise every block is candidate for scan.
    - **Use `{...}` stream filters when possible.** Block-level pruning — orders of magnitude faster than field-value.
    - **Exact over regex.** `log.level:="error"` is much faster than `log.level:~"^err"`.
    - **Filter before aggregating.** Push filters left.
    - **Avoid sorting > 10M rows.** Combine with `limit` or use `first N by (...)`.
    - **`count_uniq_hash` over `count_uniq`** for high cardinality when ±1% is acceptable.
    - **`fields` early** to drop unused columns.
    - **Stream filter at top level.** Multiple stream filters work but are less optimized.
    
    ## Query Options
    
    ```text
    _time:5m error | options concurrency=4, parallel_readers=8
    ```
    
    - **`concurrency=N`** — CPU cores.
    - **`parallel_readers=N`** — parallel block readers.
    - **`ignore_global_time_filter`** — bypass dashboard time (mostly absolute-time alert rules).
    - **`allow_partial_response`** — partial results on timeout instead of erroring.
    - **`time_offset=duration`** — shift query time perception.
    - **`global_filter`** — applies a filter to every subquery.
    
    ## Application
    
    When **writing**:
    
    - Start with `_time` + `{stream}` + content filter. Avoid free-text-only in production.
    - `unpack_json` once near top; downstream references are cheap.
    - Group by `_stream` for per-service rollups.
    - Parenthesize OR inside negation: `error -(a OR b)`, not `error -a OR b`.
    
    When **debugging**:
    
    - Drop pipes, run filter alone, sample 10 results.
    - Add pipes one at a time with `| limit 10` to keep intermediate small.
    - For unexpected stats: `| limit 5` before `stats` to inspect input.
    - Use `field_names` and `field_values` to enumerate what's available.
    
    When **reviewing**:
    
    - Flag missing `_time` in production dashboards.
    - Flag bare regex without exact pre-filter.
    - Flag `sort` without preceding `limit` or stream filter.
    
    ## Integration
    
    **metricsql** and **promql** — Prometheus metric queries. Cross-correlate at the stream level: log streams use the same
    label selectors.
    
    **observability** — general logging discipline. This skill — LogsQL syntax and performance.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.