grok-search
Enhanced web search and real-time content retrieval via Grok API with forced tool routing. Use when: (1) Web search / information retrieval / fact-checking, (2) Webpage content extraction / URL parsing, (3) Breaking knowledge cutoff limits for current information, (4) Real-time n
Install
npx skills add https://github.com/Dianel555/DSkills/tree/main/skills/grok-search
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install dianel555-dskills@llmmart
git clone https://github.com/Dianel555/DSkills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole dianel555/dskills collection as a plugin from our marketplace. Git is the plain clone.
README
GrokSearch CLI
Standalone command-line interface for Grok web search. No MCP dependency required.
Installation
pip install httpx tenacity
Layout
groksearch_cli.py # CLI entrypoint and compatibility facade
groksearch/ # Internal implementation modules
cli.py # argparse wiring
commands.py # command handlers
config.py # environment and persisted config
http.py # shared client and retry helpers
provider.py # Grok OpenAI-compatible provider
tavily.py # Tavily search/extract/map calls
formatting.py # JSON extraction and result merging
Configuration
Option 1: .env File (Recommended)
Create a .env file in the scripts directory:
cp .env.example .env
Edit .env:
GROK_API_URL=https://your-api-endpoint.com/v1
GROK_API_KEY=your-api-key-here
Option 2: Environment Variables
export GROK_API_URL="https://your-api-endpoint.com/v1"
export GROK_API_KEY="your-api-key-here"
export TAVILY_API_KEY="your-tavily-key" # optional
Option 3: Command Line Arguments
python groksearch_cli.py --api-url "https://..." --api-key "sk-..." web_search -q "query"
Commands
web_search - Web Search
python groksearch_cli.py web_search --query "search terms" [options]
Options:
-q, --query Search query (required)
-p, --platform Focus platforms, e.g., "GitHub,Reddit"
--min-results Minimum results (default: 3)
--max-results Maximum results (default: 10)
--extra-sources Additional Tavily results to merge (default: 0)
--raw Output raw response without JSON parsing
Example:
python groksearch_cli.py web_search -q "latest Python 3.12 features" --max-results 5
web_fetch - Fetch Webpage Content
python groksearch_cli.py web_fetch --url "https://..." [options]
Options:
-u, --url URL to fetch (required)
-o, --out Output file path (optional)
--via Fetch backend: grok|tavily (default: grok)
Example:
python groksearch_cli.py web_fetch -u "https://docs.python.org/3/whatsnew/3.12.html" -o python312.md
web_map - Map Website Structure
python groksearch_cli.py web_map --url "https://..." [options]
Options:
-u, --url Root URL to map (required)
--instructions Natural language filter for crawler
--max-depth Max traversal depth (default: 1)
--max-breadth Max links per page (default: 20)
--limit Total link limit (default: 50)
--timeout Operation timeout in seconds (default: 150)
get_config_info - Check Configuration
python groksearch_cli.py get_config_info [options]
Options:
--no-test Skip connection test
switch_model - Switch Grok Model
python groksearch_cli.py switch_model --model "model-id"
Options:
-m, --model Model ID to switch to (required)
Example:
python groksearch_cli.py switch_model -m "grok-2-latest"
toggle_builtin_tools - Toggle Built-in Tools
python groksearch_cli.py toggle_builtin_tools [options]
Options:
-a, --action Action: on/off/status (default: status)
-r, --root Project root path (default: auto-detect via .git)
Example:
# Disable built-in WebSearch/WebFetch
python groksearch_cli.py toggle_builtin_tools -a on
# Enable built-in tools
python groksearch_cli.py toggle_builtin_tools -a off
# Check status
python groksearch_cli.py toggle_builtin_tools -a status
Output Format
web_search: JSON array[{title, url, description, provider?}]web_fetch: Structured Markdownweb_map: JSON object{base_url, results, response_time}web_crawl: JSON object{base_url, results, response_time, usage?}web_research: JSON object{request_id, status, content, sources, usage?}- Other commands: JSON object
.env File Search Order
- Current working directory
- Script directory (
scripts/) - Parent directory of script
Configuration Persistence
- Model settings:
~/.config/grok-search/config.json - Built-in tools toggle:
<project>/.claude/settings.json
Tavily Tuning
Search and extract are tunable via .env (or environment). All are optional; defaults follow
Tavily's own agent guidance.
| Variable | Default | Values | Notes |
|---|---|---|---|
TAVILY_SEARCH_DEPTH |
advanced |
advanced / basic / fast / ultra-fast |
advanced = 2 credits, highest relevance, reaches more sources — best for grounding niche/recent/multi-facet queries. Others = 1 credit. ultra-fast returns one summary per URL instead of reranked chunks. |
TAVILY_CHUNKS_PER_SOURCE |
3 |
1–5 |
Snippets (≤500 chars) per source; joined by [...]. More = stronger evidence per URL. |
TAVILY_TOPIC |
general |
general / news / finance |
news auto-adds published_date. |
TAVILY_TIME_RANGE |
(unset) | day / week / month / year |
Publish-date window. Sources with no detectable date are kept unless filtered. |
TAVILY_INCLUDE_DOMAINS |
(unset) | comma-separated | Restrict or boost to these domains (max 300). |
TAVILY_EXCLUDE_DOMAINS |
(unset) | comma-separated | Drop these domains (max 150). |
TAVILY_INCLUDE_DOMAINS_MODE |
filter |
filter / boost |
boost also searches the wide web, so trusted sources are prioritized without risking empty results. Only sent when TAVILY_INCLUDE_DOMAINS is set. |
TAVILY_INCLUDE_ANSWER |
false |
false / basic / advanced |
LLM-generated answer. Extra cost; off by default. |
TAVILY_INCLUDE_USAGE |
false |
true / false |
Add credit usage to each response. Search attaches it as tavily_usage on the first merged result; map/crawl/research include a usage key in their JSON object. |
TAVILY_EXTRACT_DEPTH |
basic |
basic / advanced |
advanced handles tables, JS-rendered pages, structured data — higher latency and cost. Also used for crawl extraction. |
TAVILY_EXTRACT_TIMEOUT |
30 |
1.0–60.0 |
Tavily-side timeout for web_fetch --via tavily. The HTTP client waits 10s longer so Tavily's diagnosable error wins. Applies per attempt; with retries the total wall time can reach timeout × attempts. |
TAVILY_CRAWL_TIMEOUT |
150 |
10–150 |
Crawl-side timeout (API range). Client waits 10s longer. Applies per attempt; with retries the total wall time can reach timeout × attempts. |
TAVILY_CRAWL_MAX_DEPTH |
1 |
int | Crawl depth. |
TAVILY_CRAWL_MAX_BREADTH |
20 |
int | Links followed per page. |
TAVILY_CRAWL_LIMIT |
50 |
int | Total pages crawled. |
TAVILY_CRAWL_ALLOW_EXTERNAL |
true |
true / false |
false keeps the crawl on one site. |
TAVILY_CRAWL_SELECT_PATHS |
(unset) | comma-separated regexes | Include only matching paths. |
TAVILY_CRAWL_EXCLUDE_PATHS |
(unset) | comma-separated regexes | Exclude matching paths. |
TAVILY_RESEARCH_MODEL |
auto |
mini / pro / auto |
Research agent model. |
TAVILY_RESEARCH_CITATION_FORMAT |
numbered |
numbered / mla / apa / chicago |
Citation style in the report. |
TAVILY_RESEARCH_OUTPUT_LENGTH |
standard |
short / standard / long |
Report length. |
TAVILY_RESEARCH_OUTPUT_SCHEMA |
(unset) | path to a JSON file | Structured output. The file must be a JSON Schema with non-empty properties; each property needs type (object/string/integer/number/array) and description. Invalid schemas are reported before any request is sent. When set, content in the response is an object rather than a string. |
TAVILY_RESEARCH_TIMEOUT |
300 |
seconds | Total polling budget before returning status: timeout (with request_id so the task can be re-fetched). A poll HTTP/network failure returns status: poll_error with the same request_id. |
TAVILY_RESEARCH_POLL_INTERVAL |
5 |
seconds | Delay between status checks. |
Rate limits: 100 RPM (development key) / 1000 RPM (production). A 429 carries a
retry-after header, which the retry logic honors. crawl is capped at 100 RPM and
research at 20 RPM on both tiers.
Example — recent, trusted sources only:
TAVILY_TOPIC=news TAVILY_TIME_RANGE=week \
TAVILY_INCLUDE_DOMAINS="reuters.com,bloomberg.com" TAVILY_INCLUDE_DOMAINS_MODE=boost \
python groksearch_cli.py web_search -q "AI regulation" --extra-sources 5
Invalid values are reported per-key by get_config_info rather than silently ignored.
Acknowledgments
- Based on the original GuDaStudio/GrokSearch.
Skill manifest
Grok Search
Enhanced web search via Grok API. Standalone CLI only (no MCP dependency).
Implementation Layout
scripts/groksearch_cli.py- CLI entrypoint and compatibility facadescripts/groksearch/- internal modules for config, HTTP retry, Grok provider, Tavily calls, formatting, and commands
Execution Methods
Run scripts/groksearch_cli.py via Bash:
# Prerequisites: pip install httpx tenacity
# Environment: GROK_API_URL, GROK_API_KEY (required); TAVILY_API_KEY (optional)
# Web search (Grok only)
python scripts/groksearch_cli.py web_search --query "search terms" [--platform "GitHub"] [--min-results 3] [--max-results 10]
# Web search with Tavily extra sources (parallel + URL-deduplicated merge)
python scripts/groksearch_cli.py web_search --query "..." --extra-sources 5
# Fetch webpage (default: Grok)
python scripts/groksearch_cli.py web_fetch --url "https://..." [--out file.md]
# Fetch via Tavily extract endpoint
python scripts/groksearch_cli.py web_fetch --url "https://..." --via tavily
# Map a website's structure (Tavily)
python scripts/groksearch_cli.py web_map --url "https://docs.example.com" [--instructions "API only"] [--max-depth 2] [--max-breadth 20] [--limit 50] [--timeout 150]
# Crawl a website's pages with extraction (Tavily; own 100 RPM limit)
python scripts/groksearch_cli.py web_crawl --url "https://docs.example.com" [--instructions "API only"] [--max-depth 2] [--limit 50] [--select-paths "/docs/.*"] [--exclude-paths "/blog/.*"] [--timeout 150]
# Run a cited research task (Tavily; async submit + poll; own 20 RPM limit)
python scripts/groksearch_cli.py web_research --input "question to investigate" [--model mini|pro|auto] [--output-length short|standard|long] [--citation-format numbered|mla|apa|chicago] [--output-schema schema.json]
# Check config
python scripts/groksearch_cli.py get_config_info [--no-test]
# Switch model
python scripts/groksearch_cli.py switch_model --model "grok-2-latest"
# Toggle built-in tools
python scripts/groksearch_cli.py toggle_builtin_tools --action on|off|status [--root /path/to/project]
Tool Routing Policy
Forced Replacement Rules
| Scenario | Disabled | Force Use |
|---|---|---|
| Web Search | WebSearch |
CLI web_search |
| Web Fetch | WebFetch |
CLI web_fetch |
Tool Capability Matrix
| Tool | Parameters | Output |
|---|---|---|
web_search |
query(required), platform/min_results/max_results(optional), extra_sources(int, 0=disabled) |
[{title,url,description,provider?}] |
web_fetch |
url(required), out(optional), via(grok|tavily, default grok) |
Structured Markdown |
web_map |
url(required), instructions/max_depth/max_breadth/limit/timeout(optional) |
{base_url,results,response_time} JSON |
web_crawl |
url(required), instructions/max_depth/max_breadth/limit/select_paths/exclude_paths/timeout(optional; unset falls back to TAVILY_CRAWL_*) |
{base_url,results,response_time,usage?} JSON |
web_research |
input(required), model/output_length/citation_format(optional; unset falls back to TAVILY_RESEARCH_*) |
{request_id,status,content,sources,usage?} JSON |
get_config_info |
no_test(optional) |
{api_url,status,connection_test,tavily_*} |
switch_model |
model(required) |
{previous_model,current_model} |
toggle_builtin_tools |
action(on/off/status), root(optional) |
{blocked,deny_list} |
Search Workflow
Phase 1: Query Construction
- Intent Recognition: Broad search →
web_search| Deep retrieval →web_fetch - Parameter Optimization: Set
platformfor specific sources, adjust result counts
Phase 2: Search Execution
- Start with
web_searchfor structured summaries - Use
web_fetchon key URLs if summaries insufficient - Retry with adjusted query if first round unsatisfactory
Phase 3: URL Verification & Hallucination Guard (MANDATORY)
Background: Grok API calls without explicit web-search activation return results from parametric memory, which frequently fabricates URLs (observed 25% liveness rate in testing). All Grok-returned URLs MUST be verified before citation.
Verification Protocol:
- URL Liveness Check: Issue HEAD/GET request to each Grok-returned URL; non-2xx status = unreliable
- Tavily Fallback Triggers (invoke
web_searchwith--extra-sources Nwhen ANY apply):- URL liveness rate < 50% in Grok results
- Query contains version numbers, release dates, API signatures, or "latest"/"recent" temporal markers (high hallucination surface)
- Multiple Grok runs return contradictory URLs for the same factual claim
- Grok result descriptions contain specifics (dates/versions/methods) that cannot be confirmed from live URLs
- Tavily Grounding: When triggered, re-run the same query with
--extra-sources 5-10to obtain Tavily search results; prioritize these over failed Grok URLs - Content Extraction: For critical factual claims, use
web_fetch --via tavilyon verified URLs to extract authoritative source text
Citation Discipline:
- ONLY verified-live URLs may appear in final output
- Fabricated Grok URLs must be dropped entirely (do not present them with a disclaimer; omit them)
- When Tavily sources replace Grok sources, cite Tavily URLs and mark provider as
tavily - For time-sensitive queries with no live sources, state "Unable to verify current information" rather than citing dead links
Phase 4: Result Synthesis
- Cross-reference multiple sources
- Must annotate source and date for time-sensitive info
- Must include source URLs:
Title [<sup>1</sup>](URL)
Error Handling
| Error | Recovery |
|---|---|
| Connection Failure | Run get_config_info, verify API URL/Key |
| No Results | Broaden search terms |
| Fetch Timeout | Try alternative sources |
Anti-Patterns
| Prohibited | Correct |
|---|---|
| No source citation | Include Source [<sup>1</sup>](URL) |
| Give up after one failure | Retry at least once |
| Use built-in WebSearch/WebFetch | Use GrokSearch tools/CLI |
Files (dskills)
-
scripts
-
groksearch
-
cli.py 5 KB
Loading… -
commands.py 9.1 KB
Loading… -
config.py 14 KB
Loading… -
env.py 1.4 KB
Loading… -
formatting.py 3 KB
Loading… -
http.py 2.9 KB
Loading… -
prompts.py 704 B
Loading… -
provider.py 5.7 KB
Loading… -
tavily.py 18 KB
Loading… -
__init__.py 47 B
Loading…
-
-
tests
-
test_groksearch_cli.py 62.8 KB
Loading… -
test_retry_budget.py 703 B
Loading…
-
-
groksearch_cli.py 6.8 KB
Loading…
-
-
.env.example 3.2 KB · in bundle
-
README.md 8.5 KB
# GrokSearch CLI Standalone command-line interface for Grok web search. No MCP dependency required. ## Installation ```bash pip install httpx tenacity ``` ## Layout ```text groksearch_cli.py # CLI entrypoint and compatibility facade groksearch/ # Internal implementation modules cli.py # argparse wiring commands.py # command handlers config.py # environment and persisted config http.py # shared client and retry helpers provider.py # Grok OpenAI-compatible provider tavily.py # Tavily search/extract/map calls formatting.py # JSON extraction and result merging ``` ## Configuration ### Option 1: .env File (Recommended) Create a `.env` file in the scripts directory: ```bash cp .env.example .env ``` Edit `.env`: ``` GROK_API_URL=https://your-api-endpoint.com/v1 GROK_API_KEY=your-api-key-here ``` ### Option 2: Environment Variables ```bash export GROK_API_URL="https://your-api-endpoint.com/v1" export GROK_API_KEY="your-api-key-here" export TAVILY_API_KEY="your-tavily-key" # optional ``` ### Option 3: Command Line Arguments ```bash python groksearch_cli.py --api-url "https://..." --api-key "sk-..." web_search -q "query" ``` ## Commands ### web_search - Web Search ```bash python groksearch_cli.py web_search --query "search terms" [options] Options: -q, --query Search query (required) -p, --platform Focus platforms, e.g., "GitHub,Reddit" --min-results Minimum results (default: 3) --max-results Maximum results (default: 10) --extra-sources Additional Tavily results to merge (default: 0) --raw Output raw response without JSON parsing ``` Example: ```bash python groksearch_cli.py web_search -q "latest Python 3.12 features" --max-results 5 ``` ### web_fetch - Fetch Webpage Content ```bash python groksearch_cli.py web_fetch --url "https://..." [options] Options: -u, --url URL to fetch (required) -o, --out Output file path (optional) --via Fetch backend: grok|tavily (default: grok) ``` Example: ```bash python groksearch_cli.py web_fetch -u "https://docs.python.org/3/whatsnew/3.12.html" -o python312.md ``` ### web_map - Map Website Structure ```bash python groksearch_cli.py web_map --url "https://..." [options] Options: -u, --url Root URL to map (required) --instructions Natural language filter for crawler --max-depth Max traversal depth (default: 1) --max-breadth Max links per page (default: 20) --limit Total link limit (default: 50) --timeout Operation timeout in seconds (default: 150) ``` ### get_config_info - Check Configuration ```bash python groksearch_cli.py get_config_info [options] Options: --no-test Skip connection test ``` ### switch_model - Switch Grok Model ```bash python groksearch_cli.py switch_model --model "model-id" Options: -m, --model Model ID to switch to (required) ``` Example: ```bash python groksearch_cli.py switch_model -m "grok-2-latest" ``` ### toggle_builtin_tools - Toggle Built-in Tools ```bash python groksearch_cli.py toggle_builtin_tools [options] Options: -a, --action Action: on/off/status (default: status) -r, --root Project root path (default: auto-detect via .git) ``` Example: ```bash # Disable built-in WebSearch/WebFetch python groksearch_cli.py toggle_builtin_tools -a on # Enable built-in tools python groksearch_cli.py toggle_builtin_tools -a off # Check status python groksearch_cli.py toggle_builtin_tools -a status ``` ## Output Format - `web_search`: JSON array `[{title, url, description, provider?}]` - `web_fetch`: Structured Markdown - `web_map`: JSON object `{base_url, results, response_time}` - `web_crawl`: JSON object `{base_url, results, response_time, usage?}` - `web_research`: JSON object `{request_id, status, content, sources, usage?}` - Other commands: JSON object ## .env File Search Order 1. Current working directory 2. Script directory (`scripts/`) 3. Parent directory of script ## Configuration Persistence - Model settings: `~/.config/grok-search/config.json` - Built-in tools toggle: `<project>/.claude/settings.json` ## Tavily Tuning Search and extract are tunable via `.env` (or environment). All are optional; defaults follow Tavily's own agent guidance. | Variable | Default | Values | Notes | |----------|---------|--------|-------| | `TAVILY_SEARCH_DEPTH` | `advanced` | `advanced` / `basic` / `fast` / `ultra-fast` | `advanced` = 2 credits, highest relevance, reaches more sources — best for grounding niche/recent/multi-facet queries. Others = 1 credit. `ultra-fast` returns one summary per URL instead of reranked chunks. | | `TAVILY_CHUNKS_PER_SOURCE` | `3` | `1`–`5` | Snippets (≤500 chars) per source; joined by `[...]`. More = stronger evidence per URL. | | `TAVILY_TOPIC` | `general` | `general` / `news` / `finance` | `news` auto-adds `published_date`. | | `TAVILY_TIME_RANGE` | *(unset)* | `day` / `week` / `month` / `year` | Publish-date window. Sources with no detectable date are kept unless filtered. | | `TAVILY_INCLUDE_DOMAINS` | *(unset)* | comma-separated | Restrict or boost to these domains (max 300). | | `TAVILY_EXCLUDE_DOMAINS` | *(unset)* | comma-separated | Drop these domains (max 150). | | `TAVILY_INCLUDE_DOMAINS_MODE` | `filter` | `filter` / `boost` | `boost` also searches the wide web, so trusted sources are prioritized without risking empty results. Only sent when `TAVILY_INCLUDE_DOMAINS` is set. | | `TAVILY_INCLUDE_ANSWER` | `false` | `false` / `basic` / `advanced` | LLM-generated answer. Extra cost; off by default. | | `TAVILY_INCLUDE_USAGE` | `false` | `true` / `false` | Add credit `usage` to each response. Search attaches it as `tavily_usage` on the first merged result; map/crawl/research include a `usage` key in their JSON object. | | `TAVILY_EXTRACT_DEPTH` | `basic` | `basic` / `advanced` | `advanced` handles tables, JS-rendered pages, structured data — higher latency and cost. Also used for crawl extraction. | | `TAVILY_EXTRACT_TIMEOUT` | `30` | `1.0`–`60.0` | Tavily-side timeout for `web_fetch --via tavily`. The HTTP client waits 10s longer so Tavily's diagnosable error wins. Applies per attempt; with retries the total wall time can reach `timeout × attempts`. | | `TAVILY_CRAWL_TIMEOUT` | `150` | `10`–`150` | Crawl-side timeout (API range). Client waits 10s longer. Applies per attempt; with retries the total wall time can reach `timeout × attempts`. | | `TAVILY_CRAWL_MAX_DEPTH` | `1` | int | Crawl depth. | | `TAVILY_CRAWL_MAX_BREADTH` | `20` | int | Links followed per page. | | `TAVILY_CRAWL_LIMIT` | `50` | int | Total pages crawled. | | `TAVILY_CRAWL_ALLOW_EXTERNAL` | `true` | `true` / `false` | `false` keeps the crawl on one site. | | `TAVILY_CRAWL_SELECT_PATHS` | *(unset)* | comma-separated regexes | Include only matching paths. | | `TAVILY_CRAWL_EXCLUDE_PATHS` | *(unset)* | comma-separated regexes | Exclude matching paths. | | `TAVILY_RESEARCH_MODEL` | `auto` | `mini` / `pro` / `auto` | Research agent model. | | `TAVILY_RESEARCH_CITATION_FORMAT` | `numbered` | `numbered` / `mla` / `apa` / `chicago` | Citation style in the report. | | `TAVILY_RESEARCH_OUTPUT_LENGTH` | `standard` | `short` / `standard` / `long` | Report length. | | `TAVILY_RESEARCH_OUTPUT_SCHEMA` | *(unset)* | path to a JSON file | Structured output. The file must be a JSON Schema with non-empty `properties`; each property needs `type` (`object`/`string`/`integer`/`number`/`array`) and `description`. Invalid schemas are reported before any request is sent. When set, `content` in the response is an object rather than a string. | | `TAVILY_RESEARCH_TIMEOUT` | `300` | seconds | Total polling budget before returning `status: timeout` (with `request_id` so the task can be re-fetched). A poll HTTP/network failure returns `status: poll_error` with the same `request_id`. | | `TAVILY_RESEARCH_POLL_INTERVAL` | `5` | seconds | Delay between status checks. | Rate limits: 100 RPM (development key) / 1000 RPM (production). A `429` carries a `retry-after` header, which the retry logic honors. `crawl` is capped at 100 RPM and `research` at 20 RPM on both tiers. Example — recent, trusted sources only: ```bash TAVILY_TOPIC=news TAVILY_TIME_RANGE=week \ TAVILY_INCLUDE_DOMAINS="reuters.com,bloomberg.com" TAVILY_INCLUDE_DOMAINS_MODE=boost \ python groksearch_cli.py web_search -q "AI regulation" --extra-sources 5 ``` Invalid values are reported per-key by `get_config_info` rather than silently ignored. ## Acknowledgments - Based on the original [GuDaStudio/GrokSearch](https://github.com/GuDaStudio/GrokSearch). -
ruff.toml 904 B
Loading… -
SKILL.md 6.6 KB
--- name: grok-search description: | Enhanced web search and real-time content retrieval via Grok API with forced tool routing. Use when: (1) Web search / information retrieval / fact-checking, (2) Webpage content extraction / URL parsing, (3) Breaking knowledge cutoff limits for current information, (4) Real-time news and technical documentation, (5) Multi-source information aggregation. Triggers: "search for", "find information about", "latest news", "current", "fetch webpage", "get content from URL". IMPORTANT: This skill REPLACES built-in WebSearch/WebFetch with Grok Search tools. --- # Grok Search Enhanced web search via Grok API. Standalone CLI only (no MCP dependency). ## Implementation Layout - `scripts/groksearch_cli.py` - CLI entrypoint and compatibility facade - `scripts/groksearch/` - internal modules for config, HTTP retry, Grok provider, Tavily calls, formatting, and commands ## Execution Methods Run `scripts/groksearch_cli.py` via Bash: ```bash # Prerequisites: pip install httpx tenacity # Environment: GROK_API_URL, GROK_API_KEY (required); TAVILY_API_KEY (optional) # Web search (Grok only) python scripts/groksearch_cli.py web_search --query "search terms" [--platform "GitHub"] [--min-results 3] [--max-results 10] # Web search with Tavily extra sources (parallel + URL-deduplicated merge) python scripts/groksearch_cli.py web_search --query "..." --extra-sources 5 # Fetch webpage (default: Grok) python scripts/groksearch_cli.py web_fetch --url "https://..." [--out file.md] # Fetch via Tavily extract endpoint python scripts/groksearch_cli.py web_fetch --url "https://..." --via tavily # Map a website's structure (Tavily) python scripts/groksearch_cli.py web_map --url "https://docs.example.com" [--instructions "API only"] [--max-depth 2] [--max-breadth 20] [--limit 50] [--timeout 150] # Crawl a website's pages with extraction (Tavily; own 100 RPM limit) python scripts/groksearch_cli.py web_crawl --url "https://docs.example.com" [--instructions "API only"] [--max-depth 2] [--limit 50] [--select-paths "/docs/.*"] [--exclude-paths "/blog/.*"] [--timeout 150] # Run a cited research task (Tavily; async submit + poll; own 20 RPM limit) python scripts/groksearch_cli.py web_research --input "question to investigate" [--model mini|pro|auto] [--output-length short|standard|long] [--citation-format numbered|mla|apa|chicago] [--output-schema schema.json] # Check config python scripts/groksearch_cli.py get_config_info [--no-test] # Switch model python scripts/groksearch_cli.py switch_model --model "grok-2-latest" # Toggle built-in tools python scripts/groksearch_cli.py toggle_builtin_tools --action on|off|status [--root /path/to/project] ``` ## Tool Routing Policy ### Forced Replacement Rules | Scenario | Disabled | Force Use | |----------|----------|-----------| | Web Search | `WebSearch` | CLI `web_search` | | Web Fetch | `WebFetch` | CLI `web_fetch` | ### Tool Capability Matrix | Tool | Parameters | Output | |------|------------|--------| | `web_search` | `query`(required), `platform`/`min_results`/`max_results`(optional), `extra_sources`(int, 0=disabled) | `[{title,url,description,provider?}]` | | `web_fetch` | `url`(required), `out`(optional), `via`(grok\|tavily, default grok) | Structured Markdown | | `web_map` | `url`(required), `instructions`/`max_depth`/`max_breadth`/`limit`/`timeout`(optional) | `{base_url,results,response_time}` JSON | | `web_crawl` | `url`(required), `instructions`/`max_depth`/`max_breadth`/`limit`/`select_paths`/`exclude_paths`/`timeout`(optional; unset falls back to `TAVILY_CRAWL_*`) | `{base_url,results,response_time,usage?}` JSON | | `web_research` | `input`(required), `model`/`output_length`/`citation_format`(optional; unset falls back to `TAVILY_RESEARCH_*`) | `{request_id,status,content,sources,usage?}` JSON | | `get_config_info` | `no_test`(optional) | `{api_url,status,connection_test,tavily_*}` | | `switch_model` | `model`(required) | `{previous_model,current_model}` | | `toggle_builtin_tools` | `action`(on/off/status), `root`(optional) | `{blocked,deny_list}` | ## Search Workflow ### Phase 1: Query Construction - **Intent Recognition**: Broad search → `web_search` | Deep retrieval → `web_fetch` - **Parameter Optimization**: Set `platform` for specific sources, adjust result counts ### Phase 2: Search Execution 1. Start with `web_search` for structured summaries 2. Use `web_fetch` on key URLs if summaries insufficient 3. Retry with adjusted query if first round unsatisfactory ### Phase 3: URL Verification & Hallucination Guard (MANDATORY) **Background**: Grok API calls without explicit web-search activation return results from parametric memory, which frequently fabricates URLs (observed 25% liveness rate in testing). All Grok-returned URLs MUST be verified before citation. **Verification Protocol**: 1. **URL Liveness Check**: Issue HEAD/GET request to each Grok-returned URL; non-2xx status = unreliable 2. **Tavily Fallback Triggers** (invoke `web_search` with `--extra-sources N` when ANY apply): - URL liveness rate < 50% in Grok results - Query contains version numbers, release dates, API signatures, or "latest"/"recent" temporal markers (high hallucination surface) - Multiple Grok runs return contradictory URLs for the same factual claim - Grok result descriptions contain specifics (dates/versions/methods) that cannot be confirmed from live URLs 3. **Tavily Grounding**: When triggered, re-run the same query with `--extra-sources 5-10` to obtain Tavily search results; prioritize these over failed Grok URLs 4. **Content Extraction**: For critical factual claims, use `web_fetch --via tavily` on verified URLs to extract authoritative source text **Citation Discipline**: - **ONLY** verified-live URLs may appear in final output - Fabricated Grok URLs must be dropped entirely (do not present them with a disclaimer; omit them) - When Tavily sources replace Grok sources, cite Tavily URLs and mark provider as `tavily` - For time-sensitive queries with no live sources, state "Unable to verify current information" rather than citing dead links ### Phase 4: Result Synthesis 1. Cross-reference multiple sources 2. **Must annotate source and date** for time-sensitive info 3. **Must include source URLs**: `Title [<sup>1</sup>](URL)` ## Error Handling | Error | Recovery | |-------|----------| | Connection Failure | Run `get_config_info`, verify API URL/Key | | No Results | Broaden search terms | | Fetch Timeout | Try alternative sources | ## Anti-Patterns | Prohibited | Correct | |------------|---------| | No source citation | Include `Source [<sup>1</sup>](URL)` | | Give up after one failure | Retry at least once | | Use built-in WebSearch/WebFetch | Use GrokSearch tools/CLI |
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.