scriptoriumadmin/claude-skills
1
0
Fork 0
mirror of https://github.com/alirezarezvani/claude-skills.git synced 2026-08-28 04:24:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
claude-skills/docs/commands/cs-scrape.md
Claude 82c5aea9f0
Merge origin/dev: reconcile docs redesign with upstream skill changes 
- Resolve conflicts: keep redesigned skills index, take dev's cs-aeo link
  fix, union of DOMAIN_SEO_CONTEXT entries in generate-docs.py
- Regenerate catalog on the merged tree (dev's agent/command description
  updates, removed ai-seo/release-manager/command-guide, restructured
  universal-scraping-architect)
- Update counters to post-merge truth from scripts/derive_counters.py:
  345 skills, 78 plugins (14 bundles + 64 standalone), 570+ Python tools
- Add redirects for upstream-removed pages (ai-seo -> aeo,
  release-manager -> changelog-generator, command-guide -> engineering index)
- Add compliance-os bundle to bundle tables; rebuild 78-plugin table from
  marketplace.json
- Teach the generator to rewrite repo-root-relative source links to GitHub
  URLs — mkdocs build --strict now passes with zero warnings

https://claude.ai/code/session_015bYZ97nV4oRb3LbxCRFVcP
2026-06-11 15:52:42 +00:00

2.5 KiB
Raw Permalink Blame History

title description
/cs-scrape — Slash Command for AI Coding Agents Route, extract, and validate a scraping job (URL or local file) via the universal-scraping-architect skill — refuses to deliver unvalidated data.. Slash command for Claude Code, Codex CLI, Gemini CLI.

/cs-scrape

:material-console: Slash Command :material-github: Source

Run a gated extraction pipeline for $ARGUMENTS using skills/universal-scraping-architect/SKILL.md.

Pre-flight gates (stop if any fails)

  1. Target stated? If $ARGUMENTS is empty, ask for the URL or file path plus the desired output format — do not guess.
  2. Live-site etiquette: for URLs, check robots.txt and plan rate limits; refuse disallowed targets.
  3. Privacy: if the target is a local/sensitive file, do not send it to an external API — force Mode 2 (local Python).
  4. Secrets: Firecrawl key only via os.getenv('FIRECRAWL_API_KEY'); if a key appears inline anywhere, fix that first.

Workflow

  1. Route — state the mode and why (per the skill's routing rules): Mode 1 Firecrawl (public/JS-heavy URL, bulk crawl) · Mode 2 local Python (local files, private data, simple static HTML) · Mode 3 hybrid (Firecrawl extract + pandas clean).

  2. Budget — estimate API quota / token limits before multi-page jobs; add checkpointing + pagination.

  3. Extract — start from the matching runner template (run from the plugin root; --sample previews the summary shape offline):

    python3 skills/universal-scraping-architect/scripts/firecrawl_example.py --sample
python3 skills/universal-scraping-architect/scripts/local_bs4_example.py --sample

  4. Validate (mandatory, exit-code gated):

    python3 skills/universal-scraping-architect/scripts/validate_extraction.py extracted_output.json --json
    • exit 0 (status: ok) → continue
    • exit 1 (warning = empty output, error = malformed JSON) → fix and re-extract; never deliver unvalidated data

    Then check required fields and duplicates against the job spec.

  5. Deliver — CSV (tabular) / JSON (nested) / Markdown (docs, chunked), per the user's requested format, with a summary of mode chosen, row counts, empty values, and the validation verdict.