CLI-Anything-WEB
<p align="center"> <img src="assets/bannerGif10.gif" alt="CLI Anything Web" width="100%"> </p> <h3 align="center">Turn any website into a production-grade CLI — automatically.</h3> <p align="center"> <a href="#-quick-start">Quick Start</a> • <a href="#-examples">Examples</a> • <a href="#%EF%B8%8F-how-it-works">How It Works</a> • <a href="#-supported-protocols">Protocols</a> • <a href="https://itamarzand88.github.io/CLI-Anything-WEB/registry/">Registry</a> • <a href="#-whats-next">Roadmap</a> • <a href="#-contributing">Contributing</a> </p> <p align="center"> <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License: MIT"></a> <a href="https://github.com/ItamarZand88/CLI-Anything-WEB/stargazers"><img src="https://img.shields.io/github/stars/ItamarZand88/CLI-Anything-WEB?style=social" alt="GitHub Stars"></a> <a href="https://github.com/ItamarZand88/CLI-Anything-WEB/issues"><img src="https://img.shields.io/github/issues/ItamarZand88/CLI-Anything-WEB" alt="Issues"></a> <img src="https://img.shields.io/badge/python-3.10%2B-blue" alt="Python 3.10+"> <img src="https://img.shields.io/badge/CLIs_generated-19-brightgreen" alt="19 CLIs"> <img src="https://img.shields.io/badge/claude%20code-plugin-blueviolet" alt="Claude Code Plugin"> <a href="https://itamarzand88.github.io/CLI-Anything-WEB/registry/"><img src="https://img.shields.io/badge/registry-live-orange" alt="CLI Registry"></a> </p> <p align="center"> <a href="#-quick-start"><img src="https://img.shields.io/badge/Quick_Start-blue?style=for-the-badge" alt="Quick Start"></a> <a href="#-examples"><img src="https://img.shields.io/badge/19_CLIs-brightgreen?style=for-the-badge" alt="Examples"></a> <a href="#-supported-protocols"><img src="https://img.shields.io/badge/7_Protocols-orange?style=for-the-badge" alt="Protocols"></a> <a href="#-contributing"><img src="https://img.shields.io/badge/Contributing-purple?style=for-the-badge" alt="Contributing"></a> </p> --- **CLI-Anything-Web** is a [Claude Code](https://docs.anthropic.com/en/docs/claude-code) plugin that generates production-grade Python CLIs for **any** web application by capturing its live HTTP traffic. Point it at a URL, and get a fully working CLI on your PATH — with auth, REPL mode, `--json` output, and tests. > [!WARNING] > **Experimental Project — Use Responsibly** > > This project uses **undocumented web APIs** reverse-engineered from live HTTP traffic. These APIs can change without notice. > > - **Not affiliated with any website** — This is an independent open-source project > - **APIs may break** — Websites can change their internal endpoints, HTML structure, or add protections at any time > - **Respect rate limits** — Generated CLIs include exponential backoff, but heavy usage may be throttled > - **For personal use** — Best suited for prototyping, automation, research, and personal productivity > > Generated CLIs interact with real production services. Use them responsibly and in accordance with each website's terms of service. ## 💡 The Idea Most web apps don't have public APIs. **CLI-Anything-Web** changes that: <p align="center"> <img src="assets/demo.gif" alt="CLI-Anything-Web Demo" width="100%"> </p> > **No API docs needed. No reverse-engineering by hand. Just point and generate.** <br> ## 🚀 Quick Start ### Prerequisites | Requirement | Version | Why | |------------|---------|-----| | [Claude Code](https://docs.anthropic.com/en/docs/claude-code) | With plugin support | Runs the generation pipeline | | [Node.js](https://nodejs.org/) | 18+ | For playwright traffic capture | | [Python](https://python.org/) | 3.10+ | Generated CLIs are Python | ### Install the Plugin ```bash # Inside Claude Code /plugin marketplace add ItamarZand88/CLI-Anything-WEB /plugin install cli-anything-web /reload-plugins ``` ### Generate Your First CLI ```bash /cli-anything-web https://your-favorite-website.com ``` The agent opens a browser, asks you to log in if needed, captures traffic, and generates a complete CLI. That's it. <br> ## 📦 Examples <!-- fleet-table:start --> 20 real CLIs generated by the plugin — shipped as reference implementations: | CLI | Website | Protocol | Auth | Skill | Description | |-----|---------|----------|------|-------|-------------| | [`cli-web-futbin`](futbin/) | FUTBIN | HTML + JSON API | None | [📖 Skill](.claude/skills/futbin-cli/SKILL.md) | EA FC player database — search, compare, prices, market analysis, arbitrage, trading signals | | [`cli-web-notebooklm`](notebooklm/) | Google NotebookLM | batchexecute RPC | Google SSO | [📖 Skill](.claude/skills/notebooklm-cli/SKILL.md) | Notebooks, sources, chat, 9 artifact types (audio, video, slides, quiz, mindmap) | | [`cli-web-gh-trending`](gh-trending/) | GitHub Trending | HTML scraping | None | [📖 Skill](.claude/skills/gh-trending-cli/SKILL.md) | Trending repos & developers with language/time filters | | [`cli-web-producthunt`](producthunt/) | Product Hunt | HTML scraping (curl_cffi) | None | [📖 Skill](.claude/skills/producthunt-cli/SKILL.md) | Today's launches, leaderboards, product details | | [`cli-web-unsplash`](unsplash/) | Unsplash | REST API (curl_cffi) | None | [📖 Skill](.claude/skills/unsplash-cli/SKILL.md) | Photo search, download, topics, collections, profiles | | [`cli-web-booking`](booking/) | Booking.com | GraphQL + HTML (curl_cffi) | WAF cookies | [📖 Skill](.claude/skills/booking-cli/SKILL.md) | Hotel search, property details, destination resolution | | [`cli-web-stitch`](stitch/) | Google Stitch | batchexecute RPC | Google SSO | [📖 Skill](.claude/skills/stitch-cli/SKILL.md) | AI UI design — generate mobile/web app mockups from text prompts | | [`cli-web-pexels`](pexels/) | Pexels | SSR + __NEXT_DATA__ (curl_cffi) | None | [📖 Skill](.claude/skills/pexels-cli/SKILL.md) | Free stock photos & videos — search, download, collections, profiles | | [`cli-web-reddit`](reddit/) | Reddit | REST JSON API (curl_cffi) | Optional (OAuth) | [📖 Skill](.claude/skills/reddit-cli/SKILL.md) | Feeds, subreddits, search, vote, comment, submit, save, inbox | | [`cli-web-gai`](gai/) | Google AI Mode | Browser-rendered (Playwright) | None | [📖 Skill](.claude/skills/gai-cli/SKILL.md) | AI-powered search with source references | | [`cli-web-youtube`](youtube/) | YouTube | InnerTube REST API (httpx) | None | [📖 Skill](.claude/skills/youtube-cli/SKILL.md) | Search videos, video details, trending, channel info | | [`cli-web-hackernews`](hackernews/) | Hacker News | REST API — Firebase + Algolia (httpx) | Cookie (optional) | [📖 Skill](.claude/skills/hackernews-cli/SKILL.md) | Stories, search, comments, users, upvote, submit, comment, favorite | | [`cli-web-codewiki`](codewiki/) | Google Code Wiki | batchexecute RPC | None | [📖 Skill](.claude/skills/codewiki-cli/SKILL.md) | AI-generated repo docs, wiki sections, Gemini chat, download as .md | | [`cli-web-chatgpt`](chatgpt/) | ChatGPT | REST API + Camoufox browser | Browser (OpenAI SSO) | [📖 Skill](.claude/skills/chatgpt-cli/SKILL.md) | Ask questions, generate/download images, conversations, models | | [`cli-web-airbnb`](airbnb/) | Airbnb | SSR HTML + niobeClientData (curl_cffi) | None | [📖 Skill](.claude/skills/airbnb-cli/SKILL.md) | Search stays, listing details, amenities, host info, autocomplete locations | | [`cli-web-amazon`](amazon/) | amazon.com | SSR HTML + REST JSON | None | [📖 Skill](.claude/skills/amazon-cli/SKILL.md) | Search products, view details, browse bestsellers | | [`cli-web-tripadvisor`](tripadvisor/) | TripAdvisor | SSR HTML + JSON-LD (curl_cffi) | None | [📖 Skill](.claude/skills/tripadvisor-cli/SKILL.md) | Search locations, hotels, restaurants, and attractions | | [`cli-web-linkedin`](linkedin/) | LinkedIn | GraphQL + Voyager REST (curl_cffi) | Cookie | [📖 Skill](.claude/skills/linkedin-cli/SKILL.md) | Search, feed, profiles, jobs, posts, reactions, comments, network, messaging (26 cmds) | | [`cli-web-capitoltrades`](capitoltrades/) | Capitol Trades | SSR HTML + BFF JSON (curl_cffi) | None | [📖 Skill](.claude/skills/capitoltrades-cli/SKILL.md) | US congressional stock trades (STOCK Act) — trades, politicians, issuers, price history | | [`cli-web-worldcup`](worldcup/) | ESPN / FIFA World Cup 2026 | REST JSON (ESPN + The Odds API) | None | [📖 Skill](.claude/skills/worldcup-cli/SKILL.md) | FIFA World Cup 2026 — fixtures, nations, squads, group standings, and bookmaker odds (read-only) | <!-- fleet-table:end --> ### Try them yourself <!-- fleet-install:start --> ```bash # Install all CLIs at once pip install -e futbin/agent-harness -e notebooklm/agent-harness \ -e gh-trending/agent-harness -e producthunt/agent-harness -e unsplash/agent-harness \ -e booking/agent-harness -e stitch/agent-harness -e pexels/agent-harness \ -e reddit/agent-harness -e gai/agent-harness -e youtube/agent-harness \ -e hackernews/agent-harness -e codewiki/agent-harness -e chatgpt/agent-harness \ -e airbnb/agent-harness -e amazon/agent-harness -e tripadvisor/agent-harness \ -e linkedin/agent-harness -e capitoltrades/agent-harness -e worldcup/agent-harness ``` <!-- fleet-install:end --> ```bash # Required browsers playwright install chromium # for CLIs that require browser-based auth or rendering python -m camoufox fetch # for CLIs that use stealth browser (anti-bot bypass) ``` **GitHub Trending** — no auth, great first test ```bash pip install -e gh-trending/agent-harness cli-web-gh-trending repos list --language python --since weekly --json ``` **FUTBIN** — search EA FC players ```bash pip install -e futbin/agent-harness cli-web-futbin players search --name "Messi" --json ``` **NotebookLM** — requires Google login ```bash pip install -e notebooklm/agent-harness cli-web-notebooklm auth login cli-web-notebooklm notebooks list --json ``` **Product Hunt** — no auth, bypasses Cloudflare ```bash pip install -e producthunt/agent-harness cli-web-producthunt posts list --json ``` **Unsplash** — photo search ```bash pip install -e unsplash/agent-harness cli-web-unsplash photos search "mountains" --json ``` **Booking.com** — hotel search ```bash pip install -e booking/agent-harness cli-web-booking auth login # required — fetches WAF browser cookies cli-web-booking search find "Paris" --json ``` **Google Stitch** — requires Google SSO login ```bash pip install -e stitch/agent-harness cli-web-stitch auth login cli-web-stitch projects list --json ``` **Pexels** — free stock photos & videos ```bash pip install -e pexels/agent-harness cli-web-pexels photos search "mountains" --json ``` **Reddit** — feeds, search, vote, comment, submit ```bash pip install -e reddit/agent-harness cli-web-reddit auth login # optional — required for voting, commenting, submitting cli-web-reddit feed hot --limit 5 --json ``` **Hacker News** — stories, search, upvote, submit ```bash pip install -e hackernews/agent-harness cli-web-hackernews auth login # optional — required for upvoting, submitting, commenting cli-web-hackernews stories top -n 5 --json ``` **Google Code Wiki** — AI-generated repo docs + Gemini chat ```bash pip install -e codewiki/agent-harness cli-web-codewiki wiki sections excalidraw/excalidraw --json cli-web-codewiki chat ask "How does rendering work?" --repo excalidraw/excalidraw --json ``` **Airbnb** — search stays, get listing details ```bash pip install -e airbnb/agent-harness cli-web-airbnb search stays "London, UK" --checkin 2024-06-01 --checkout 2024-06-05 --adults 2 --json cli-web-airbnb listings get 770993223449115417 --json cli-web-airbnb autocomplete locations "New Yor" --json ``` **TripAdvisor** — search hotels, restaurants, and attractions (no auth, DataDome bypass) ```bash pip install -e tripadvisor/agent-harness cli-web-tripadvisor hotels search "Paris" --json cli-web-tripadvisor attractions search "London" --json ``` **ChatGPT** — ask questions, generate images ```bash pip install -e chatgpt/agent-harness cli-web-chatgpt auth login # optional — required for image generation & conversations cli-web-chatgpt chat ask "Explain quantum computing in one sentence" --json cli-web-chatgpt chat image "A sunset over mountains" --output sunset.png --json ``` **Google AI Mode** — AI-powered search (requires Playwright browser) ```bash pip install -e gai/agent-harness playwright install chromium cli-web-gai search ask "What is quantum computing" --json ``` **Amazon** — search products, bestsellers, autocomplete (no auth required) ```bash pip install -e amazon/agent-harness # Search Amazon products cli-web-amazon search "laptop" --json # View product details cli-web-amazon product get B0GRZ78683 --json # Browse Best Sellers cli-web-amazon bestsellers electronics --json ``` **LinkedIn** — search people/jobs/companies, feed, profiles, post, network, messaging (auth required) ```bash pip install -e linkedin/agent-harness cli-web-linkedin auth login # required — browser-based LinkedIn SSO # Search people and jobs cli-web-linkedin search people "python developer" --limit 5 --json cli-web-linkedin jobs search "software engineer" --json # View feed, profile, connections cli-web-linkedin feed --count 5 --json cli-web-linkedin profile me --json cli-web-linkedin network connections --limit 10 --json ``` **Capitol Trades** — US congressional stock trades (no auth, CloudFront bypass) ```bash pip install -e capitoltrades/agent-harness # Latest trades across Congress cli-web-capitoltrades --json trades list --page-size 20 # Trades for a specific ticker cli-web-capitoltrades --json trades by-ticker NVDA # Top politicians by trade volume cli-web-capitoltrades --json politicians top --by trades --page-size 10 # Rich issuer data with 1-year price history cli-web-capitoltrades --json issuers search nvidia ``` ### Agent-Native: Claude uses the CLIs automatically Every generated CLI comes with a **skill file** (`.claude/skills/<app>-cli/SKILL.md`) that Claude Code loads automatically. The skill tells Claude exactly how to use the CLI — every command, every option, every output format. When you ask Claude a question like *"find me a hotel in Paris"*, it reads the booking skill and runs `cli-web-booking search find "Paris"` on its own. Each skill includes: - All available commands with arguments and options - Output format for each command - Auth requirements and setup instructions - Common agent patterns (piping, filtering, chaining commands) ### MCP Server Mode: every CLI is also an MCP server Every generated CLI doubles as a **Model Context Protocol server** — run `cli-web-<app> mcp-serve` and any MCP client (Claude Desktop, IDEs, other agents) gets every command as a typed MCP tool, with input schemas derived from the CLI's own options and the standard `--json` envelope as tool output: ```jsonc // e.g. in an MCP client config { "mcpServers": { "hackernews": { "command": "cli-web-hackernews", "args": ["mcp-serve"] } } } ``` No extra dependencies, no separate server code — the adapter walks the Click command tree at startup, so new commands automatically become new tools. Every CLI also ships a `doctor` command (`cli-web-<app> doctor [--json]`) that diagnoses the local setup — install, Python version, auth file and its permissions, optional dependencies — before you ever need to open an issue. <p align="center"> <img src="assets/agentic-demo.gif" alt="Agentic Pattern Demo" width="100%"> </p> <br> ## ⚙️ How It Works The plugin runs a 4-phase pipeline, fully automated by Claude: <p align="center"> <img src="assets/pipeline.png" alt="CLI-Anything-Web Pipeline" width="100%"> </p> | Phase | What Happens | |-------|-------------| | **1. Capture** | Opens browser via Playwright, records all HTTP traffic while you use the site | | **2. Analyze & Generate** | Identifies protocol (REST/GraphQL/RPC/HTML), maps endpoints, generates full Python CLI | | **3. Test** | Writes unit tests (mocked) + E2E tests (live API) + subprocess tests | | **4. Publish** | `pip install -e .` → on your PATH → smoke tested → Claude skill generated | <br> ## 🔧 What Every Generated CLI Includes | Feature | Details | |---------|---------| | **Click commands** | `cli-web-<app> <group> <command> [options]` | | **Interactive REPL** | Run with no args — history, autocomplete, branded prompt | | **`--json` output** | Machine-readable on every command — pipe into `jq`, scripts, or agents | | **Auth management** | Python Playwright browser login → cookie extraction → `auth.json` | | **Error handling** | Typed exceptions → structured JSON errors: `{"error": true, "code": "AUTH_EXPIRED"}` | | **Tests** | Unit (mocked HTTP) + E2E (live API) + subprocess (`_resolve_cli`) | | **Installable** | `pip install -e .` puts it on your PATH immediately | ## 📋 Commands | Command | Description | |---------|-------------| | `/cli-anything-web <url>` | Full pipeline — capture, analyze, generate, publish | | `/cli-anything-web:record <url>` | Capture traffic only (for exploration) | | `/cli-anything-web:refine <path>` | Add more commands to an existing CLI | | `/cli-anything-web:test <path>` | Run tests and update TEST.md | | `/cli-anything-web:validate <path>` | Validate against 75-check quality standards | | `/cli-anything-web:list` | List all generated CLIs | <br> ## 🌐 Supported Protocols The plugin auto-detects and handles multiple web architectures: | Protocol | Example Sites | How It's Handled | |----------|--------------|-----------------| | **REST / JSON API** | Monday.com, Dev.to | `httpx` client, JSON response parsing | | **Server-rendered HTML** | GitHub, FUTBIN, Hacker News | `BeautifulSoup4` + CSS selectors | | **Cloudflare-protected** | Product Hunt, Unsplash | `curl_cffi` with Chrome TLS impersonation | | **GraphQL** | Shopify, GitHub API v4 | Query abstraction into CLI commands | | **GraphQL + AWS WAF** | Booking.com | `curl_cffi` + WAF cookie bypass | | **Google batchexecute** | NotebookLM, Google Docs, Keep | Custom RPC encoder/decoder | | **Browser-rendered (Playwright)** | Google AI Mode | Headless browser rendering + content extraction | <br> ## 🏗️ Repository Structure ``` CLI-Anything-WEB/ ├── cli-anything-web-plugin/ # 🔌 The installable Claude Code plugin │ ├── commands/ # Slash command definitions │ ├── skills/ # 4-phase pipeline skills │ │ ├── capture/ # Phase 1: browser + traffic capture │ │ ├── methodology/ # Phase 2: analysis + code generation │ │ ├── testing/ # Phase 3: test generation │ │ └── standards/ # Phase 4: validation + publishing │ ├── scripts/ # Shared utilities │ └── HARNESS.md # Complete methodology SOP │ ├── stitch/ # 🎨 Google Stitch (batchexecute RPC) ├── reddit/ # 💬 Reddit (REST JSON API) ├── booking/ # 🏨 Booking.com (GraphQL + AWS WAF) ├── gai/ # 🤖 Google AI Mode (Browser-rendered) ├── notebooklm/ # 📓 NotebookLM (Google batchexecute) ├── pexels/ # 📸 Pexels (SSR + __NEXT_DATA__) ├── unsplash/ # 📷 Unsplash (REST API + curl_cffi) ├── producthunt/ # 🚀 Product Hunt (curl_cffi) ├── futbin/ # 🎮 FUTBIN (HTML + JSON) ├── gh-trending/ # 📈 GitHub Trending (HTML scraping) ├── youtube/ # 🎬 YouTube (InnerTube REST API) ├── hackernews/ # 📰 Hacker News (Firebase + Algolia API) ├── codewiki/ # 📚 Google Code Wiki (batchexecute RPC) ├── chatgpt/ # 🤖 ChatGPT (REST API + Camoufox) ├── airbnb/ # 🏠 Airbnb (SSR HTML + niobeClientData) ├── amazon/ # 🛒 Amazon (SSR HTML + REST JSON) ├── tripadvisor/ # 🌍 TripAdvisor (SSR HTML + JSON-LD, curl_cffi) ├── linkedin/ # 💼 LinkedIn (GraphQL + Voyager REST, curl_cffi) └── capitoltrades/ # 🏛️ Capitol Trades (SSR HTML + BFF JSON, curl_cffi) ``` <br> ## 🔗 Inspiration This project is directly inspired by [**CLI-Anything**](https://github.com/HKUDS/CLI-Anything) by HKUDS — a Claude Code plugin that makes **desktop software** (GIMP, Blender, LibreOffice, OBS Studio) agent-native by analyzing source code and generating CLI wrappers. **CLI-Anything-Web** extends the same vision to the **web** — where there's no source code to analyze, only live HTTP traffic to capture. | | [**CLI-Anything**](https://github.com/HKUDS/CLI-Anything) | **CLI-Anything-Web** | |---|---|---| | **Target** | Desktop apps (GIMP, Blender, OBS) | Web apps (NotebookLM, Booking.com, any website) | | **Input** | Source code, GUI APIs, plugin systems | Live HTTP traffic from browser | | **Analysis** | Static code analysis + API mapping | Network traffic capture + protocol detection | | **Auth** | N/A (local software) | Browser login, cookies, WAF bypass, API keys | | **Output** | Click CLI + REPL + `--json` + tests | Click CLI + REPL + `--json` + tests | Together they cover the full spectrum: **CLI-Anything** for desktop, **CLI-Anything-Web** for the web. <br> ## 🗺️ What's Next We're actively building more CLIs and improving the plugin: - 🎯 **More CLIs** — Jira, Notion, Monday.com, Spotify - 🎵 **Content generation** — Suno, ElevenLabs, Midjourney - 🧠 **Smarter analysis** — Auto-detect auth flows, pagination, WebSocket streams - 🔄 **CI/CD integration** — Run CLIs in GitHub Actions with env-var auth - 📦 **Community registry** — Share and install CLIs built by other users **Want a specific website?** [Open an issue](https://github.com/ItamarZand88/CLI-Anything-WEB/issues/new) with the URL — we'll prioritize the most requested ones. <br> ## 🤝 Contributing We'd love your help! See **[CONTRIBUTING.md](CONTRIBUTING.md)** for the full guide. The quickest ways to contribute: | What | How | |------|-----| | **Build a new CLI** | Run `/cli-anything-web <url>` on any website, submit a PR | | **Improve the plugin** | Add patterns to `cli-anything-web-plugin/skills/` | | **Report bugs** | [Open an issue](https://github.com/ItamarZand88/CLI-Anything-WEB/issues) with `--json` output | | **Fix a broken CLI** | Website changed? Update the scraper/client | <br> <br> ## 📄 License This project is licensed under the [MIT License](LICENSE). --- <p align="center"> Built with <a href="https://docs.anthropic.com/en/docs/claude-code">Claude Code</a> · Inspired by <a href="https://github.com/HKUDS/CLI-Anything">CLI-Anything</a> · <a href="https://github.com/ItamarZand88/CLI-Anything-WEB/issues/new">Request a CLI</a> </p>