AutoCLI

Formerly known as opencli-rs. Renamed to AutoCLI starting from v0.2.4.

https://autocli.ai — AI-powered adapter marketplace & cloud API

What's New

v0.3.2

Chrome Extension Selector Tool — Just select the core data you need, visually pick elements from any page and build precise CSS selectors to target specific content
AI-Powered Generation via AutoCLI.ai — Based on your selected data, AI automatically expands and discovers related fields, generating complete scraping rules
Seamless Local + Cloud Sync — Generated adapters are saved locally and synced to AutoCLI.ai, ready to use immediately

Blazing fast, memory-safe command-line tool — Fetch information from any website with a single command. Covers Twitter/X, Reddit, YouTube, HackerNews, Bilibili, Zhihu, Xiaohongshu, and 55+ sites, with support for controlling Electron desktop apps, integrating local CLI tools (gh, docker, kubectl), powered by browser session reuse and AI-native discovery capabilities.

A complete rewrite in pure Rust based on OpenCLI (TypeScript). Feature-equivalent, up to 12x faster, 10x less memory, single 4.7MB binary, zero runtime dependencies.

The perfect companion for OpenClaw/Agent — Give your AI Agent the ability to reach information across the entire web, fetching real-time data from 55+ sites with a single command. Built for AI Agents: Configure autocli list in AGENT.md or .cursorrules, and AI can automatically discover all available tools. Register your local CLI (autocli register mycli), and AI can seamlessly invoke all your tools.

🚀 Performance Comparison

Metric	🦀 autocli (Rust)	📦 opencli (Node.js)	Improvement
💾 Memory Usage (Public Commands)	15 MB	99 MB	6.6x
💾 Memory Usage (Browser Commands)	9 MB	95 MB	10.6x
📏 Binary Size	4.7 MB	~50 MB (node_modules)	10x
🔗 Runtime Dependencies	None	Node.js 20+	Zero deps
✅ Test Pass Rate	103/122 (84%)	104/122 (85%)	Near parity

⚡ Real-world Command Timing Comparison:

Command	🦀 autocli	📦 opencli	Speedup
`bilibili hot`	1.66s	20.1s	🔥 12x
`zhihu hot`	1.77s	20.5s	🔥 11.6x
`xueqiu search 茅台`	1.82s	9.2s	⚡ 5x
`xiaohongshu search`	5.1s	14s	⚡ 2.7x

Based on automated testing of 122 commands (55 sites), macOS Apple Silicon environment.

Features

55 sites, 333 commands — Covers Bilibili, Twitter, Reddit, Zhihu, Xiaohongshu, YouTube, Hacker News, and more
Browser session reuse — Reuse logged-in sessions via Chrome extension, no need to manage tokens
Declarative YAML Pipeline — Describe data scraping workflows in YAML, add new adapters with zero code
AI-native discovery — explore analyzes website APIs, generate auto-creates adapters with one command, cascade probes authentication strategies
AI-powered generation — generate --ai uses LLM to analyze any website and create working adapters automatically, with cloud sharing via autocli.ai
Download media & articles — Download videos (via yt-dlp), articles as Markdown with images localized
External CLI passthrough — Integrate GitHub CLI, Docker, Kubernetes, and other tools
Multi-format output — table, JSON, YAML, CSV, Markdown
Single binary — Compiles to a 4MB static binary with zero runtime dependencies

Installation

Just one file, download and use. No Node.js, Python, or any runtime needed — just put it in your PATH and go.

One-line Install Script (macOS / Linux)

curl -fsSL https://raw.githubusercontent.com/nashsu/autocli/main/scripts/install.sh | sh

Automatically detects your system and architecture, downloads the corresponding binary, and installs to /usr/local/bin/.

Windows (PowerShell)

Invoke-WebRequest -Uri "https://github.com/nashsu/autocli/releases/latest/download/autocli-x86_64-pc-windows-msvc.zip" -OutFile autocli.zip
Expand-Archive autocli.zip -DestinationPath .
Move-Item autocli.exe "$env:LOCALAPPDATA\Microsoft\WindowsApps\"

Manual Download (Simplest)

Download the file for your platform from GitHub Releases:

Platform	File
macOS (Apple Silicon)	`autocli-aarch64-apple-darwin.tar.gz`
macOS (Intel)	`autocli-x86_64-apple-darwin.tar.gz`
Linux (x86_64)	`autocli-x86_64-unknown-linux-musl.tar.gz`
Linux (ARM64)	`autocli-aarch64-unknown-linux-musl.tar.gz`
Windows (x64)	`autocli-x86_64-pc-windows-msvc.zip`

After extracting, place autocli (or autocli.exe on Windows) in your system PATH.

Build from Source

git clone https://github.com/nashsu/autocli.git
cd autocli
cargo build --release
cp target/release/autocli /usr/local/bin/   # macOS / Linux

Update

Simply re-run the install command or download the latest release to overwrite the existing binary.

Chrome Extension Setup (required for browser commands)

Download autocli-chrome-extension.zip from GitHub Releases
Extract to any directory
Open Chrome and go to chrome://extensions
Enable "Developer mode" (top right toggle)
Click "Load unpacked" and select the extracted folder
The extension will automatically connect to the autocli daemon

Public mode commands (hackernews, devto, lobsters, etc.) work without the extension.

Skill Install

One-click install autocli skill for your AI Agent:

npx skills add https://github.com/nashsu/autocli-skill

Quick Start

# View all available commands
autocli --help

# View commands for a specific site
autocli hackernews --help

# Get Hacker News top stories (public API, no browser needed)
autocli hackernews top --limit 10

# JSON format output
autocli hackernews top --limit 5 --format json

# Get Bilibili trending videos (requires browser + Cookie)
autocli bilibili hot --limit 20

# Search Twitter (requires browser + login)
autocli twitter search "rust lang" --limit 10

# Run diagnostics
autocli doctor

# Generate shell completions
autocli completion bash >> ~/.bashrc
autocli completion zsh >> ~/.zshrc
autocli completion fish > ~/.config/fish/completions/autocli.fish

AI Commands

Powered by autocli.ai — Get your API token, share adapters with the community, and let AI generate adapters for any website.

Step 1: Authenticate

autocli auth

This will:

Open your browser to https://autocli.ai/get-token
Prompt you to enter the token
Verify the token with the server
Save it to ~/.autocli/config.json

Step 2: Use the Chrome Extension to precisely select the data you need from any website. Click the Generate button, and AI will automatically analyze the page, expand related data, and generate an adapter:

Once generation is complete, you can use autocli with the newly generated command to retrieve the data you need.

Step 3: Search Existing Adapters

# Search by URL
autocli search https://www.example.com

# Domain name also works (auto-prepends https://)
autocli search example.com

Searches autocli.ai for community-shared adapters matching the URL. Select one from the interactive list to download and save it locally — ready to use immediately.

Environment Variables

Variable	Description	Default
`AUTOCLI_API_BASE`	Override server URL	`https://www.autocli.ai`

Built-in Commands

Run autocli --help to see all available commands.

Site	Commands	Mode
hackernews	`top` `new` `best` `ask` `show` `jobs` `search` `user`	Public
devto	`top` `tag` `user`	Public
lobsters	`hot` `newest` `active` `tag`	Public
stackoverflow	`hot` `search` `bounties` `unanswered`	Public
steam	`top-sellers`	Public
linux-do	`hot` `latest` `search` `categories` `category` `topic`	Public
arxiv	`search` `paper`	Public
wikipedia	`search` `summary` `random` `trending`	Public
apple-podcasts	`search` `episodes` `top`	Public
xiaoyuzhou	`podcast` `podcast-episodes` `episode`	Public
bbc	`news`	Public
hf	`top`	Public
sinafinance	`news`	Public
google	`news` `search` `suggest` `trends`	Public / Browser
v2ex	`hot` `latest` `topic` `node` `user` `member` `replies` `nodes` `daily` `me` `notifications`	Public / Browser
bloomberg	`main` `markets` `economics` `industries` `tech` `politics` `businessweek` `opinions` `feeds` `news`	Public / Browser
twitter	`trending` `bookmarks` `profile` `search` `timeline` `thread` `following` `followers` `notifications` `post` `reply` `delete` `like` `article` `follow` `unfollow` `bookmark` `unbookmark` `download` `accept` `reply-dm` `block` `unblock` `hide-reply`	Browser
bilibili	`hot` `search` `me` `favorite` `history` `feed` `subtitle` `dynamic` `ranking` `following` `user-videos` `download`	Browser
reddit	`hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `save` `comment` `subscribe` `saved` `upvoted`	Browser
zhihu	`hot` `search` `question` `download`	Browser
xiaohongshu	`search` `notifications` `feed` `user` `download` `publish` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats`	Browser
xueqiu	`feed` `hot-stock` `hot` `search` `stock` `watchlist` `earnings-date`	Browser
weibo	`hot` `search`	Browser
douban	`search` `top250` `subject` `marks` `reviews` `movie-hot` `book-hot`	Browser
weread	`shelf` `search` `book` `highlights` `notes` `notebooks` `ranking`	Browser
youtube	`search` `video` `transcript`	Browser
medium	`feed` `search` `user`	Browser
substack	`feed` `search` `publication`	Browser
sinablog	`hot` `search` `article` `user`	Browser
boss	`search` `detail` `recommend` `joblist` `greet` `batchgreet` `send` `chatlist` `chatmsg` `invite` `mark` `exchange` `resume` `stats`	Browser
jike	`feed` `search` `create` `like` `comment` `repost` `notifications` `post` `topic` `user`	Browser
facebook	`feed` `profile` `search` `friends` `groups` `events` `notifications` `memories` `add-friend` `join-group`	Browser
instagram	`explore` `profile` `search` `user` `followers` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `saved`	Browser
tiktok	`explore` `search` `profile` `user` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `live` `notifications` `friends`	Browser
yollomi	`generate` `video` `edit` `upload` `models` `remove-bg` `upscale` `face-swap` `restore` `try-on` `background` `object-remover`	Browser
yahoo-finance	`quote`	Browser
barchart	`quote` `options` `greeks` `flow`	Browser
linkedin	`search`	Browser
reuters	`search`	Browser
smzdm	`search`	Browser
ctrip	`search`	Browser
coupang	`search` `add-to-cart`	Browser
grok	`ask`	Browser
jimeng	`generate` `history`	Browser
chaoxing	`assignments` `exams`	Browser
weixin	`download`	Browser
doubao	`status` `new` `send` `read` `ask`	Browser
cursor	`status` `send` `read` `new` `dump` `composer` `model` `extract-code` `ask` `screenshot` `history` `export`	Desktop
codex	`status` `send` `read` `new` `dump` `extract-diff` `model` `ask` `screenshot` `history` `export`	Desktop
chatwise	`status` `new` `send` `read` `ask` `model` `history` `export` `screenshot`	Desktop
chatgpt	`status` `new` `send` `read` `ask`	Desktop
doubao-app	`status` `new` `send` `read` `ask` `screenshot` `dump`	Desktop
notion	`status` `search` `read` `new` `write` `sidebar` `favorites` `export`	Desktop
discord-app	`status` `send` `read` `channels` `servers` `search` `members`	Desktop
antigravity	`status` `send` `read` `new` `dump` `extract-code` `model` `watch`	Desktop

Mode legend: Public = No browser needed, calls API directly; Browser = Requires Chrome + extension; Desktop = Requires the desktop app to be running

AI Discovery Capabilities

Two approaches to auto-generate adapters:

# 🤖 AI-powered (recommended): LLM analyzes page and generates adapter
autocli generate https://www.example.com --goal hot --ai
# Searches autocli.ai for existing adapters first, then generates with AI if needed

# 🔧 Rule-based: heuristic analysis without AI
autocli generate https://www.example.com --goal hot

# Explore website API surface (endpoints, framework, stores)
autocli explore https://www.example.com --site mysite

# With interactive fuzzing (click buttons to trigger hidden APIs)
autocli explore https://www.example.com --auto --click "Comments,CC"

# Auto-detect authentication strategy (PUBLIC → COOKIE → HEADER)
autocli cascade https://api.example.com/hot

Discovery features:

.json suffix probing (Reddit-style REST discovery)
__INITIAL_STATE__ extraction (SSR sites like Bilibili, Xiaohongshu)
Pinia/Vuex store discovery and action mapping
Auto search endpoint discovery with --goal search
Framework detection (Vue/React/Next.js/Nuxt)

Download

Download media and articles from supported sites:

# Download Bilibili video (requires yt-dlp)
autocli bilibili download BV1xxx --output ./videos --quality 1080p

# Download Zhihu article as Markdown with images
autocli zhihu download "https://zhuanlan.zhihu.com/p/xxx" --output ./articles

# Download WeChat article as Markdown with images
autocli weixin download "https://mp.weixin.qq.com/s/xxx" --output ./articles

# Download Twitter/X media (images + videos)
autocli twitter download nash_su --limit 10 --output ./twitter
autocli twitter download --tweet-url "https://x.com/user/status/123" --output ./twitter

Download features:

Videos via yt-dlp (cookies extracted from browser automatically, no Keychain prompt)
Articles as Markdown with YAML frontmatter (title, author, date, source)
Images downloaded and localized (remote URLs replaced with local images/img_001.jpg)
Output directory structure: output/article_title/title.md + output/article_title/images/

External CLI Integration

Integrated external tools (passthrough execution):

Tool	Description
`gh`	GitHub CLI
`docker`	Docker CLI
`kubectl`	Kubernetes CLI
`obsidian`	Obsidian note management
`readwise`	Readwise reading management
`gws`	Google Workspace CLI

# Passthrough to GitHub CLI
autocli gh repo list

# Passthrough to kubectl
autocli kubectl get pods

Output Formats

Switch output format via the --format global flag:

autocli hackernews top --format table    # ASCII table (default)
autocli hackernews top --format json     # JSON
autocli hackernews top --format yaml     # YAML
autocli hackernews top --format csv      # CSV
autocli hackernews top --format md       # Markdown table

Authentication Strategies

Each command uses a different authentication strategy:

Strategy	Description	Requires Browser
`public`	Public API, no authentication needed	No
`cookie`	Requires browser Cookie	Yes
`header`	Requires specific request headers	Yes
`intercept`	Requires network request interception	Yes
`ui`	Requires UI interaction	Yes

Custom Adapters

Add custom adapters by creating YAML files under ~/.autocli/adapters/:

# ~/.autocli/adapters/mysite/hot.yaml
site: mysite
name: hot
description: My site hot posts
strategy: public
browser: false

args:
  limit:
    type: int
    default: 20
    description: Number of items

columns: [rank, title, score]

pipeline:
  - fetch: https://api.mysite.com/hot
  - select: data.posts
  - map:
      rank: "${{ index + 1 }}"
      title: "${{ item.title }}"
      score: "${{ item.score }}"
  - limit: "${{ args.limit }}"

Pipeline Steps

Step	Function	Example
`fetch`	HTTP request	`fetch: https://api.example.com/data`
`evaluate`	Execute JS in browser	`evaluate: "document.title"`
`navigate`	Page navigation	`navigate: https://example.com`
`click`	Click element	`click: "#button"`
`type`	Type text	`type: { selector: "#input", text: "hello" }`
`wait`	Wait	`wait: 2000`
`select`	Select nested data	`select: data.items`
`map`	Data mapping	`map: { title: "${{ item.title }}" }`
`filter`	Data filtering	`filter: "item.score > 10"`
`sort`	Sort	`sort: { by: score, order: desc }`
`limit`	Truncate	`limit: "${{ args.limit }}"`
`intercept`	Network interception	`intercept: { pattern: "/api/" }`
`tap`	State management bridge	`tap: { action: "store.fetch", url: "/api/" }`
`download`	Download	`download: { type: media }`

Template Expressions

Pipelines use the ${{ expression }} syntax:

# Variable access
"${{ args.limit }}"
"${{ item.title }}"
"${{ index + 1 }}"

# Comparison and logic
"${{ item.score > 10 }}"
"${{ item.title && !item.deleted }}"

# Ternary expressions
"${{ item.active ? 'yes' : 'no' }}"

# Pipe filters
"${{ item.title | truncate(30) }}"
"${{ item.tags | join(', ') }}"
"${{ item.name | lower | trim }}"

# String interpolation
"https://api.com/${{ item.id }}.json"

# Fallback
"${{ item.subtitle || 'N/A' }}"

# Math functions
"${{ Math.min(args.limit, 50) }}"

Built-in filters (16): default, join, upper, lower, trim, truncate, replace, keys, length, first, last, json, slugify, sanitize, ext, basename

Configuration

Environment Variables

Variable	Default	Description
`OPENCLI_VERBOSE`	-	Enable verbose output
`OPENCLI_DAEMON_PORT`	`19825`	Daemon port
`OPENCLI_CDP_ENDPOINT`	-	CDP direct endpoint (bypasses Daemon)
`OPENCLI_BROWSER_COMMAND_TIMEOUT`	`60`	Command timeout (seconds)
`OPENCLI_BROWSER_CONNECT_TIMEOUT`	`30`	Browser connection timeout (seconds)
`OPENCLI_BROWSER_EXPLORE_TIMEOUT`	`120`	Explore timeout (seconds)

File Paths

Path	Description
`~/.autocli/adapters/`	User custom adapters
`~/.autocli/plugins/`	User plugins
`~/.autocli/external-clis.yaml`	User external CLI registry

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                       User / AI Agent                           │
│                     autocli <site> <command>                  │
└─────────────────────┬───────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                      CLI Layer (clap)                            │
│  main.rs → discovery → clap dynamic subcommands → execution.rs  │
│  ┌───────────┐  ┌───────────────┐  ┌──────────────────┐        │
│  │ Built-in   │  │ Site adapter  │  │ External CLI     │        │
│  │ commands   │  │ commands      │  │ passthrough      │        │
│  │ explore    │  │ bilibili hot  │  │ gh, docker, k8s  │        │
│  │ doctor     │  │ twitter feed  │  │                  │        │
│  └───────────┘  └───────┬───────┘  └──────────────────┘        │
└─────────────────────────┼───────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────────┐
│                   Execution Engine (execution.rs)                │
│             Arg validation → Capability routing → Timeout ctrl  │
│                    ┌─────────┼─────────┐                        │
│                    ▼         ▼         ▼                        │
│              YAML Pipeline  Rust Func  External CLI              │
└────────────────┬────────────────────────────────────────────────┘
                 │
                 ▼
┌──────────────────────────────────────────────────────────────────┐
│  Pipeline Engine                    Browser Bridge                │
│  ┌────────────┐                      ┌─────────────────────┐    │
│  │ fetch      │                      │ BrowserBridge       │    │
│  │ evaluate   │  ──── IPage ────▶    │ DaemonClient (HTTP) │    │
│  │ navigate   │                      │ CdpPage (WebSocket) │    │
│  │ map/filter │                      └──────────┬──────────┘    │
│  │ sort/limit │                                 │               │
│  │ intercept  │                      Daemon (axum:19825)        │
│  │ tap        │                        HTTP + WebSocket          │
│  └────────────┘                                 │               │
│                                                 ▼               │
│  Expression Engine (pest)            Chrome Extension (CDP)      │
│  ${{ expr | filter }}                chrome.debugger API         │
└──────────────────────────────────────────────────────────────────┘

Workspace Structure

autocli/
├── crates/
│   ├── autocli-core/        # Core data models: Strategy, CliCommand, Registry, IPage trait, Error
│   ├── autocli-pipeline/    # Pipeline engine: pest expressions, executor, 14 step types
│   ├── autocli-browser/     # Browser bridge: Daemon, DaemonPage, CdpPage, DOM helpers
│   ├── autocli-output/      # Output rendering: table, json, yaml, csv, markdown
│   ├── autocli-discovery/   # Adapter discovery: YAML parsing, build.rs compile-time embedding
│   ├── autocli-external/    # External CLI: loading, detection, passthrough execution
│   ├── autocli-ai/          # AI capabilities: explore, synthesize, cascade, generate
│   └── autocli-cli/         # CLI entry point: clap, execution orchestration, doctor, completion
├── adapters/                   # 333 YAML adapter definitions
│   ├── hackernews/
│   ├── bilibili/
│   ├── twitter/
│   └── ...(55 sites)
└── resources/
    └── external-clis.yaml      # External CLI registry

Improvements over the TypeScript Original

Improvement	Original (TypeScript)	autocli (Rust)
Distribution	Node.js + npm install (~100MB)	Single binary (4.1MB)
Startup speed	Read manifest JSON → parse → register	Compile-time embedding, zero file I/O
Template engine	JS eval (security risk)	pest PEG parser (type-safe)
Concurrent fetch	Non-browser mode pool=5	FuturesUnordered, concurrency=10
Error system	Single hint string	Structured error chain + multiple suggestions
HTTP connections	New fetch each time	reqwest connection pool reuse
Memory safety	GC	Ownership system, zero GC pauses

Development

# Build
cargo build

# Test (166 tests)
cargo test --workspace

# Release build (with LTO, ~4MB)
cargo build --release

# Add a new adapter
# 1. Create a YAML file under adapters/<site>/
# 2. Recompile (build.rs auto-embeds)
cargo build

Supported Sites

Click to expand all 55 sites

Site	Commands	Strategy
hackernews	8	public
bilibili	12	cookie
twitter	24	cookie/intercept
reddit	15	public/cookie
zhihu	2	cookie
xiaohongshu	11	cookie
douban	7	cookie
weibo	2	cookie
v2ex	11	public/cookie
bloomberg	10	cookie
youtube	4	cookie
wikipedia	4	public
google	4	public/cookie
facebook	10	cookie
instagram	14	cookie
tiktok	15	cookie
notion	8	ui
cursor	12	ui
chatgpt	6	public
stackoverflow	4	public
devto	3	public
lobsters	4	public
medium	3	cookie
substack	3	cookie
weread	7	cookie
xueqiu	7	cookie
boss	14	cookie
jike	10	cookie
Other 27 sites	...	...

Star History

Acknowledgements

This project is built on top of OpenCLI by jackwener. We gratefully acknowledge the original work that made this project possible.

License

Apache-2.0

Name		Name	Last commit message	Last commit date
Latest commit History 106 Commits
.cargo		.cargo
.github/workflows		.github/workflows
adapters		adapters
assets		assets
crates		crates
docs/design		docs/design
extension		extension
logo		logo
prompts		prompts
scripts		scripts
.DS_Store		.DS_Store
.gitignore		.gitignore
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
Makefile		Makefile
NOTICE		NOTICE
README.ja.md		README.ja.md
README.md		README.md
README.zh.md		README.zh.md

Folders and files

Latest commit

History

Repository files navigation

AutoCLI

What's New

v0.3.2

🚀 Performance Comparison

Features

Installation

One-line Install Script (macOS / Linux)

Windows (PowerShell)

Manual Download (Simplest)

Build from Source

Update

Chrome Extension Setup (required for browser commands)

Skill Install

Quick Start

AI Commands

Step 1: Authenticate

Step 2: Use the Chrome Extension to precisely select the data you need from any website. Click the Generate button, and AI will automatically analyze the page, expand related data, and generate an adapter:

Step 3: Search Existing Adapters

Environment Variables

Built-in Commands

AI Discovery Capabilities

Download

External CLI Integration

Output Formats

Authentication Strategies

Custom Adapters

Pipeline Steps

Template Expressions

Configuration

Environment Variables

File Paths

Architecture

Workspace Structure

Improvements over the TypeScript Original

Development

Supported Sites

Star History

Acknowledgements

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 13

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages