sitefetch

Fetch a site and extract its readable content as Markdown (to be used with AI models).

Install

Install globally

go install github.com/salman0ansari/sitefetch@latest

Usage

sitefetch https://hisalman.in --outfile site.txt

# or better concurrency
sitefetch https://hisalman.in --outfile site.txt --concurrency 10

Match specific pages

Use the --match flag to specify the pages you want to fetch:

sitefetch https://vite.dev --match "/blog/**,/guide/**"

Content selector

sitefetch https://vite.dev --content-selector ".content"

Credit

This project is a Go implementation of the original sitefetch written in TypeScript by egoist.