Skip to content

Add automated Vercel-style llms.txt generation#179

Open
fcakyon wants to merge 5 commits intomainfrom
add-llms-txt
Open

Add automated Vercel-style llms.txt generation#179
fcakyon wants to merge 5 commits intomainfrom
add-llms-txt

Conversation

@fcakyon
Copy link
Member

@fcakyon fcakyon commented Dec 20, 2025

Adds automatic Vercel-style llms.txt file generation after site build.

Generated file: llms.txt

ultralytics-mkdocs-llmstxt.mp4

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Adds automatic generation of a Vercel-style llms.txt file after MkDocs builds complete. 🤖📄

📊 Key Changes

  • Introduces a new plugin option add_llms_txt (default True) and hooks into on_post_build to generate llms.txt.
  • Implements generate_llms_txt() in plugin/postprocess.py to create site/llms.txt from nav (preferred) or by scanning docs/**/*.md.
  • Extracts optional page descriptions from Markdown frontmatter and includes them alongside links for richer LLM-friendly indexing.
  • Extends standalone postprocess_site() to also generate llms.txt when add_llms_txt=True.

🎯 Purpose & Impact

  • Improves LLM consumption of the docs by providing a single structured index of key pages and sections.
  • Keeps llms.txt automatically in sync with the site navigation and build output, reducing manual maintenance.
  • Enables consistent output both in-plugin (MkDocs build) and in standalone postprocessing flows.

@UltralyticsAssistant UltralyticsAssistant added documentation Improvements or additions to documentation enhancement New feature or request labels Dec 20, 2025
@UltralyticsAssistant
Copy link
Member

👋 Hello @fcakyon, thank you for submitting a ultralytics/mkdocs 🚀 PR! This is an automated message—an engineer will assist with review shortly. 🤖✨

  • Define a Purpose: Clearly explain the purpose of your fix or feature in your PR description, and link to any relevant issues. Ensure your commit messages are clear, concise, and adhere to the project's conventions.
  • Synchronize with Source: Confirm your PR is synchronized with the ultralytics/mkdocs main branch. If it's behind, update it by clicking the 'Update branch' button or by running git pull and git merge main locally.
  • Ensure CI Checks Pass: Verify all Ultralytics Continuous Integration (CI) checks are passing. If any checks fail, please address the issues.
  • Update Documentation: Update the relevant documentation for any new or modified features.
  • Add Tests: If applicable, include or update tests to cover your changes, and confirm that all tests are passing.
  • Sign the CLA: Please ensure you have signed our Contributor License Agreement if this is your first Ultralytics PR by writing "I have read the CLA Document and I sign the CLA" in a new message.
  • Minimize Changes: Limit your changes to the minimum necessary for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

For more guidance, please refer to our Contributing Guide. Don't hesitate to leave a comment if you have any questions. Thank you for contributing to Ultralytics! 🚀

Copy link
Member

@UltralyticsAssistant UltralyticsAssistant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 PR Review

Made with ❤️ by Ultralytics Actions

Good addition overall: the feature is scoped, integrates cleanly via on_post_build, and the standalone flow is updated consistently. Main issues to address are correctness of nav traversal (top-level strings / single-page sections are currently skipped) and ensuring Path/path normalization is reliable at runtime. Minor note: avoid unconditional print() noise in build logs.

💬 Posted 4 inline comments

@fcakyon fcakyon requested a review from glenn-jocher December 20, 2025 22:21
@fcakyon fcakyon self-assigned this Dec 21, 2025
fcakyon and others added 3 commits December 24, 2025 16:43
Add seen_urls set to track URLs already added, skip duplicates.
Each URL now appears only once (first occurrence wins).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants