Conversation
… guide The old recipe described what to do but left the agent to figure out the boilerplate and token discovery. Since adding a reasoning parser is almost entirely mechanical (4 inputs → 4 files touched), the new recipe provides: Token discovery guide (6-step priority order): 1. Check vLLM (17 models) and SGLang implementations first — tokens already validated against real model output 2. HuggingFace tokenizer_config.json → added_tokens_decoder 3. HuggingFace tokenizer_config.json → chat_template Jinja2 4. HuggingFace model card / README 5. Provider API docs 6. Send test request and observe raw output Reference tables: - 9 model families currently in SMG with exact tokens and flags - 8 model families in vLLM/SGLang not yet in SMG (candidates to add) - initial_in_reasoning decision guide Template: - Complete parameterized Rust parser file with 7 tests - Exact registration steps for mod.rs, lib.rs, factory.rs - Custom logic section for MiniMax-style edge cases - Common mistakes table with consequences and fixes Signed-off-by: Simo Lin <slin1237@gmail.com> Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewrite the reasoning-parser recipe from a vague description into a parameterized template with a concrete token discovery guide referencing vLLM and SGLang.
Problem
The old recipe told you to "check model card or API docs for reasoning delimiters" — but reasoning tokens live in different places depending on the model. Contributors had to rediscover this every time.
Changes
skills/implement/reasoning-parser.md(321 insertions, 44 deletions):tokenizer_config.json→added_tokens_decodertokenizer_config.json→chat_templatehttps://github.com/vllm-project/vllm/tree/main/vllm/reasoninghttps://github.com/sgl-project/sglang/blob/main/python/sglang/srt/parser/reasoning_parser.pyinitial_in_reasoningguide: Checkadd_generation_promptor observe first tokenTest Plan