ITR (Instruction-Tool Retrieval) is a sophisticated system for efficiently retrieving and assembling the most relevant instructions and tools for agentic Large Language Models (LLMs). It enables intelligent context management for AI agents by dynamically selecting the optimal subset of instructions and tools based on user queries and token budget constraints.
💡 Key Benefits: Reduce context bloat, improve tool selection and latency, and optimize token usage through intelligent hybrid retrieval and budget-aware selection algorithms.
⚠️ Note: This framework was developed to support research findings. For production use, it is recommended to either create a custom solution tailored to your specific needs or, better yet, contribute to improving this framework.
- ITR: Instruction-Tool Retrieval
- Hybrid Retrieval: Combines dense (embedding-based) and sparse (BM25) retrieval methods
- Budget-Aware Selection: Intelligent token budget management with greedy optimization
- Dynamic Chunking: Smart text chunking with configurable size ranges
- Fallback Mechanisms: Automatic expansion of tool sets for better coverage
- CLI Interface: Easy-to-use command-line interface for testing and integration
- Flexible Configuration: Configurable parameters for different use cases
- Type Safety: Full type hints for better development experience
# Install ITR
uv add instruction-tool-retrieval
# Or install from source
git clone https://github.com/uriafranko/ITR.git
cd ITR
uv syncpip install instruction-tool-retrievalgit clone https://github.com/uriafranko/ITR.git
cd ITR
uv sync --devfrom itr import ITR, ITRConfig
# Initialize ITR with custom configuration
config = ITRConfig(
k_a_instructions=3, # Max instructions to select
k_b_tools=2, # Max tools to select
token_budget=1500 # Total token budget
)
itr = ITR(config)
# Add instructions
itr.add_instruction(
"You are a helpful AI assistant. Be concise and accurate.",
metadata={"source": "base_personality", "priority": 1}
)
itr.add_instruction(
"Always prioritize safety and ethical considerations in your responses.",
metadata={"source": "safety_guidelines", "priority": 2}
)
# Add tools
calculator_tool = {
"name": "calculator",
"description": "Perform mathematical calculations and arithmetic operations",
"schema": {
"type": "object",
"properties": {
"expression": {"type": "string", "description": "Math expression to evaluate"}
}
}
}
itr.add_tool(calculator_tool)
# Perform retrieval
query = "What is 15% of 240?"
result = itr.step(query)
print(f"Selected {len(result.instructions)} instructions and {len(result.tools)} tools")
print(f"Total tokens: {result.total_tokens}")
print(f"Confidence: {result.confidence_score:.2f}")
# Get assembled prompt
prompt = itr.get_prompt(query)
print("\\nAssembled Prompt:")
print(prompt)from itr import ITR
itr = ITR()
# Load instructions from text file
itr.load_instructions("path/to/instructions.txt")
# Load tools from JSON file
itr.load_tools("path/to/tools.json")
# Perform retrieval
result = itr.step("How do I process a CSV file?")from itr import ITR, InstructionFragment, FragmentType
# Create fragments manually
fragments = [
InstructionFragment(
id="custom_1",
content="Use clear and concise language",
token_count=6,
fragment_type=FragmentType.STYLE_RULE,
metadata={"custom": True}
),
InstructionFragment(
id="custom_2",
content="Provide step-by-step explanations for complex topics",
token_count=9,
fragment_type=FragmentType.DOMAIN_SPECIFIC
)
]
itr = ITR()
itr.add_instruction_fragments(fragments)
result = itr.step("Explain machine learning")ITR provides a command-line interface for easy testing and integration:
itr retrieve --query "How do I calculate compound interest?" \\
--instructions instructions.txt \\
--tools tools.json \\
--show-promptitr interactiveitr init-config --output my-config.jsonitr retrieve --config my-config.json \\
--query "Analyze this dataset" \\
--instructions data_instructions.txt \\
--tools analysis_tools.jsonITR uses a flexible configuration system. You can customize behavior through the ITRConfig class:
from itr import ITRConfig
config = ITRConfig(
# Retrieval parameters
top_m_instructions=20, # Candidates to retrieve
top_m_tools=15, # Tool candidates to retrieve
k_a_instructions=4, # Max instructions to select
k_b_tools=2, # Max tools to select
# Scoring weights for hybrid retrieval
dense_weight=0.4, # Embedding similarity weight
sparse_weight=0.3, # BM25 score weight
rerank_weight=0.3, # Reranking weight
# Budget management
token_budget=2000, # Total token limit
safety_overlay_tokens=200, # Reserved tokens
# Fallback settings
confidence_threshold=0.7, # Trigger fallback below this
discovery_expansion_factor=2.0, # Tool expansion multiplier
# Model settings
embedding_model="all-MiniLM-L6-v2",
reranker_model="cross-encoder/ms-marco-MiniLM-L-6-v2"
)ITR supports JSON, YAML and .env configuration files:
{
"k_a_instructions": 3,
"k_b_tools": 2,
"token_budget": 1500,
"confidence_threshold": 0.8,
"embedding_model": "all-MiniLM-L6-v2"
}ITR uses a multi-stage pipeline for instruction and tool retrieval:
- Indexing: Instructions are chunked and indexed with both dense embeddings and sparse representations
- Retrieval: User queries are processed through hybrid retrieval (dense + sparse)
- Selection: Budget-aware selection algorithm picks optimal subset within token limits
- Assembly: Selected instructions and tools are assembled into final prompt
- Fallback: If confidence is low, expanded tool sets are used
ITR automatically categorizes instruction fragments:
ROLE_GUIDANCE: Defines AI agent roles and personasSTYLE_RULE: Formatting and communication style guidelinesSAFETY_POLICY: Safety and ethical constraintsDOMAIN_SPECIFIC: Task-specific instructionsEXEMPLAR: Examples and demonstrations
ITR is designed for efficiency:
- Fast Retrieval: Optimized hybrid search with caching
- Memory Efficient: Lazy loading and smart chunking
- Scalable: Handles large instruction/tool corpora
- Token Aware: Precise token counting and budget management
ITR achieves significant token reduction while maintaining high confidence and functionality across diverse analysis types.
We welcome contributions! Please see our Contributing Guidelines for details.
git clone https://github.com/uria-franko/ITR.git
cd ITR
uv sync --dev
# Run tests
uv run pytest
# Format code
uv run black .
uv run isort .
# Type checking
uv run mypy itr/This project is licensed under the MIT License - see the LICENSE file for details.
- PyPI Package: Coming Soon
- Issue Tracker: https://github.com/uriafranko/ITR/issues
Check out the examples directory for more detailed usage examples:
- Basic retrieval workflows
- Custom instruction creation
- Tool integration patterns
- Configuration examples
- Performance optimization tips
"Module not found" errors: Make sure you've installed all dependencies:
uv syncMemory issues with large corpora: Use chunking and increase available memory:
config = ITRConfig(chunk_size_range=(100, 400)) # Smaller chunksPoor retrieval quality: Try adjusting scoring weights:
config = ITRConfig(
dense_weight=0.6, # Increase embedding weight
sparse_weight=0.4 # Decrease keyword weight
)For more help, please open an issue on GitHub.
Built with ❤️ for the AI Agent community
⭐ Star this repo if you find ITR useful!