Service layer validation in tool classifier

[nuwangeek]

This pull request introduces significant improvements to the tool classifier's workflow orchestration, service enrichment, and service discovery logic. The main changes include implementing a layer-wise fallback chain for workflow execution, enhancing service enrichment by combining metadata and generated context for embeddings, and providing SQL and workflow logic for adaptive service discovery based on active service count. Constants for configuration have also been centralized for easier management.

Workflow orchestration and fallback logic:

• Implemented a layer-wise fallback chain in both _execute_with_fallback_async and _execute_with_fallback_streaming methods, allowing the classifier to try each workflow layer in order and only proceed to the next if the previous returns None. This ensures robust handling and proper fallback to RAG/OOD workflows. [1] [2] [3] [4]
• Updated the classify method to start with the SERVICE workflow and cascade through layers, reflecting the new fallback chain logic and improving intent detection flow. [1] [2]

Service enrichment improvements:

• Enhanced the enrichment process by combining generated context with original service metadata (name, description, examples, entities) before creating embeddings, resulting in richer vector representations for semantic search.
• Adjusted enrichment workflow step numbering to reflect the new process, and clarified logging for each step.
• Updated enrichment prompt to instruct context generation in the same language as the service description, supporting multilingual enrichment.

Service discovery and adaptive workflow:

• Added SQL scripts for counting active services (count-active-services.sql), retrieving all active services (get-all-active-services.sql), and fetching a service by ID (get-service-by-id.sql), supporting adaptive search strategies. [1] [2] [3]
• Introduced a new Ruuter workflow (get-services.yml) that returns either all services or signals to use semantic search based on the active service count, enabling dynamic intent detection strategies.

Configuration and constants:

• Centralized Qdrant, semantic search, and Ruuter configuration in constants.py, making thresholds and endpoints easier to manage and update.

Cleanup and refactoring:

• Removed the obsolete enrich.yml.backup file, reflecting a shift to new enrichment and indexing workflows.
• Passed the orchestration_service to ServiceWorkflowExecutor for improved orchestration.

These changes collectively modernize the classifier's workflow handling, improve service enrichment quality, and enable adaptive service discovery for more scalable and accurate intent detection.

[Copilot]

Pull request overview

This pull request implements a comprehensive service workflow layer (Layer 1) for the tool classifier system, introducing intelligent service intent detection, semantic search capabilities, and adaptive service discovery. The changes enable the system to route external API/service call queries through a sophisticated LLM-based intent detection pipeline before falling back to context or RAG workflows.

Changes:

• Implemented a complete service workflow executor with semantic search, intent detection via DSPy, entity extraction/validation, and layer-wise fallback logic
• Enhanced service enrichment to combine original metadata with LLM-generated context for richer vector embeddings
• Added adaptive service discovery workflow that switches between returning all services or signaling semantic search based on active service count
• Centralized configuration constants for Qdrant, semantic search, and Ruuter endpoints

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
src/tool_classifier/workflows/service_workflow.py	Complete implementation of service workflow with semantic search, intent detection, entity validation, and cost tracking
src/tool_classifier/intent_detector.py	New DSPy-based intent detection module for matching user queries to services and extracting entities
src/tool_classifier/constants.py	Centralized configuration constants for Qdrant, semantic search thresholds, and Ruuter endpoints
src/tool_classifier/classifier.py	Updated fallback chain to cascade through SERVICE → CONTEXT → RAG → OOD layers
src/intent_data_enrichment/main_enrichment.py	Enhanced enrichment to combine service metadata with generated context for embeddings
src/intent_data_enrichment/constants.py	Added multilingual instruction to enrichment prompt
DSL/Ruuter.public/rag-search/GET/services/get-services.yml	New workflow for adaptive service discovery based on count threshold
DSL/Resql/rag-search/POST/count-active-services.sql	SQL query to count active services
DSL/Resql/rag-search/POST/get-all-active-services.sql	SQL query to retrieve all active services for intent detection
DSL/Resql/rag-search/POST/get-service-by-id.sql	SQL query to fetch specific service by ID for validation
docs/TOOL_CLASSIFIER_AND_SERVICE_WORKFLOW.md	Comprehensive documentation of architecture and implementation details
enrich.yml.backup	Removed obsolete backup file

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The code extracts nested response data from discovery_result.get("response", {}) but then also tries to extract from the top level with response_data.get("use_semantic_search", False). However, based on the Ruuter workflow implementation in get-services.yml, the response structure returns data at the top level (not nested under "response"). This mismatch could cause the logic to fail. Verify the actual response structure from the Ruuter endpoint and adjust accordingly.

[Copilot]

When semantic search fails and services_from_ruuter is available, the code uses it directly (line 468). However, services_from_ruuter was extracted from the potentially incorrect nested path response_data.get("services", []). If the response structure issue at line 438 is corrected, this fallback path should also be verified to ensure it's extracting services from the correct location.

[Thirunayan22]

This is already implemented in service module right @nuwangeek ? Any reason we are having a resql service here too?

[nuwangeek]

@Thirunayan22 yes since we haven't integrated with service module we need this as mock resql

[Thirunayan22]

then can we add a "mock-" prefix just to be clear?

[Thirunayan22]

This is also already implemented in service module

[nuwangeek]

@Thirunayan22 Same as above