Skip to content

Conversation

@EmanueleDeRossi1
Copy link
Collaborator

@EmanueleDeRossi1 EmanueleDeRossi1 commented Dec 1, 2025

This PR introduces changes from the feature/add-test-connection-tool branch.

📝 Summary

This PR introduces a “Test Connection” feature for MCP tools — users can now verify their MCP credentials before saving a tool.

Backend: new POST /mcp/test-connection endpoint. The MCP agent attempts to authenticate using the provided credentials and checks for a 401 Unauthorized response. Success or failure is returned accordingly.

Frontend: added a “Test Connection” button in the MCP connection dialog. Inline alerts show success or error. Testing the MCP configuration is now required: users can not save the MCP tool without having successfully tested the connection.

Smaller changes: Removed the robot icon from the button and removed duplicate toast notifications for cleaner UX.

📁 Files Changed ( 8 files)

apps/backend/src/rhesis/backend/app/routers/services.py
apps/backend/src/rhesis/backend/app/schemas/services.py
apps/backend/src/rhesis/backend/app/services/mcp_service.py
apps/backend/src/rhesis/backend/app/templates/mcp_test_auth_prompt.jinja2
apps/frontend/src/app/(protected)/knowledge/components/MCPImportDialog.tsx
apps/frontend/src/app/(protected)/mcp/components/MCPConnectionDialog.tsx
apps/frontend/src/utils/api-client/interfaces/test-configuration.ts
apps/frontend/src/utils/api-client/services-client.ts

📋 Commit Details

c0d6f0c7 - feat(mcp): require connection test before saving tools (Emanuele De Rossi, 2025-12-03 15:56)
7b4a2ebf - feat(frontend): make TestConfigurationUpdate a partial of TestConfigurationBase (Emanuele De Rossi, 2025-12-03 11:04)
53889f60 - feat(mcp): test connections without creating tools (Emanuele De Rossi, 2025-12-03 10:54)
551acc16 - feat(frontend): remove duplicate success notifications in MCP Import Dialog and MCP Connect Dialog (Emanuele De Rossi, 2025-12-02 16:28)
c9279f77 - feat(frontend): remove robot icon from Test Connection button (Emanuele De Rossi, 2025-12-02 16:11)
e854b53c - feat(frontend): add test connection button to MCP connection dialog (Emanuele De Rossi, 2025-12-01 11:03)
6d1d37dc - feat(mcp): simplify test connection response to use JSON from LLM (Emanuele De Rossi, 2025-12-01 10:01)
e5497763 - feat(mcp): add test connection endpoint (Emanuele De Rossi, 2025-12-01 09:10)

@codecov
Copy link

codecov bot commented Dec 1, 2025

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
1440 1 1439 86
View the full list of 1 ❄️ flaky test(s)
backend/services/connector/test_handler.py::TestSDKMessageHandler::test_sync_function_endpoints_error

Flake rate in main: 100.00% (Passed 0 times, Failed 10 times)

Stack Traces | 0.062s run time
.../services/connector/test_handler.py:137: in test_sync_function_endpoints_error
    assert result["status"] == "success"  # Registration still succeeds
E   AssertionError: assert 'error' == 'success'
E     
E     - success
E     + error

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@EmanueleDeRossi1 EmanueleDeRossi1 marked this pull request as draft December 1, 2025 09:00
@EmanueleDeRossi1 EmanueleDeRossi1 linked an issue Dec 1, 2025 that may be closed by this pull request
5 tasks
@EmanueleDeRossi1 EmanueleDeRossi1 force-pushed the feature/add-test-connection-tool branch from 1ab3618 to 551acc1 Compare December 2, 2025 15:55
@EmanueleDeRossi1 EmanueleDeRossi1 marked this pull request as ready for review December 2, 2025 16:02
@EmanueleDeRossi1 EmanueleDeRossi1 marked this pull request as draft December 2, 2025 16:30
@EmanueleDeRossi1 EmanueleDeRossi1 marked this pull request as ready for review December 3, 2025 10:21
@EmanueleDeRossi1 EmanueleDeRossi1 marked this pull request as draft December 3, 2025 13:30
@EmanueleDeRossi1 EmanueleDeRossi1 force-pushed the feature/add-test-connection-tool branch from c0d6f0c to 33971e7 Compare December 4, 2025 07:22
- Add /mcp/test-connection endpoint to verify MCP authentication
- Implement test_mcp_authentication service function using MCPAgent
- Add TestMCPConnectionRequest and TestMCPConnectionResponse schemas
- Create mcp_test_auth_prompt.jinja2 template for authentication testing
- Parse LLM response to determine authentication status (Yes/No)
- Change TestMCPConnectionResponse to use string is_authenticated (Yes/No) and message
- Update prompt template to return JSON format with is_authenticated and message
- Simplify service function to directly parse and return JSON from final_answer
- Remove complex text parsing logic in favor of direct JSON parsing
- Add testMCPConnection method to ServicesClient
- Add test connection button in MCPConnectionDialog for both create and edit modes
- In create mode, temporarily create tool for testing and clean up on failure
- Display test results with success/error alerts showing authentication status and message
- Handle tool creation when test succeeds in create mode
Allow testing MCP connections using provider_type_id, credentials, and tool_metadata parameters instead of requiring a tool in the database.

- Backend: Add parameter-based testing support to test_mcp_authentication()
- Frontend: Simplify MCPConnectionDialog to use direct parameters
- Maintains backward compatibility with tool_id-based calls
Add validation to require successful connection test before saving MCP tools, matching the models workflow.
@harry-rhesis
Copy link
Contributor

@EmanueleDeRossi1 thanks for implementing this feature, it makes it more convenient and easy for user!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Backend Frontend Integrations Issues around integrations with tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add connection testing for tool integrations

3 participants