Skip to content

fix: self-heal stale url-ingestion provider state#1207

Open
Wallgau wants to merge 5 commits intorelease-0.4.0from
ingest-url-from-chat
Open

fix: self-heal stale url-ingestion provider state#1207
Wallgau wants to merge 5 commits intorelease-0.4.0from
ingest-url-from-chat

Conversation

@Wallgau
Copy link
Collaborator

@Wallgau Wallgau commented Mar 20, 2026

This PR fixes URL-ingestion failures caused by stale provider/runtime state in Langflow and MCP wiring.

  • Scopes ingestion and MCP provider credentials to the selected embedding provider instead of sending all provider vars.
  • Removes stale provider headers (for example old OLLAMA_BASE_URL) from MCP server args so non-selected providers are not accidentally used.
  • Adds conditional self-heal for URL ingestion: on known stale-state signatures (for example “Failed to connect to Ollama” while provider is not Ollama), reconcile once and retry once.
  • Improves retry-failure errors with provider-aware messaging so failures are easier to diagnose.

Issue: #1170

  • Users could still hit “Failed to connect to Ollama” even when OpenAI was selected, due to stale runtime/MCP state and mixed provider headers. This PR makes provider selection authoritative and adds safe, targeted recovery.

What Changed

  • src/utils/langflow_headers.py
  • Provider-scoped credential/header/global-var generation.
  • src/services/langflow_mcp_service.py
  • Prunes stale provider headers not in the desired selected-provider set.
  • src/services/langflow_file_service.py
  • Uses selected provider when building ingestion headers.
  • Supports dynamic URL-ingest flow-id resolution when available.
  • Adds conditional stale-state reconcile + single retry path.
  • Adds provider-specific retry error formatting.

Tests
Added TDD coverage with mocked HTTP (CI-safe, no real external calls):

  • tests/unit/test_url_ingestion_pipeline_tdd.py
  • selected-provider-only ingestion headers
  • prefers resolved flow-id over stale configured id
  • reconcile-and-retry once on known stale-state errors
  • no retry on non-reconcilable errors
  • provider-specific error message after retry failure
  • integration-style mocked chain for provider selection + self-heal

tests/unit/test_mcp_provider_selection_tdd.py

  • provider-scoped MCP global vars
  • stale provider header pruning
  • server update applies selected-provider vars only

Before user will get error trying to ingest url:
Screenshot 2026-03-20 at 11 03 30 AM

Now urls are ingest properly and reflect in the knowledge table:
Screenshot 2026-03-20 at 11 02 35 AM

Scope provider credentials and MCP global vars to the selected embedding provider, prune stale provider headers, and retry URL ingestion once after targeted stale-state reconciliation errors.

Made-with: Cursor
@github-actions github-actions bot added bug 🔴 Something isn't working. backend 🔷 Issues related to backend services (OpenSearch, Langflow, APIs) tests labels Mar 20, 2026
Olfa Maslah and others added 2 commits March 20, 2026 17:37
Include all configured provider credentials in Langflow global vars and add tests for single-provider and multi-provider ingestion scenarios to prevent regressions when documents are embedded with different models.

Made-with: Cursor
@github-actions github-actions bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Mar 20, 2026
@github-actions github-actions bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Mar 24, 2026
@github-actions github-actions bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend 🔷 Issues related to backend services (OpenSearch, Langflow, APIs) bug 🔴 Something isn't working. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants