Market Signal Terminal

×

Market Signal Terminal

Ask one market question. Get a live-scraped evidence graph.

An AI research terminal that plans queries, scrapes the web in real time via Bright Data, and builds a linked knowledge graph — streamed live to a map-first workspace.

---

What It Does

You type a market question — "Why is BTC down today?" or "NVDA post-earnings: what's the evidence?" — and the terminal:

Plans 4–6 targeted search queries using an LLM
Searches Google SERP concurrently via Bright Data (news + web verticals)
Scrapes the top source pages for full-text markdown via Bright Data Web Unlocker (deep mode)
Extracts structured evidence: titles, excerpts, entities, catalysts, sentiment
Builds a knowledge graph — assets, events, sources, entities — linked with typed, confidence-scored edges
Streams everything live to a map-first workspace via Server-Sent Events
Answers follow-up questions grounded strictly in the collected evidence

No static datasets. No hallucinated sources. Every answer traces back to a scraped URL.

Architecture

┌───────────────────────────────────────────────────────────────────┐
│                          Browser  (SSE)                           │
│  ┌────────────────┐  ┌─────────────────┐  ┌────────────────────┐ │
│  │  Evidence Map  │  │    Timeline      │  │  Chat (grounded)   │ │
│  │  (graph viz)   │  │  (tape + events) │  │  /api/chat         │ │
│  └───────┬────────┘  └────────┬─────────┘  └─────────┬──────────┘ │
└──────────┼───────────────────┼──────────────────────┼────────────┘
           │                   │   SSE stream          │
           ▼                   ▼                       ▼
┌─────────────────────────────────────────────────────────────────────┐
│                    /api/run  — 8-stage pipeline                     │
│                                                                     │
│   plan → search → scrape → extract → summaries → artifacts         │
│                                     → link → render → ready        │
└──────────┬─────────────────┬──────────────────────┬────────────────┘
           │                 │                      │
           ▼                 ▼                      ▼
   ┌──────────────┐  ┌──────────────────┐  ┌───────────────┐
   │  Bright Data │  │  LLM             │  │    Convex     │
   │  SERP +      │  │  OpenRouter      │  │  session DB + │
   │  Unlocker    │  │  (Gemini 3)      │  │  event log    │
   └──────────────┘  └──────────────────┘  └───────────────┘

Pipeline Stages

Stage	What Happens
plan	LLM generates 4–6 targeted search queries from your topic
search	4–6 concurrent Bright Data SERP requests (news → web fallback)
scrape	Top 4 URLs extracted to markdown via Bright Data Web Unlocker (deep mode)
extract	SERP snippets + markdown merged into structured evidence items
summaries	LLM extracts bullets, entities, catalysts, sentiment per item (deep mode)
artifacts	LLM builds knowledge graph nodes + edges with confidence scores
link	Heuristic enrichment: connectivity repair, domain tagging, tape events
render → ready	Final artifacts streamed to client, session persisted to Convex

The Evidence Graph

The core output is a typed knowledge graph — not a flat list of links.

Node types

Type	Examples
`asset`	`BITCOIN`, `NVDA`, `DXY`
`entity`	`Federal Reserve`, `BlackRock`, `MicroStrategy`
`event`	`ETF Approval`, `Fed Rate Hike`, `Earnings Miss`
`source`	`reuters.com`, `bloomberg.com`, `coindesk.com`, `forexfactory.com`

Edge types

Type	Meaning
`hypothesis`	Event → Asset (this catalyst may explain this move)
`mentions`	Source → Event (this outlet covered this catalyst)
`co_moves`	Entity ↔ Asset (correlated in the evidence)
`same_story`	Event ↔ Event (same narrative, different angles)

Every edge carries a confidence score [0, 1] and cites the evidence IDs that support it.

Evidence Views

The terminal offers four ways to explore the same underlying data:

View	Description
Graph	Interactive force-directed knowledge graph — drag, zoom, click nodes
Flow	Directional flow diagram showing causal chains
Mind Map	Hierarchical view radiating from the primary asset
Timeline	Chronological tape of events ordered by publish date

Tech Stack

Layer	Technology
Framework	Next.js 16 (App Router) + React 19
Styling	Tailwind CSS 4
Realtime DB	Convex — sessions, event log, TTL cleanup
Web Data	Bright Data — SERP + Web Unlocker
AI	OpenRouter — Gemini 3 Flash Preview (configurable per pipeline stage)
Graph Viz	`react-force-graph-2d`
Validation	Zod
Icons	lucide-react

Project Structure

market-terminal/
├── src/
│   ├── app/
│   │   ├── api/
│   │   │   ├── run/              # Main pipeline — SSE orchestrator
│   │   │   ├── chat/             # Evidence-grounded Q&A
│   │   │   ├── sessions/         # Listing, event trace, snapshots
│   │   │   ├── health/           # Config check + live dependency probes
│   │   │   └── serp/             # Direct SERP search (debug/test)
│   │   ├── terminal/             # Interactive evidence workspace
│   │   ├── dashboard/            # Session history + event replay
│   │   └── how-it-works/         # Architecture documentation page
│   ├── components/
│   │   ├── terminal/
│   │   │   ├── Terminal.tsx          # Workspace orchestrator
│   │   │   ├── EvidenceGraph.tsx     # Force-directed graph
│   │   │   ├── EvidenceFlow.tsx      # Flow diagram
│   │   │   ├── EvidenceMindMap.tsx   # Mind map
│   │   │   ├── EvidenceTimeline.tsx  # Chronological tape
│   │   │   └── PipelineTimeline.tsx  # Live pipeline progress bar
│   │   └── dashboard/
│   │       └── SessionDashboard.tsx  # History + replay UI
│   ├── lib/
│   │   ├── brightdata.ts         # SERP + Web Unlocker client
│   │   ├── ai.ts                 # LLM client + JSON parsing utilities
│   │   ├── modelRouting.ts       # Provider/model selection per stage
│   │   └── env.ts                # Typed environment configuration
│   └── prompts/                  # Multi-stage LLM prompt templates
│       ├── signalTerminalPlan.ts
│       ├── signalTerminalSummaries.ts
│       ├── signalTerminalArtifacts.ts
│       ├── signalTerminalImpact.ts
│       └── signalTerminalChat.ts
├── convex/
│   ├── schema.ts                 # sessions + sessionEvents table definitions
│   ├── sessions.ts               # CRUD, search, 24h TTL cleanup
│   └── sessionEvents.ts          # Event log insert + query
└── public/
    ├── brightdata.svg
    └── convex.svg

Getting Started

Prerequisites

Node.js 20+
A Bright Data account with SERP and Web Unlocker zones configured
An OpenRouter API key
A Convex project (free tier works)

1. Install dependencies

cd market-terminal
npm install

2. Configure environment

cp .env.local.example .env.local

# Bright Data
BRIGHTDATA_API_TOKEN=your_token_here
BRIGHTDATA_WEB_UNLOCKER_ZONE=unblocker
BRIGHTDATA_SERP_ZONE=serp

# AI — OpenRouter
OPENROUTER_API_KEY=sk-or-...
OPENROUTER_MODEL=google/gemini-3-flash-preview

# Convex
CONVEX_DEPLOYMENT=prod:your-deployment-slug
NEXT_PUBLIC_CONVEX_URL=https://your-deployment.convex.cloud

3. Deploy Convex schema

npx convex dev

4. Run locally

npm run dev

Open http://localhost:3000/market-terminal

API Reference

`POST /api/run`

Starts the research pipeline. Returns a Server-Sent Events stream.

{
  "topic": "Bitcoin",
  "question": "Why is BTC down today?",
  "mode": "fast"
}

Mode	Behavior
`fast`	SERP only, broad time window, lighter AI model — ~3–5s
`deep`	SERP + full-page markdown extraction, richer AI summaries — ~8–15s

SSE event types: session · plan · search.partial · search · scrape.page · data · step · warn · perf.mark · perf.summary · ai.usage

# Example curl
curl -N -X POST http://localhost:3000/market-terminal/api/run \
  -H 'Content-Type: application/json' \
  -d '{"topic":"Bitcoin","mode":"fast"}'

`POST /api/chat`

Answer a follow-up question grounded in a session's collected evidence.

{
  "sessionId": "uuid",
  "message": "What is the strongest bullish signal?",
  "focusEvidenceIds": ["ev_1", "ev_2"]
}

`GET /api/sessions`

List sessions with artifact counts and narrative map tags.

GET /api/sessions?limit=50&q=bitcoin&status=ready

`GET /api/health?probe=1`

Live health check — actively tests Bright Data SERP, Web Unlocker, AI model, and Convex connectivity.

How Bright Data Powers This

Bright Data is the data backbone of every run:

SERP — Concurrent multi-query Google search with news/web vertical routing, domain diversity scoring, and automatic staleness filtering. Results are scored for editorial authority and catalyst-signal density before being fed to the LLM.
Web Unlocker — Full-page markdown extraction from any URL, bypassing bot detection to deliver clean, readable content for deep-mode evidence enrichment.
Resilience — Exponential backoff on 429/5xx responses, news→web fallback on empty verticals, best-effort scrape failures so the pipeline always completes and streams a result.

How Convex Powers This

Convex is the persistent backbone behind each research session:

Session records — Full metadata, pipeline progress, and final artifacts (evidence, tape, nodes, edges, clusters) stored in meta.artifacts
Event log — Every SSE event is persisted to sessionEvents, enabling full replay, debugging, and performance analysis from the Dashboard
Auto-expiry — Sessions are cleaned up after 24 hours via a Convex scheduled mutation
Non-blocking writes — Convex persistence is fire-and-forget; the pipeline streams to the client regardless of DB latency

Dashboard & Replay

Every run is logged. The /dashboard page shows all sessions with:

Topic, status, mode, model used
Artifact counts (evidence items, graph nodes/edges, narrative clusters)
Map tags (top entities, catalysts, narrative themes extracted automatically)
Full event trace with per-stage timing breakdowns

Click any session to re-open its evidence workspace — no re-run required.

Observability

The pipeline emits granular perf.mark events for every sub-operation:

plan           →   450ms
serp[0]        →   820ms
serp[1]        →   790ms
scrape[0]      →  1240ms
ai.summaries   →  2100ms
ai.artifacts   →  3300ms
──────────────────────────
total          →   ~6.1s

ai.usage events log input/output token counts per LLM call. The /api/health?probe=1 endpoint actively probes all external dependencies and reports configuration status.

Resources

Building a Live AI Market Research Terminal: How Bright Data and Convex Replace Polling With Real-Time Everything — Full architecture walkthrough covering both Bright Data paths, Convex roles, the reflection stage, and model selection logic.

Built with and

Name		Name	Last commit message	Last commit date
Latest commit History 10 Commits
.github/workflows		.github/workflows
convex		convex
public		public
src		src
.env.local.example		.env.local.example
.gitignore		.gitignore
README.md		README.md
eslint.config.mjs		eslint.config.mjs
next.config.ts		next.config.ts
package-lock.json		package-lock.json
package.json		package.json
postcss.config.mjs		postcss.config.mjs
tsconfig.json		tsconfig.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Market Signal Terminal

What It Does

Architecture

Pipeline Stages

The Evidence Graph

Evidence Views

Tech Stack

Project Structure

Getting Started

Prerequisites

1. Install dependencies

2. Configure environment

3. Deploy Convex schema

4. Run locally

API Reference

`POST /api/run`

`POST /api/chat`

`GET /api/sessions`

`GET /api/health?probe=1`

How Bright Data Powers This

How Convex Powers This

Dashboard & Replay

Observability

Resources

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

Market Signal Terminal

What It Does

Architecture

Pipeline Stages

The Evidence Graph

Evidence Views

Tech Stack

Project Structure

Getting Started

Prerequisites

1. Install dependencies

2. Configure environment

3. Deploy Convex schema

4. Run locally

API Reference

POST /api/run

POST /api/chat

GET /api/sessions

GET /api/health?probe=1

How Bright Data Powers This

How Convex Powers This

Dashboard & Replay

Observability

Resources

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

`POST /api/run`

`POST /api/chat`

`GET /api/sessions`

`GET /api/health?probe=1`

Packages