add technical documentation command

[karian7]

Add sc-document.md command for generating both developer and user documentation with automated screenshot integration and cross-referencing capabilities.

Result - dev document

Question Feature Implementation Document

Overview

This is a chat system that enables AI question-and-answer based on video file subtitles. Users can click the "Question" button from the video list to converse with OpenAI GPT about the subtitle content of that video.

Architecture

Frontend (React/TypeScript)

src/
├── components/
│   ├── ChatPanel.tsx         # Chat UI panel component
│   └── VideoListItem.tsx     # Video item with question button
├── types/index.ts            # ChatSession, ChatMessage type definitions
└── services/tauri.ts         # Tauri backend communication service

Backend (Rust/Tauri)

src-tauri/src/
├── main.rs                   # Tauri commands and event handling
├── chat.rs                   # Chat session and message structures
├── ai.rs                     # OpenAI API client
└── database.rs               # SQLite schema and queries

Database Schema

chat_sessions table

CREATE TABLE chat_sessions (
    id TEXT PRIMARY KEY,
    video_id TEXT NOT NULL,
    ai_provider TEXT NOT NULL DEFAULT 'openai',
    model TEXT NOT NULL DEFAULT 'gpt-4.1',
    system_prompt TEXT NOT NULL,
    subtitle_snapshot TEXT,
    created_at TEXT NOT NULL,
    updated_at TEXT NOT NULL,
    FOREIGN KEY (video_id) REFERENCES video_items (id)
);

chat_messages table

CREATE TABLE chat_messages (
    id TEXT PRIMARY KEY,
    session_id TEXT NOT NULL,
    role TEXT NOT NULL CHECK (role IN ('system', 'user', 'assistant', 'developer')),
    content TEXT NOT NULL,
    created_at TEXT NOT NULL,
    FOREIGN KEY (session_id) REFERENCES chat_sessions (id)
);

API Specification

Tauri Commands

get_or_create_chat_session

Retrieves or creates a new chat session.

Parameters:

• video_id: String - Video ID
• ai_provider: Option<String> - AI provider (default: "openai")
• model: Option<String> - Model to use (default: "gpt-4.1")

Return value:

struct ChatSessionInitPayload {
    session: ChatSessionView,
    messages: Vec<ChatMessageView>,
    ai_ready: bool,
}

Implementation logic:

1. Search for existing session (based on video_id, provider, model)
2. Create new session if none exists
3. Read subtitle file to generate subtitle_snapshot
4. Set system prompt
5. Return existing message list

submit_chat_message

Sends user message and receives AI response via streaming.

Parameters:

• session_id: String - Chat session ID
• message: String - User message

Return value:

struct SubmitChatResponsePayload {
    session: ChatSessionView,
    user_message: ChatMessageView,
    assistant_message: ChatMessageView,
}

Implementation logic:

1. Save user message to DB
2. Retrieve recent message history (max 10)
3. Call OpenAI Responses API (streaming)
4. Emit Tauri events during streaming
5. Save AI response to DB when complete

Tauri Events

chat_stream_start

Emitted when AI response streaming starts

interface ChatStreamStartEvent {
  session_id: string;
  assistant_message_id: string;
}

chat_stream_delta

Emitted when AI response chunk is received

interface ChatStreamDeltaEvent {
  session_id: string;
  assistant_message_id: string;
  delta: string;
}

chat_stream_end

Emitted when AI response is complete

interface ChatStreamEndEvent {
  session_id: string;
  assistant_message_id: string;
}

chat_stream_error

Emitted when error occurs during AI response

interface ChatStreamErrorEvent {
  session_id: string;
  assistant_message_id: string;
  error: string;
}

Frontend Implementation

ChatPanel Component

Main component responsible for chat UI.

Key features:

• Display message list (user/AI distinction)
• Text input and send (Enter/Shift+Enter support)
• Real-time streaming response display
• Error state display
• API key setup guidance

Props:

interface ChatPanelProps {
  open: boolean;
  videoTitle?: string;
  messages: ChatMessage[];
  aiReady: boolean;
  onClose: () => void;
  onSend: (content: string) => Promise<void>;
  sending: boolean;
  streamingAssistantId?: string;
  error?: string;
  loading: boolean;
  apiKeyStatus?: ApiKeyStatus;
  onOpenSettings: () => void;
}

State Management

Centralized chat-related state management in App.tsx:

// Chat state
const [chatOpen, setChatOpen] = useState(false);
const [chatSession, setChatSession] = useState<ChatSession | null>(null);
const [chatMessages, setChatMessages] = useState<ChatMessage[]>([]);
const [chatVideo, setChatVideo] = useState<VideoItem | null>(null);
const [chatError, setChatError] = useState<string | null>(null);
const [chatSending, setChatSending] = useState(false);
const [chatLoading, setChatLoading] = useState(false);

Event Listeners

Subscribe to Tauri events for streaming response handling:

useEffect(() => {
  let unlistenStart: (() => void) | undefined;
  let unlistenDelta: (() => void) | undefined;
  let unlistenEnd: (() => void) | undefined;
  let unlistenError: (() => void) | undefined;

  const setupListeners = async () => {
    unlistenStart = await listen<ChatStreamStartEvent>('chat_stream_start', ({ payload }) => {
      // Create empty AI message
    });
    
    unlistenDelta = await listen<ChatStreamDeltaEvent>('chat_stream_delta', ({ payload }) => {
      // Add content to AI message
    });
    
    unlistenEnd = await listen<ChatStreamEndEvent>('chat_stream_end', ({ payload }) => {
      // Handle streaming completion
    });
    
    unlistenError = await listen<ChatStreamErrorEvent>('chat_stream_error', ({ payload }) => {
      // Handle errors
    });
  };
  
  setupListeners();
  return () => {
    // Cleanup
  };
}, []);

Backend Implementation

OpenAI Client

OpenAI Responses API integration in ai.rs:

pub struct OpenAiClient {
    api_key: String,
    http_client: reqwest::Client,
}

impl OpenAiClient {
    pub async fn chat_stream(
        &self,
        request: serde_json::Value,
    ) -> Result<impl Stream<Item = Result<String, OpenAiError>>, OpenAiError> {
        // SSE streaming implementation
    }
}

Subtitle Snapshot Generation

Read video subtitle files for AI context:

fn load_subtitle_snapshot(scan_dir: &Path, video: &VideoItem) -> Result<String, String> {
    let subtitle_file_name = video.subtitle_file_name.as_ref()
        .ok_or_else(|| "No subtitle file name available.")?;
    
    let subtitle_path = scan_dir
        .join(&video.relative_path)
        .parent()
        .unwrap()
        .join(subtitle_file_name);
        
    let content = fs::read_to_string(&subtitle_path)
        .map_err(|e| format!("Cannot read subtitle file: {}", e))?;
        
    if content.len() > MAX_SUBTITLE_CHARS {
        Ok(content.chars().take(MAX_SUBTITLE_CHARS).collect())
    } else {
        Ok(content)
    }
}

System Prompt

Prompt explaining the AI's role and context:

fn build_system_prompt(video_title: &str) -> String {
    format!(
        "You are an AI assistant that answers questions based on the subtitles of the '{}' video. \
        When users ask about video content, please refer to the provided subtitles to give accurate and helpful answers. \
        For content not in the subtitles, do not speculate and only answer within the scope of the subtitles.",
        video_title
    )
}

Security Considerations

API Key Management

• Store OpenAI API key encrypted in environment variables or database
• Prevent key exposure in frontend
• Key status verification and validation logic

Input Validation

• Limit user message length
• Limit subtitle snapshot size (16,000 characters)
• Prevent SQL injection (use parameterized queries)

Performance Optimization

Database

-- Add indexes
CREATE INDEX idx_chat_sessions_video_id ON chat_sessions(video_id);
CREATE INDEX idx_chat_messages_session_id ON chat_messages(session_id);
CREATE INDEX idx_chat_messages_created_at ON chat_messages(created_at DESC);

Memory Management

• Limit conversation history (max 10 messages)
• Limit subtitle snapshot size
• Process streaming responses chunk by chunk

Error Handling

Common Error Cases

1. No subtitle file: "Cannot start conversation as there is no connected subtitle file."
2. No API key: "OPENAI_API_KEY environment variable is not set."
3. No session: "Cannot find conversation session."
4. Network error: Retry logic for OpenAI API call failures

Error Recovery

• Remove partial messages on streaming errors
• Guide user to retry on network errors
• Rollback processing on database errors

Testing

Frontend Testing

src/components/__tests__/ChatPanel.test.tsx:

• Message display testing
• Input send testing
• Error state testing
• Streaming display testing

Backend Testing

Using Cargo test framework:

• Chat session creation testing
• Message save/retrieve testing
• Subtitle snapshot loading testing

Deployment Considerations

Environment Setup

# Set OpenAI API key
export OPENAI_API_KEY="sk-..."

# Development mode
pnpm tauri:dev

# Production build
pnpm tauri:build

Dependencies

• Frontend: React 18, TypeScript, Tauri API
• Backend: Rust, Tauri, SQLite, reqwest, tokio

Future Improvements

1. Model Selection: UI feature to change GPT models
2. Multiple Providers: Support for Anthropic Claude, Google Gemini, etc.
3. Conversation Management: Conversation list, search, favorites
4. Subtitle Highlighting: Display subtitle sections referenced in AI answers
5. API Usage Monitoring: Track token usage and cost management

References

• OpenAI Responses API
• Tauri Events
• SQLite FTS (for future search functionality)

---

Result - user document

How to Use the Question Feature Guide

Overview

The video organizer's question feature allows you to chat with AI based on the subtitle content of videos. It's useful when you need additional explanations or have questions about the content after watching a video.

Prerequisites

1. OpenAI API Key Setup

You need an OpenAI API key to use the question feature.

Method 1: Environment Variable Setup

export OPENAI_API_KEY="sk-your-api-key-here"

Method 2: Register in App Settings

1. Click the settings button (⚙️) in the top right of the app
2. Enter your key in the "OpenAI API Key" field
3. Click the "Save" button

2. Subtitle File Verification

The question feature works based on subtitle files linked to videos. Make sure the following files exist in your video folder:

• meta.json file (video metadata)
• Subtitle files like .srt, .vtt, .ass, etc.

Only videos with subtitle files will have the "❓ Question" button enabled.

Basic Usage

1. Starting Questions

1. Find the video you want to ask questions about on the main screen
2. Click the "❓ Question" button in the video card's metadata area
3. A chat panel will open on the right side of the screen

Note: Videos without subtitle files will have the "❓ Question" button disabled and show a tooltip saying "No linked subtitle files available."

2. Entering and Sending Questions

1. Type your question in the input field at the bottom of the chat panel
2. Click the "Send" button or press Enter
3. AI will provide answers based on the subtitle content

Input Tips:

• Enter: Send message
• Shift + Enter: Line break (without sending)
• Empty messages will not be sent

3. Continuing Conversations

• Previous conversation content is automatically maintained
• Conversation history continues to be saved for the same video
• Up to 10 recent messages are used as context

Advanced Features

Question Count Display

You can see how many questions you've asked about a video on the video card.

• "❓ Question" → No questions asked yet
• "❓ Question (3)" → 3 questions asked

Streaming Responses

AI responses are displayed in real-time streaming:

• Response start: Empty message box appears
• Response progress: Text is gradually added
• Response complete: Cursor (▌) indicator disappears

Troubleshooting

API Key Related Errors

Symptom: "Please register your OpenAI API key in settings" message appears

Solutions:

1. Enter the correct OpenAI API key in the settings screen
2. Check the OPENAI_API_KEY environment variable setting
3. Verify the API key is valid and has remaining balance in the OpenAI dashboard

No Subtitle Files

Symptom: Question button is disabled

Solutions:

1. Check if subtitle files (.srt, .vtt, .ass, etc.) exist in the video folder
2. Verify that subtitle file information is correctly set in the meta.json file
3. Rescan videos if necessary

Chat Session Errors

Symptom: "Failed to prepare chat session" error

Solutions:

1. Try restarting the app
2. Check if the subtitle file for that video hasn't been deleted
3. If it's a database issue, reset the scan directory

Slow Responses

Causes:

• OpenAI API server status
• Network connection status
• Very large subtitle files

Solutions:

• Check network connection
• Try again later
• If subtitle files exceed 16,000 characters, only a portion will be used

Usage Tips

Effective Questioning Methods

1. Ask Specifically

   • ❌ "How is this video?"
   • ✅ "Explain the concept mentioned around 5:30 in simple terms"

2. Focus on Subtitle Content

   • ✅ "What are the three methods the speaker mentioned?"
   • ❌ "Recommend other related materials" (outside subtitle scope)

3. Utilize Context

   • Can ask follow-up questions referring to previous conversations
   • "Give examples of what you just explained"

Precautions

• AI answers are based only on subtitle content
• Does not speculate about content not in subtitles
• Conversation content is stored only in the local database
• Charges may apply based on OpenAI API usage

Related Documentation

• Technical Implementation Document - Detailed implementation information for developers
• subtitle-chat.md - Feature development notes

Inquiries and Feedback

If you encounter problems using the question feature or have suggestions for improvements, please let us know through GitHub Issues.