scene_explorer

Interactive scene exploration application built with React (Vite) frontend and FastAPI backend. Generate and explore interconnected scenes using AI-powered image generation and analysis.

Features

• Scene Generation: AI-powered scene creation with image generation, visual/sound/smell keywords, and vibe detection
• Route Building: Create and manage routes through multiple scenes
• Ghost Scenes: Automatic pre-generation of next scenes for seamless exploration
• User Authentication: JWT-based authentication with user isolation
• Real-time Updates: Polling and optimistic UI updates for smooth user experience

Quick Start

Prerequisites

• Docker and Docker Compose
• OpenAI API key (for scene generation)

Running Locally

1. Copy the example environment file:

cp example.env .env

2. Edit .env and set your OPENAI_API_KEY:

OPENAI_API_KEY=your-api-key-here

3. Start the application:

docker compose up --build

4. Access the application:

• Frontend: http://localhost:5173
• Backend API: http://localhost:8000
• API Documentation: http://localhost:8000/docs
• Uploaded files: http://localhost:8000/uploads/<name>

Default Credentials

On first startup, an admin user is automatically created. The default credentials are admin/admin, but you should change these in production by setting ADMIN_USERNAME and ADMIN_PASSWORD environment variables.

Authentication

All API endpoints (except /api/health and /api/auth/login) require authentication via Bearer token.

Login

POST /api/auth/login
Content-Type: application/json

{
  "username": "admin",
  "password": "admin"
}

Response:

{
  "access_token": "eyJ...",
  "token_type": "bearer"
}

Using the Token

Include the token in the Authorization header:

Authorization: Bearer <token>

User Management

• Users can only see and modify their own scenes and routes
• Admin user is automatically created on first startup
• Set ADMIN_USERNAME and ADMIN_PASSWORD environment variables to customize admin credentials

Environment Variables

Required

• OPENAI_API_KEY: Your OpenAI API key (required for scene generation)
• JWT_SECRET_KEY: Secret key for JWT token signing (MUST be changed in production)

Optional

• ADMIN_USERNAME: Admin username (default: admin)
• ADMIN_PASSWORD: Admin password (default: admin)
• MONGO_URI: MongoDB connection URI (default: mongodb://mongodb:27017)
• MONGO_DB: MongoDB database name (default: scene_explorer)
• UPLOAD_DIR: Directory for uploaded files (default: uploads)
• CELERY_BROKER_URL: Redis URL for Celery (default: redis://redis:6379/0)
• LLM_PROVIDER: stub or openai (default: stub)
• OPENAI_BASE_URL: OpenAI API base URL (default: https://api.openai.com/v1)
• OPENAI_MODEL: OpenAI model name (default: gpt-4o-mini)
• OPENAI_IMAGE_MODEL: OpenAI image model (default: dall-e-3)
• OPENAI_VISION_MODEL: OpenAI vision model (default: gpt-4o-mini)

Generating a Secure JWT Secret

python3 -c "import secrets; print(secrets.token_urlsafe(32))"

Backend Tests

Run tests locally:

python3 -m venv .venv
source .venv/bin/activate
pip install -r backend/requirements.txt
cd backend
pytest -v

Uploads

• Upload: POST /api/uploads (multipart form field: file)
• Serve: GET /uploads/<name>
• Stored on disk in: ./data/uploads/ (bind-mounted into the backend container)

Scenes

• Create: POST /api/scenes
• List: GET /api/scenes
• Delete: DELETE /api/scenes/<scene_id>
  • Stored in MongoDB (see docker-compose.yml)

Create payload:

{
  "name": "string | null",
  "initial_prompt": "string | null",
  "image_url": "https://example.com/image.png",
  "visual": "string",
  "sound": "string",
  "smell": "string",
  "vibe": "string | null"
}

MongoDB (local):

• URI: mongodb://localhost:27017
• DB: scene_explorer
• Data directory: ./data/mongodb/

LLM wrapper

Backend includes a small LLM wrapper (backend/app/llm.py) with a single method:

• process(user_prompt, system_prompt=None) -> str

Configuration (env vars):

• LLM_PROVIDER: stub (default) or openai
• OPENAI_API_KEY: required if LLM_PROVIDER=openai
• OPENAI_BASE_URL: defaults to https://api.openai.com/v1
• OPENAI_MODEL: defaults to gpt-4o-mini
• OPENAI_IMAGE_MODEL: defaults to dall-e-3
• OPENAI_VISION_MODEL: defaults to gpt-4o-mini

To run with OpenAI locally:

cp example.env .env
# edit .env and set OPENAI_API_KEY
docker compose up --build

API:

• POST /api/llm/process with JSON { "user_prompt": "...", "system_prompt": "..." }

Scene generation

Backend includes SceneGenerator (backend/app/scene_generation.py) which:

• Takes an initial_prompt
• Generates an image first (OpenAI images) from the initial prompt and saves it to ./data/uploads/
• Uses the generated image to derive visual, sound, smell, optional vibe (comma-separated keywords / short phrases, max 5)
• Persists the created scene in MongoDB with image_url=/uploads/<name>.png

API:

• POST /api/scenes/generate with JSON { "initial_prompt": "..." }

Routes

• Create: POST /api/routes with JSON { "name": "string | null", "direction": "string | null", "scene_ids": ["<scene_id>", "..."] }
• Update: PUT /api/routes/<route_id> with JSON { "name": "string | null", "direction": "string | null", "scene_ids": ["<scene_id>", "..."] }
• List: GET /api/routes (returns routes with embedded scenes)
• Get: GET /api/routes/<route_id> (returns a single route with embedded scenes)
• Delete: DELETE /api/routes/<route_id>
• Proceed: POST /api/routes/<route_id>/proceed (generates and appends a new scene to the route)

Ghost Scenes

Routes automatically generate "ghost scenes" - the next scene is pre-generated in the background for faster user experience.

• Check ghost status: GET /api/routes/<route_id>/ghost
• Claim ghost: POST /api/routes/<route_id>/claim-ghost (adds the pre-generated scene to the route)

When you proceed past the last scene in a route, if a ghost scene is ready, it's automatically claimed. Otherwise, a new scene is generated synchronously.

Migration Scripts

The repository includes migration scripts for data updates:

backend/migrate_add_users.py

Migrates existing scenes and routes to belong to the admin user. Run this if you have existing data from before user authentication was added.

cd backend
python3 migrate_add_users.py

backend/migrate_is_ghost.py

Sets the is_ghost flag consistently across all scenes. Run this to ensure ghost scenes are properly marked.

cd backend
python3 migrate_is_ghost.py

Both scripts use environment variables MONGO_URI and MONGO_DB (defaults: mongodb://localhost:27017 and scene_explorer).

Frontend E2E Tests (Playwright)

From frontend/:

npm install
npx playwright install
npm run test:e2e

Notes:

• Tests assume the stack is already running (docker compose up --build) and will seed data via the backend API.
• Tests log in with admin/admin credentials.

Production Deployment

Security Checklist

Before deploying to production:

1. Change JWT Secret: Set JWT_SECRET_KEY to a strong random secret (use the command above to generate one)
2. Change Admin Credentials: Set ADMIN_USERNAME and ADMIN_PASSWORD to secure values
3. Review CORS Settings: Currently no CORS middleware is configured - add appropriate CORS settings for your frontend domain
4. Use Environment Variables: Never commit .env files - use your deployment platform's secret management
5. Database Security: Ensure MongoDB is not exposed publicly without authentication
6. HTTPS: Always use HTTPS in production

Generating Secrets

# JWT Secret
python3 -c "import secrets; print(secrets.token_urlsafe(32))"

# Admin Password (use a password manager or generate securely)

Architecture

• Frontend: React with Redux Toolkit for state management, React Router for navigation
• Backend: FastAPI with MongoDB for data persistence
• Background Jobs: Celery with Redis for asynchronous scene generation
• Authentication: JWT tokens with bcrypt password hashing
• Image Generation: OpenAI DALL-E for scene images
• Scene Analysis: OpenAI Vision API for extracting keywords and vibe

API Endpoints

Public Endpoints

• GET /api/health - Health check
• POST /api/auth/login - User login

Protected Endpoints (require Bearer token)

• GET /api/scenes - List user's scenes
• POST /api/scenes - Create a scene
• POST /api/scenes/generate - Generate a new scene from prompt
• DELETE /api/scenes/<scene_id> - Delete a scene
• GET /api/routes - List user's routes
• POST /api/routes - Create a route
• GET /api/routes/<route_id> - Get a route
• PUT /api/routes/<route_id> - Update a route
• DELETE /api/routes/<route_id> - Delete a route
• POST /api/routes/<route_id>/proceed - Proceed route (generate next scene)
• GET /api/routes/<route_id>/ghost - Get ghost scene status
• POST /api/routes/<route_id>/claim-ghost - Claim pre-generated ghost scene
• POST /api/uploads - Upload a file
• POST /api/llm/process - Process text with LLM

Development

Code Style

• Python: Follow PEP 8, use type hints
• JavaScript: Use single quotes, follow existing patterns
• Tests: Write tests before implementation (TDD approach)
• Documentation: Avoid comments unless necessary (code should be self-documenting)

Running Tests

# Backend tests
cd backend
pytest -v

# Frontend E2E tests
cd frontend
npm run test:e2e

License

See LICENSE file for details.