🔍 SMARTDRIVE - Search Service

Intelligent Search & Discovery Service with High-Performance Caching

📋 Table of Contents

Overview
Architecture
Features
Tech Stack
Quick Start
API Endpoints
Configuration
Caching Strategy
Monitoring & Observability
Security
Testing
Deployment
Contributing
License
Support

🎯 Overview

The SMARTDRIVE Search Service is a high-performance, intelligent search engine that provides advanced file discovery capabilities across the SMARTDRIVE ecosystem. It leverages Elasticsearch for powerful full-text search and Redis for lightning-fast caching, enabling users to find files quickly and efficiently.

Key Capabilities

🔍 Intelligent Search: Full-text search across file names, content, and AI-generated metadata
🚀 High-Performance Caching: Redis-based caching for instant search results
📊 Advanced Analytics: Search analytics and insights
🔗 Multi-Service Integration: Seamless integration with other SMARTDRIVE services
📈 Scalable Architecture: Built for high availability and performance

🏗️ Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Search API    │    │   Elasticsearch │    │      Redis      │
│   (Spring Boot) │◄──►│   (Search Index)│    │   (Caching)     │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         ▼                       ▼                       ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Search Logic  │    │   Index Schema  │    │   Cache TTL     │
│   & Filters     │    │   & Mapping     │    │   & Invalidation│
└─────────────────┘    └─────────────────┘    └─────────────────┘

Service Components

Search Controller: REST API endpoints for search operations
Search Service: Core search logic and business rules
Elasticsearch Integration: Direct querying and indexing
Cache Manager: Redis-based caching with TTL
Circuit Breaker: Resilience patterns for external dependencies

✨ Features

🔍 Search Capabilities

Full-Text Search: Search across file names, content types, and metadata
AI-Enhanced Search: Search through AI-generated tags, summaries, and labels
Image Content Search: Search through image labels, objects, and extracted text
Document Content Search: Search through extracted text and document metadata
Advanced Filters: Filter by content type, upload date, file size, and more
Pagination: Efficient result pagination with configurable page sizes

🚀 Performance Features

Redis Caching: Intelligent caching with configurable TTL
Search Results Cache: 15-minute TTL for repeated searches
File Details Cache: 1-hour TTL for file metadata
Cache Invalidation: Automatic cache cleanup on file operations
Circuit Breaker: Resilience against Elasticsearch failures

📊 Analytics & Monitoring

Search Analytics: Track search patterns and performance
Response Time Monitoring: Monitor search response times
Cache Hit Rates: Track cache effectiveness
OpenTelemetry Integration: Distributed tracing and metrics
Health Checks: Comprehensive health monitoring

🛠️ Tech Stack

Core Framework

Java 17: Modern Java with enhanced performance
Spring Boot 3.2: Rapid application development framework
Spring Web: RESTful web services
Spring Data Elasticsearch: Elasticsearch integration
Spring Data Redis: Redis caching integration

Search & Caching

Elasticsearch 8.x: Distributed search and analytics engine
Redis 7.x: In-memory data structure store for caching
Spring Cache: Caching abstraction layer

Resilience & Monitoring

Resilience4j: Circuit breaker and resilience patterns
OpenTelemetry: Observability framework
Micrometer: Application metrics
Spring Boot Actuator: Health checks and monitoring

Documentation & Testing

OpenAPI 3: API documentation
Swagger UI: Interactive API documentation
JUnit 5: Unit testing framework
Testcontainers: Integration testing

🚀 Quick Start

Prerequisites

Java 17 or higher
Docker and Docker Compose
Elasticsearch 8.x
Redis 7.x

Local Development Setup

Clone the Repository

git clone <repository-url>
cd search-service

Environment Configuration

# Copy environment template
cp .env.example .env

# Configure your environment variables
ELASTICSEARCH_HOST=elasticsearch
ELASTICSEARCH_PORT=9200
REDIS_HOST=redis
REDIS_PORT=6379

Start Dependencies

# Start Elasticsearch and Redis
docker-compose up -d elasticsearch redis

Run the Application

# Using Gradle
./gradlew bootRun

# Or using Docker
docker-compose up search-service

Verify Installation

# Health check
curl http://localhost:8084/actuator/health

# API documentation
open http://localhost:8084/swagger-ui.html

📡 API Endpoints

Search Operations

🔍 Search Files

POST /api/v1/search
Content-Type: application/json

{
  "query": "document",
  "page": 1,
  "size": 10,
  "filters": {
    "contentType": "application/pdf",
    "uploadDateFrom": "2024-01-01",
    "uploadDateTo": "2024-12-31"
  }
}

📄 Get File Details

GET /api/v1/search/files/{contentId}

Cache Management

🗑️ Invalidate Specific Cache

DELETE /api/v1/search/cache/{cacheKey}

🗑️ Invalidate All Search Cache

DELETE /api/v1/search/cache

🗑️ Invalidate All File Cache

DELETE /api/v1/search/cache/files

🗑️ Invalidate Cache on File Upload

POST /api/v1/search/cache/invalidate/upload

🗑️ Invalidate Cache on File Update

POST /api/v1/search/cache/invalidate/update/{contentId}

🗑️ Invalidate Cache on File Delete

POST /api/v1/search/cache/invalidate/delete/{contentId}

Health & Monitoring

💚 Health Check

GET /actuator/health

📊 Metrics

GET /actuator/metrics

⚙️ Configuration

Application Properties

spring:
  application:
    name: search-service
  
  elasticsearch:
    uris: http://elasticsearch:9200
  
  data:
    redis:
      host: redis
      port: 6379
      timeout: 2000ms
      lettuce:
        pool:
          max-active: 8
          max-idle: 8
          min-idle: 0

app:
  elasticsearch:
    index-name: file_metadata
  
  cache:
    ttl:
      search-results: 900    # 15 minutes
      file-details: 3600     # 1 hour

management:
  endpoints:
    web:
      exposure:
        include: health,metrics,info
  endpoint:
    health:
      show-details: always

Environment Variables

Variable	Description	Default
`ELASTICSEARCH_HOST`	Elasticsearch host	`elasticsearch`
`ELASTICSEARCH_PORT`	Elasticsearch port	`9200`
`REDIS_HOST`	Redis host	`redis`
`REDIS_PORT`	Redis port	`6379`
`APP_CACHE_TTL_SEARCH_RESULTS`	Search cache TTL (seconds)	`900`
`APP_CACHE_TTL_FILE_DETAILS`	File cache TTL (seconds)	`3600`

🗄️ Caching Strategy

Cache Levels

Search Results Cache
- TTL: 15 minutes
- Key: query:page:size
- Purpose: Cache repeated search queries
File Details Cache
- TTL: 1 hour
- Key: contentId
- Purpose: Cache individual file metadata

Cache Invalidation

File Upload: Invalidates all search cache
File Update: Invalidates search cache and specific file cache
File Delete: Invalidates search cache and specific file cache
Manual Invalidation: Admin endpoints for cache management

Cache Configuration

@Configuration
@EnableCaching
public class CacheConfig {
    
    @Bean
    public CacheManager cacheManager(RedisConnectionFactory connectionFactory) {
        // Configure TTL and serialization
        // Handle Java 8 time types
        // Enable polymorphic deserialization
    }
}

📊 Monitoring & Observability

OpenTelemetry Integration

Distributed Tracing: Track requests across services
Metrics Collection: Application and business metrics
Log Correlation: Link logs with trace IDs

Health Checks

Elasticsearch Health: Connection and index status
Redis Health: Connection and memory status
Application Health: Overall service status

Metrics

Search Performance: Response times and throughput
Cache Metrics: Hit rates and eviction rates
Error Rates: Failed searches and exceptions

🔒 Security

API Security

Input Validation: Comprehensive request validation
Rate Limiting: Protect against abuse
CORS Configuration: Cross-origin resource sharing
Error Handling: Secure error responses

Data Security

Elasticsearch Security: TLS encryption and authentication
Redis Security: Network isolation and authentication
Sensitive Data: No storage of sensitive information

🧪 Testing

Unit Tests

# Run unit tests
./gradlew test

# Run with coverage
./gradlew test jacocoTestReport

Integration Tests

# Run integration tests
./gradlew integrationTest

API Tests

# Test search functionality
curl -X POST http://localhost:8084/api/v1/search \
  -H "Content-Type: application/json" \
  -d '{"query": "test", "page": 1, "size": 10}'

🚀 Deployment

Docker Deployment

# Build Docker image
docker build -t smartdrive-search-service .

# Run container
docker run -p 8084:8084 \
  -e ELASTICSEARCH_HOST=elasticsearch \
  -e REDIS_HOST=redis \
  smartdrive-search-service

Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: search-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: search-service
  template:
    metadata:
      labels:
        app: search-service
    spec:
      containers:
      - name: search-service
        image: smartdrive/search-service:latest
        ports:
        - containerPort: 8084
        env:
        - name: ELASTICSEARCH_HOST
          value: "elasticsearch"
        - name: REDIS_HOST
          value: "redis"

Production Considerations

High Availability: Multiple replicas across zones
Auto-scaling: Horizontal pod autoscaling
Resource Limits: CPU and memory constraints
Monitoring: Prometheus and Grafana integration
Logging: Centralized log aggregation

🤝 Contributing

We welcome contributions to the SMARTDRIVE Search Service! Please follow these guidelines:

Development Workflow

Fork the Repository
Create a Feature Branch: git checkout -b feature/amazing-feature
Make Your Changes: Follow coding standards and add tests
Run Tests: Ensure all tests pass
Submit a Pull Request: Provide clear description and context

Code Standards

Java Code Style: Follow Google Java Style Guide
Documentation: Update README and API docs
Testing: Maintain >80% code coverage
Security: Follow security best practices

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🆘 Support

Getting Help

Documentation: Check this README and API docs
Issues: Report bugs and feature requests on GitHub
Discussions: Join community discussions
Email: support@smartdrive.com

Community

GitHub: SMARTDRIVE Search Service
Discord: Join our community server
Blog: Latest updates and tutorials

Built with ❤️ by the SMARTDRIVE Team
Empowering intelligent file management and discovery

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
gradle/wrapper		gradle/wrapper
src/main		src/main
.dockerignore		.dockerignore
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
build.gradle		build.gradle
gradlew		gradlew
gradlew.bat		gradlew.bat
settings.gradle		settings.gradle

SmartDrive-Platform/smartdrive-search-service

Folders and files

Latest commit

History

Repository files navigation