From 2ca2e0fae4bc7380fbe9c7dce4e2d4e6d59388a0 Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Fri, 18 Jul 2025 18:21:28 +0300 Subject: [PATCH 01/11] docs(evops-rest): draft overview --- crates/evops-rest/README.md | 41 +++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 crates/evops-rest/README.md diff --git a/crates/evops-rest/README.md b/crates/evops-rest/README.md new file mode 100644 index 0000000..301ab3c --- /dev/null +++ b/crates/evops-rest/README.md @@ -0,0 +1,41 @@ +# evops-rest + +## Overview + +Provides REST API endpoints using `axum` crate. Automatically generates OpenAPI documentation using `aide` crate. Organized into logical service groups (events, tags, users, etc.). + +### Key Files + +- `lib.rs`: Main entry point + - Sets up API routes + - Configures OpenAPI generation + +- `routes/`: Endpoint implementations + - Each file handles related operations + - Uses file-system routing (nested routes like `/tags/{tag-id}`) + +- `docs.rs`: OpenAPI configuration + - Defines API tags (e.g., `"TagService"`, `"UserService"`) + - Configures OpenAPI 3.0 schema generation + +- `types.rs`: Request/response data structures + - Request/Response types: + - Strict validation: + ```rust + #[schemars(length(min = 2, max = 50))] + struct TagName(String); + ``` + - Pagination support: + - `last_id`, `limit` `parameters` + +- `error.rs`: Error handling + Standardizes API errors: + - `400` Bad Request + - `404` Not Found + - `409` Conflict + - `422` Validation Error + - `500` Server Error + +### Adding New Endpoint + +1. From 5db05548e9e555538c2b812bd5eee79da1638df5 Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Fri, 18 Jul 2025 20:24:21 +0300 Subject: [PATCH 02/11] docs(evops-storage): draft overview --- crates/evops-storage/README.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 crates/evops-storage/README.md diff --git a/crates/evops-storage/README.md b/crates/evops-storage/README.md new file mode 100644 index 0000000..082b838 --- /dev/null +++ b/crates/evops-storage/README.md @@ -0,0 +1,20 @@ +# evops-storage + +## Overview + +Provides object storage operations for event images using MinIO. Handles image uploads, streaming downloads, and deletions. + +### Key Files + +- `lib.rs`: Client Initialization + Sets up the storage client and ensures required buckets exist. + - Creates MinIO/S3 client with authentication + - Checks storage connection + - Auto-creates `event-images` bucket if missing + +- `logic.rs`: Business Logic +Implements CRUD operations for event images. Uses WebP format for storage. +Key methods: + - `upload_event_image` - Encodes image as WebP and uploads to storage + - `stream_event_image` - Streams image bytes chunk-by-chunk + - `delete_event_image` - Removes image from storage From 7426a69dc5ad047d0a4e338af1dd81c44fe26c3f Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Fri, 18 Jul 2025 20:46:49 +0300 Subject: [PATCH 03/11] docs(evops-ml-client): draft overview --- crates/evops-ml-client/README.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 crates/evops-ml-client/README.md diff --git a/crates/evops-ml-client/README.md b/crates/evops-ml-client/README.md new file mode 100644 index 0000000..88379ba --- /dev/null +++ b/crates/evops-ml-client/README.md @@ -0,0 +1,24 @@ +# `evops-ml-client` + +## Overview + +Provides gRPC client implementation for communicating with Machine Learning services. Handles connection management, request serialization, and response parsing. + +### Key Files + +- `lib.rs`: Core Client Setup + Main entry point that establishes gRPC connection to ML server. + Key Responsibilities: + - Creates `MlClient` struct wrapping gRPC channel + - Implements connection retry logic with 5-second intervals + - Initializes gRPC client stub + +- `logic.rs`: Business Logic + Contains domain-specific operations and data transformations. + 1. Converts `EventDescription` to protobuf format + 2. Executes gRPC call to ML service + +- `pb.rs`: Protobuf Bindings +Auto-generated file (via `tonic-build`) containing: + - gRPC client stubs (`MlServiceClient`) + - Protobuf message definitions \ No newline at end of file From ad7ab90c0ca9aba873e92dcba86ec37709ff5fc8 Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Fri, 18 Jul 2025 22:39:44 +0300 Subject: [PATCH 04/11] docs(evops-core): draft overview --- crates/evops-core/README.md | 50 +++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 crates/evops-core/README.md diff --git a/crates/evops-core/README.md b/crates/evops-core/README.md new file mode 100644 index 0000000..af6cac7 --- /dev/null +++ b/crates/evops-core/README.md @@ -0,0 +1,50 @@ +# `evops-core` + +## Overview + +The core service layer containing business logic and integrations. Orchestrates database operations (via `evops_db`), file storage (MinIO via `evops_storage`), and ML services (via `evops_ml_client`). Used by `evops-rest` and `evops-grpc`. + +### Key Files + +- `lib.rs`: +Initializes application state and dependencies + - Sets up API routes + - Key components: + - `AppState`: Shared application state (thread-safe via `Arc`) + - `State`: Internal struct holding: + - Database connection (`evops_db::Database`) + - File storage client (`evops_storage::Storage`) + - ML service client (`evops_ml_client::MlClient`) + - Methods: + - `new()`: Creates application state with DB/storage/ML connections + - `arc_clone()`: Explicitly clones shared state +- `services.rs` +Aggregates service modules (event, language, tag, user) + +### Service Modules + +- `event.rs` +Handles event operations and image management. +Key Methods: + - `list_tags()` - Paginated tag listing + - `create_tag()` - Creates new tag + - `get_tags_by_description()` - Predicts tags using ML service + - `delete_tag()` - Deletes tag + +- `user.rs` +Manages user accounts. +Key Methods: + - `list_users()` - Gets all users + - `create_user()` - Creates new user + - `find_user()` - Gets user by ID + +- `language.rs` +Currently placeholder implementation +Future use: Manage multilingual content + +### Critical Workspace Dependencies + +1. `evops-db`: Database operations (PostgreSQL) +2. `evops-storage`: File storage (MinIO) +3. `evops-ml-client`: Machine learning service integration +4. `evops-models`: Domain data structures From b3b5125d19033889086f81714c0bd7ba24f9c441 Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Fri, 18 Jul 2025 22:59:02 +0300 Subject: [PATCH 05/11] docs(evops-grpc): draft overview --- crates/evops-grpc/README.md | 51 +++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 crates/evops-grpc/README.md diff --git a/crates/evops-grpc/README.md b/crates/evops-grpc/README.md new file mode 100644 index 0000000..e508e35 --- /dev/null +++ b/crates/evops-grpc/README.md @@ -0,0 +1,51 @@ +# `evops-grpc` + +## Overview + +Provides gRPC API endpoints using `tonic` framework. Handles protocol buffer serialization, gRPC-web compatibility, and service implementations. Organized into: +- Protocol buffer definitions (`pb`) +- Service implementations (`services`) +- gRPC-specific utilities (`headers`) +### Key Files + +- `lib.rs`: Main entry point + - Creates gRPC router with `axum` + - Registers all services + - Sets up gRPC reflection service for debugging + - Configures CORS with gRPC-specific headers + +- `pb.rs`: Protocol Buffer Hub + - Auto-generated protobuf code + - Contains all gRPC service and message definitions + - Includes file descriptor set for reflection + - Manual validation implementations + + - `pb/impls.rs`: Protobuf ↔ Model Conversions + - Handles validation and type conversions + - Key responsibilities + - String to UUID conversion + - Field validation + - Model to gRPC response transformations + +- `services/event.rs`: Event Service + - Implements gRPC methods: + - CRUD operations + - Image management (push/find/delete) + +- `services/language.rs`: Language Service + +- `services/tag.rs`: Tag Service + - Core methods: + - `create/list/find/delete` + - `suggest` - ML-powered tag suggestions` + +- `services/user.rs`: User Service + - Standard CRUD operations + - UUID handling for all requests + +- `headers.rs`: gRPC Header Constants + - Defines standard gRPC headers + - Used for CORS configuration and error handling + +- `services.rs`: Service Registry +Aggregates all service implementations \ No newline at end of file From 89a67c3d4d6b9289a1b6b5bb94e5e870126741d4 Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Fri, 18 Jul 2025 23:14:32 +0300 Subject: [PATCH 06/11] docs(evops): draft overview --- crates/evops/README.md | 44 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 crates/evops/README.md diff --git a/crates/evops/README.md b/crates/evops/README.md new file mode 100644 index 0000000..2ce2ae9 --- /dev/null +++ b/crates/evops/README.md @@ -0,0 +1,44 @@ +# `evops` + +## Overview + +The main executable crate serving as the backend entry point. Handles: +- Configuration loading +- Logging initialization +- Server startup (REST + gRPC) +- Graceful shutdown +- Dependency integration + +### Key Files + +- `config.rs` +Loads configuration from environment variables: + - Validates URL formats + - Uses `eyre` for error handling + +- `main.rs` +Entry point with async runtime setup +Key components: + - `init_logging()`: Configures `tracing` with env-based filters + - `tcp_listener()`: Binds to all interfaces on specified port + - `rest_grpc_app()`: Combines REST and gRPC routers + +### Key Dependencies + +1. `evops-core` +Core business logic crate. Contains shared application state, database/storage/ML integrations, and domain logic reused across REST/gRPC. + +2. `evops-rest` +Dedicated REST API implementation. Defines HTTP routes, request/validation handlers, and OpenAPI 3 docs. + +3. `evops-grpc` +gRPC service definitions and implementations. Generates protocol buffers code and handles RPC calls. + +4. `axum` +Modern web framework for building REST APIs. Handles HTTP routing, middleware, and request/response processing. + +5. `axum_tonic` +Critical integration crate that enables simultaneous REST + gRPC services on the same port. Coordinates traffic between the two protocols. + +6. `tokio` +Async runtime. Enables concurrent operations. \ No newline at end of file From b90cb563bb1d2253c33994538d11732212b9ab22 Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Sun, 20 Jul 2025 15:27:10 +0300 Subject: [PATCH 07/11] docs(evops-grpc): update content --- crates/evops-grpc/README.md | 123 +++++++++++++++++++++++++++++------- 1 file changed, 99 insertions(+), 24 deletions(-) diff --git a/crates/evops-grpc/README.md b/crates/evops-grpc/README.md index e508e35..806e049 100644 --- a/crates/evops-grpc/README.md +++ b/crates/evops-grpc/README.md @@ -6,6 +6,102 @@ Provides gRPC API endpoints using `tonic` framework. Handles protocol buffer ser - Protocol buffer definitions (`pb`) - Service implementations (`services`) - gRPC-specific utilities (`headers`) + +### Service Example + +Event Service (`services/event.rs`) +- Creating an event: +```rust +async fn create( + &self, + request: Request, +) -> Result, Status> { + let request_data = request.into_inner(); + + let form = { + request_data + .form + .ok_or(ApiError::InvalidArgument({ + "EventServiceCreateRequest.form must not be null.".to_owned() + }))? + .try_into()? + }; + let event_id = self.state.create_event(form).await?; + + let response_data = EventServiceCreateResponse { + event_id: event_id.to_string(), + }; + Ok(Response::new(response_data)) +} +``` + +- Streaming image download: +```rust +async fn push_image( + &self, + request: Request>, +) -> Result, Status> { + use crate::pb::event_service_push_image_request::Message; + + let mut request_stream = request.into_inner(); + + let err_msg_not_null = "EventServicePushImageRequest.message must not be null."; + + let event_id = { + let err_msg_1 = "The first message must contain metadata."; + let message = { + request_stream + .next() + .await + .ok_or(ApiError::InvalidArgument(err_msg_1.to_owned()))?? + .message + .ok_or(ApiError::InvalidArgument(err_msg_not_null.to_owned()))? + }; + let metadata = match message { + Message::Metadata(data) => data, + Message::Chunk(_) => { + return Err(ApiError::InvalidArgument(err_msg_1.to_owned()).into()); + } + }; + + evops_models::EventId::new({ + metadata + .event_id + .parse::() + .map_err(|e| ApiError::InvalidArgument(e.to_string()))? + }) + }; + let image = { + let mut buffer = BytesMut::new(); + while let Some(message) = request_stream.message().await? { + let message = { + message + .message + .ok_or(ApiError::InvalidArgument(err_msg_not_null.to_owned()))? + }; + let chunk = match message { + Message::Metadata(_) => { + let err_msg = "Expected chunk, found metadata. Only the first message needs to contain metadata."; + return Err(ApiError::InvalidArgument(err_msg.to_owned()).into()); + } + Message::Chunk(c) => c, + }; + buffer.put(&*chunk); + } + let buffer = buffer.freeze(); + + evops_models::EventImage::try_from(buffer)? + }; + + let image_id = self.state.push_event_image(event_id, image).await?; + let response_data = EventServicePushImageResponse { + image_id: image_id.to_string(), + }; + + Ok(Response::new(response_data)) +} +``` + ### Key Files - `lib.rs`: Main entry point @@ -14,38 +110,17 @@ Provides gRPC API endpoints using `tonic` framework. Handles protocol buffer ser - Sets up gRPC reflection service for debugging - Configures CORS with gRPC-specific headers -- `pb.rs`: Protocol Buffer Hub - - Auto-generated protobuf code +- `pb.rs`: PContains auto-generated Protobuf bindings - Contains all gRPC service and message definitions - Includes file descriptor set for reflection - Manual validation implementations - - `pb/impls.rs`: Protobuf ↔ Model Conversions + - `pb/impls.rs`: Protobuf ↔ Domain types - Handles validation and type conversions - Key responsibilities - String to UUID conversion - Field validation - Model to gRPC response transformations -- `services/event.rs`: Event Service - - Implements gRPC methods: - - CRUD operations - - Image management (push/find/delete) - -- `services/language.rs`: Language Service - -- `services/tag.rs`: Tag Service - - Core methods: - - `create/list/find/delete` - - `suggest` - ML-powered tag suggestions` - -- `services/user.rs`: User Service - - Standard CRUD operations - - UUID handling for all requests - -- `headers.rs`: gRPC Header Constants - - Defines standard gRPC headers - - Used for CORS configuration and error handling - - `services.rs`: Service Registry -Aggregates all service implementations \ No newline at end of file +Aggregates all service implementations From 4a49d4dc41fb805e6fdd8b2e9957de051923a313 Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Sun, 20 Jul 2025 15:28:27 +0300 Subject: [PATCH 08/11] docs(evops-core): update content --- crates/evops-core/README.md | 49 +++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 26 deletions(-) diff --git a/crates/evops-core/README.md b/crates/evops-core/README.md index af6cac7..1c7092d 100644 --- a/crates/evops-core/README.md +++ b/crates/evops-core/README.md @@ -2,49 +2,46 @@ ## Overview -The core service layer containing business logic and integrations. Orchestrates database operations (via `evops_db`), file storage (MinIO via `evops_storage`), and ML services (via `evops_ml_client`). Used by `evops-rest` and `evops-grpc`. +Acts as the application's source of truth, containing all core business logic and service integrations. This crate is reused by both `evops-rest` and `evops-grpc` to ensure consistent behavior across API boundaries. Orchestrates database operations (via `evops_db`), file storage (MinIO via `evops_storage`), and ML services (via `evops_ml_client`) + +### Core Mechanism + +1. **State Initialization** +During `AppState` construction the application is wiring together: + - Database connections + - Storage clients + - ML service integrations + +2. **API Layer Integration** + - REST: All endpoints inject AppState via axum's State mechanism + - gRPC: Service handlers hold shared references to AppState + +3. **Business Logic Execution** +Both REST controllers and gRPC services directly call `AppState` methods like `create_event()` ### Key Files - `lib.rs`: Initializes application state and dependencies - - Sets up API routes - Key components: + - Sets up API routes - `AppState`: Shared application state (thread-safe via `Arc`) - `State`: Internal struct holding: - - Database connection (`evops_db::Database`) - - File storage client (`evops_storage::Storage`) - - ML service client (`evops_ml_client::MlClient`) - - Methods: - - `new()`: Creates application state with DB/storage/ML connections - - `arc_clone()`: Explicitly clones shared state + - `evops_db::Database` + - `evops_storage::Storage` + - `evops_ml_client::MlClient` + - `services.rs` Aggregates service modules (event, language, tag, user) ### Service Modules - `event.rs` -Handles event operations and image management. -Key Methods: - - `list_tags()` - Paginated tag listing - - `create_tag()` - Creates new tag - - `get_tags_by_description()` - Predicts tags using ML service - - `delete_tag()` - Deletes tag +Handles event operations and image management - `user.rs` -Manages user accounts. -Key Methods: - - `list_users()` - Gets all users - - `create_user()` - Creates new user - - `find_user()` - Gets user by ID +Manages user accounts - `language.rs` Currently placeholder implementation Future use: Manage multilingual content - -### Critical Workspace Dependencies - -1. `evops-db`: Database operations (PostgreSQL) -2. `evops-storage`: File storage (MinIO) -3. `evops-ml-client`: Machine learning service integration -4. `evops-models`: Domain data structures From 2a60671fc757e83be7802b7e8f06c458deabeae7 Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Sun, 20 Jul 2025 15:30:46 +0300 Subject: [PATCH 09/11] docs(evops-storage): improve code formatting --- crates/evops-storage/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crates/evops-storage/README.md b/crates/evops-storage/README.md index 082b838..8dd1a18 100644 --- a/crates/evops-storage/README.md +++ b/crates/evops-storage/README.md @@ -1,4 +1,4 @@ -# evops-storage +# `evops-storage` ## Overview From 44914dcd1673f9c58b5a5bf85a8c25613431bfac Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Sun, 20 Jul 2025 15:32:24 +0300 Subject: [PATCH 10/11] docs(evops-rest): update content --- crates/evops-rest/README.md | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/crates/evops-rest/README.md b/crates/evops-rest/README.md index 301ab3c..61180f8 100644 --- a/crates/evops-rest/README.md +++ b/crates/evops-rest/README.md @@ -1,4 +1,4 @@ -# evops-rest +# `evops-rest` ## Overview @@ -16,17 +16,9 @@ Provides REST API endpoints using `axum` crate. Automatically generates OpenAPI - `docs.rs`: OpenAPI configuration - Defines API tags (e.g., `"TagService"`, `"UserService"`) - - Configures OpenAPI 3.0 schema generation + - Configures OpenAPI 3 schema generation - `types.rs`: Request/response data structures - - Request/Response types: - - Strict validation: - ```rust - #[schemars(length(min = 2, max = 50))] - struct TagName(String); - ``` - - Pagination support: - - `last_id`, `limit` `parameters` - `error.rs`: Error handling Standardizes API errors: From 7d5080db4bae548547cd0a21f7b4f3b9d56c7ab7 Mon Sep 17 00:00:00 2001 From: Maksim Ilin Date: Sun, 20 Jul 2025 15:34:00 +0300 Subject: [PATCH 11/11] docs(evops): remove unnecessary content --- crates/evops/README.md | 27 +++++---------------------- 1 file changed, 5 insertions(+), 22 deletions(-) diff --git a/crates/evops/README.md b/crates/evops/README.md index 2ce2ae9..875fd43 100644 --- a/crates/evops/README.md +++ b/crates/evops/README.md @@ -18,27 +18,10 @@ Loads configuration from environment variables: - `main.rs` Entry point with async runtime setup -Key components: - - `init_logging()`: Configures `tracing` with env-based filters - - `tcp_listener()`: Binds to all interfaces on specified port - - `rest_grpc_app()`: Combines REST and gRPC routers -### Key Dependencies +### Dependencies -1. `evops-core` -Core business logic crate. Contains shared application state, database/storage/ML integrations, and domain logic reused across REST/gRPC. - -2. `evops-rest` -Dedicated REST API implementation. Defines HTTP routes, request/validation handlers, and OpenAPI 3 docs. - -3. `evops-grpc` -gRPC service definitions and implementations. Generates protocol buffers code and handles RPC calls. - -4. `axum` -Modern web framework for building REST APIs. Handles HTTP routing, middleware, and request/response processing. - -5. `axum_tonic` -Critical integration crate that enables simultaneous REST + gRPC services on the same port. Coordinates traffic between the two protocols. - -6. `tokio` -Async runtime. Enables concurrent operations. \ No newline at end of file +1. `evops-db`: Database operations (PostgreSQL) +2. `evops-storage`: File storage (MinIO) +3. `evops-ml-client`: Machine learning service integration +4. `evops-models`: Domain data structures \ No newline at end of file