Conversation
- Remove MLflow/FastAPI dependencies and resolve Pydantic warnings - Implement comprehensive Dash-based model serving dashboard - Add complete pytest test suite with 100% pass rate - Clean up codebase and remove unused MLOps infrastructure - Update documentation and project structure - Maintain all core ML functionality while simplifying architecture - Fix type annotations and security warnings
- Simplified Dash app UI to only use manual input form with actual personality features - Removed JSON input and file upload tabs for cleaner, focused interface - Enhanced prediction result display with confidence scores, probability bars, and personality descriptions - Updated model loader to properly load ensemble model with feature ordering and metadata - Added visual styling with personality-specific colors and improved formatting - Created trained ensemble model (VotingClassifier) with LR, RF, and LGBM estimators - Added comprehensive metadata for feature mapping and proper prediction decoding - Fixed feature encoding for Stage_fear and Drained_after_socializing categorical variables - Improved error handling and fallback mechanisms in model loading - Increased pre-commit file size limit to 25MB for trained models
- Moved train_and_save_models.py to scripts/ directory for better organization - Simplified Dockerfile by removing unnecessary build dependencies (gcc, g++) - Updated Docker configuration to use ensemble model and copy trained models - Removed MLflow service from docker-compose.yml (no longer needed) - Updated .dockerignore to include trained models while excluding unnecessary files - Added train-models target to Makefile for running model training script - Cleaned up examples directory by removing obsolete mlops_demo.py
- Only runs on main branch pushes and PRs - Triggers only when relevant files are modified (src/, dash_app/, tests/, scripts/, etc.) - Uses single Python version (3.11) instead of matrix - Sets UV_SYSTEM_PYTHON=1 to avoid virtual environment creation - Combines all tests into single job for efficiency - Includes code quality checks, tests, and Dash app validation
- Auto-fix import sorting in scripts/train_and_save_models.py - Auto-fix import sorting in tests/modules/test_model_builders.py - Remove unused import (unittest.mock.patch) from test file - Ensure code passes Ruff checks for CI pipeline
- Fixed formatting issues in 9 files across dash_app/, src/, scripts/, and tests/ - Ensured all code passes ruff format --check for CI pipeline - Maintained consistent code style throughout the project
- Fixed label mapping in model_loader.py to use metadata-driven approach - Added explicit label_mapping to model metadata during training - Fixed Makefile dash target to include --model-name ensemble parameter - Increased pre-commit large file limit to 150MB for ML model files - Added large stack model files to .gitignore (can be regenerated) - Predictions now correctly show 'Extrovert'/'Introvert' instead of 'Class_2' - Probabilities are now properly aligned with prediction classes - Kept ensemble model (20MB) in version control, excluded large stack models
- Auto-formatted scripts/train_and_save_models.py to comply with ruff standards - All linting and formatting checks now pass
There was a problem hiding this comment.
Pull Request Overview
This PR integrates a Dash-based interactive dashboard, enhances the main pipeline with logging and default handling, introduces a model training script, expands test coverage, and updates documentation and CI configurations.
- Added unit and integration tests for model builders and Dash application
- Introduced
dash_app/directory with a full Dash app, Docker setup, and CI workflow - Enhanced
src/main_modular.pywith improved logging, None defaults, and MLOps hooks - Added
scripts/train_and_save_models.pyto train and persist individual stacks and an ensemble - Updated docs and project configuration for new dependencies and best practices
Reviewed Changes
Copilot reviewed 35 out of 39 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/modules/test_model_builders.py | Added unit tests for model builder functions |
| tests/modules/init.py | Initialized modules test package |
| tests/dash_app/test_dash_app.py | Added integration tests for Dash app creation/layout/server |
| tests/dash_app/init.py | Initialized dash_app test package |
| tests/conftest.py | Expanded pytest fixtures and configuration |
| tests/init.py | Initialized root test package |
| tests/README.md | Added test suite documentation and usage instructions |
| src/modules/config.py | Refactored configuration docstring |
| src/main_modular.py | Enhanced main pipeline with logging and default parameter handling |
| scripts/train_and_save_models.py | Added training and model saving script for stacks and ensemble |
| pytest.ini | Configured pytest options and markers |
| pyproject.toml | Updated dependencies (imbalanced-learn, sdv, dash, joblib, bandit, pydocstyle) |
| docs/technical-guide.md | Added detailed Dash application architecture documentation |
| docs/mlops-integration-summary.md | Introduced MLOps integration summary |
| docs/mlops-infrastructure.md | New comprehensive MLOps infrastructure documentation |
| dash_app/src/model_loader.py | Implemented ModelLoader and LocalModelLoader for Dash |
| dash_app/src/layout.py | Created Dash UI layout and input components |
| dash_app/src/callbacks.py | Defined callback functions for predictions and history updates |
| dash_app/src/app.py | Developed Dash application initialization and run logic |
| dash_app/src/init.py | Defined dash_app package exports |
| dash_app/main.py | Entry point for running Dash app via CLI |
| dash_app/docker-compose.yml | Docker Compose config for Dash service |
| dash_app/Dockerfile | Dockerfile for building Dash app container |
| dash_app/.dockerignore | Dockerignore rules for Dash app build context |
| Makefile | Updated build targets (run, test, dash, train-models, stop-dash) |
| .pre-commit-config.yaml | Increased file size limit for large model files |
| .github/workflows/test.yml | CI workflow for tests, lint, type and security checks |
| README.md | Updated project README to highlight Dash and production infrastructure |
Comments suppressed due to low confidence (3)
tests/modules/test_model_builders.py:9
- Tests cover
build_stack,build_stack_c, andbuild_sklearn_stack, but do not includebuild_neural_stackorbuild_noisy_stack. Consider adding test cases for these builder functions to ensure full coverage.
from modules.model_builders import build_sklearn_stack, build_stack, build_stack_c
tests/modules/test_model_builders.py:25
- [nitpick] This fixture name duplicates the
sample_datafixture inconftest.py, which may lead to fixture resolution conflicts. Consider renaming it (e.g.,model_builders_sample_data).
def sample_data(self):
dash_app/src/callbacks.py:154
- The callback
sync_json_inputreferences component IDsjson-inputandjson-input-displaywhich are not defined in the layout, causing runtime errors. Consider removing or updating this callback to match actual component IDs.
@app.callback(
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes
Provide a clear and concise description of the changes made in this pull request. Include any relevant context or background information.
Issue ticket number and link
Type of Change
Check the type of change your pull request introduces:
Checklist before requesting a review
Before submitting your pull request, ensure the following: