Complete MATLAB integration and system optimization (25 commits) #8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ve information architecture - Transform CLAUDE.md from detailed design document to focused dispatch hub - Remove redundant design philosophy, system capabilities, and extension points sections - Add clear navigation to dedicated design documents with enhanced references - Dramatically streamline README.md from 327 to 118 lines with research focus - Streamline basic_design.md from 254 to 192 lines removing enterprise language - Remove obsolete design files (current_design.md, current_design_ja.md) - Add GitHub Pages link (https://napinoco.github.io/optimization-solver-benchmark/) to README.md - Update development status messaging for proper research tool context - Fix detail_design.md status value case consistency (UPPERCASE vs lowercase) - Remove unused problem_types arrays from MATLAB solver configurations - Strengthen design document references throughout CLAUDE.md 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
## Overview Dashboard (index.html) Improvements: - Remove unreliable status/success rate information (self-reported values) - Focus on objective performance metrics: solver count, problem count, libraries - Add comprehensive solver list display with proper formatting - Replace performance analysis with TBD section (evaluation criteria in development) - Add detailed environment information (Git commit, platform, Python, MATLAB) ## Data Export System: - Create dedicated data/index.html with professional download interface - Remove summary.json generation and references (redundant with main reports) - Provide JSON and CSV formats with clear documentation - Remove Usage License section per requirements ## Technical Improvements: - Convert all timestamp displays to UTC with explicit timezone indication - Fix bullet point positioning issues in lists (add padding-left: 1.5rem) - Add new result processor methods for library/type problem counting - Maintain data independence: HTML generation uses database directly, not export files ## System Architecture: - Preserve data integrity: all reports generated from same database source - Enable independent data format changes without affecting HTML generation - Streamline export process: only essential files (JSON/CSV) generated Generated files: - docs/pages/index.html (redesigned overview) - docs/pages/results_matrix.html (UTC timestamps) - docs/pages/raw_data.html (UTC timestamps) - docs/pages/data/index.html (new data export page) - docs/pages/data/benchmark_results.json - docs/pages/data/benchmark_results.csv 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed unexpected summary.json generation by disabling export_summary_only() in test function - Added summary.json to .gitignore to prevent accidental tracking - Changed all results export to sort by ID (ORDER BY id ASC) for database restoration - Updated main.py to use export_both_versions() for both latest and all results - Updated HTML generator to display links to _latest and _all versions - Removed backward compatibility code and old benchmark_results files - Data exports now provide two versions: - benchmark_results_latest.*: Latest run sorted by business logic - benchmark_results_all.*: Complete history sorted by ID for DB restoration 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Modified run_single_benchmark to detect "Unknown solver" errors - Skip database insertion when solver doesn't exist to prevent DB pollution - Continue storing error results for other types of execution failures - Added warning log for skipped unknown solver attempts 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed K.l validation to handle empty array [] by converting to 0 - Fixed K.q validation to skip validation when array is empty - Resolves "K.l must be a non-negative integer" error for pure SDP problems - Enables SDPT3 solver to work with truss5 and similar problems with no linear variables Test case: python main.py --benchmark --solvers matlab_sdpt3 --problems truss5 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
…iles
- Extended Python dat_loader.py to handle array format: {+0.0,+1.0,+1.0,...}
- Extended MATLAB dat_loader.m to handle array format with comma parsing
- Added support for both standard space-separated and array formats
- Resolves parsing error for gpp100.dat-s and similar problems
- Added example comments (gpp100 uses array format, arch0 uses standard format)
Test case: python main.py --benchmark --solvers matlab_sdpt3 --problems gpp100
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
…_value signs Problem Registry Changes: - Fixed negative signs for 8 hamming/torus problems: * hamming_7_5_6: 42.666667 → -42.666667 * hamming_8_3_4: 25.6 → -25.6 * hamming_9_8: 224 → -224 * hamming_10_2: 102.4 → -102.4 * toruspm3-8-50: 527.808663 → -527.808663 * toruspm3-15-50: 3474.4 → -3474.4 * torusg3-8: 457.358179 → -457.358179 * torusg3-15: 3134.6 → -3134.6 - Commented out 18 problematic problems: * Memory issues (4): hamming_11_2, hamming_9_5_6, maxG55, maxG60 * Unsupported formats (2): biomedP, industry2 (.dat instead of .dat-s) * Old versions (6): nql30old, nql60old, nql180old, qssp30old, qssp60old, qssp180old * Solver issues (6): fap36, filtinf1, infd1, infd2, infp1, infp2 - Active problems reduced from 142 → 120 for more reliable benchmarking System Improvements: - Enhanced result processor to exclude commented-out problems from reports - Updated HTML reports to reflect new active problem count (120) - Improved Java warning suppression in MATLAB interface 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
…cording - Fixed BenchmarkRunner to pass timeout argument to Python solver interfaces - Enhanced CVXPY solver timeout detection with backend-specific parameter handling - Added manual timeout detection logic for solvers without native timeout support - Ensured TIMEOUT status is properly returned and stored in database Key Changes: - scripts/benchmark/runner.py: Added default_timeout parameter and pass to solve() calls - scripts/solvers/python/cvxpy_runner.py: Enhanced timeout handling for all CVXPY backends This resolves the issue where Python solver timeouts were not being recorded in the database while MATLAB solver timeouts were working correctly. Now both Python and MATLAB solvers have consistent timeout behavior. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Added --timeout/-t argument to control solver timeout from command line - Modified run_benchmark() function to accept timeout parameter - Updated BenchmarkRunner initialization to use command-line timeout value - Added help examples showing timeout usage (--timeout 60, --timeout 300) - Default timeout remains 120.0 seconds when not specified Usage Examples: - python main.py --benchmark --timeout 60 # Set 1-minute timeout - python main.py --all --timeout 300 # Set 5-minute timeout - python main.py --benchmark --problems nb --timeout 1.0 # Quick timeout test This allows users to control solver timeout behavior without modifying code, useful for testing with different time constraints or handling problem sets with varying computational complexity. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Updated documentation across all levels to reflect timeout argument implementation: Core Code Documentation: - main.py: Enhanced --timeout help text and added 4 usage examples (60s-1800s) - BenchmarkRunner: Documented default_timeout parameter with usage guidelines - SolverInterface: Detailed timeout parameter documentation with backend differences Technical Documentation: - basic_design.md: Added timeout management to technical principles and validation strategy - detail_design.md: New comprehensive "Timeout Implementation" section with: * 4-level timeout architecture (CLI → Runner → Interface → Solver) * Solver-specific timeout handling (CVXPY native vs manual detection) * MATLAB subprocess timeout implementation * Complete code examples User Documentation: - README.md: Added timeout configuration examples for different use cases Coverage: - Command-line usage examples with recommended timeout values - Implementation details for Python (CVXPY/SciPy) and MATLAB solvers - Architecture explanation from user interface to solver execution - Best practices for different problem types (60s quick tests, 1800s for large SDP) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
… Python/MATLAB interfaces Major architectural improvements for system stability and crash protection: ## Subprocess Isolation Implementation - Add subprocess-based execution for both Python and MATLAB solvers - Implement ulimit-based memory limits (8GB Python, 16GB MATLAB default) - Add comprehensive timeout control with configurable CLI parameter - Enable process isolation to prevent system crashes from solver failures ## Symmetric Interface Architecture - Rename matlab_interface.py → matlab_process_interface.py (consistency) - Rename matlab_interface.m → matlab_solver_runner.m (subprocess entry point) - Rename python_interface.py → python_solver_runner.py (subprocess entry point) - Create python_process_interface.py (mirrors matlab_process_interface.py) - Implement identical error detection and resource control across both interfaces ## Enhanced Error Detection and Resource Control - Add SIGKILL detection for memory/resource limit violations (returncode -9/137) - Implement TIMEOUT detection for subprocess execution time limits - Add SUBPROCESS_ERROR detection for non-zero exit codes - Create comprehensive SolverResult factory methods for all error types - Add resource_limits.py utility for cross-platform ulimit integration ## System Architecture Updates - Update BenchmarkRunner to use new process interfaces - Add Mermaid diagrams showing entry point and execution flow - Document sequential execution steps: (1) solve → (2) subprocess.run → (3-5) result flow - Implement internationalization (①②③④⑤ → (1)(2)(3)(4)(5)) for global compatibility - Add comprehensive CLI documentation with timeout usage examples ## Testing and Validation - Add test_error_detection.py for comprehensive error scenario testing - Test SIGKILL, TIMEOUT, SUBPROCESS_ERROR, and normal execution paths - Validate resource limits and error detection across Python and MATLAB solvers This refactoring provides robust crash protection while maintaining identical interfaces and symmetric architecture between Python and MATLAB solver execution paths. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Remove extensive implementation code examples (Python/MATLAB) - Preserve essential architecture diagrams (mermaid process flow) - Maintain directory structure as design documentation - Eliminate redundancy with basic_design.md - Reduce file from 1117 lines (47KB) to 494 lines (23KB) The document now focuses on technical specifications and design decisions rather than implementation details, creating clear separation between basic design (vision/overview) and technical design (architecture/interfaces). 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Reduce problem loading tests from 139 to 4 problems (for_test_flag: true only) - Replace solver execution tests with import-based validation - Add _test_python_solver_import() and _test_matlab_solver_availability() methods - Improve validation performance from ~2 minutes to ~5 seconds - Achieve 11/11 solver validation success rate - Maintain comprehensive problem loading validation (4/4 problems) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Updated language specification to MATLAB R2024+ - Added MVP Development Principles section - Clarified development approach and user confirmation requirements - Improved development workflow documentation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Docker environment implementation with Dockerfile, docker-compose, and entrypoint - Multi-platform support: x86_64 (pre-built) and ARM64 (source build) for SCIP solver - Container memory detection via cgroup integration (fixes 7.8GB vs 16GB issue) - MATLAB R2024a version consistency across all components - Build optimization: FROM AS casing fixes, timezone prompt resolution - Comprehensive execution scripts: docker-run.sh and setup-matlab-license.sh - Octave integration for MATLAB-compatible solvers without license restrictions - Memory management: --memory-swap controls and OOM killer configuration - Host system protection through controlled container isolation - Complete documentation: design specifications and usage guides Key features: - Architecture detection for optimal SCIP installation strategy - Resource control with timeout and memory limits - Secure user permission handling across platforms - Efficient build process with optional SCIP exclusion (--no-scip) Files added: - docker/: Complete Docker environment setup - scripts/docker-run.sh: Container execution helper - scripts/setup-matlab-license.sh: License configuration utility - requirements-no-scip.txt: SCIP-free dependency list - docs/: Comprehensive Docker documentation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
… support" This reverts commit e02a47342f003f384d36505e534c81987c1cca75.
- Remove memory_limit_gb parameter from Python and MATLAB process interfaces - Delete scripts/utils/resource_limits.py entirely - Remove psutil dependency from requirements.txt - Retain SIGKILL detection for proper error classification - Return to simpler, more reliable system without complex memory management 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Updated database/results.db with new benchmark results - Database size increased from 4.6MB to 5.6MB - Contains additional solver performance data 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Report Generator Improvements: - Updated results matrix legend to include all status types (TIMEOUT, SIGKILL, SUBPROCESS_ERROR) - Added gray styling for non-solution statuses to improve visual clarity - Organized legend in two-column layout for better readability - Enhanced status detection logic for comprehensive error handling Data Updates: - Refreshed all HTML reports with latest benchmark results - Updated CSV/JSON exports with comprehensive solver performance data - Enhanced problem registry configuration - Improved result processor with better data handling Generated Reports: - docs/pages/index.html: Updated overview dashboard - docs/pages/results_matrix.html: Enhanced matrix with improved legend - docs/pages/raw_data.html: Comprehensive detailed results table - docs/pages/data/: Updated JSON/CSV exports 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
SDPT3 Integration Improvements: - Add custom_SDPT3soln_SEDUMIsoln.m with compatibility fixes for MATLAB/Octave - Initialize sblk array to prevent variable access errors - Update sdpt3_runner.m to use custom conversion function instead of submodule version - Maintain compatibility with SDPT3 solver while avoiding submodule modifications Files Modified: - scripts/solvers/matlab_octave/sdpt3_runner.m: Use custom conversion function - scripts/solvers/matlab_octave/custom_SDPT3soln_SEDUMIsoln.m: Custom version with fixes Testing Results: - SDP problem arch0: OPTIMAL in 1.692s, objective: -5.665173e-01 - SOCP problem nb: OPTIMAL in 0.910s, objective: -5.070309e-02 Benefits: - Preserves essential SDPT3 functionality with required fixes - Keeps submodules clean and unmodified - Ensures reliable solver operation across different MATLAB/Octave versions 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Directory Structure Changes: - Rename scripts/solvers/matlab_octave → scripts/solvers/matlab - Rename scripts/data_loaders/matlab_octave → scripts/data_loaders/matlab - Update .gitmodules to reflect new submodule paths Code Updates: - Update all Python imports from matlab_octave to matlab - Fix imports in runner.py, test files, and other Python modules - Maintain all MATLAB solver functionality (SeDuMi, SDPT3) Documentation Updates: - CLAUDE.md: Remove Octave references, update directory structure - README.md: Focus on MATLAB-only installation and setup - docs/development/*.md: Update architecture diagrams and references - Remove all installation instructions for Octave Testing Results: - SDP problem arch0: OPTIMAL in 1.597s, objective: -5.665173e-01 - SOCP problem nb: OPTIMAL in 0.919s, objective: -5.070309e-02 Benefits: - Simplified MVP approach focusing on MATLAB only - Reduced confusion from dual MATLAB/Octave support - Cleaner codebase and documentation - Maintained full MATLAB solver functionality 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Final Octave Cleanup: - Remove use_octave parameter and Octave command construction from MatlabProcessInterface - Update all file comments and docstrings to remove Octave references - Remove Octave installation section from configuration guide - Update test files to remove Octave-specific test cases - Fix all MATLAB file paths from matlab_octave to matlab - Update HTML generator to reflect MATLAB-only support Files Modified: - scripts/solvers/matlab/matlab_process_interface.py: Remove Octave support code - scripts/solvers/matlab/.gitkeep: Update directory comments - scripts/solvers/matlab/__init__.py: Update package description - scripts/solvers/matlab/custom_SDPT3soln_SEDUMIsoln.m: Update compatibility comments - scripts/reporting/html_generator.py: Remove Octave from MATLAB note - docs/guides/CONFIGURATION.md: Remove Octave installation guide - docs/development/tasks.md: Update solver descriptions - tests/unit/test_matlab_solver.py: Remove Octave tests, update imports - All MATLAB test files: Update path references - scripts/solvers/matlab/setup_matlab_solvers.m: Fix directory paths Result: Complete MATLAB-only implementation - Zero Octave references remain in codebase (excluding development history) - Simplified and cleaner architecture focused on MATLAB - All functionality preserved with MATLAB solver support 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Remove results.db from git tracking to reduce repository size (5.4MB binary file) - Add database/results.db to .gitignore for future exclusion - Enhance table_restorer.py with CLI interface supporting configurable input/output paths - Add database restoration documentation to README.md and detail_design.md - Update system architecture diagram with JSON→DB restoration flow Benefits: - Significant repository size reduction after git gc - Meaningful version control through JSON/CSV exports - Simple database restoration from published benchmark data - Maintains complete data recovery capability 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add .gitkeep to database/ directory to preserve empty directory structure - Update detail_design.md mermaid diagram: - Remove intermediate Database SQLite node for cleaner architecture - Move Report Generator outside Parent Process Environment - Direct insert DB operations to results.db file - Update node legends and execution flow descriptions - Simplify data flow from BenchmarkRunner directly to database file Architecture improvement: - Clearer data flow representation - Simplified database operations model - Report Generator as independent component - Maintained JSON restoration capabilities 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Delete docs/development/history.md (610 lines of development history) - Clear docs/development/tasks.md with completion status and future work section - Remove CLAUDE.md reference to history.md and update numbering - Delete entire tests/ directory contents (integration, performance, unit test files) - Streamline documentation to focus on current system state 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
napinoco
changed the title
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
📊 Preview Reports Ready!Your preview has been deployed and is available at: Preview Details
Available Reports
Notes
Preview deployment powered by GitHub Actions |
- Fix validation workflow failure due to missing psutil import - psutil is required by scripts/utils/environment_info.py for system info collection - Pin to version 6.1.0 for reproducible builds 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
📊 Preview Reports Ready!Your preview has been deployed and is available at: Preview Details
Available Reports
Notes
Preview deployment powered by GitHub Actions |
- Remove redundant "Source Code" heading from site config overview - Update HTML report generation timestamps - Improve author information display in overview section - Regenerate all HTML reports with updated formatting 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
📊 Preview Reports Ready!Your preview has been deployed and is available at: Preview Details
Available Reports
Notes
Preview deployment powered by GitHub Actions |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR brings 25 commits of major system improvements, completing MATLAB solver integration and implementing comprehensive architecture optimizations.
Major Features Added
Technical Changes
🔧 MATLAB Integration
matlab_octave/tomatlab/directory structure🏗️ Architecture Improvements
📊 Database & Reporting
🧹 System Cleanup
🚀 Performance & Reliability
Files Changed
Current System Status
Test Plan
🤖 Generated with Claude Code