Skip to content

Commit c7cec33

Browse files
saleh-mirsaleh-mir
committed
Add build scripts for cross-compilation and comprehensive builds
- Introduced `build-all-wheels.sh` for cross-compilation across supported platforms. - Added `build-comprehensive.sh` for building wheels for multiple Python versions and platforms. - Created `build-quick.sh` for fast builds on common platforms. - Documented cross-compilation setup in `CROSS_COMPILE_SETUP.md` for local builds. - These scripts enhance the build process and provide flexibility for developers.
1 parent 4bd3fe1 commit c7cec33

File tree

4 files changed

+570
-0
lines changed

4 files changed

+570
-0
lines changed

‎CROSS_COMPILE_SETUP.md‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
# Cross-Compilation Setup for Jesse Rust
2+
3+
This guide explains how to set up cross-compilation for building Python wheels locally instead of using GitHub Actions.
4+
5+
## Quick Start
6+
7+
The easiest way to get started is to use the provided build script:
8+
9+
```bash
10+
./build-all-wheels.sh
11+
```
12+
13+
This will give you options for different build methods.
14+
15+
## Build Methods
16+
17+
### 1. Native Build (Simplest)
18+
Builds only for your current platform:
19+
```bash
20+
maturin build --release --out dist
21+
```
22+
23+
### 2. Cross-Compilation (Fast)
24+
Uses Rust's cross-compilation capabilities. Works best on macOS for multiple targets:
25+
26+
```bash
27+
# Install targets
28+
rustup target add x86_64-apple-darwin aarch64-apple-darwin
29+
30+
# Build for both architectures
31+
maturin build --release --target x86_64-apple-darwin --out dist
32+
maturin build --release --target aarch64-apple-darwin --out dist
33+
```
34+
35+
### 3. cibuildwheel (Most Comprehensive)
36+
Uses Docker containers to build for multiple platforms:
37+
38+
```bash
39+
pip install cibuildwheel
40+
cibuildwheel --platform auto --output-dir dist
41+
```
42+
43+
## Platform-Specific Setup
44+
45+
### macOS (Recommended for cross-compilation)
46+
On macOS, you can easily build for both Intel and Apple Silicon:
47+
48+
```bash
49+
# Install Xcode command line tools (if not already installed)
50+
xcode-select --install
51+
52+
# Install targets
53+
rustup target add x86_64-apple-darwin
54+
rustup target add aarch64-apple-darwin
55+
56+
# Build
57+
./build-all-wheels.sh
58+
```
59+
60+
### Linux
61+
For Linux, you can build for multiple architectures but need additional setup:
62+
63+
```bash
64+
# Install cross-compilation tools
65+
sudo apt-get install gcc-aarch64-linux-gnu # For ARM64
66+
67+
# Install targets
68+
rustup target add aarch64-unknown-linux-gnu
69+
70+
# Build
71+
./build-all-wheels.sh
72+
```
73+
74+
### Windows
75+
On Windows, use the PowerShell script or WSL:
76+
77+
```powershell
78+
# Use the existing PowerShell script
79+
./build-local.ps1
80+
```
81+
82+
## Environment Variables
83+
84+
Set these environment variables for PyPI publishing:
85+
86+
```bash
87+
export MATURIN_PYPI_TOKEN="your_pypi_token_here"
88+
```
89+
90+
## Comparison: Local vs GitHub Actions
91+
92+
### Local Cross-Compilation Advantages:
93+
- ✅ **Faster**: No waiting for CI queue
94+
- ✅ **Immediate feedback**: Test builds instantly
95+
- ✅ **Control**: Full control over build environment
96+
- ✅ **Debugging**: Easier to debug build issues
97+
- ✅ **Cost**: No CI minutes usage
98+
99+
### GitHub Actions Advantages:
100+
- ✅ **Comprehensive**: Builds for all platforms reliably
101+
- ✅ **Testing**: Automatic testing on all platforms
102+
- ✅ **Automation**: Triggers on tags/releases
103+
- ✅ **Consistency**: Same environment every time
104+
- ✅ **Windows support**: Better Windows cross-compilation
105+
106+
## Recommended Workflow
107+
108+
1. **Development**: Use local native builds for quick iteration
109+
2. **Testing**: Use cross-compilation for testing on multiple platforms
110+
3. **Release**: Use either local build + manual upload or keep GitHub Actions
111+
112+
## Publishing to PyPI
113+
114+
After building wheels locally:
115+
116+
```bash
117+
# Publish all wheels
118+
maturin publish --skip-existing dist/*
119+
120+
# Or with specific token
121+
MATURIN_PYPI_TOKEN=your_token maturin publish --skip-existing dist/*
122+
```
123+
124+
## Troubleshooting
125+
126+
### Common Issues:
127+
1. **Missing cross-compilation tools**: Install platform-specific toolchains
128+
2. **Target not found**: Run `rustup target add <target>`
129+
3. **Permission denied**: Run `chmod +x build-all-wheels.sh`
130+
4. **Docker not running**: Start Docker for cibuildwheel
131+
132+
### Platform-Specific Issues:
133+
- **macOS**: Make sure Xcode CLI tools are installed
134+
- **Linux**: Install cross-compilation GCC toolchains
135+
- **Windows**: Consider using WSL or keep using GitHub Actions
136+
137+
## Next Steps
138+
139+
1. Try the `./build-all-wheels.sh` script
140+
2. Choose the build method that works best for your setup
141+
3. Consider keeping GitHub Actions for releases but using local builds for development

‎build-all-wheels.sh‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
#!/bin/bash
2+
3+
# Jesse Rust - Build All Wheels Script
4+
# Cross-compilation for all supported platforms (no Docker)
5+
6+
set -e
7+
8+
echo "🦀 Jesse Rust - Build All Wheels (Cross-compilation)"
9+
echo "==================================================="
10+
11+
# Check if maturin is installed
12+
if ! command -v maturin &> /dev/null; then
13+
echo "📦 Installing maturin..."
14+
pip install maturin
15+
fi
16+
17+
# Create dist directory
18+
mkdir -p dist
19+
rm -rf dist/*
20+
21+
echo "🔧 System Information:"
22+
echo " Rust version: $(rustc --version)"
23+
echo " Python version: $(python3 --version)"
24+
echo " Maturin version: $(maturin --version)"
25+
echo " Host platform: $OSTYPE"
26+
echo ""
27+
28+
# Install all cross-compilation targets
29+
echo "🎯 Installing cross-compilation targets..."
30+
TARGETS=()
31+
32+
if [[ "$OSTYPE" == "darwin"* ]]; then
33+
# macOS can cross-compile for macOS targets
34+
TARGETS+=("x86_64-apple-darwin")
35+
TARGETS+=("aarch64-apple-darwin")
36+
echo " Installing macOS targets..."
37+
rustup target add x86_64-apple-darwin aarch64-apple-darwin
38+
39+
elif [[ "$OSTYPE" == "linux-gnu"* ]]; then
40+
# Linux can cross-compile for Linux targets
41+
TARGETS+=("x86_64-unknown-linux-gnu")
42+
TARGETS+=("aarch64-unknown-linux-gnu")
43+
echo " Installing Linux targets..."
44+
rustup target add x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu
45+
46+
# Check for cross-compilation tools
47+
if command -v aarch64-linux-gnu-gcc &> /dev/null; then
48+
echo " ✅ aarch64 cross-compiler found"
49+
else
50+
echo " ⚠️ aarch64-linux-gnu-gcc not found - ARM64 build may fail"
51+
fi
52+
fi
53+
54+
# Always try to add the current platform as fallback
55+
CURRENT_TARGET=$(rustc -vV | grep host | cut -d' ' -f2)
56+
TARGETS+=("$CURRENT_TARGET")
57+
echo " Current platform target: $CURRENT_TARGET"
58+
59+
# Remove duplicates
60+
TARGETS=($(printf "%s\n" "${TARGETS[@]}" | sort -u))
61+
62+
echo ""
63+
echo "🔨 Building wheels for targets: ${TARGETS[*]}"
64+
echo ""
65+
66+
# Build counter and results tracking
67+
SUCCESSFUL_BUILDS=()
68+
FAILED_BUILDS=()
69+
BUILD_COUNT=0
70+
71+
# Build wheels for each target
72+
for target in "${TARGETS[@]}"; do
73+
BUILD_COUNT=$((BUILD_COUNT + 1))
74+
echo "[$BUILD_COUNT/${#TARGETS[@]}] Building for $target..."
75+
76+
if maturin build --release --target "$target" --out dist 2>/dev/null; then
77+
echo " ✅ SUCCESS: $target"
78+
SUCCESSFUL_BUILDS+=("$target")
79+
else
80+
echo " ❌ FAILED: $target"
81+
FAILED_BUILDS+=("$target")
82+
fi
83+
echo ""
84+
done
85+
86+
# Build source distribution
87+
echo "📦 Building source distribution..."
88+
if maturin sdist --out dist 2>/dev/null; then
89+
echo " ✅ SUCCESS: Source distribution"
90+
else
91+
echo " ❌ FAILED: Source distribution"
92+
fi
93+
94+
echo ""
95+
echo "🎯 BUILD SUMMARY"
96+
echo "================"
97+
98+
# Get version from built packages
99+
if ls dist/*.whl &> /dev/null; then
100+
WHEEL_FILE=$(ls dist/*.whl | head -n1)
101+
VERSION=$(basename "$WHEEL_FILE" | cut -d'-' -f2)
102+
echo "📋 Package: jesse_rust v$VERSION"
103+
else
104+
echo "📋 Package: jesse_rust (version unknown)"
105+
fi
106+
107+
echo "🐍 Python: $(python3 --version | cut -d' ' -f2)"
108+
echo "🦀 Rust: $(rustc --version | cut -d' ' -f2)"
109+
echo "��️ Host: $(uname -m)-$(uname -s | tr '[:upper:]' '[:lower:]')"
110+
111+
echo ""
112+
echo "✅ SUCCESSFUL BUILDS (${#SUCCESSFUL_BUILDS[@]}):"
113+
if [ ${#SUCCESSFUL_BUILDS[@]} -eq 0 ]; then
114+
echo " None"
115+
else
116+
for target in "${SUCCESSFUL_BUILDS[@]}"; do
117+
echo " ✓ $target"
118+
done
119+
fi
120+
121+
if [ ${#FAILED_BUILDS[@]} -gt 0 ]; then
122+
echo ""
123+
echo "❌ FAILED BUILDS (${#FAILED_BUILDS[@]}):"
124+
for target in "${FAILED_BUILDS[@]}"; do
125+
echo " ✗ $target"
126+
done
127+
fi
128+
129+
echo ""
130+
echo "📦 BUILT PACKAGES:"
131+
if ls dist/* &> /dev/null; then
132+
for file in dist/*; do
133+
filename=$(basename "$file")
134+
size=$(du -h "$file" | cut -f1)
135+
136+
if [[ "$filename" == *.whl ]]; then
137+
# Extract platform info from wheel name
138+
platform=$(echo "$filename" | rev | cut -d'-' -f1 | rev | sed 's/.whl//')
139+
python_tag=$(echo "$filename" | cut -d'-' -f3)
140+
echo " 🎯 $filename ($size) - Python $python_tag, $platform"
141+
elif [[ "$filename" == *.tar.gz ]]; then
142+
echo " 📦 $filename ($size) - Source distribution"
143+
else
144+
echo " 📄 $filename ($size)"
145+
fi
146+
done
147+
else
148+
echo " No packages built"
149+
fi
150+
151+
echo ""
152+
echo "🚀 NEXT STEPS:"
153+
echo " Test locally: pip install --force-reinstall dist/*.whl"
154+
echo " Publish to PyPI: maturin publish --skip-existing dist/*"
155+
156+
# Exit with error if no successful builds
157+
if [ ${#SUCCESSFUL_BUILDS[@]} -eq 0 ]; then
158+
echo ""
159+
echo "⚠️ No successful builds! Check the error messages above."
160+
exit 1
161+
fi
162+
163+
echo ""
164+
echo "🎉 Cross-compilation completed successfully!"

0 commit comments

Comments
 (0)