Skip to content

Commit 49d85d0

Browse files
valeriy42valeriy42
committed
code formating docker for aarch64
1 parent 6af0694 commit 49d85d0

File tree

4 files changed

+351
-0
lines changed

4 files changed

+351
-0
lines changed

‎build-setup/vscode/using_vscode.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,11 +105,35 @@ Here is an example of user tasks specified in `tasks.json`:
105105
"kind": "build",
106106
"isDefault": true
107107
}
108+
},
109+
{
110+
"label": "Format changed files (last 20 commits)",
111+
"type": "shell",
112+
"command": "docker run --rm -v ${workspaceFolder}:/ml-cpp -u $(id -u):$(id -g) docker.elastic.co/ml-dev/ml-check-style-aarch64:1 bash -c 'cd /ml-cpp && git diff --name-only --diff-filter=ACMRT HEAD~20 HEAD | grep -E \"\\.(cc|h)$\" | grep -v \"^3rd_party\" | grep -v \"^build-setup\" | xargs -r clang-format -i'",
113+
"problemMatcher": [],
114+
"presentation": {
115+
"reveal": "always",
116+
"panel": "shared"
117+
}
108118
}
109119
]
110120
}
111121
```
112122

123+
### Formatting Changed Files with Docker
124+
125+
You can use a Docker container with the correct clang-format version to format only files that have changed in recent commits. The example task above formats files changed in the last 20 commits. The task:
126+
127+
- Uses the `ml-check-style-aarch64` Docker image with clang-format 5.0.1
128+
- Mounts the workspace as a volume at `/ml-cpp`
129+
- Uses `-u $(id -u):$(id -g)` to preserve file ownership
130+
- Gets files changed in the last 20 commits using `git diff`
131+
- Filters for `.cc` and `.h` files only
132+
- Excludes `3rd_party` and `build-setup` directories
133+
- Runs `clang-format -i` in-place on each file
134+
135+
You can adjust the number of commits by changing `HEAD~20` to your desired value (e.g., `HEAD~10` for the last 10 commits).
136+
113137
## Debugging
114138

115139
To use [debugging](https://code.visualstudio.com/docs/editor/debugging), you need to specify the binary and the

‎dev-tools/docker/build_check_style_image_aarch64.sh‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/bin/bash
2+
#
3+
# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
4+
# or more contributor license agreements. Licensed under the Elastic License
5+
# 2.0 and the following additional limitation. Functionality enabled by the
6+
# files subject to the Elastic License 2.0 may only be used in production when
7+
# invoked by an Elasticsearch process with a license key installed that permits
8+
# use of machine learning features. You may not use this file except in
9+
# compliance with the Elastic License 2.0 and the foregoing additional
10+
# limitation.
11+
#
12+
13+
# Builds the Docker image that contains the correct version of clang-format 5.0.1
14+
# for checking the code style on aarch64 architecture.
15+
#
16+
# This script is not intended to be run regularly. When changing the
17+
# clang-format version, increment the image version, change the Dockerfile and
18+
# build a new image to be used for subsequent builds on this branch.
19+
#
20+
# Note: This image should be built on an aarch64 host or using Docker buildx
21+
# for cross-platform builds.
22+
23+
HOST=docker.elastic.co
24+
ACCOUNT=ml-dev
25+
REPOSITORY=ml-check-style-aarch64
26+
VERSION=1
27+
28+
set -e
29+
30+
cd `dirname $0`
31+
32+
. ./prefetch_docker_image.sh
33+
CONTEXT=check_style_image_aarch64
34+
prefetch_docker_base_image $CONTEXT/Dockerfile
35+
docker build --no-cache -t $HOST/$ACCOUNT/$REPOSITORY:$VERSION $CONTEXT
36+
# Get a username and password for this by visiting
37+
# https://docker-auth.elastic.co and allowing it to authenticate against your
38+
# GitHub account
39+
docker login $HOST
40+
docker push $HOST/$ACCOUNT/$REPOSITORY:$VERSION
41+

‎dev-tools/docker/check_style_image_aarch64/Dockerfile‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
#
2+
# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
3+
# or more contributor license agreements. Licensed under the Elastic License
4+
# 2.0 and the following additional limitation. Functionality enabled by the
5+
# files subject to the Elastic License 2.0 may only be used in production when
6+
# invoked by an Elasticsearch process with a license key installed that permits
7+
# use of machine learning features. You may not use this file except in
8+
# compliance with the Elastic License 2.0 and the foregoing additional
9+
# limitation.
10+
#
11+
12+
FROM alpine:3.8
13+
14+
# Docker image containing the correct clang-format 5.0.1 for check-style.sh on aarch64
15+
16+
MAINTAINER Valeriy Khakhutskyy <valeriy.khakhutskyy@elastic.co>
17+
18+
# Install build dependencies
19+
RUN apk update && \
20+
apk add --no-cache \
21+
bash \
22+
git \
23+
build-base \
24+
cmake \
25+
ninja \
26+
python3 \
27+
libc-dev \
28+
linux-headers \
29+
wget \
30+
tar \
31+
xz
32+
33+
# Build clang-format 5.0.1 from LLVM source
34+
# We need to build from source since Alpine 3.8 doesn't have clang-format 5.0.1 for aarch64
35+
ARG LLVM_VERSION=5.0.1
36+
ARG BUILD_DIR=/tmp/llvm-build
37+
38+
RUN mkdir -p ${BUILD_DIR} && \
39+
cd ${BUILD_DIR} && \
40+
# Download LLVM 5.0.1 source from official releases (using the old repository structure before monorepo)
41+
# LLVM and Clang were separate repositories for version 5.0.1
42+
wget -q https://releases.llvm.org/5.0.1/llvm-5.0.1.src.tar.xz && \
43+
tar -xJf llvm-5.0.1.src.tar.xz && \
44+
mv llvm-5.0.1.src llvm && \
45+
cd llvm/tools && \
46+
wget -q https://releases.llvm.org/5.0.1/cfe-5.0.1.src.tar.xz && \
47+
tar -xJf cfe-5.0.1.src.tar.xz && \
48+
mv cfe-5.0.1.src clang && \
49+
cd ../.. && \
50+
# Configure and build only clang-format (minimal build)
51+
mkdir build && \
52+
cd build && \
53+
cmake -G Ninja \
54+
-DCMAKE_BUILD_TYPE=Release \
55+
-DLLVM_TARGETS_TO_BUILD="AArch64" \
56+
-DLLVM_ENABLE_ASSERTIONS=OFF \
57+
-DCMAKE_INSTALL_PREFIX=/usr/local \
58+
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
59+
../llvm && \
60+
# Build only clang-format to reduce build time and image size
61+
ninja clang-format && \
62+
# Install clang-format
63+
install -m 755 bin/clang-format /usr/local/bin/clang-format && \
64+
# Clean up build artifacts
65+
cd / && \
66+
rm -rf ${BUILD_DIR} && \
67+
# Install runtime libraries needed by clang-format before removing build-base
68+
# libstdc++ and libgcc_s are required for C++ runtime
69+
apk add --no-cache libstdc++ libgcc && \
70+
# Remove build-only dependencies (keep runtime libraries installed above)
71+
apk del build-base cmake ninja python3 wget tar xz
72+
73+
# Verify clang-format version
74+
RUN clang-format --version | grep -q "5.0.1" || (echo "ERROR: clang-format version mismatch" && exit 1)
75+
76+
WORKDIR /ml-cpp
77+
ENV CPP_SRC_HOME=/ml-cpp
78+

‎dev-tools/docker/check_style_image_aarch64/README.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Building and Testing the clang-format 5.0.1 Docker Container for aarch64
2+
3+
This directory contains the Dockerfile and build script for creating a Docker container with clang-format 5.0.1 for Linux aarch64/ARM64 architecture.
4+
5+
## Prerequisites
6+
7+
- Docker installed and running
8+
- For aarch64 builds: Docker running on an aarch64 host, or Docker buildx configured for cross-platform builds
9+
- Git repository with at least 20 commits (for testing the format task)
10+
11+
## Building the Container
12+
13+
### Option 1: Build Locally (Recommended for Testing)
14+
15+
If you're on an aarch64 machine or have buildx configured:
16+
17+
```bash
18+
cd dev-tools/docker
19+
docker build --no-cache -t ml-check-style-aarch64:local check_style_image_aarch64
20+
```
21+
22+
This will:
23+
- Build the image with tag `ml-check-style-aarch64:local`
24+
- Take approximately 30-60 minutes (building LLVM from source)
25+
- Create an image with clang-format 5.0.1 installed
26+
27+
### Option 2: Using the Build Script
28+
29+
The build script is designed for pushing to the Elastic registry:
30+
31+
```bash
32+
cd dev-tools/docker
33+
./build_check_style_image_aarch64.sh
34+
```
35+
36+
**Note**: This requires authentication to `docker.elastic.co`. For local testing, use Option 1.
37+
38+
### Option 3: Cross-Platform Build (if not on aarch64)
39+
40+
If you're not on an aarch64 machine, you can use Docker buildx:
41+
42+
```bash
43+
# Enable buildx (if not already enabled)
44+
docker buildx create --name multiarch --use
45+
docker buildx inspect --bootstrap
46+
47+
# Build for aarch64
48+
cd dev-tools/docker
49+
docker buildx build --platform linux/arm64 --no-cache -t ml-check-style-aarch64:local -f check_style_image_aarch64/Dockerfile check_style_image_aarch64 --load
50+
```
51+
52+
## Testing the Container
53+
54+
### 1. Verify clang-format Version
55+
56+
First, verify that clang-format 5.0.1 is correctly installed:
57+
58+
```bash
59+
docker run --rm ml-check-style-aarch64:local clang-format --version
60+
```
61+
62+
Expected output:
63+
```
64+
clang-format version 5.0.1 (tags/RELEASE_501/final)
65+
```
66+
67+
### 2. Test Formatting a Single File
68+
69+
Test that clang-format works on a sample file:
70+
71+
```bash
72+
# Create a test file with formatting issues
73+
cat > /tmp/test_format.cc << 'EOF'
74+
int main(){return 0;}
75+
EOF
76+
77+
# Format it
78+
docker run --rm -v /tmp:/tmp ml-check-style-aarch64:local clang-format -i /tmp/test_format.cc
79+
80+
# Check the result
81+
cat /tmp/test_format.cc
82+
```
83+
84+
### 3. Test Formatting Files from Recent Commits
85+
86+
Test the complete workflow that will be used in the VS Code task:
87+
88+
```bash
89+
# From the repository root
90+
cd /path/to/ml-cpp
91+
92+
# Get files changed in last 20 commits and format them
93+
docker run --rm \
94+
-v $(pwd):/ml-cpp \
95+
-u $(id -u):$(id -g) \
96+
ml-check-style-aarch64:local \
97+
bash -c 'cd /ml-cpp && git diff --name-only --diff-filter=ACMRT HEAD~20 HEAD | grep -E "\.(cc|h)$" | grep -v "^3rd_party" | grep -v "^build-setup" | xargs -r clang-format -i'
98+
99+
# Check git status to see what was changed
100+
git status
101+
```
102+
103+
### 4. Test with a Dry Run (Preview Changes)
104+
105+
To see what files would be formatted without actually changing them:
106+
107+
```bash
108+
cd /path/to/ml-cpp
109+
110+
# Get list of files that would be formatted
111+
docker run --rm \
112+
-v $(pwd):/ml-cpp \
113+
ml-check-style-aarch64:local \
114+
bash -c 'cd /ml-cpp && git diff --name-only --diff-filter=ACMRT HEAD~20 HEAD | grep -E "\.(cc|h)$" | grep -v "^3rd_party" | grep -v "^build-setup"'
115+
```
116+
117+
### 5. Test the VS Code Task Command
118+
119+
You can test the exact command that will be used in the VS Code task:
120+
121+
```bash
122+
cd /path/to/ml-cpp
123+
124+
docker run --rm \
125+
-v $(pwd):/ml-cpp \
126+
-u $(id -u):$(id -g) \
127+
ml-check-style-aarch64:local \
128+
bash -c 'cd /ml-cpp && git diff --name-only --diff-filter=ACMRT HEAD~20 HEAD | grep -E "\.(cc|h)$" | grep -v "^3rd_party" | grep -v "^build-setup" | xargs -r clang-format -i'
129+
```
130+
131+
## Using in VS Code
132+
133+
### Setting Up the Task
134+
135+
1. Create or edit `.vscode/tasks.json` in your workspace root:
136+
137+
```json
138+
{
139+
"version": "2.0.0",
140+
"tasks": [
141+
{
142+
"label": "Format changed files (last 20 commits)",
143+
"type": "shell",
144+
"command": "docker run --rm -v ${workspaceFolder}:/ml-cpp -u $(id -u):$(id -g) ml-check-style-aarch64:local bash -c 'cd /ml-cpp && git diff --name-only --diff-filter=ACMRT HEAD~20 HEAD | grep -E \"\\.(cc|h)$\" | grep -v \"^3rd_party\" | grep -v \"^build-setup\" | xargs -r clang-format -i'",
145+
"problemMatcher": [],
146+
"presentation": {
147+
"reveal": "always",
148+
"panel": "shared"
149+
}
150+
}
151+
]
152+
}
153+
```
154+
155+
**Note**: Change `ml-check-style-aarch64:local` to `docker.elastic.co/ml-dev/ml-check-style-aarch64:1` if using the official registry image.
156+
157+
### Running the Task
158+
159+
1. Open Command Palette (`Cmd+Shift+P` on macOS, `Ctrl+Shift+P` on Linux/Windows)
160+
2. Type "Tasks: Run Task"
161+
3. Select "Format changed files (last 20 commits)"
162+
4. The task will format all `.cc` and `.h` files changed in the last 20 commits
163+
164+
## Troubleshooting
165+
166+
### Issue: "clang-format version mismatch"
167+
- **Solution**: The build may have failed. Check the Docker build logs for errors.
168+
- Rebuild the container with `--no-cache` flag
169+
170+
### Issue: "Permission denied" when formatting files
171+
- **Solution**: Ensure you're using `-u $(id -u):$(id -g)` in the docker run command
172+
- This ensures files are created with your user ID
173+
174+
### Issue: "No files to format"
175+
- **Solution**: Check that you have at least 20 commits in your repository
176+
- Try reducing the number: change `HEAD~20` to `HEAD~10` or `HEAD~5`
177+
178+
### Issue: Build fails with "No such file or directory" for LLVM
179+
- **Solution**: The LLVM release URLs might have changed. Check `https://releases.llvm.org/5.0.1/` for correct filenames
180+
181+
### Issue: Build takes too long
182+
- **Solution**: This is expected - building LLVM from source takes 30-60 minutes
183+
- The build only compiles clang-format (not full LLVM) to minimize time
184+
- Consider using a pre-built image from the registry if available
185+
186+
## Verifying the Build
187+
188+
After building, you can inspect the image:
189+
190+
```bash
191+
# Check image size
192+
docker images ml-check-style-aarch64:local
193+
194+
# Inspect the image
195+
docker inspect ml-check-style-aarch64:local
196+
197+
# Check what's installed
198+
docker run --rm ml-check-style-aarch64:local ls -la /usr/local/bin/clang-format
199+
```
200+
201+
## Next Steps
202+
203+
Once the container is built and tested:
204+
1. Test formatting on a few files manually
205+
2. Add the VS Code task to your workspace
206+
3. Run the task to format files from recent commits
207+
4. Review the changes with `git diff` before committing
208+

0 commit comments

Comments
 (0)