Skip to content

feat(tikv/migration): migrate CI artifacts from fileserver to OCI registry #4272

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ti-chi-bot merged 9 commits into from
Mar 6, 2026
Merged

feat(tikv/migration): migrate CI artifacts from fileserver to OCI registry #4272

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a4dff08
Initial plan
Copilot Feb 28, 2026
4de60e6
feat(pipelines/tikv/migration): migrate artifacts from fileserver to …
Copilot Feb 28, 2026
a51df4d
Merge branch 'main' into copilot/migrate-artifacts-to-oci-please-work
wuhuizuo Mar 4, 2026
37a2c45
Merge branch 'main' into copilot/migrate-artifacts-to-oci-please-work
ti-chi-bot Mar 6, 2026
21ce4b2
ci(tikv/migration): revert pull-unit-test changes from oci migration pr
wuhuizuo Mar 6, 2026
4e1f83f
chore(tikv/migration): keep pull_unit_test stanza unchanged
wuhuizuo Mar 6, 2026
28235e5
ci(tikv/migration): point OCI_ARTIFACT_HOST to hub-zot mirror
wuhuizuo Mar 6, 2026
0175ffe
Merge branch 'main' into copilot/migrate-artifacts-to-oci-please-work
wuhuizuo Mar 6, 2026
4f85537
fix(pipelines/tikv/migration): cover prepare_test_binaries artifacts …
wuhuizuo Mar 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions pipelines/tikv/migration/latest/pod-pull_integration_kafka_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,16 @@ spec:
limits:
memory: 128Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
volumes:
- emptyDir: {}
name: volume-0
Expand Down
10 changes: 10 additions & 0 deletions pipelines/tikv/migration/latest/pod-pull_integration_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,16 @@ spec:
limits:
memory: 256Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
Expand Down
29 changes: 27 additions & 2 deletions pipelines/tikv/migration/latest/pull_integration_kafka_test.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ final GIT_FULL_REPO_NAME = 'tikv/migration'
final GIT_CREDENTIALS_ID = 'github-sre-bot-ssh'
final POD_TEMPLATE_FILE = 'pipelines/tikv/migration/latest/pod-pull_integration_kafka_test.yaml'
final REFS = readJSON(text: params.JOB_SPEC).refs
final OCI_TAG_TIDB = component.computeArtifactOciTagFromPR('tidb', REFS.base_ref, REFS.pulls[0].title, 'master')
final OCI_TAG_TIKV = component.computeArtifactOciTagFromPR('tikv', REFS.base_ref, REFS.pulls[0].title, 'master')
final OCI_TAG_PD = component.computeArtifactOciTagFromPR('pd', REFS.base_ref, REFS.pulls[0].title, 'master')
final OCI_TAG_ETCD = 'v3.5.15'
final OCI_TAG_YCSB = 'v1.0.3'

pipeline {
agent {
Expand All @@ -18,7 +23,11 @@ pipeline {
}
}
environment {
FILE_SERVER_URL = 'http://fileserver.pingcap.net'
// OCI artifact registry: hub-zot.pingcap.net/mirrors/hub
// tidb-server: hub-zot.pingcap.net/mirrors/hub/pingcap/tidb/package:<tag>_linux_amd64
// tikv-server: hub-zot.pingcap.net/mirrors/hub/tikv/tikv/package:<tag>_linux_amd64
// pd-server: hub-zot.pingcap.net/mirrors/hub/tikv/pd/package:<tag>_linux_amd64
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

download_pingcap_oci_artifact.sh uses OCI_ARTIFACT_HOST_COMMUNITY (defaulting to us-docker.pkg.dev/.../hub) for third-party artifacts like --etcdctl and --ycsb. If these jobs are intended to pull all artifacts via the hub-zot mirror, also set OCI_ARTIFACT_HOST_COMMUNITY (or export it just for the script invocation) to the mirror registry; otherwise these downloads will still hit the community registry.

Suggested change
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
OCI_ARTIFACT_HOST_COMMUNITY = 'hub-zot.pingcap.net/mirrors/hub'

Copilot uses AI. Check for mistakes.
}
options {
timeout(time: 65, unit: 'MINUTES')
Expand Down Expand Up @@ -66,10 +75,26 @@ pipeline {
steps {
dir('migration') {
cache(path: "./cdc", includes: '**/*', key: "ws/${BUILD_TAG}/tikvcdc") {
container("utils") {
sh label: 'download test binaries via OCI', script: """
mkdir -p ./cdc/scripts/bin
cd ./cdc/scripts/bin
${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh \
--tidb=${OCI_TAG_TIDB} \
--tikv=${OCI_TAG_TIKV} \
--pd=${OCI_TAG_PD} \
--pd-ctl=${OCI_TAG_PD} \
--etcdctl=${OCI_TAG_ETCD} \
--ycsb=${OCI_TAG_YCSB}
Comment on lines +82 to +88
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sh invocation interpolates OCI_TAG_* values (computed from PR metadata via component.computeArtifactOciTagFromPR) directly into a shell command as unquoted arguments like --tidb=${OCI_TAG_TIDB}. Since the PR title can include shell metacharacters (e.g., ;, &) that are preserved by computeArtifactOciTagFromPR, a malicious PR author could craft a title that turns these arguments into a command-injection vector, executing arbitrary commands in the CI utils container. Please treat these tag values as untrusted: constrain them to a safe character set and ensure they are safely quoted/passed to the script so injected shell syntax cannot alter the command flow.

Copilot uses AI. Check for mistakes.
chmod +x tidb-server tikv-server pd-server pd-ctl etcdctl go-ycsb
cd ../../
touch prepare_test_binaries
ls -alh
"""
Comment on lines +79 to +93
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OCI download step is doing network fetches but isn’t wrapped in retry(...), unlike similar pipelines in this repo. Consider adding a small retry around the download script invocation to reduce CI flakiness from transient registry/network errors.

Suggested change
sh label: 'download test binaries via OCI', script: """
mkdir -p ./cdc/scripts/bin
cd ./cdc/scripts/bin
${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh \
--tidb=${OCI_TAG_TIDB} \
--tikv=${OCI_TAG_TIKV} \
--pd=${OCI_TAG_PD} \
--pd-ctl=${OCI_TAG_PD} \
--etcdctl=${OCI_TAG_ETCD} \
--ycsb=${OCI_TAG_YCSB}
chmod +x tidb-server tikv-server pd-server pd-ctl etcdctl go-ycsb
cd ../../
touch prepare_test_binaries
ls -alh
"""
retry(2) {
sh label: 'download test binaries via OCI', script: """
mkdir -p ./cdc/scripts/bin
cd ./cdc/scripts/bin
${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh \
--tidb=${OCI_TAG_TIDB} \
--tikv=${OCI_TAG_TIKV} \
--pd=${OCI_TAG_PD} \
--pd-ctl=${OCI_TAG_PD} \
--etcdctl=${OCI_TAG_ETCD} \
--ycsb=${OCI_TAG_YCSB}
chmod +x tidb-server tikv-server pd-server pd-ctl etcdctl go-ycsb
cd ../../
touch prepare_test_binaries
ls -alh
"""
}

Copilot uses AI. Check for mistakes.
}
container("golang") {
sh label: 'integration test prepare', script: """#!/usr/bin/env bash
cd cdc/
make prepare_test_binaries
make check_third_party_binary
make integration_test_build
"""
Expand Down
29 changes: 27 additions & 2 deletions pipelines/tikv/migration/latest/pull_integration_test.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ final GIT_FULL_REPO_NAME = 'tikv/migration'
final GIT_CREDENTIALS_ID = 'github-sre-bot-ssh'
final POD_TEMPLATE_FILE = 'pipelines/tikv/migration/latest/pod-pull_integration_test.yaml'
final REFS = readJSON(text: params.JOB_SPEC).refs
final OCI_TAG_TIDB = component.computeArtifactOciTagFromPR('tidb', REFS.base_ref, REFS.pulls[0].title, 'master')
final OCI_TAG_TIKV = component.computeArtifactOciTagFromPR('tikv', REFS.base_ref, REFS.pulls[0].title, 'master')
final OCI_TAG_PD = component.computeArtifactOciTagFromPR('pd', REFS.base_ref, REFS.pulls[0].title, 'master')
final OCI_TAG_ETCD = 'v3.5.15'
final OCI_TAG_YCSB = 'v1.0.3'

pipeline {
agent {
Expand All @@ -18,7 +23,11 @@ pipeline {
}
}
environment {
FILE_SERVER_URL = 'http://fileserver.pingcap.net'
// OCI artifact registry: hub-zot.pingcap.net/mirrors/hub
// tidb-server: hub-zot.pingcap.net/mirrors/hub/pingcap/tidb/package:<tag>_linux_amd64
// tikv-server: hub-zot.pingcap.net/mirrors/hub/tikv/tikv/package:<tag>_linux_amd64
// pd-server: hub-zot.pingcap.net/mirrors/hub/tikv/pd/package:<tag>_linux_amd64
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

download_pingcap_oci_artifact.sh uses OCI_ARTIFACT_HOST_COMMUNITY (defaulting to us-docker.pkg.dev/.../hub) for third-party artifacts like --etcdctl and --ycsb. If these jobs are intended to pull all artifacts via the hub-zot mirror, also set OCI_ARTIFACT_HOST_COMMUNITY (or export it just for the script invocation) to the mirror registry; otherwise these downloads will still hit the community registry.

Suggested change
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
// Community / third-party artifacts (e.g. etcdctl, ycsb) via the same mirror
OCI_ARTIFACT_HOST_COMMUNITY = 'hub-zot.pingcap.net/mirrors/hub'

Copilot uses AI. Check for mistakes.
}
options {
timeout(time: 65, unit: 'MINUTES')
Expand Down Expand Up @@ -66,10 +75,26 @@ pipeline {
steps {
dir('migration') {
cache(path: "./cdc", includes: '**/*', key: "ws/${BUILD_TAG}/tikvcdc") {
container("utils") {
sh label: 'download test binaries via OCI', script: """
mkdir -p ./cdc/scripts/bin
cd ./cdc/scripts/bin
${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh \
--tidb=${OCI_TAG_TIDB} \
--tikv=${OCI_TAG_TIKV} \
--pd=${OCI_TAG_PD} \
--pd-ctl=${OCI_TAG_PD} \
--etcdctl=${OCI_TAG_ETCD} \
--ycsb=${OCI_TAG_YCSB}
Comment on lines +82 to +88
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sh step builds a shell command line where --tidb=${OCI_TAG_TIDB}, --tikv=${OCI_TAG_TIKV}, and other flags interpolate values derived from the PR title into an unquoted shell context. Because OCI_TAG_* values come from component.computeArtifactOciTagFromPR, which can be influenced via the PR title and allows shell metacharacters like ;, an attacker can craft a PR title such that the resulting tag injects arbitrary commands executed in the CI container. Treat OCI_TAG_* as untrusted: strictly validate/whitelist allowed characters for tags and ensure they are passed as safely quoted arguments (or via an array/parameter mechanism) so no shell metacharacters can break out of the intended argument.

Copilot uses AI. Check for mistakes.
chmod +x tidb-server tikv-server pd-server pd-ctl etcdctl go-ycsb
cd ../../
touch prepare_test_binaries
ls -alh
"""
Comment on lines +79 to +93
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OCI download step is doing network fetches but isn’t wrapped in retry(...), unlike similar pipelines in this repo. Consider adding a small retry around the download script invocation to reduce CI flakiness from transient registry/network errors.

Suggested change
sh label: 'download test binaries via OCI', script: """
mkdir -p ./cdc/scripts/bin
cd ./cdc/scripts/bin
${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh \
--tidb=${OCI_TAG_TIDB} \
--tikv=${OCI_TAG_TIKV} \
--pd=${OCI_TAG_PD} \
--pd-ctl=${OCI_TAG_PD} \
--etcdctl=${OCI_TAG_ETCD} \
--ycsb=${OCI_TAG_YCSB}
chmod +x tidb-server tikv-server pd-server pd-ctl etcdctl go-ycsb
cd ../../
touch prepare_test_binaries
ls -alh
"""
retry(3) {
sh label: 'download test binaries via OCI', script: """
mkdir -p ./cdc/scripts/bin
cd ./cdc/scripts/bin
${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh \
--tidb=${OCI_TAG_TIDB} \
--tikv=${OCI_TAG_TIKV} \
--pd=${OCI_TAG_PD} \
--pd-ctl=${OCI_TAG_PD} \
--etcdctl=${OCI_TAG_ETCD} \
--ycsb=${OCI_TAG_YCSB}
chmod +x tidb-server tikv-server pd-server pd-ctl etcdctl go-ycsb
cd ../../
touch prepare_test_binaries
ls -alh
"""
}

Copilot uses AI. Check for mistakes.
}
container("golang") {
sh label: 'integration test prepare', script: """#!/usr/bin/env bash
cd cdc/
make prepare_test_binaries
make check_third_party_binary
make integration_test_build
"""
Expand Down
34 changes: 16 additions & 18 deletions prow-jobs/tikv/migration/latest-presubmits.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,29 +1,27 @@
global_definitions:
brancher: &brancher
branches:
- ^main$
- ^cdc-release-.*$
- ^br-release-.*$
jenkins_job: &jenkins_job
agent: jenkins
labels:
master: "0"
decorate: false

# struct ref: https://pkg.go.dev/sigs.k8s.io/prow/pkg/config#Presubmit
presubmits:
tikv/migration:
- name: tikv/migration/pull_integration_test
agent: jenkins
labels:
master: "0"
decorate: false # need add this.
- <<: [*brancher, *jenkins_job]
name: tikv/migration/pull_integration_test
always_run: true
context: pull-integration-test
trigger: "(?m)^/test (?:.*? )?pull-integration-test(?: .*?)?$"
rerun_command: "/test pull-integration-test"
branches:
- ^main$
- ^cdc-release-.*$
- ^br-release-.*$
- name: tikv/migration/pull_integration_kafka_test
agent: jenkins
labels:
master: "0"
decorate: false # need add this.
- <<: [*brancher, *jenkins_job]
name: tikv/migration/pull_integration_kafka_test
always_run: true
context: pull-integration-kafka-test
trigger: "(?m)^/test (?:.*? )?pull-integration-kafka-test(?: .*?)?$"
rerun_command: "/test pull-integration-kafka-test"
branches:
- ^main$
- ^cdc-release-.*$
- ^br-release-.*$