chore: task reexecution #4443

Elvis339 · 2025-10-30T20:15:22Z

Why this should be merged

This patch is breakdown of #4435

This patch contains a custom action that doesn't rely on relative imports, workflows that externalize reexecute configuration to tasks.

…reusable

Copilot

Pull Request Overview

This PR refactors the C-Chain re-execution benchmark infrastructure to use externalized configuration and a custom GitHub action that doesn't rely on relative imports. The changes streamline task configuration by moving runtime variables to environment-based configuration and introduce predefined task configurations for common benchmark scenarios.

Externalized runtime variables to environment configuration rather than explicit task parameters
Added custom GitHub action for C-Chain re-execution benchmarks with improved dependency management
Created predefined task configurations for common benchmark scenarios (hashdb, firewood, different block ranges)

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
scripts/copy_dir.sh	Enhanced to support S3 object key expansion for simplified source specification
scripts/benchmark_cchain_range.sh	Added default values for optional variables and improved environment variable documentation
Taskfile.yml	Refactored task configuration to use externalized variables and added predefined benchmark tasks
.github/actions/run-monitored-tmpnet-cmd/output-metrics-url.sh	Added warning comment about file duplication
.github/actions/c-chain-reexecution-benchmark/output-metrics-url.sh	New duplicate of metrics URL generation script
.github/actions/c-chain-reexecution-benchmark/action.yml	New custom GitHub action for C-Chain re-execution benchmarks

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.github/actions/c-chain-reexecution-benchmark/output-metrics-url.sh

.github/actions/c-chain-reexecution-benchmark/action.yml

scripts/benchmark_cchain_range.sh

…esolving error on compare benchmark results job

.github/actions/c-chain-reexecution-benchmark/action.yml

.github/workflows/c-chain-reexecution-benchmark-container.yml

.github/workflows/c-chain-reexecution-benchmark-gh-native.yml

scripts/copy_dir.sh

scripts/benchmark_cchain_range.sh

.github/actions/c-chain-reexecution-benchmark/action.yml

scripts/benchmark_cchain_range.sh

…to S3`

.github/actions/c-chain-reexecution-benchmark/action.yml

scripts/benchmark_cchain_range.sh

…ER_NAME`

- inline set task env - remove unrelated `TMPNET_START_METRICS_COLLECTOR` env var

…k-reexecution

maru-ava

Thank you!

.github/workflows/c-chain-reexecution-benchmark-container.yml

Taskfile.yml

scripts/copy_dir.sh

aaronbuchwald

Nice work!

Nice improvement to provide well-defined tasks for specific jobs in CI. We should imho continue to support running dynamically defined jobs via workflow dispatch as this has proven to be extremely helpful imo.

…perations

…e (custom) approach to task based operations

…k-reexecution

Elvis339 · 2025-11-06T20:48:52Z

Triggered workflow with custom inputs.

 gh workflow run "C-Chain Re-Execution Benchmark GH Native" \       
  --ref es/task-reexecution \
  -f task="" \
  -f block-dir-src=cchain-mainnet-blocks-1m-ldb \
  -f current-state-dir-src=cchain-current-state-hashdb-full-100 \
  -f start-block=101 \
  -f end-block=250000 \
  -f config=default \
  -f runner=ubuntu-latest \
  -f timeout-minutes=360

Run: https://github.com/ava-labs/avalanchego/actions/runs/19149324486/job/54735171547

…ile env making copy_dir.sh reusable for any S3 bucket

maru-ava · 2025-11-10T15:06:04Z

Taskfile.yml

-      METRICS_COLLECTOR_ENABLED={{.METRICS_COLLECTOR_ENABLED}} \
+      EXECUTION_DATA_DIR={{.EXECUTION_DATA_DIR}} \
      bash -x ./scripts/benchmark_cchain_range.sh
+    # Runtime context variables are read from environment by the script:


(No action required) Not entirely sure why these details need to be embedded in the taskfile. Or, to be honest, why this task makes sense at all. It might be cleaner to only add tasks for pre-defined test configurations and leave ad-hoc usage to direct script invocation.

aaronbuchwald · 2025-11-11T03:33:10Z

Triggered workflow with custom inputs.

 gh workflow run "C-Chain Re-Execution Benchmark GH Native" \       
  --ref es/task-reexecution \
  -f task="" \
  -f block-dir-src=cchain-mainnet-blocks-1m-ldb \
  -f current-state-dir-src=cchain-current-state-hashdb-full-100 \
  -f start-block=101 \
  -f end-block=250000 \
  -f config=default \
  -f runner=ubuntu-latest \
  -f timeout-minutes=360

Run: https://github.com/ava-labs/avalanchego/actions/runs/19149324486/job/54735171547

Could we update the re-execution README to make sure that instructions to run this via the GH CLI are up to date?

Also, does this support both running a named task and granularly defined task via GH CLI?

aaronbuchwald · 2025-11-11T03:17:52Z

.github/actions/c-chain-reexecution-benchmark/action.yml

  using: composite
  steps:
-    - name: Set task env
+    - uses: cachix/install-nix-action@02a151ada4993995686f9ed4f1be7cfbb229e56f #v31


So we are duplicating the code from run monitored tmpnet command here to avoid the relative import? Is there no better way to share code across these two actions than to duplicate the shared code?

The alternative is duplicating the custom actions into each repo's .github/actions/ directory. Inlining avoids that while giving us:

Local reproducibility: ./scripts/run_task.sh <task-name> just works

Zero cross-repo dependencies

Self-documenting workflows

The logic seems simple enough that duplication beats dependency management here, especially during the polyrepo migration where independence is valuable.

aaronbuchwald · 2025-11-11T03:19:48Z

.github/actions/c-chain-reexecution-benchmark/action.yml

      with:
        tool: 'go'
-        output-file-path: ${{ env.BENCHMARK_OUTPUT_FILE }}
+        output-file-path: benchmark-output.txt


What's the reason for this change?

Good catch I used it for testing relative paths on self-hosted environments.

Reverted this change: 0cb04dc

aaronbuchwald · 2025-11-11T03:22:07Z

.github/actions/c-chain-reexecution-benchmark/action.yml

+            START_BLOCK=${{ inputs.start-block }} \
+            END_BLOCK=${{ inputs.end-block }} \
+            BENCHMARK_OUTPUT_FILE="${{ env.BENCHMARK_OUTPUT_FILE }}" \
+            TASK_NAME="custom-reexecution"


Why do we need a task name here?

Without TASK_NAME, parallel local execution of c-chain-reexecution-* tasks would conflict they'd all try to create file with the same name. TASK_NAME ensures each task gets unique context rich identifiers.

This isn't required for CI because each matrix job runs on a separate runner with isolated resources, but it's required for local development where you might run multiple tasks simultaneously on the same machine.

aaronbuchwald · 2025-11-11T03:22:46Z

.github/actions/c-chain-reexecution-benchmark/action.yml

+            BENCHMARK_OUTPUT_FILE="${{ env.BENCHMARK_OUTPUT_FILE }}" \
+            TASK_NAME="custom-reexecution"
+        fi
+      env:


Why were some env vars removed in this change? It seems like metrics server enabled and labels were dropped here

In current master labels were never set, not even in the parent action for example we can see labels are empty on this run.

Regarding metrics server enabled we consolidated it making it implicitly controlled by whether the collector is enabled.

The benchmark_cchain_range.sh script reads these from the environment using the ${VAR:+--flag} pattern, so they're still available just passed through environment variables instead of explicit task parameters. This makes the workflow cleaner since monitoring config is grouped in the env: section.

aaronbuchwald · 2025-11-11T03:26:56Z

.github/workflows/c-chain-reexecution-benchmark-container.yml

+        options:
+          - ubuntu-latest
+          - blacksmith-4vcpu-ubuntu-2404
+          - avalanche-avalanchego-runner-2ti
+          - avago-runner-m6i-4xlarge-ebs-fast
+          - avago-runner-i4i-4xlarge-local-ssd


Can we remove this restriction for now?

We are still adding new labels behind ARC on a somewhat regular basis as we test different instance / storage types and this would require merging a code change to test those new labels, whereas right now we can easily trigger manual workflows.

Same comment for the other file that mirrors this one.

Valid, done.

3483737

…t file path

…k-reexecution

…ing default options

Elvis339 added 3 commits October 30, 2025 15:37

chore: externalize c-chain-reexecution json based workflow into tasks

5964641

ci(c-chain-reexecution): remove relative imports to make this action …

70fbc9f

…reusable

ci(c-chain-reexecution): use correct shells

d5bc227

Elvis339 self-assigned this Oct 30, 2025

Copilot AI review requested due to automatic review settings October 30, 2025 20:15

Elvis339 requested review from aaronbuchwald, joshua-kim and maru-ava as code owners October 30, 2025 20:15

github-project-automation bot added this to avalanchego Oct 30, 2025

Copilot AI reviewed Oct 30, 2025

View reviewed changes

.github/actions/c-chain-reexecution-benchmark/output-metrics-url.sh Outdated Show resolved Hide resolved

.github/actions/c-chain-reexecution-benchmark/action.yml Outdated Show resolved Hide resolved

Elvis339 added 2 commits October 30, 2025 16:24

ci(c-chain-reexecution)!:

ff86c39

copilot PR comments

b88c5ea

Elvis339 commented Oct 30, 2025

View reviewed changes

scripts/benchmark_cchain_range.sh Outdated Show resolved Hide resolved

ci(c-chain-reexecution): set absolute path to benchmark output file r…

c753cf9

…esolving error on compare benchmark results job

maru-ava reviewed Oct 30, 2025

View reviewed changes

Elvis339 added 3 commits November 3, 2025 09:06

address pr review

7b3a231

address PR

c0281ed

ci(c-chain-reexecution-benchmark): revert change on `Push Post-State …

10c5ef7

…to S3`

Elvis339 requested a review from maru-ava November 3, 2025 15:21

Elvis339 added 3 commits November 3, 2025 22:59

Merge branch 'master' into es/task-reexecution

7d7956c

Merge branch 'master' into es/task-reexecution

a42e26b

Merge branch 'master' into es/task-reexecution

785278f

maru-ava reviewed Nov 5, 2025

View reviewed changes

.github/actions/c-chain-reexecution-benchmark/action.yml Outdated Show resolved Hide resolved

.github/actions/c-chain-reexecution-benchmark/action.yml Outdated Show resolved Hide resolved

scripts/benchmark_cchain_range.sh Outdated Show resolved Hide resolved

Elvis339 and others added 5 commits November 6, 2025 00:05

chore(benchamrk_cchain_range): avoid duplicating the default of `RUNN…

c5b650a

…ER_NAME`

ci(c-chain-reexecution):

20c4ad0

- inline set task env - remove unrelated `TMPNET_START_METRICS_COLLECTOR` env var

Merge branch 'master' into es/task-reexecution

f9b3ce0

chore: rm doc

519e92a

Merge remote-tracking branch 'origin/es/task-reexecution' into es/tas…

f3d56fb

…k-reexecution

Elvis339 requested a review from maru-ava November 5, 2025 20:21

maru-ava moved this to Ready 🚦 in avalanchego Nov 6, 2025

maru-ava added testing This primarily focuses on testing ci This focuses on changes to the CI process labels Nov 6, 2025

maru-ava approved these changes Nov 6, 2025

View reviewed changes

Merge branch 'master' into es/task-reexecution

69c38b5

aaronbuchwald reviewed Nov 6, 2025

View reviewed changes

.github/workflows/c-chain-reexecution-benchmark-container.yml Show resolved Hide resolved

aaronbuchwald reviewed Nov 6, 2025

View reviewed changes

Taskfile.yml Show resolved Hide resolved

aaronbuchwald reviewed Nov 6, 2025

View reviewed changes

scripts/copy_dir.sh Outdated Show resolved Hide resolved

aaronbuchwald requested changes Nov 6, 2025

View reviewed changes

github-project-automation bot moved this from Ready 🚦 to In Progress 🏗️ in avalanchego Nov 6, 2025

Elvis339 added 4 commits November 7, 2025 00:33

ci(c-chain-reexecution): support alternative approach to task based o…

a800ff1

…perations

ci(c-chain-reexecution): support both container and native altenrativ…

f7f0ccf

…e (custom) approach to task based operations

Merge remote-tracking branch 'origin/es/task-reexecution' into es/tas…

ef5cae7

…k-reexecution

lint

7921a03

Elvis339 requested review from aaronbuchwald and maru-ava November 6, 2025 20:49

Elvis339 and others added 3 commits November 7, 2025 00:58

refactor(scripts): extract S3 bucket config from copy_dir.sh to Taskf…

e703fe0

…ile env making copy_dir.sh reusable for any S3 bucket

fix

d359c7a

Merge branch 'master' into es/task-reexecution

30e7b0c

maru-ava approved these changes Nov 10, 2025

View reviewed changes

aaronbuchwald requested changes Nov 11, 2025

View reviewed changes

Elvis339 and others added 5 commits November 12, 2025 13:30

ci(c-chain-reexecution): use dynamic env variable for benchmark outpu…

0cb04dc

…t file path

Merge remote-tracking branch 'origin/es/task-reexecution' into es/tas…

8082a9e

…k-reexecution

ci(c-chain-reexecution): remove unused TASK_NAME environment variable

3b2a82f

ci(c-chain-reexecution): enforce required runner parameter by remov…

3483737

…ing default options

Merge branch 'master' into es/task-reexecution

c14c088

Elvis339 requested a review from aaronbuchwald November 13, 2025 07:04

chore: task reexecution #4443

Are you sure you want to change the base?

chore: task reexecution #4443

Uh oh!

Conversation

Elvis339 commented Oct 30, 2025

Why this should be merged

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maru-ava left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aaronbuchwald left a comment

Choose a reason for hiding this comment

Uh oh!

Elvis339 commented Nov 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aaronbuchwald commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Elvis339 Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Elvis339 Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aaronbuchwald commented Nov 11, 2025 •

edited

Loading

Elvis339 Nov 12, 2025 •

edited

Loading

Elvis339 Nov 12, 2025 •

edited

Loading