-
Notifications
You must be signed in to change notification settings - Fork 850
Description
Is there an existing issue for this?
I couldn't find one.
Describe the bug
Rolling builds on main are getting stretched out for hours because macos-latest jobs appear to be competing for a very limited runner pool. In some cases the workflow ends up getting cancelled after spending a large amount of time waiting for macOS capacity.
Example run:
https://github.com/microsoft/aspire/actions/runs/23576613463
A few concrete data points from that run:
- The workflow started at
2026-03-26T03:51:39Zand was not fully updated until2026-03-26T07:50:07Z. - There were
101macOS jobs in the run. - The first macOS jobs did not start until about
51minutes after the workflow started. - The median macOS job start delay was about
91minutes. 35macOS jobs started more than2hours after the workflow began.10macOS jobs started more than3hours after the workflow began.- The latest macOS job in this run (
Tests / Templates-XUnit_V3MTP_NewUpAndBuildSupportProjectTemplatesTests / Templates-XUnit_V3MTP_NewUpAndBuildSupportProjectTemplatesTests (macos-latest)) did not start until about233minutes after the workflow started:
https://github.com/microsoft/aspire/actions/runs/23576613463/job/68654796150
Even when the jobs eventually run, this level of queueing makes rolling validation much slower and seems to increase the odds that the overall workflow gets cancelled before everything is meaningfully complete.
Expected Behavior
Rolling builds should be able to start macOS jobs promptly enough that the workflow finishes in a predictable amount of time instead of spending hours waiting on runner availability.
Steps To Reproduce
- Open a rolling
mainworkflow run with a large macOS matrix, such as:
https://github.com/microsoft/aspire/actions/runs/23576613463 - Compare the workflow start time to the
started_attimestamps of themacos-latestjobs. - Observe that many macOS jobs do not begin for 1-4 hours, and the overall run eventually ends in
cancelled.
Exceptions (if any)
No response
.NET Version info
No response