Fix databricks labs list to include all projects
#3935
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is currently incomplete because there are more projects now than the default pagination size, and pagination was not taking place. Here we implement support for pagination.
This reduces the number of API calls that we make.
pietern
reviewed
Nov 20, 2025
Contributor
pietern
left a comment
pietern
left a comment
There was a problem hiding this comment.
If the max result size is 100, then the change could also just set that instead of adding the pagination. It probably works but won't be exercised until we actually have more than 100 repositories under labs.
Contributor
Author
That's fair, and I'm on the fence on this: when the code was originally implemented we were at (I believe) about a third of what we have now. So not supporting it feels like waiting for it to break again. The options I see are:
@pietern: With it laid out like this, what's your opinion? |
Contributor
|
I'm ok going with what's already implemented. Can you add a few more unit tests specifically for the code that extracts the next page link. Also please include a link to the GitHub docs that describe how this should work. |
Contributor
Author
Thanks. I'll implement those unit tests and let you know, likely tomorrow or the day after. |
|
An authorized user can trigger integration tests manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
pietern
approved these changes
Nov 26, 2025
Contributor
Author
|
@pietern: The integration tests need to be triggered for this to merge, I think. |
Contributor
|
@asnare Thanks for flagging. They don't need to run for this change. |
Collaborator
|
Commit: 52a3fcf
23 failing tests:
Top 50 slowest tests (at least 2 minutes):
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
This PR fixes the
databricks labs listcommand so that all projects from Databricks Labs are listed: prior to this PR only the first 30 projects are listed.The root cause of the problem is that the GitHub API being used is paginated, and defaults to only 30 results per page, and pagination was not implemented. This PR resolves the issue by:
Incidental changes include:
Out of scope for now:
Why
The
databricks labs listcommand was returning an incomplete list; projects that could be installed (eg.ucx) were not included in the list.Tests