Skip to content

[ISSUE] CLI-based auth fails when the Databricks CLI is installed using certain package managers on Windows #1274

@sugibuchi

Description

@sugibuchi

Description
When CLI-based authentication is used, the current version of the SDK checks the file size of the installed Databricks CLI executable to verify that it is not older than version 0.100.0.

https://github.com/databricks/databricks-sdk-py/blob/v0.90.0/databricks/sdk/credentials_provider.py#L901-L906

            # The new Databricks CLI is a single binary with size > 1MB.
            # We use the size as a signal to determine which Databricks CLI is installed.
            stat = path.stat()
            if stat.st_size < (1024 * 1024):
                err = FileNotFoundError("Databricks CLI version <0.100.0 detected")
                continue

However, this version check does not work when the Databricks CLI is installed using certain Windows package managers, such as Scoop and Chocolatey.

These package managers create "shim files" that work as symbolic links to actual executable files. They use these shim files to install and manage multiple versions of the same package on the Windows file system, which does not support symbolic links in the same way as Unix file systems do.

As the shim file is usually smaller than the original executable file, the Databricks SDK incorrectly recognises the latest version of the Databricks CLI installed by the package manager as an older version.

C:\Users\TSUGIBUCHI>WHERE databricks
C:\ProgramData\chocolatey\bin\databricks.exe

C:\Users\TSUGIBUCHI>DIR C:\ProgramData\chocolatey\bin\databricks.exe
 Volume in drive C is Windows
 Volume Serial Number is 3674-B47B

 Directory of C:\ProgramData\chocolatey\bin

02/18/2026  10:54 AM           392,704 databricks.exe
               1 File(s)        392,704 bytes

Consequently, CLI-based authentication always fails.

I imagine the reason the current implementation uses file size to check the CLI version is because this method is much lighter than running databricks --version. However, checking the version by file size is neither straightforward nor reliable, as this example demonstrates.

I think the SDK should check the version of the installed CLI by running databricks --version and cache the result as a class field of DatabricksCliTokenSource, for example.

Alternatively, we could simply drop this CLI version check altogether. As the last version before 0.100.0 was released almost three years ago, the risk of using an old, unsupported version is probably negligible.

Reproduction

  • Windows environment
  • Python environment with installation of the Databricks SDK
  • Databricks CLI installed by Scoop or Chocolatey
  • [DEFAULT] profile with auth_type = databricks-cli in ~/.databrickscfg
from databricks.sdk import WorkspaceClient

client = WorkspaceClient()

Output

...
ValueError: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=https://adb-xxxxx.azuredatabricks.net, auth_type=databricks-cli

Expected behavior
WorkspaceClient is instantiated without errors.

Is it a regression?
I don't think so. This version check logic using file sizes has existed in the SDK code for years.

Debug Logs

INFO:databricks.sdk:loading DEFAULT profile from ~/.databrickscfg: host, auth_type
DEBUG:databricks.sdk:Ignoring pat auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring basic auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring metadata-service auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring oauth-m2m auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring env-oidc auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring file-oidc auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring github-oidc auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring azure-client-secret auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring github-oidc-azure auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring azure-cli auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring azure-devops-oidc auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring external-browser auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Attempting to configure auth: databricks-cli
DEBUG:databricks.sdk:Databricks CLI version <0.100.0 detected                                          <- :( :( :(
DEBUG:databricks.sdk:Ignoring runtime-oauth auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring runtime auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring google-credentials auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring google-id auth, because databricks-cli is preferred
DEBUG:databricks.sdk:Ignoring model-serving auth, because databricks-cli is preferred

Other Information

  • OS: Windows 11
  • Version: OS build 26100.7781

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions