-
Notifications
You must be signed in to change notification settings - Fork 148
Fix pandas version constraint to prevent numpy binary incompatibility #454
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix pandas version constraint to prevent numpy binary incompatibility #454
Conversation
Update pandas minimum version from 0.25 to 2.2.0 to ensure compatibility with numpy 2.2+. Pandas versions prior to 2.2.0 have pre-compiled C extensions that cause binary incompatibility errors when used with numpy 2.2.6, resulting in 'numpy.dtype size changed' errors on fresh installs. Fixes test collection and import errors in fresh environments.
|
@KRRT7 Thanks for flagging this and opening up a PR. I just made a new conda env using the following: conda create -n test python=3.10 ipython uv pip conda openjdk=8and then within the env I installed > v pip install datacompy
Using Python 3.10.19 environment at: miniconda3/envs/dc_test
Resolved 12 packages in 4.56s
Prepared 1 package in 6.89s
Installed 12 packages in 174ms
+ datacompy==0.18.1
+ jinja2==3.1.6
+ markupsafe==3.0.3
+ numpy==2.2.6
+ ordered-set==4.1.0
+ pandas==2.3.3
+ polars==1.33.1
+ pyarrow==22.0.0
+ python-dateutil==2.9.0.post0
+ pytz==2025.2
+ six==1.17.0
+ tzdata==2025.2It seems to resolve fine for me. I do know the error you mentioned, I've experienced this before myself. We are working on a new 1.0 release here: Lines 36 to 42 in cb82bda
where we are bumping up releases to the latest. not sure if you have any thoughts on this but maybe end users should be pinning on their end to ensure compatible versions? @ak-gupta @rhaffar any thoughts? |
|
Capturing this change in https://github.com/capitalone/datacompy/tree/refactor/init-setup. Appreciate the PR! |
* Refactor imports to remove __init__ calls * fix: update import paths for Fugue, Pandas, Polars, and Spark usage documentation * fix: update numpy and polars version constraints in dependencies * fix from #454 for binary incompatability * chore: remove version checks and warnings. * fix: update numpy version constraints for compatibility * refactor: simplify Spark installation workflow by removing Pandas and NumPy versioning * feat: add availability checks for optional dependencies in fugue, snowflake, and spark modules * feat: implement decorators to check availability of fugue, snowflake, and spark extras * typo: move to init * feat: implement utility function to check module availability and refactor existing checks for fugue, snowflake, and spark * test: add unit tests for module availability checks
Problem
When installing datacompy in a fresh environment, the dependency resolver installs
pandas==2.1.1alongsidenumpy>=2.2.6, which causes a binary incompatibility error:This prevents any tests from running and breaks imports.
Root Cause
The current dependency specification allows
pandas>=0.25,<=2.3.3, but pandas 2.1.1 has pre-compiled C extensions that are incompatible with numpy 2.2.6. Pandas needs to be at least version 2.2.0 to work with numpy 2.2+.Changes
>=0.25to>=2.2.0inpyproject.tomlTesting
Verified that after this change:
uv syncinstalls pandas 2.3.3 (compatible version)pytest --collect-onlysuccessfully collects 269 tests without errorsEnvironment Tested