Improve installation instructions to account for unuspported Python versions

[mambocab]

Describe the bug

p5 doesn't support newer versions of Python yet, but this isn't reflected in the installation docs, which say "p5 requires Python 3.6 or above".

I'm seeing maintainers recommend 3.8 in bug reports, which is ultimately what worked for me.

If there's some way to pin a Cython or numpy version and get this working on newer Pythons, that could also be a nice improvement. I couldn't find one but based my attempts off the recommendations here.

To Reproduce
I tried a few ways to install on newer Pythons but ran into bugs that are already reported elsewhere. Pinning a Cython version ultimately didn't work.

System information:

• p5 release (version number or latest commit):
• Python version: Latest 3.10, 3.11, and 3.12.
• Operating system: macos, M1 Mac Mini.

Thanks for your work! I'm up and running now, but it would've been a smoother experience with this small improvement to installation instructions.

[github-actions]

Thank you for submitting your first issue to p5py

[tushar5526]

Hi, thank you for the feedback. Feel free to make a PR, we will try to extend support for other Python versions. However, p5py should work fine for versions <= 3.10, as we have a CI suite running for all these versions.

[mambocab]

TL;DR -- would you welcome a PR that sets a floor for the numpy version and adds Pythons 3.11 and 3.12 to CI?

---

Thanks for setting expectations there. Looks like my issues on 3.10 and 3.11 were transient environmental things.

p5py should work fine for versions <= 3.10, as we have a CI suite running for all these versions.

Worth noting that your tests don't test against Python 3.11 or .12. My running on Apple Silicon also introduces some drift from your CI environment, which could matter while Cythonizing dependencies.

I can consistently reproduce the installation failure for 3.12. Looks like pip decides that p5 needs numpy==1.18.5, which doesn't work on my machine:

numpy installation failure

p5py-3.12-scratch ❯ pip install p5
Collecting p5
  Using cached p5-0.8.2-py3-none-any.whl (144 kB)
Collecting freetype-py==2.1.0.post1 (from p5)
  Using cached freetype_py-2.1.0.post1-py2.py3-none-any.whl (69 kB)
Collecting glfw==1.11.2 (from p5)
  Using cached glfw-1.11.2.tar.gz (27 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy (from p5)
  Obtaining dependency information for numpy from https://files.pythonhosted.org/packages/7a/72/6d1cbdf0d770016bc9485f9ef02e73d5cb4cf3c726f8e120b860a403d307/numpy-1.26.0-cp312-cp312-macosx_11_0_arm64.whl.metadata
  Using cached numpy-1.26.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (53 kB)
Collecting Pillow==9.0.1 (from p5)
  Using cached Pillow-9.0.1.tar.gz (49.5 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting vispy==0.10.0 (from p5)
  Using cached vispy-0.10.0.tar.gz (2.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting PyOpenGL-accelerate==3.1.6 (from p5)
  Using cached PyOpenGL-accelerate-3.1.6.tar.gz (550 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting PyOpenGL==3.1.6 (from p5)
  Using cached PyOpenGL-3.1.6-py3-none-any.whl (2.4 MB)
Collecting requests>=2.25.0 (from p5)
  Obtaining dependency information for requests>=2.25.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
INFO: pip is looking at multiple versions of p5 to determine which version is compatible with other requirements. This could take a while.
Collecting p5
  Using cached p5-0.8.1-py3-none-any.whl (144 kB)
  Using cached p5-0.8.0-py3-none-any.whl (136 kB)
Collecting numpy==1.18.5 (from p5)
  Using cached numpy-1.18.5.zip (5.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [56 lines of output]
      Running from numpy source directory.
      <string>:461: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
      /private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-install-0obuzrtr/numpy_bfb32028a5454c8ea7bc409b2cb07ca5/tools/cythonize.py:75: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        required_version = LooseVersion('0.29.14')
      /private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-install-0obuzrtr/numpy_bfb32028a5454c8ea7bc409b2cb07ca5/tools/cythonize.py:77: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        if LooseVersion(cython_version) < required_version:

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
                  self.rng_state.ctr.v[i] = counter[i]

              self._reset_state_variables()

              self._bitgen.state = <void *>&self.rng_state
              self._bitgen.next_uint64 = &philox_uint64
                                         ^
      ------------------------------------------------------------

      _philox.pyx:195:35: Cannot assign type 'uint64_t (*)(void *) except? -1 nogil' to 'uint64_t (*)(void *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to type 'uint64_t (void *) except? -1 nogil'.
      Processing numpy/random/_bounded_integers.pxd.in
      Processing numpy/random/_philox.pyx
      Traceback (most recent call last):
        File "/private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-install-0obuzrtr/numpy_bfb32028a5454c8ea7bc409b2cb07ca5/tools/cythonize.py", line 238, in <module>
          main()
        File "/private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-install-0obuzrtr/numpy_bfb32028a5454c8ea7bc409b2cb07ca5/tools/cythonize.py", line 234, in main
          find_process_files(root_dir)
        File "/private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-install-0obuzrtr/numpy_bfb32028a5454c8ea7bc409b2cb07ca5/tools/cythonize.py", line 225, in find_process_files
          process(root_dir, fromfile, tofile, function, hash_db)
        File "/private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-install-0obuzrtr/numpy_bfb32028a5454c8ea7bc409b2cb07ca5/tools/cythonize.py", line 191, in process
          processor_function(fromfile, tofile)
        File "/private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-install-0obuzrtr/numpy_bfb32028a5454c8ea7bc409b2cb07ca5/tools/cythonize.py", line 80, in process_pyx
          subprocess.check_call(
        File "/Users/tmp-06-09/.pyenv/versions/3.12.0/lib/python3.12/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/Users/tmp-06-09/.pyenv/versions/3.12.0/envs/p5py-3.12-scratch/bin/python3.12', '-m', 'cython', '-3', '--fast-fail', '-o', '_philox.c', '_philox.pyx']' returned non-zero exit status 1.
      Cythonizing sources
      Traceback (most recent call last):
        File "/Users/tmp-06-09/.pyenv/versions/3.12.0/envs/p5py-3.12-scratch/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/tmp-06-09/.pyenv/versions/3.12.0/envs/p5py-3.12-scratch/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/tmp-06-09/.pyenv/versions/3.12.0/envs/p5py-3.12-scratch/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-build-env-cuhm2vqz/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 396, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-build-env-cuhm2vqz/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 507, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "/private/var/folders/w0/lm8_dkr17379t6mb63m46knm0000gn/T/pip-build-env-cuhm2vqz/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 488, in <module>
        File "<string>", line 469, in setup_package
        File "<string>", line 275, in generate_cython
      RuntimeError: Running cythonize failed!
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

(I'll note that this is consistent with my professional experience with Cython. Disappointing to see that errors haven't gotten clearer in the past 5 years. But I guess that's C/Cython/cpython for ya.)

By manually specifying a numpy version, I was able to address the issue on my machine.

successfull installation

$ pip install 'numpy==1.26.0' p5
Collecting numpy==1.26.0
  Obtaining dependency information for numpy==1.26.0 from https://files.pythonhosted.org/packages/7a/72/6d1cbdf0d770016bc9485f9ef02e73d5cb4cf3c726f8e120b860a403d307/numpy-1.26.0-cp312-cp312-macosx_11_0_arm64.whl.metadata
  Using cached numpy-1.26.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (53 kB)
Collecting p5
  Using cached p5-0.8.2-py3-none-any.whl (144 kB)
Collecting freetype-py==2.1.0.post1 (from p5)
  Using cached freetype_py-2.1.0.post1-py2.py3-none-any.whl (69 kB)
Collecting glfw==1.11.2 (from p5)
  Using cached glfw-1.11.2.tar.gz (27 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting Pillow==9.0.1 (from p5)
  Using cached Pillow-9.0.1.tar.gz (49.5 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting vispy==0.10.0 (from p5)
  Using cached vispy-0.10.0.tar.gz (2.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting PyOpenGL-accelerate==3.1.6 (from p5)
  Using cached PyOpenGL-accelerate-3.1.6.tar.gz (550 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting PyOpenGL==3.1.6 (from p5)
  Using cached PyOpenGL-3.1.6-py3-none-any.whl (2.4 MB)
Collecting requests>=2.25.0 (from p5)
  Obtaining dependency information for requests>=2.25.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
INFO: pip is looking at multiple versions of p5 to determine which version is compatible with other requirements. This could take a while.
Collecting p5
  Using cached p5-0.8.1-py3-none-any.whl (144 kB)
  Using cached p5-0.8.0-py3-none-any.whl (136 kB)
  Using cached p5-0.7.1-py3-none-any.whl (108 kB)
Collecting freetype-py (from p5)
  Using cached freetype_py-2.4.0-py3-none-macosx_10_9_universal2.whl (1.8 MB)
Collecting glfw (from p5)
  Obtaining dependency information for glfw from https://files.pythonhosted.org/packages/5c/de/bf2d9553ac82553b18f18882e93beac565c715b0a3fcc291f21df9fb58fa/glfw-2.6.2-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-macosx_11_0_arm64.whl.metadata
  Using cached glfw-2.6.2-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-macosx_11_0_arm64.whl.metadata (5.4 kB)
Collecting Pillow (from p5)
  Obtaining dependency information for Pillow from https://files.pythonhosted.org/packages/5c/72/6d0a22c3f499e966f8b6b89842dfb915fb9449d0c9aa2b4e336259a7f00c/Pillow-10.0.1-cp312-cp312-macosx_11_0_arm64.whl.metadata
  Using cached Pillow-10.0.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (9.5 kB)
Collecting vispy (from p5)
  Obtaining dependency information for vispy from https://files.pythonhosted.org/packages/a3/44/e8e0be8810459af66321fae2eb3dca9d4e23fa3e14c9c1457e16a2790ff3/vispy-0.14.1-cp312-cp312-macosx_11_0_arm64.whl.metadata
  Using cached vispy-0.14.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (8.3 kB)
Collecting PyOpenGL (from p5)
  Obtaining dependency information for PyOpenGL from https://files.pythonhosted.org/packages/99/48/00e31747821d3fc56faddd00a4725454d1e694a8b67d715cf20f531506a5/PyOpenGL-3.1.7-py3-none-any.whl.metadata
  Using cached PyOpenGL-3.1.7-py3-none-any.whl.metadata (3.2 kB)
Collecting PyOpenGL-accelerate (from p5)
  Using cached PyOpenGL_accelerate-3.1.7-cp312-cp312-macosx_13_0_arm64.whl
Collecting charset-normalizer<4,>=2 (from requests>=2.25.0->p5)
  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/b8/db/2374cb7e1736f7f11651c706d44c4a2428e58f9376d7f1666431ce876259/charset_normalizer-3.3.0-cp312-cp312-macosx_11_0_arm64.whl.metadata
  Using cached charset_normalizer-3.3.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests>=2.25.0->p5)
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests>=2.25.0->p5)
  Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
  Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests>=2.25.0->p5)
  Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
  Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting hsluv (from vispy->p5)
  Obtaining dependency information for hsluv from https://files.pythonhosted.org/packages/96/36/5bddefea3d7adf22a64f9aa9701492f8a9fe6948223f5cf2602c22ec9be7/hsluv-5.0.4-py2.py3-none-any.whl.metadata
  Using cached hsluv-5.0.4-py2.py3-none-any.whl.metadata (2.7 kB)
Collecting kiwisolver (from vispy->p5)
  Obtaining dependency information for kiwisolver from https://files.pythonhosted.org/packages/26/61/58bb691f6880588be3a4801d199bd776032ece07203faf3e4a8b377f7d9b/kiwisolver-1.4.5-cp312-cp312-macosx_11_0_arm64.whl.metadata
  Using cached kiwisolver-1.4.5-cp312-cp312-macosx_11_0_arm64.whl.metadata (6.4 kB)
Collecting packaging (from vispy->p5)
  Obtaining dependency information for packaging from https://files.pythonhosted.org/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl.metadata
  Using cached packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
Using cached numpy-1.26.0-cp312-cp312-macosx_11_0_arm64.whl (13.7 MB)
Using cached glfw-2.6.2-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-macosx_11_0_arm64.whl (92 kB)
Using cached Pillow-10.0.1-cp312-cp312-macosx_11_0_arm64.whl (3.3 MB)
Using cached PyOpenGL-3.1.7-py3-none-any.whl (2.4 MB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached vispy-0.14.1-cp312-cp312-macosx_11_0_arm64.whl (1.5 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.3.0-cp312-cp312-macosx_11_0_arm64.whl (116 kB)
Using cached urllib3-2.0.6-py3-none-any.whl (123 kB)
Using cached hsluv-5.0.4-py2.py3-none-any.whl (5.3 kB)
Using cached kiwisolver-1.4.5-cp312-cp312-macosx_11_0_arm64.whl (64 kB)
Using cached packaging-23.2-py3-none-any.whl (53 kB)
Installing collected packages: PyOpenGL-accelerate, PyOpenGL, glfw, urllib3, Pillow, packaging, numpy, kiwisolver, idna, hsluv, freetype-py, charset-normalizer, certifi, vispy, requests, p5
Successfully installed Pillow-10.0.1 PyOpenGL-3.1.7 PyOpenGL-accelerate-3.1.7 certifi-2023.7.22 charset-normalizer-3.3.0 freetype-py-2.4.0 glfw-2.6.2 hsluv-5.0.4 idna-3.4 kiwisolver-1.4.5 numpy-1.26.0 p5-0.7.1 packaging-23.2 requests-2.31.0 urllib3-2.0.6 vispy-0.14.1

So from my perspective, the correct PR here would be one that restricts numpy versions to 1.26.0 and up. That seems appropriate anyway, as the numpy maintainers claim that 1.26.0 is the first version compatible with Python 3.12.0.

[tushar5526]

Hi @mambocab , thank you for the detailed research. Please feel free to make a PR for this and we can have a release with updated docs.

[audiocommander]

I also failed installing P5.py on my mac silicon M2; I have Python 3.11 and 3.10, failed to install with both versions. I now created an environment for python 3.7, but I hope you guys are aware that 3.7 is older than silicon, so it is really troublesome to find all libraries matching 3.7.
FYI, I eventually gave up on this after fiddling around for almost a day, as my initial intention was to provide an easy intro workshop for students, that are new to Jupyter/Python as there are a lot of resources for Processing. But the setup seems way too complicated for beginners. The recommended workaround with python <= 3.7 is a tough one, at least for Mac Silicons.

[tushar5526]

Can you share the errors you faced while installing?