Remove use of `python setup.py develop/install` in the project #2172

ndgrigorian · 2025-10-15T00:30:59Z

A warning that python setup.py develop is deprecated and will no longer be supported at the end of October, 2025 has been ongoing for some time when using dpctl build driver scripts, and can be seen in the CI

This PR proposes instead relying on pip for installing dpctl in scripts, and reworks the scripts to maintain use of scikit-build

In the future, this will also simplify a transition to scikit-build-core

The PR also introduces options

--clean and --skip-editable to build_locally driver
--clean and --skip-pytest to gen_coverage driver
--clean to gen_docs driver

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
Have you added documentation for your changes, if necessary?
Have you added your changes to the changelog?
If this PR is a work in progress, are you opening the PR as a draft?

github-actions · 2025-10-15T01:06:39Z

View rendered docs @ https://intelpython.github.io/dpctl/pulls/2172/index.html

github-actions · 2025-10-15T01:11:01Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_29 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-15T01:14:34Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_29 ran successfully.
Passed: 1116
Failed: 5
Skipped: 119

coveralls · 2025-10-15T01:15:53Z

coverage: 86.277%. remained the same
when pulling 1800644 on do-not-use-setup-py-develop
into 091beaa on master.

github-actions · 2025-10-15T05:30:47Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_30 ran successfully.
Passed: 1114
Failed: 7
Skipped: 119

github-actions · 2025-10-15T05:58:07Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_30 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-15T06:22:57Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_30 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

scripts/_build_helper.py

scripts/build_locally.py

antonwolfy · 2025-10-15T10:10:51Z

scripts/build_locally.py

+    p.add_argument(
        "--debug",
-        default="Release",
+        dest="build_type",


The debug build produces a lot of the warning which probably has to be resolved in the follow-up PR.

a number of them are
icpx: warning: argument unused during compilation: '-Xsycl-target-frontend=spir64 -g0'
which will need some looking at

the remainder are
warning: comparison of integers of different signs: 'py::ssize_t' (aka 'long') and 'std::size_t'
from
assert(dst.get_size() == iter_nelems)
in sorting functions

should be a simple enough fix

github-actions · 2025-10-15T16:14:26Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_32 ran successfully.
Passed: 1116
Failed: 5
Skipped: 119

github-actions · 2025-10-15T16:21:18Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_31 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-15T21:39:04Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_32 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-28T23:19:35Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_33 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-28T23:26:34Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_34 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-28T23:43:44Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_34 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-29T00:38:14Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_35 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

antonwolfy · 2025-10-30T15:31:24Z

.github/workflows/conda-package.yml

            pushd $d
            conda activate --stack ${{ env.BUILD_ENV_NAME }}
-            CC=icx CXX=icpx python setup.py develop -G Ninja  || exit 1
+            CC=icx CXX=icpx python setup.py build_ext --inplace -G Ninja  || exit 1


CONTRIBUTING.md needs to be updated also

and docs/_legacy/docfiles/user_guides/QuickStart.rst and docs/doc_sources/contributor_guides/building.rst

some README.md in the examples folder as well

.github/workflows/generate-docs.yml

.github/workflows/os-llvm-sycl-build.yml

antonwolfy · 2025-10-30T15:49:51Z

scripts/build_locally.py

 import sys

+# add scripts dir to Python path so we can import _build_helper
+sys.path.insert(0, os.path.abspath("scripts"))


I wonder for what use case we need that? Simple python scripts/build_locally.py works fine without that.

It didn't work for me locally without this, essentially importing from _build_helper wouldn't work without this or an __init__.py

Probably it would be better to add __init__.py then. It looks clearer.

scripts/gen_coverage.py

scripts/gen_docs.py

scripts/build_locally.py

antonwolfy · 2025-10-30T17:08:15Z

scripts/build_locally.py

+        dest="no_level_zero",
+        action="store_true",
+        default=False,
+        help="Disable Level Zero backend (deprecated: use --target-level-zero "


--target-level-zero OFF does not work

scripts/build_locally.py

to be reused in gen_docs and gen_coverage

github-actions · 2025-10-30T21:30:13Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_31 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-30T21:37:22Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_31 ran successfully.
Passed: 1114
Failed: 7
Skipped: 119

github-actions · 2025-10-30T21:37:40Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_31 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

vlad-perevezentsev · 2025-10-31T13:24:10Z

scripts/_build_helper.py

+        f"-DCMAKE_C_COMPILER:PATH={c_compiler}" if c_compiler else "",
+        f"-DCMAKE_CXX_COMPILER:PATH={cxx_compiler}" if cxx_compiler else "",
+        f"-DDPCTL_ENABLE_L0_PROGRAM_CREATION={'ON' if level_zero else 'OFF'}",
+        f"-DDPTL_ENABLE_GLOG:BOOL={'ON' if glog else 'OFF'}",


Suggested change

f"-DDPTL_ENABLE_GLOG:BOOL={'ON' if glog else 'OFF'}",

f"-DDPCTL_ENABLE_GLOG:BOOL={'ON' if glog else 'OFF'}",

vlad-perevezentsev · 2025-10-31T13:50:42Z

scripts/_build_helper.py

+    if other_opts:
+        args.extend(other_opts.split())
+
+    return args


Is it safe to pass a list containing empty ("") arguments to CMake?

vlad-perevezentsev · 2025-10-31T13:58:45Z

scripts/build_locally.py

+    if args.target_hip is not None and not args.target_hip.strip():
+        err("--target-hip requires an explicit architecture", "build_locally")
+
+    # CUDA/HIP targets


The previous implementation included a safeguard to prevent using --target-cuda together with -DDPCTL_TARGET_CUDA=... (HIP as well) in --cmake-opts

if target_cuda is not None: if not target_cuda.strip(): raise ValueError( "--target-cuda can not be an empty string. " "Use --target-cuda=<arch> or --target-cuda" ) if any(opt.startswith("-DDPCTL_TARGET_CUDA=") for opt in cmake_args): raise ValueError( "Both --target-cuda and -DDPCTL_TARGET_CUDA in --cmake-opts " "were specified. Please use only one method " "to avoid ambiguity" ) cmake_args += [ f"-DDPCTL_TARGET_CUDA={target_cuda}", ]

Why did you decide to remove this check?

vlad-perevezentsev · 2025-10-31T14:07:48Z

scripts/_build_helper.py

+
+
+def err(msg: str, script: str):
+    print(f"[{script}][error] {msg}", file=sys.stderr)


This function only prints an error message.
Should we also terminate build_locally script when err() is called?

vlad-perevezentsev · 2025-10-31T14:31:34Z

scripts/build_locally.py

-        "cxx_compiler",
-        "compiler_root",
-    ]
+    p.add_argument(


It does not work

$ python scripts/build_locally.py --clean $ ls | grep _skbuild > _skbuild $ ls _skbuild/linux-x86_64-3.13/ > cmake-build cmake-install setuptools

ndgrigorian force-pushed the do-not-use-setup-py-develop branch from 6d87c72 to 124f5f7 Compare October 15, 2025 00:34

ndgrigorian force-pushed the do-not-use-setup-py-develop branch 2 times, most recently from fd45fec to 13409b7 Compare October 15, 2025 05:18

ndgrigorian force-pushed the do-not-use-setup-py-develop branch from 13409b7 to 32f1509 Compare October 15, 2025 05:42

ndgrigorian changed the title ~~Do not use python setup.py develop in the project~~ Remove use of python setup.py develop/install in the project Oct 15, 2025

antonwolfy reviewed Oct 15, 2025

View reviewed changes

ndgrigorian force-pushed the do-not-use-setup-py-develop branch from 00161f1 to 989df1b Compare October 15, 2025 15:40

ndgrigorian requested a review from antonwolfy October 15, 2025 15:46

ndgrigorian force-pushed the do-not-use-setup-py-develop branch from acb4584 to a113d01 Compare October 28, 2025 23:03

ndgrigorian marked this pull request as ready for review October 28, 2025 23:54

ndgrigorian requested a review from vlad-perevezentsev as a code owner October 28, 2025 23:54

antonwolfy reviewed Oct 30, 2025

View reviewed changes

ndgrigorian added 4 commits October 30, 2025 13:35

update build_locally script to avoid python setup.py develop call

9b7deae

refactor common build functionality out into separate file

70e4638

to be reused in gen_docs and gen_coverage

remove --build-dir option and fix --clean option

d7e561e

do not return compiler root unnecessarily from resolve_compilers

365cd43

ndgrigorian added 9 commits October 30, 2025 13:35

update gen_coverage script to align with build_locally

4a3ab99

resolve bin_llvm from default compiler layout when not provided

268351f

keep find_objects defined within main

181a51b

generalize err and warn utilities for different build scripts

5b4c927

use common resolve_compilers utility

136d318

update gen_docs script

775394e

try using pip install -e in place of setup.py develop in CI

08d9cd3

use python setup.py build_ext in Cython extension building

0e0c1ba

add types and descriptions for script args

7c8e35e

ndgrigorian force-pushed the do-not-use-setup-py-develop branch 2 times, most recently from 6abda67 to f49a274 Compare October 30, 2025 20:54

do not override CMAKE_ARGS in build scripts

1800644

ndgrigorian force-pushed the do-not-use-setup-py-develop branch from f49a274 to 1800644 Compare October 30, 2025 20:56

vlad-perevezentsev reviewed Oct 31, 2025

View reviewed changes

	f"-DDPTL_ENABLE_GLOG:BOOL={'ON' if glog else 'OFF'}",
	f"-DDPCTL_ENABLE_GLOG:BOOL={'ON' if glog else 'OFF'}",



		def err(msg: str, script: str):
		print(f"[{script}][error] {msg}", file=sys.stderr)

Remove use of python setup.py develop/install in the project #2172

Are you sure you want to change the base?

Remove use of python setup.py develop/install in the project #2172

Conversation

ndgrigorian commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

coveralls commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

github-actions bot commented Oct 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Oct 30, 2025

Uh oh!

github-actions bot commented Oct 30, 2025

Uh oh!

github-actions bot commented Oct 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Remove use of `python setup.py develop/install` in the project #2172

Remove use of `python setup.py develop/install` in the project #2172

ndgrigorian commented Oct 15, 2025 •

edited

Loading

coveralls commented Oct 15, 2025 •

edited

Loading