Skip to content

GH-49503: [Docs][Python] Documenting .pxi doctests are tested via lib.pyx#49515

Open
bhavanaeh wants to merge 1 commit intoapache:mainfrom
bhavanaeh:docs-pxi-doctest-info
Open

GH-49503: [Docs][Python] Documenting .pxi doctests are tested via lib.pyx#49515
bhavanaeh wants to merge 1 commit intoapache:mainfrom
bhavanaeh:docs-pxi-doctest-info

Conversation

@bhavanaeh
Copy link

@bhavanaeh bhavanaeh commented Mar 15, 2026

Rationale for this change

Running python -m pytest --doctest-cython directly on .pxi files doesn't work because Cython .pxi files are included in .pyx files at compile time. This isn't documented, which could potentially confuse contributors.

Please see discussion in #49279 (comment) by @AlenkaF for additional context.

What changes are included in this PR?

Added a note to the Doctest section of the Developing PyArrow documentation explaining that:

  • .pxi files cannot be tested directly with --doctest-cython
  • Doctests should be run on the .pyx file they are included in (e.g., lib.pyx)
  • Errors will appear under the .pyx file, not the original .pxi filename

Are these changes tested?

Documentation-only change. No code changes.

Are there any user-facing changes?

No.

@github-actions
Copy link

⚠️ GitHub issue #49503 has been automatically assigned in GitHub to PR creator.

for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.

.. note::
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with a .. note:: block to make this stand out, but happy to change it to inline text if that's preferred.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine 👍

@bhavanaeh bhavanaeh changed the title GH-49503: [Python][Doc] Documenting .pxi doctests are tested via lib.pyx GH-49503: [Docs][Python] Documenting .pxi doctests are tested via lib.pyx Mar 15, 2026
@github-actions
Copy link

⚠️ GitHub issue #49503 has been automatically assigned in GitHub to PR creator.

@AlenkaF
Copy link
Member

AlenkaF commented Mar 19, 2026

@github-actions crossbow submit preview-docs

@github-actions
Copy link

Revision: ecff1a1

Submitted crossbow builds: ursacomputing/crossbow @ actions-bfd9f4b8a6

Task Status
preview-docs GitHub Actions

Copy link
Member

@AlenkaF AlenkaF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!
Adding one comment from my side.

for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.

.. note::
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine 👍

Comment on lines +199 to +207
Cython ``.pxi`` files are included in ``.pyx`` files at compile time,
so ``--doctest-cython`` cannot be run directly on ``.pxi`` files.
Instead, run doctests on the ``.pyx`` file they are included in, for
example ``lib.pyx``::

$ python -m pytest --doctest-cython pyarrow/lib.pyx

Any doctest errors originating from ``.pxi`` files will appear under
the corresponding ``.pyx`` file, not the original ``.pxi`` filename.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In PyArrow all the .pxi files are included into one .pyx file, namely lib.pyx, see:

# Assorted compatibility helpers
include "compat.pxi"
# Exception types and Status handling
include "error.pxi"
# Configuration information
include "config.pxi"
# pandas API shim
include "pandas-shim.pxi"
# Memory pools and allocation
include "memory.pxi"
# Device type and memory manager
include "device.pxi"
# DataType, Field, Schema
include "types.pxi"
# Array scalar values
include "scalar.pxi"
# Array types
include "array.pxi"
# Builders
include "builder.pxi"
# Column, Table, Record Batch
include "table.pxi"
# Tensors
include "tensor.pxi"
# DLPack
include "_dlpack.pxi"
# File IO
include "io.pxi"
# IPC / Messaging
include "ipc.pxi"
# Micro-benchmark routines
include "benchmark.pxi"
# Public API
include "public-api.pxi"

Therefore we can make the more general (though correct) statement here into a more concrete and clear that all the .pxi files in this case will be part of the lib.pyx and not any other .pyx file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants