Skip to content

Fix connection generator code to work with PyNEST-NG#3784

Draft
jessica-mitchell wants to merge 2 commits intonest:mainfrom
jessica-mitchell:fix-csa
Draft

Fix connection generator code to work with PyNEST-NG#3784
jessica-mitchell wants to merge 2 commits intonest:mainfrom
jessica-mitchell:fix-csa

Conversation

@jessica-mitchell
Copy link
Copy Markdown
Contributor

This is a solution found by Claude code.
csa_example.py and csa_spatial_example.py both worked in Docker container with these modifications
The following text and code changes were done with AI:

File Change
pynest/pynestkernel_aux.h Remove conngenmodule.h include; replace Datum*-returning CYTHON_unpackConnectionGeneratorDatum with CYTHON_insertConnectionGenerator that stores shared_ptr<ConnectionGenerator> into a boost::any Dictionary slot
pynest/nestkernel_api.pxd Add cdef extern declarations for CYTHON_isConnectionGenerator and CYTHON_insertConnectionGenerator
pynest/nestkernel_api.pyx Add elif CYTHON_isConnectionGenerator branch in pydict_to_Dictionary before the AttributeError fallthrough
pynest/CMakeLists.txt Add ${LIBNEUROSIM_LIBRARIES} to nestkernel_api link libraries
nestkernel/conn_builder_conngen.cpp Read params_map indices as long not size_t; static_cast to size_t

Fix: ConnectionGenerator (libneurosim/CSA) support in pynest-ng

Background

When pynest-ng (Cython) was merged into main, the ConnectionGenerator
integration from the old SWIG-based pynest was not fully ported. There are
three distinct problems:

Problem 1 — pynestkernel_aux.h exists but is broken and unused

The file pynest/pynestkernel_aux.h was carried over from the SWIG era but
was never wired into pynest-ng. It has two bugs that make it unusable as-is:

Bug A — includes a module that no longer exists:

// Includes from conngen:
#include "conngenmodule.h"    // <-- ConngenModule was removed from NEST main

This line causes a compile error if the header is ever included while
HAVE_LIBNEUROSIM is defined.

Bug B — returns Datum*, which is incompatible with pynest-ng:

Datum*
CYTHON_unpackConnectionGeneratorDatum( PyObject* obj )
{
  ...
  ret = static_cast< Datum* >( new nest::ConnectionGeneratorDatum( cg ) );
  return ret;
}

pynest-ng does not use the SLI Datum type. All values are stored in
boost::any fields inside a Dictionary. A function returning Datum* cannot
be used to insert a ConnectionGenerator into a pynest-ng Dictionary.

Problem 2 — pydict_to_Dictionary has no ConnectionGenerator branch

nestkernel_api.pyx falls through to raise AttributeError for any type it
does not recognise. pynestkernel_aux.h is not included and its macros are
never called:

AttributeError: value of key (cg) is not a known type,
got <class 'csa.connset.ConnectionSet'>

Problem 3 — conn_builder_conngen.cpp reads params_map as size_t

boost::any requires an exact type match. pydict_to_Dictionary stores Python
int values as long, but the conngen builder reads them as size_t:

NESTErrors.TypeMismatch: Failed to cast 'delay' from long to type unsigned long

Reproducer

With NEST built with libneurosim support:

import nest
import csa

pre  = nest.Create("iaf_psc_alpha", 4)
post = nest.Create("iaf_psc_alpha", 4)

cg = csa.cset(csa.oneToOne, 10000.0, 2.0)
params_map = {"weight": 0, "delay": 1}
connspec = {"rule": "conngen", "cg": cg, "params_map": params_map}

nest.Connect(pre, post, connspec)  # raises AttributeError  (problem 2)
                                   # after fixing problem 2: raises TypeMismatch (problem 3)

The existing test suite in
testsuite/pytests/connect/test_connect_conngen.py covers all four conngen
scenarios and serves as the regression suite for this fix.

@jessica-mitchell jessica-mitchell added S: Normal Handle this with default priority T: Maintenance Work to keep up the quality of the code and documentation. labels Mar 18, 2026
@github-project-automation github-project-automation bot moved this to In progress in PyNEST-NG Mar 18, 2026
@jessica-mitchell jessica-mitchell added the I: No breaking change Previously written code will work as before, no one should note anything changing (aside the fix) label Mar 18, 2026
@jessica-mitchell jessica-mitchell marked this pull request as draft March 18, 2026 10:16
@heplesser
Copy link
Copy Markdown
Contributor

Solving this will be more complicated, since we need to figure out how to interface ConnectionGenerator from libneurosim with NEST. That should only happen once we have #3748 merged, so we do not first create a solution based on boost::any which we then immediately need to convert to variant.

It turns out that our CSA tests have not run for a long time in the CI, they are always skipped because either CSA or libneurosim is not properly installed.

@heplesser heplesser added the P: Blocked Work on this can not continue, see comments for the particular reason label Mar 18, 2026
@jessica-mitchell
Copy link
Copy Markdown
Contributor Author

OK should i Just close this one then?

@heplesser
Copy link
Copy Markdown
Contributor

OK should i Just close this one then?

Leave it open, it contains some useful information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

I: No breaking change Previously written code will work as before, no one should note anything changing (aside the fix) P: Blocked Work on this can not continue, see comments for the particular reason S: Normal Handle this with default priority T: Maintenance Work to keep up the quality of the code and documentation.

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

2 participants