Get HemeLB RBC code running on Cirrus post merge into main

[mobernabeu]

Similar to #770.

Let's start by compiling the code manually on the login nodes of the machine and then we will try to sort out builds with Fabric afterwards.

Document module configuration here.

[CharlesQiZhou]

Minimum requirement for the dependencies to be compiled on cirrus:

module load cmake/3.17.3
module load gcc/8.2.0
module load intel-mpi-19/19.0.0.117

@mobernabeu @rupertnash
Currently, dependencies fail during MAKE with the errors below:
stderr.txt

• rm -rf autom4te.cache
• autoreconf --force --install --warnings all,no-obsolete
  configure.ac:25: installing './ar-lib'
  configure.ac:24: installing './compile'
  configure.ac:18: installing './config.guess'
  configure.ac:18: installing './config.sub'
  configure.ac:19: installing './install-sh'
  configure.ac:19: installing './missing'
  Makefile.am: installing './INSTALL'
  Makefile.am:69: error: Libtool library used but 'LIBTOOL' is undefined
  Makefile.am:69: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
  Makefile.am:69: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
  Makefile.am:69: If 'LT_INIT' is in 'configure.ac', make sure
  Makefile.am:69: its definition is in aclocal's search path.
  Makefile.am: installing './depcomp'
  parallel-tests: installing './test-driver'
  autoreconf: automake failed with exit status: 1
• rm -rf autom4te.cache
• autoreconf --force --install --warnings all,no-obsolete
  Makefile.am:69: error: Libtool library used but 'LIBTOOL' is undefined
  Makefile.am:69: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
  Makefile.am:69: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
  Makefile.am:69: If 'LT_INIT' is in 'configure.ac', make sure
  Makefile.am:69: its definition is in aclocal's search path.
  autoreconf: automake failed with exit status: 1
  make[2]: *** [CMakeFiles/ctemplate.dir/build.make:142: ctemplate-prefix/src/ctemplate-stamp/ctemplate-autogen] Error 1
  make[1]: *** [CMakeFiles/Makefile2:155: CMakeFiles/ctemplate.dir/all] Error 2
  make: *** [Makefile:104: all] Error 2

stdout.txt

Scanning dependencies of target ctemplate
[ 11%] Creating directories for 'ctemplate'
[ 22%] Performing download step (download, verify and extract) for 'ctemplate'
-- Downloading...
dst='/lustre/home/ec136/qizhou/hemelb/dependencies/build/ctemplate-prefix/src/ctemplate-2.4.tar.gz'
timeout='none'
-- Using src='http://github.com/OlafvdSpek/ctemplate/archive/ctemplate-2.4.tar.gz'
-- [download 100% complete]
-- Downloading... done
-- extracting...
src='/lustre/home/ec136/qizhou/hemelb/dependencies/build/ctemplate-prefix/src/ctemplate-2.4.tar.gz'
dst='/lustre/home/ec136/qizhou/hemelb/dependencies/build/ctemplate-prefix/src/ctemplate'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[ 33%] Performing patch step for 'ctemplate'
noalign
novacopy
[ 44%] Running ./autogen.sh, possibly twice

[mobernabeu]

@rupertnash is there any good reason not to go with gcc 10.2 in Cirrus? (rather than ancient 8.2)

[CharlesQiZhou]

Just tried gcc 10.2, namely with the below modules

module load cmake/3.17.3
module load gcc/10.2.0
module load intel-mpi-19/19.0.0.117

Dependencies (make after cmake -DHEMELB_BUILD_RBC=ON ..) failed with seemingly the same errors as using gcc 8.2 (Note that previously the red_blood_cells/merge branch successfully compiled on Cirrus with gcc 8.2).

See the complete stderr.txt and stdout.txt below for this attempt with gcc 10.2:
stderr.txt
stdout.txt
:

[rupertnash]

I'm pro updating to 10: newer is usually better for the compilers

[rupertnash]

I think you need to load the libtool module on Cirrus: module load libtool/2.4.6

[CharlesQiZhou]

Thanks @rupertnash. Adding module load libtool/2.4.6 works for the dependencies to compile without errors. The Code also compiles and hemelb executable is generated, but with the errors (stderr.txt) below:

/lustre/home/ec136/qizhou/hemelb/Code/redblood/Mesh.cc:691:2: warning: multi-line comment [-Wcomment]
691 | // | \ |
| ^
/lustre/home/ec136/qizhou/hemelb/Code/redblood/MeshIO.cc: In member function ‘virtual hemelb::redblood::MeshIO::MeshPtr hemelb::redblood::KruegerMeshIO::read(hemelb::redblood::MeshIO::Mode, const string&, bool) const’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/MeshIO.cc:207:5: warning: control reaches end of non-void function [-Wreturn-type]
207 | }
| ^
/lustre/home/ec136/qizhou/hemelb/Code/redblood/MeshIO.cc: In member function ‘virtual std::string hemelb::redblood::KruegerMeshIO::write(hemelb::redblood::MeshIO::Mode, const string&, const Vertices&, const Facets&, const hemelb::util::UnitConverter&, const PointScalarData&) const’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/MeshIO.cc:264:5: warning: control reaches end of non-void function [-Wreturn-type]
264 | }
| ^
/lustre/home/ec136/qizhou/hemelb/Code/redblood/MeshIO.cc: In member function ‘virtual std::string hemelb::redblood::VTKMeshIO::write(hemelb::redblood::MeshIO::Mode, const string&, const Vertices&, const Facets&, const hemelb::util::UnitConverter&, const PointScalarData&) const’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/MeshIO.cc:356:51: warning: control reaches end of non-void function [-Wreturn-type]
356 | auto pd = vtkSmartPointer::New();
| ^
/lustre/home/ec136/qizhou/hemelb/Code/redblood/Cell.cc: In member function ‘hemelb::LatticeEnergy hemelb::redblood::Cell::facetBending() const’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/Cell.cc:64:24: warning: comparison of integer expressions of different signedness: ‘long long int’ and ‘std::size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
64 | if (neighbor > static_caststd::size_t(current_facet))
| ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/redblood/Cell.cc: In member function ‘hemelb::LatticeEnergy hemelb::redblood::Cell::facetBending(std::vector<hemelb::util::Vector3D >&) const’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/Cell.cc:92:24: warning: comparison of integer expressions of different signedness: ‘long long int’ and ‘std::size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
92 | if (neighbor > current_facet)
| ~~~~~~~~~^~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/redblood/xmlIO.cc:90:23: warning: ‘hemelb::LatticePosition hemelb::redblood::{anonymous}::GetPosition(const hemelb::io::xml::Element&, const string&, const hemelb::util::UnitConverter&)’ defined but not used [-Wunused-function]
90 | LatticePosition GetPosition(const io::xml::Element& parent, const std::string &elemname,
| ^~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/redblood/parallel/CellParallelization.cc: In member function ‘void hemelb::redblood::parallel::ExchangeCells::AddDisownedToLocalSendBuffers(int, int, std::set<std::shared_ptrhemelb::redblood::CellBase, hemelb::redblood::details::CellUUIDComparison>::const_reference, const mapped_type&)’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/parallel/CellParallelization.cc:355:23: warning: unused variable ‘inserted’ [-Wunused-variable]
355 | auto const &inserted = formelyOwned[neighbor].emplace(std::move(lentCell));
| ^~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/extraction/LocalPropertyOutput.cc: In member function ‘void hemelb::extraction::LocalPropertyOutput::Write(long unsigned int)’:
/lustre/home/ec136/qizhou/hemelb/Code/extraction/LocalPropertyOutput.cc:280:35: warning: comparison of integer expressions of different signedness: ‘int’ and ‘unsigned int’ [-Wsign-compare]
280 | for (int i = 0; i < numComponents; i++)
| ~~^~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/extraction/LocalDistributionInput.cc: In member function ‘void hemelb::extraction::LocalDistributionInput::LoadDistribution(hemelb::geometry::LatticeData*, boost::optional&)’:
/lustre/home/ec136/qizhou/hemelb/Code/extraction/LocalDistributionInput.cc:153:20: warning: comparison of integer expressions of different signedness: ‘int’ and ‘const unsigned int’ [-Wsign-compare]
153 | for (int i = 0; i < NUMVECTORS; i++) {
| ~~^~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/extraction/LocalDistributionInput.cc: In member function ‘void hemelb::extraction::LocalDistributionInput::ReadOffsets(const string&)’:
/lustre/home/ec136/qizhou/hemelb/Code/extraction/LocalDistributionInput.cc:274:13: warning: comparison of integer expressions of different signedness: ‘uint32_t’ {aka ‘unsigned int’} and ‘int’ [-Wsign-compare]
274 | if (nRanks != comms.Size())
| ~~~~~~~^~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/lb/iolets/InOutLetFile.cc: In member function ‘void hemelb::lb::iolets::InOutLetFile::CalculateTable(hemelb::LatticeTimeStep, hemelb::PhysicalTime)’:
/lustre/home/ec136/qizhou/hemelb/Code/lb/iolets/InOutLetFile.cc:109:13: warning: unused variable ‘TimeStepsInInletPressureProfile’ [-Wunused-variable]
109 | int TimeStepsInInletPressureProfile = times.back() / timeStepLength;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/net/MpiCommunicator.cc:9:
/lustre/home/ec136/qizhou/hemelb/Code/net/MpiCommunicator.h: In constructor ‘hemelb::net::MpiCommunicator::MpiCommunicator(int, int)’:
/lustre/home/ec136/qizhou/hemelb/Code/net/MpiCommunicator.h:229:13: warning: ‘hemelb::net::MpiCommunicator::localRankInCommunicator’ will be initialized after [-Wreorder]
229 | int localRankInCommunicator;
| ^~~~~~~~~~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/net/MpiCommunicator.h:227:13: warning: ‘int hemelb::net::MpiCommunicator::communicatorSize’ [-Wreorder]
227 | int communicatorSize;
| ^~~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/net/MpiCommunicator.cc:68:5: warning: when initialized here [-Wreorder]
68 | MpiCommunicator::MpiCommunicator(int localRankInCommunicator, int communicatorSize) :
| ^~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/io/writers/xdr/XdrFileReader.cc: In member function ‘virtual const char* hemelb::io::writers::xdr::XdrFileReader::get_bytes(size_t)’:
/lustre/home/ec136/qizhou/hemelb/Code/io/writers/xdr/XdrFileReader.cc:33:9: warning: unused variable ‘nread’ [-Wunused-variable]
33 | auto nread = std::fread(buf.data(), 1, n, fh);
| ^~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/colloids/BoundaryConditions.cc:7:
/lustre/home/ec136/qizhou/hemelb/Code/colloids/LubricationBC.h: In member function ‘virtual const bool hemelb::colloids::LubricationBC::DoSomethingToParticle(hemelb::colloids::Particle&, std::vector<hemelb::util::Vector3D >)’:
/lustre/home/ec136/qizhou/hemelb/Code/colloids/LubricationBC.h:34:27: warning: ‘lubricationVelocityAdjustment.hemelb::util::Vector3D::x’ may be used uninitialized in this function [-Wmaybe-uninitialized]
34 | LatticeVelocity lubricationVelocityAdjustment;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/colloids/LubricationBC.h:34:27: warning: ‘lubricationVelocityAdjustment.hemelb::util::Vector3D::y’ may be used uninitialized in this function [-Wmaybe-uninitialized]
In file included from /lustre/home/ec136/qizhou/hemelb/Code/colloids/BoundaryConditions.h:13,
from /lustre/home/ec136/qizhou/hemelb/Code/colloids/BoundaryConditions.cc:6:
/lustre/home/ec136/qizhou/hemelb/Code/colloids/Particle.h:175:41: warning: ‘lubricationVelocityAdjustment.hemelb::util::Vector3D::z’ may be used uninitialized in this function [-Wmaybe-uninitialized]
175 | lubricationVelocityAdjustment = adjustment;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/colloids/BoundaryConditions.cc:7:
/lustre/home/ec136/qizhou/hemelb/Code/colloids/LubricationBC.h:34:27: note: ‘lubricationVelocityAdjustment.hemelb::util::Vector3D::z’ was declared here
34 | LatticeVelocity lubricationVelocityAdjustment;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/main.cc:10:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/SimulationMasterTests.cc:9:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/tests/extraction/LocalPropertyOutputTests.cc:19:
/lustre/home/ec136/qizhou/hemelb/Code/tests/extraction/DummyDataSource.h:126:23: warning: invalid covariant return type for ‘virtual hemelb::distribn_t hemelb::tests::DummyDataSource::GetDistribution() const’
126 | distribn_t GetDistribution() const
| ^~~~~~~~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/extraction/GeometrySelector.h:10,
from /lustre/home/ec136/qizhou/hemelb/Code/extraction/PropertyOutputFile.h:13,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/extraction/LocalPropertyOutputTests.cc:13:
/lustre/home/ec136/qizhou/hemelb/Code/extraction/IterableDataSource.h:98:35: note: overridden function is ‘virtual const distribn_t* hemelb::extraction::IterableDataSource::GetDistribution() const’
98 | virtual const distribn_t* GetDistribution() const = 0;
| ^~~~~~~~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/tests/extraction/LocalPropertyOutputTests.cc:19:
/lustre/home/ec136/qizhou/hemelb/Code/tests/extraction/DummyDataSource.h: In member function ‘virtual hemelb::distribn_t* hemelb::tests::DummyDataSource::GetDistribution() const’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/extraction/DummyDataSource.h:130:20: warning: function returns address of local variable [-Wreturn-local-addr]
130 | return p_fval;
| ^~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/tests/extraction/DummyDataSource.h:128:20: note: declared here
128 | double fval = 1.0;
| ^~~~
/lustre/home/ec136/qizhou/hemelb/Code/tests/io/xdr_test_data.cc: In function ‘float hemelb::tests::{anonymous}::binflt(uint32_t)’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/io/xdr_test_data.cc:18:17: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
18 | return reinterpret_cast<float>(&x);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/tests/io/xdr_test_data.cc: In function ‘double hemelb::tests::{anonymous}::bindbl(uint64_t)’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/io/xdr_test_data.cc:21:17: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
21 | return reinterpret_cast<double>(&x);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/tests/io/XdrReaderTests.cc: In function ‘void hemelb::tests::____C_A_T_C_H____T_E_S_T____6()’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/io/XdrReaderTests.cc:66:12: warning: variable ‘make_ones’ set but not used [-Wunused-but-set-variable]
66 | auto make_ones = [](size_t n) -> UPC {
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/tests/lb/BroadcastMocks.cc:6:
/lustre/home/ec136/qizhou/hemelb/Code/tests/lb/BroadcastMocks.h:125: warning: ignoring ‘#pragma intel optimization_level’ [-Wunknown-pragmas]
125 | #pragma intel optimization_level 0
|
In file included from /lustre/home/ec136/qizhou/hemelb/Code/tests/lb/IncompressibilityCheckerTests.cc:10:
/lustre/home/ec136/qizhou/hemelb/Code/tests/lb/BroadcastMocks.h:125: warning: ignoring ‘#pragma intel optimization_level’ [-Wunknown-pragmas]
125 | #pragma intel optimization_level 0
|
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/multiscale/MultiscaleSimulationMaster.h:10,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/multiscale/MockIntercommunicatorTests.cc:11:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/tests/reporting/ReporterTests.cc:20:
/lustre/home/ec136/qizhou/hemelb/Code/tests/lb/BroadcastMocks.h:125: warning: ignoring ‘#pragma intel optimization_level’ [-Wunknown-pragmas]
125 | #pragma intel optimization_level 0
|
In file included from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/CellArmyTests.cc:8:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q15, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/CellArmyTests.cc:215:26: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc: In instantiation of ‘void hemelb::tests::____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____F_U_N_C____9() [with TestType = hemelb::redblood::stencil::FourPoint]’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:226:5: required from ‘void hemelb::tests::{anonymous}::ns_____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____10::____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____10::reg_tests() [with Types = {hemelb::redblood::stencil::FourPoint, hemelb::redblood::stencil::CosineApprox, hemelb::redblood::stencil::ThreePoint, hemelb::redblood::stencil::TwoPoint}]’
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:226:5: required from here
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:256:27: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
256 | LatticeDistance const x(*reinterpret_cast<LatticeDistance const *>(&ux));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc: In instantiation of ‘void hemelb::tests::____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____F_U_N_C____9() [with TestType = hemelb::redblood::stencil::CosineApprox]’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:226:5: required from ‘void hemelb::tests::{anonymous}::ns_____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____10::____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____10::reg_tests() [with Types = {hemelb::redblood::stencil::FourPoint, hemelb::redblood::stencil::CosineApprox, hemelb::redblood::stencil::ThreePoint, hemelb::redblood::stencil::TwoPoint}]’
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:226:5: required from here
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:256:27: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc: In instantiation of ‘void hemelb::tests::____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____F_U_N_C____9() [with TestType = hemelb::redblood::stencil::ThreePoint]’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:226:5: required from ‘void hemelb::tests::{anonymous}::ns_____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____10::____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____10::reg_tests() [with Types = {hemelb::redblood::stencil::FourPoint, hemelb::redblood::stencil::CosineApprox, hemelb::redblood::stencil::ThreePoint, hemelb::redblood::stencil::TwoPoint}]’
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:226:5: required from here
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:256:27: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc: In instantiation of ‘void hemelb::tests::____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____F_U_N_C____9() [with TestType = hemelb::redblood::stencil::TwoPoint]’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:226:5: required from ‘void hemelb::tests::{anonymous}::ns_____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____10::____C_A_T_C_H____T_E_M_P_L_A_T_E____T_E_S_T____10::reg_tests() [with Types = {hemelb::redblood::stencil::FourPoint, hemelb::redblood::stencil::CosineApprox, hemelb::redblood::stencil::ThreePoint, hemelb::redblood::stencil::TwoPoint}]’
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:226:5: required from here
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/InterpolationTests.cc:256:27: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/LoadDeformedCellTests.cc:11:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/NodeIntegrationTests.cc:13:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::TwoPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::TwoPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::CosineApprox>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::CosineApprox>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/CellIntegrationTests.cc:11:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/FadeInOutIntegrationTests.cc:10:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/SadCellIntegrationTests.cc:10:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/MulticellBenchmarkTests.cc:10:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/FedosovValidationTests.cc:10:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/parallel/ParallelFixtureTests.cc:16:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/parallel/MPISpreadForcesTests.cc:13:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::TwoPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::TwoPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::CosineApprox>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::CosineApprox>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/parallel/MPIIntegrateVelocities.cc:14:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::TwoPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::TwoPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::CosineApprox>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::CosineApprox>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/parallel/MPILockStepTests.cc:13:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::TwoPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::TwoPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::CosineApprox>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::CosineApprox>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.impl.h:33,
from /lustre/home/ec136/qizhou/hemelb/Code/SimulationMaster.h:151,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/parallel/ParallelFixtures.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/parallel/GraphCommsTests.cc:9:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/parallel/RBCInserterWithPerturbationParallelTests.cc: In member function ‘void hemelb::tests::{anonymous}::____C_A_T_C_H____T_E_S_T____0::test()’:
/lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/parallel/RBCInserterWithPerturbationParallelTests.cc:42:18: warning: unused variable ‘r’ [-Wunused-variable]
42 | auto volatile r = rand();
| ^
In file included from /lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:11,
from /lustre/home/ec136/qizhou/hemelb/Code/tests/redblood/parallel/MPIParallelIntegrationTests.cc:12:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h: In instantiation of ‘void hemelb::redblood::CellArmy::CellRemoval() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’:
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:49:40: required from ‘void hemelb::redblood::CellController::EndIteration() [with TRAITS = hemelb::Traits<hemelb::lb::lattices::D3Q19, hemelb::lb::GuoForcingLBGK, hemelb::lb::collisions::Normal, hemelb::lb::streamers::SimpleCollideAndStream, hemelb::lb::BFL, hemelb::lb::LADDIOLET, hemelb::lb::NASHZEROTHORDERPRESSUREIOLET, hemelb::lb::LADDIOLETBFL, hemelb::lb::NASHZEROTHORDERPRESSUREBFL, hemelb::redblood::stencil::FourPoint>]’
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellController.h:45:14: required from here
/lustre/home/ec136/qizhou/hemelb/Code/redblood/CellArmy.h:328:22: warning: unused variable ‘numErased’ [-Wunused-variable]
328 | auto const numErased = nodeDistributions.erase((*i_current)->GetTag());
| ^~~~~~~~~
_ZNSt17_Function_handlerIFSt3setIiSt4lessIiESaIiEERKN6hemelb4util8Vector3DIdEEEZNS5_8redblood8parallel7detailsL17AssessMPIFunctionINSC_7stencil10ThreePointEEESt8functionISB_ERKSt3mapINS7_IlEEiNSD_29VectorLexicographicalOrderingISL_EESaISt4pairIKSL_iEEEEUlSA_E_E9_M_invokeERKSt9_Any_dataSA_' referenced in section .rodata.cst8' of CMakeFiles/mpi_redblood_tests.dir/redblood/parallel/GraphCommsTests.cc.o: defined in discarded section .text._ZNSt17_Function_handlerIFSt3setIiSt4lessIiESaIiEERKN6hemelb4util8Vector3DIdEEEZNS5_8redblood8parallel7detailsL17AssessMPIFunctionINSC_7stencil10ThreePointEEESt8functionISB_ERKSt3mapINS7_IlEEiNSD_29VectorLexicographicalOrderingISL_EESaISt4pairIKSL_iEEEEUlSA_E_E9_M_invokeERKSt9_Any_dataSA_[_ZN6hemelb8redblood8CellArmyINS_6TraitsINS_2lb8lattices5D3Q19ENS3_14GuoForcingLBGKENS3_10collisions6NormalENS3_9streamers22SimpleCollideAndStreamENS3_3BFLENS3_9LADDIOLETENS3_28NASHZEROTHORDERPRESSUREIOLETENS3_12LADDIOLETBFLENS3_26NASHZEROTHORDERPRESSUREBFLENS0_7stencil9FourPointEEEEC5ERNS_8geometry11LatticeDataERKSt3setISt10shared_ptrINS0_8CellBaseEENS0_7details18CellUUIDComparisonESaISQ_EESO_ISt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESQ_St4lessIS13_ESaISt4pairIKS13_SQ_EEEERNS_9reporting10TimersBaseINS1C_17HemeLBClockPolicyENS1C_14MPICommsPolicyEEEdRKNS0_14Node2NodeForceES1K_RKNS_3net15MpiCommunicatorE]' of CMakeFiles/mpi_redblood_tests.dir/redblood/parallel/GraphCommsTests.cc.o ZNSt17_Function_handlerIFSt3setIiSt4lessIiESaIiEERKN6hemelb4util8Vector3DIdEEEZNS5_8redblood8parallel7detailsL17AssessMPIFunctionINSC_7stencil10ThreePointEEESt8functionISB_ERKSt3mapINS7_IlEEiNSD_29VectorLexicographicalOrderingISL_EESaISt4pairIKSL_iEEEEUlSA_E_E9_M_invokeERKSt9_Any_dataSA' referenced in section .rodata.cst8' of CMakeFiles/mpi_redblood_tests.dir/redblood/parallel/MPIParallelIntegrationTests.cc.o: defined in discarded section .text.ZNSt17_Function_handlerIFSt3setIiSt4lessIiESaIiEERKN6hemelb4util8Vector3DIdEEEZNS5_8redblood8parallel7detailsL17AssessMPIFunctionINSC_7stencil10ThreePointEEESt8functionISB_ERKSt3mapINS7_IlEEiNSD_29VectorLexicographicalOrderingISL_EESaISt4pairIKSL_iEEEEUlSA_E_E9_M_invokeERKSt9_Any_dataSA[_ZN6hemelb8redblood8CellArmyINS_6TraitsINS_2lb8lattices5D3Q19ENS3_14GuoForcingLBGKENS3_10collisions6NormalENS3_9streamers22SimpleCollideAndStreamENS3_3BFLENS3_9LADDIOLETENS3_28NASHZEROTHORDERPRESSUREIOLETENS3_12LADDIOLETBFLENS3_26NASHZEROTHORDERPRESSUREBFLENS0_7stencil9FourPointEEEEC5ERNS_8geometry11LatticeDataERKSt3setISt10shared_ptrINS0_8CellBaseEENS0_7details18CellUUIDComparisonESaISQ_EESO_ISt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESQ_St4lessIS13_ESaISt4pairIKS13_SQ_EEEERNS_9reporting10TimersBaseINS1C_17HemeLBClockPolicyENS1C_14MPICommsPolicyEEEdRKNS0_14Node2NodeForceES1K_RKNS_3net15MpiCommunicatorE]' of CMakeFiles/mpi_redblood_tests.dir/redblood/parallel/MPIParallelIntegrationTests.cc.o
collect2: error: ld returned 1 exit status
make[2]: *** [tests/CMakeFiles/mpi_redblood_tests.dir/build.make:307: tests/mpi_redblood_tests] Error 1
make[1]: *** [CMakeFiles/Makefile2:1333: tests/CMakeFiles/mpi_redblood_tests.dir/all] Error 2
make: *** [Makefile:164: all] Error 2

I'm also attaching the stdout.txt here:
stdout.txt

[mobernabeu]

Thanks @CharlesQiZhou, I suggest that you try running the non-MPI unit tests (assuming they have compiled?) and then test with the main executable whether you encounter the issue with RBC vertex coordinates in #770.

Moving forward, please parse the compilation output and only add to tickets the actual errors, rather than a long list of warnings.

[CharlesQiZhou]

Thanks @mobernabeu. Unfortunately only two executable hemelb and hemelb-confcheck were compiled, and the unit tests were not. I tested the main executable (hemelb) after the compiling Code already. The simulation did run on Cirrus (single-core job tested), but encountered the RBC vertex issue similarly as on ARCHER2

[rupertnash]

You've got a linking error there. Which symbol is it complaining about? You can demangle the name with c++filt

[CharlesQiZhou]

Well spotted @rupertnash. The demangled functions are:

std::_Function_handler<std::set<int, std::less, std::allocator > (hemelb::util::Vector3D const&), std::function<std::set<int, std::less, std::allocator > (hemelb::util::Vector3D const&)> hemelb::redblood::parallel::details::AssessMPIFunctionhemelb::redblood::stencil::ThreePoint(std::map<hemelb::util::Vector3D, int, hemelb::redblood::parallel::VectorLexicographicalOrdering<hemelb::util::Vector3D >, std::allocator<std::pair<hemelb::util::Vector3D const, int> > > const&)::{lambda(hemelb::util::Vector3D const&)#1}>::_M_invoke(std::_Any_data const&, hemelb::util::Vector3D const&)' referenced in section

[mobernabeu]

Thanks @CharlesQiZhou.

@rupertnash, could it be a lambda function that it's been optimised out by error? Perhaps we could try setting CMAKE_BUILD_TYPE to Debug and start adding optimisations incrementally? I can also see a HEMELB_OPTIMISATION CMake variable that I'm not sure how it combines with the previous. Any thoughts welcome!

[CharlesQiZhou]

Hi @rupertnash, thank for your recent changes before Christmas. I found the superbuild quite handy. However, even with superbuild, the code is still compiled only for serial jobs on Cirrus (with srun --ntasks=1). Parallel jobs with srun would fail with the error below:

/lustre/home/ec136/qizhou/hemelb/Code/net/MpiFile.cc:38: MPI_File_open: Other MPI error, error stack:
MPI_FILE_OPEN(67): Invalid MPI_Info

and mpirun would fail with libstdc++.so.6 error instead:

/lustre/sw/gcc/8.2.0/lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /lustre/home/ec136/qizhou/hemelb/build/hemelb-prefix/src/hemelb-build/hemelb)

The mpirun error is beyond me as gcc10.2.0 rather than 8.2.0 was loaded for both the compilation and the job submission.
Note we used to have the srun issue for parallel jobs for red_blood_cells/merge branch too, but mpirun worked as an alternative at that time. Any thoughts about the current error?

FYI, the modules I loaded for compiling the code were:

module load cmake/3.17.3
module load gcc/10.2.0
module load intel-mpi-19/19.0.0.117
module load libtool/2.4.6

The hemelb main executable did compile, but errors occurred during cmake --build build:

[100%] Built target hemelb-tests
make[4]: Leaving directory '/lustre/home/ec136/qizhou/hemelb/build/hemelb-prefix/src/hemelb-build'
make[3]: *** [Makefile:161: all] Error 2
make[3]: Leaving directory '/lustre/home/ec136/qizhou/hemelb/build/hemelb-prefix/src/hemelb-build'
gmake[2]: *** [CMakeFiles/hemelb.dir/build.make:135: hemelb-prefix/src/hemelb-stamp/hemelb-build] Error 2
gmake[2]: Leaving directory '/lustre/home/ec136/qizhou/hemelb/build'
gmake[1]: *** [CMakeFiles/Makefile2:137: CMakeFiles/hemelb.dir/all] Error 2
gmake[1]: Leaving directory '/lustre/home/ec136/qizhou/hemelb/build'
gmake: *** [Makefile:107: all] Error 2

[CharlesQiZhou]

Maybe it's also useful to post the error from conventional build, i.e. manually compiling the dependencies and then the code itself. Previously in last November both would compile and the main executable was generated. Now make the dependencies gives the error as below:

CMake Error at ThirdParty/loguru/vtkloguru/cmake_install.cmake:41 (file):
file cannot create directory: /usr/local/include/vtk-9.1/vtkloguru. Maybe
need administrative privileges.
Call Stack (most recent call first):
ThirdParty/loguru/cmake_install.cmake:42 (include)
cmake_install.cmake:42 (include)
make[3]: *** [Makefile:106: install] Error 1
make[2]: *** [CMakeFiles/dep_VTK.dir/build.make:94: dep_VTK-prefix/src/dep_VTK-stamp/dep_VTK-install] Error 2
make[1]: *** [CMakeFiles/Makefile2:108: CMakeFiles/dep_VTK.dir/all] Error 2
make: *** [Makefile:104: all] Error 2

[mobernabeu]

Hi Charles, while this gets looked into properly. Can you check if directories /usr/local/include/ and /usr/local/include/vtk-9.1 exist and if so, what are their permissions?