"initialization_openacc" error

[LSX2491995]

I was trying to compile the code on a HPC (to run simulations using gpu) and I kept encountering the same error message:
"src/Smilei.cpp", line 88: error: identifier "initialization_openacc" is undefined
initialization_openacc();
^

1 error detected in the compilation of "src/Smilei.cpp".

Below are the command lines I used for compilation:
conda activate smilei_env ##environment for python 3.9 and other python modules
module load compiler/nvhpc/24.5
module load openmpi
module load cuda

export SMILEICXX=mpicxx
export HDF5_ROOT_DIR=$CONDA_PREFIX
export PYTHONEXE=$(which python)
export CXXFLAGS="-O3 -march=native -I$CONDA_PREFIX/include/python3.9"
export GPU_COMPILER=nvc++
export GPU_COMPILER_FLAGS="-acc -gpu=cc70"

make config=gpu_nvidia -j4

which mpicxx gives:
/util/opt/nvidia-hpc-sdk/2024.245/Linux_x86_64/24.5/comm_libs/mpi/bin/mpicxx

and mpicxx --version gives:
nvc++ 24.5-1 64-bit target on x86-64 Linux -tp icelake-server
NVIDIA Compilers and Tools
Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

Any help is appreciated.

[charlesprouveur]

Hi,
There are a couple issues with your compilation process.

1. You are missng an hdf5 install: here is an example of an hdf5 installation process using nvhpc 23.11 https://smileipic.github.io/Smilei/Use/install_linux_GPU.html

Note that nvhpc 24.5 works just fine, this is the version I am using currently.

2. Your environment / machine file is incorrect: see /scripts/compile_tools/machine/jean_zay_gpu_V100 as an example
   The biggest issue is "export GPU_COMPILER=nvc++" it should be "nvcc" here, not "nvc++"

Hope this helps.

[LSX2491995]

Hello,

Thanks for the reply! I forgot to mention that I have hdf5 (version: 1.14.3) installed in the environment.

I have changed "nvc++" to "nvcc", modified the flag based on the example you provided, and now see the message "compilation completed with warnings". However, I cannot find the executable in the main Smilei directory or any subdirectories, including build/src. Does the compilation need to finish without any warnings to produce the executable?

Below are the latest command lines I used for compilation:
conda activate smilei_env
module load compiler/nvhpc/24.5
module load openmpi ##version 4.1
module load cuda ##version: 12.4

export SMILEICXX=mpicxx
export HDF5_ROOT_DIR=$CONDA_PREFIX
export PYTHONEXE=$(which python)
export CXXFLAGS="-O3 -std=c++14 -acc=gpu -gpu=cc70 -I$CONDA_PREFIX/include/python3.9"
export GPU_COMPILER=nvcc
export GPU_COMPILER_FLAGS="-O3 --std=c++14 -gpu=cc70 --expt-relaxed-constexpr"
make config=gpu_nvidia -j4

[charlesprouveur]

Hi,

The compilation can be finished with warnings, the fact you do not have an executable means there is an error somewhere.

I would do a "make clean" then try to compile again with the following changes:

For GPU_COMPILER_FLAGS you should change "-O3 --std=c++14 -gpu=cc70 --expt-relaxed-constexpr" to
" -O3 --std c++14 -arch=sm_70 --expt-relaxed-constexpr"

You are also probably missing the
LDFLAGS += -acc=gpu -gpu=cc70 -std=c++14 -cudalib=curand -lcudart -lcurand -lacccuda

[LSX2491995]

Hello,

I have made the modification according to your suggestion. However, I am still unable to find the executable after compilation.
I noticed errors like the ones below appearing during the early stages of compilation. Could these errors be preventing the code from compiling correctly?
Checking dependencies for src/Diagnostic/DiagnosticPerformances.cpp
nvc++-Error-Unknown switch: -MFbuild/src/Diagnostic/DiagnosticPerformances.d
nvc++-Error-Unknown switch: -MTbuild/src/Diagnostic/DiagnosticPerformances.d build/src/Diagnostic/DiagnosticPerformances.o

Moreover, I also saw the warning lines like this:
"src/Diagnostic/DiagnosticParticleList.h", line 12: warning: overloaded virtual function "Diagnostic::run" is only partially overridden in class "DiagnosticParticleList" [partial_override]
class DiagnosticParticleList : public Diagnostic

Finally, the compilation completed with the following output:
NVC++-W-1057-Static variables are not supported in acc routine - best_prime (src/Collisions/BinaryProcesses.cpp: 54)
NVC++/x86-64 Linux 24.5-1: compilation completed with warnings

Below are the latest command lines I used for compilation:
conda activate smilei_env
module load compiler/nvhpc/24.5
module load openmpi ##version 4.1
module load cuda ##version: 12.4

export SMILEICXX=mpicxx
export HDF5_ROOT_DIR=$CONDA_PREFIX
export PYTHONEXE=$(which python)
export CXXFLAGS="-O3 -std=c++14 -acc=gpu -gpu=cc70 -I$CONDA_PREFIX/include/python3.9"
export GPU_COMPILER=nvcc
export GPU_COMPILER_FLAGS="-O3 --std c++14 -arch=sm_70 --expt-relaxed-constexpr"
export LDFLAGS="$LDFLAGS -acc=gpu -gpu=cc70 -std=c++14 -cudalib=curand -lcudart -lcurand -lacccuda"
make config=gpu_nvidia -j4

[charlesprouveur]

the "nvc++-Error-Unknown switch" are solved by specifying
SMILEICXX_DEPS = g++
nvc++ does not support the -M option
Partial override warnings are not a problem
This output though is not something i have seen before: "NVC++-W-1057-Static variables are not supported in acc routine - best_prime (src/Collisions/BinaryProcesses.cpp: 54)"
Are you using the latest SMILEI version?

Can you add verbose to your make command and export the output in a file, ie:
make clean
make -j 12 config="gpu_nvidia verbose" > output
And then post the output file here?
By curiosity i compiled for V100 with nvhpc 24.5 and i don't see any issues

[LSX2491995]

My Smilei was installed using the git clone command:
git clone https://github.com/SmileiPIC/Smilei.git

Here is the output file with the same command lines I used before:
Smilei_output.txt

[charlesprouveur]

My guess is your openmpi module is not compiled with nvc++ and / or your gcc is too old.
can you show the output of

mpicxx -v

and

g++ -v

furthemore: to remove the nvc++-Error-Unknown switch messages, use:

export SMILEICXX_DEPS = g++ -I$(HDF5_INCLUDE)

Finally can you show the output of the compilation with this in sequential once you made sure the openmpi and hdf5 modules are compiled with nvc++, and that g++ is not something like 4.8.5