VASP 6.2.1 with Intel oneAPI (Base+HPC) Toolkit on Ubuntu 20.04.3 - Tests Failed

[NIDAL@UWM.EDU]

Hi,

I successfully compiled VASP 6.2.1 using the Intel oneAPI (Base+HPC) toolkits on Ubuntu 20.04.3 LTS by modifying the default makefile:

Code: Select all

makefile.include.linux_intel

I am aware of the AVX-512 issue so I modified the

Code: Select all

FFLAGS     = -assume byterecl -w -xHOST

line in the makefile.include to

Code: Select all

FFLAGS     = -assume byterecl -w -march=core-avx2

to restrict to AVX-2 instruction sets.

My CPU is a dual Intel Xeon Silver 4210R. I also "unlimited" my stack size by adding

Code: Select all

ulimit -s unlimited

to my

Code: Select all

.profile

and sourcing it via

Code: Select all

source ~/.profile

I implemented the above following the discussion in this thread: Compiling VASP 6.2.1 with IntelOneAPI 2021.3.0 - Failed tests

However I still have all the following tests failing:

andersen_nve andersen_nve_constrain_fixed andersen_nve_constrain_fixed_RPR andersen_nve_RPR andersen_nvt andersen_nvt_fixed andersen_nvt_RPR bulk_GaAs_ACFDT bulk_GaAs_G0W0_sym bulk_GaAs_G0W0_sym_RPR bulk_GaAs_scGW0_ALGO=D_sym bulk_GaAs_scGW0_ALGO=D_sym_RPR bulk_GaAs_scGW0_sym bulk_GaAs_scGW0_sym_RPR bulk_GaAs_scGW_ALGO=D_sym bulk_GaAs_scGW_ALGO=D_sym_RPR bulk_GaAs_scGW_sym bulk_GaAs_scGW_sym_RPR bulk_InP_SOC_G0W0_sym bulk_InP_SOC_G0W0_sym_RPR bulk_SiO2_elastic_properties_ibrion6 bulk_SiO2_elastic_properties_ibrion6_RPR bulk_SiO2_elastic_properties_ibrion8 bulk_SiO2_elastic_properties_ibrion8_RPR bulk_SiO2_HSE CrS_RPR GaAs_LEPSILON GaAs_LEPSILON_LPEAD GaAs_LEPSILON_LPEAD_RPR GaAs_LEPSILON_RPR HEG_333_LW md_8_Cu_IALGO=38 md_8_Cu_IALGO=38_RPR md_8_Cu_IALGO=48 md_8_Cu_IALGO=48_RPR NiOsLDAU=2_x NiOsLDAU=2_z_RPR SiC8_GW0R SiC_HSE06_ALGO=A SiC_HSE06_ALGO=A_RPR SiC_HSE06_ALGO=D SiC_HSE06_ALGO=D_RPR SiC_phon SiC_phon_RPR Tl_y

The exact same tests were also failing when I used the default makefile:

Code: Select all

makefile.include.linux_intel

without any other modifications.

I'm attaching the

testsuite.log, OUTCAR of anderson_nve, and makefile.include

I would be much obliged if I could get this issue resolved.

Regards,
Nidal

[martin.schlipf]

Something is wrong with your MPI setup. You can see that in the OUTCAR file that you send (it runs on 1 core) and also in the testsuite.log file where you see a lot of output duplicated.
My suspicion is that your mpirun command actually uses openmpi and not the intelmpi that you build your code with. You can check
mpirun --version
and see if that is compatible with your makefile.include. That sometimes happen if you source the Intel path, but forget to redo it in a new shell and have a different MPI available.

[NIDAL@UWM.EDU]

Hi Martin,

Thanks for the answer. You are correct, mpirun command is using openmpi on my computer:

Code: Select all

mpirun --version
mpirun (Open MPI) 3.1.5

This is the output of my $PATH:

Code: Select all

/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin:/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/mpi/bin:/opt/intel/oneapi/vtune/2021.8.0/bin64:/opt/intel/oneapi/vpl/2021.6.0/bin:/opt/intel/oneapi/mpi/2021.4.0//libfabric/bin:/opt/intel/oneapi/mpi/2021.4.0//bin:/opt/intel/oneapi/mkl/2021.4.0/bin/intel64:/opt/intel/oneapi/itac/2021.4.0/bin:/opt/intel/oneapi/intelpython/latest/bin:/opt/intel/oneapi/intelpython/latest/condabin:/opt/intel/oneapi/inspector/2021.4.0/bin64:/opt/intel/oneapi/dpcpp-ct/2021.4.0/bin:/opt/intel/oneapi/dev-utilities/2021.4.0/bin:/opt/intel/oneapi/debugger/10.2.4/gdb/intel64/bin:/opt/intel/oneapi/compiler/2021.4.0/linux/lib/oclfpga/llvm/aocl-bin:/opt/intel/oneapi/compiler/2021.4.0/linux/lib/oclfpga/bin:/opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64:/opt/intel/oneapi/compiler/2021.4.0/linux/bin:/opt/intel/oneapi/clck/2021.4.0/bin/intel64:/opt/intel/oneapi/advisor/2021.4.0/bin64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

It does have the Intel MPI in it. So, how do I make the mpirun use the Intel MPI instead?

Also how do I configure MPI correctly? Do I need to do any further modification to the makefile.include so that

Code: Select all

make test

use multiple MPI cores?

Regards,
Nidal

[martin.schlipf]

How did you setup the path? The recommended way is to source the script provided by Intel
https://www.intel.com/content/www/us/en ... -you-begin
See the section "Set Environment Variables for CLI Development"
If you already did that you may need to check where the other mpirun is installed and manipulate the PATH such that the intelmpi one is found first.

If that doesn't work you can use an explicit path to the mpirun. You would have to change which command the testsuite uses to run the code. Please check the Readme of the testsuite how to achieve that if it is necessary.

Make test will use multiple cores, in fact it already ran on multiple cores. It was just that you used an OpenMpi version while the Mpi instructions where linked to the Intelmpi so that the cores didn't share the work and were all doing the same.