Help reduce memory usage for VASP calculation

[csabilca]

Good day, I trust you are well and I thank you for taking your time reading through my problem.

I am currently trying to calculate a single point energy calculation with the HSE06 functional for a system with 200+ atoms. It contains atom types: H, O, Al, Si, Ti, Cu

Here is my INCAR file:
NSW = 0
ISIF = 2
ISMEAR = 0
SIGMA = 0.05
ISPIN=2
LORBIT=10
IBRION = -1
ISTART = 1
NEDOS = 30001
NELMIN = 1
LREAL=Auto
NCORE = 4
LWAVE = .False.
IDIPOL = 3
LHFCALC = .TRUE.
HFSCREEN = 0.2
AEXX = 0.25
GGA = PE
IVDW = 12

K-points is a single gamma point. At this point I have tried using up to 5 TB of memory, and the job would get killed at the 5th SCF cycle step due to "out of memory" issues. I am now trying to see if 12 TB will help, but that means a very long queue time.

I was wondering if you see a way to reduce the memory usage, without having an effect on the accuracy of the calculation. I know the system size is large and I should expect a lot of resources and time being required, but I am just trying my luck to see if I can't reduce the resource requirements a bit further.

Further information, cell size is 50x50x50 and it can't be reduced further. I am using vasp_gam.

[jonathan_lahnsteiner2]

Dear csabilca,

I would recommend to take a look at the following VASP wiki pages:
Memory requirements
and Not enough memory
and the tag NPAR
I think those should be of help to you. To which values you should set your input tags will strongly depend on the computer architecture on which you are running.

All the best Jonathan

[csabilca]

Dear Jonathan

Thank you for your response. Regarding the "Not enough memory" link, the recommendations are to use:
NCORE >= 4
LREAL = Auto
KPAR = 1 # uses the least memory per core
ISYM = 0
LSCALAPACK = .TRUE.

I am already using NCORE = 4, but I am currently experimenting with larger values, such as 8, 16 and 24.
I am already using LREAL = Auto and KPAR = 1 (default).
By default, if VASP is compiled with scaLAPACK support, LSCALAPACK = .TRUE. I believe it has been compiled, but even if it hasn't, I can't change that on the system.

I am not sure I should use ISYM = 0 though, since it is recommended for IBRION = 0.

I am also explicitly decreasing the NBANDS tag value, but beyond that I guess there isn't any way?

[jonathan_lahnsteiner2]

Dear csabilca,

You have to check the makefile.include to see if VASP was compiled with scalapack. You have to look if -DscaLAPACK was added to the CPP_OPTIONS. The makefile.include can be found in the VASP base folder.
Regarding your second question. You should run molecular dynamics simulations with ISYIM=0 because the atoms will move during the MD simulation which will destroy the symmetry of your structure. Therefore, it would not make any sense to include symmetry.

I hope this is of help.
All the best Jonathan

[csabilca]

Dear Jonathan

I am performing a single point energy calculation, not molecular dynamics, so I don't believe using ISYM=0 in that case is appropriate. Correct me if I am wrong.

And thank you, I just checked and VASP was compiled with scalapack.

[jonathan_lahnsteiner2]

Dear csabilca,

I am sorry for the confusion. When doing molecular dynamics runs it is not recommended to use symmetry during the calculation. But in your case you can definitely switch on symmetry. I would recommend to try ISYM=2 since this will reduce the memory consumption. See also the vasp wiki page about the ISYM tag

All the best Jonathan

[csabilca]

Dear Jonathan

I have LHFCALC=.TRUE. and the default from VASP for this setting is ISYM = 3. Is it a good idea to steer away from this very specific default value? If so, I will test it out.