Long time - 1,000 MC cycles GOMC vs Towhee - binary mixture

[mleao882]

I am working with binary and ternary mixtures of the system Nitric Acid - Sulfuric Acid - Water for comparison of the VLE with experimental data. When attempting the pure molecules' simultations in GOMC using the GEMC_NVT ensemble (with only the CPU), I get great results when comparing to Towhee's simulation time (19 min - GOMC; 1h21min - Towhee) for a simulation of 100 molecules of sulfuric acid and 1,000 MC cycles (100,000 MC steps for GOMC). However, when I attempt the binary mixture HNO3 (nitric acid) and H2O (water) in the GEMC_NpT ensemble, 1,000 MC cycles for 200 molecules (100 HNO3/100 H2O), I get better results with Towhee (1h3min - Towhee; 8h00 - GOMC).

Input files
The input files used for the binary simulation in GOMC are attached in the compact folder HNO3_H2O.
HNO3_H2O.zip

Please complete the following information:

• OS: Cray Linux Environment
• Ensemble: GEMC
• Code version: 2.70

[mleao882]

The commands for submitting the Job on Cray are in the file AH2O.pbs (PBS script file) as follows:
#!/bin/bash
#PBS -j oe
#PBS -l select=1:ncpus=16:mpiprocs=16:ompthreads=4
cd $PBS_O_WORKDIR
aprun -n4 -N4 -d4 ./GOMC_CPU_GEMC in3.conf >ANHOH_output

[msoroush]

@mleao882 would you please attach the simulation files for your pure molecule system as well?

[mleao882]

@msoroush I attached bellow the compacted files for the simulations of pure nitric acid (HNO3) and pure sulfuric acid (H2SO4).
H2SO4.zip
HNO3.zip

[msoroush]

@mleao882 My immediate suggestion, based on your binary mixture, specially the gas phase (800 atoms in a cubic box with length of 150 A) is to increase the RcutCoulomb for box 1 (gas phase). If you dont specify the the rcut value for short range coulomb potential in Ewald Summation method, the default value would the the value of Rcut=10 which would result in huge number of reciprocal vectors:

Box: 0, RecipVectors:   1595, kmax: 10
Box: 1, RecipVectors: 347794, kmax: 55

and your simulation would take almost 3.5 hrs (16 step/sec) to finish.

But if you increase the RcutCoulomb for box 1 (gas phase) to something like 60 RcutCoulomb 1 60, you will end up with much smaller reciprocal vectors:

Box: 0, RecipVectors:   1595, kmax: 10
Box: 1, RecipVectors:   1595, kmax: 10

and your simulation would take almost 7 min (700 step/sec) to finish.

You can try to run quick simulation to optimize the value of RcutCoulomb. Increasing RcutCoulomb would shift the Ewald computation work from long-range to short-range coulomb calculation. In this case, since twe are dealing with a gas phase, this shift in work would actually improves your performance.

[msoroush]

@msoroush I attached bellow the compacted files for the simulations of pure nitric acid (HNO3) and pure sulfuric acid (H2SO4).
H2SO4.zip
HNO3.zip

@mleao882 The difference between your pure system and binary mixture is increase in gas phase box dimension, which increased from 50 A to 150 A. If you increase the RcutCoulomb for gas phase, GOMC performance would increase tremendously.

[jpotoff]

@msoroush beat me to the punchline, but I also, want to mention that when comparing to Towhee, it's important to make sure you're comparing two equivalent electrostatic calculations. In GOMC, we set the tolerance of the Ewald sum, so the code automatically adjusts the number of k-vectors needed based on system size. In Towhee, it looks like that is set with the ewald_fixed_cutoff flag, but I know a lot of older calculations were run with ewald_fixed_kmax = 5 or 6, which would give a much lower precision than the default GOMC settings (or Towhee's with 'ewald_fixed_cutoff').

One other comment, your system probably isn't big enough to fully utilize 16 CPU cores, and you may actually get better performance by using fewer CPUs.

[mleao882]

@msoroush I have just performed a new simulation with the RcutCoulomb for the gas phase you suggested and the total simulation time was 9 min. Thanks very much for the help !

[LSchwiebert]

Hope you see this even though the issue is closed. To use multiple cores with OpenMP with GOMC, you need to include +p# in your command line before specifying the input file, where # is the number of cores. I don't think GOMC will recognize the cores you make available with your PBS script. You can verify this by looking at the top of your output file. So, for instance, if you have 4 cores, you will want to use the following instead:

aprun -n4 -N4 -d4 ./GOMC_CPU_GEMC +p4 in3.conf >ANHOH_output

[LSchwiebert]

As the issue was already closed, I'm not sure if you saw my comment, but I included some information on how to make sure that GOMC is using multiple cores. I don't think it will use more than one core with your current setup, because you need the parameter +p# as a command-line argument before the config filename, where # is the number of cores. Best, Loren

…

-- Loren Schwiebert, Ph.D. Chair, Dept. of Computer Science [email protected] Wayne State University (313) 577-5474 I am working remotely. Messages to my phone are checked only occasionally. Disclaimer: Opinions expressed are mine and not necessarily shared by WSU.

________________________________ From: mleao882 <[email protected]> Sent: Wednesday, February 10, 2021 2:20 PM To: GOMC-WSU/GOMC <[email protected]> Cc: Subscribed <[email protected]> Subject: [GOMC-WSU/GOMC] Long time - 1,000 MC cycles GOMC vs Towhee - binary mixture (#323) [EXTERNAL] I am working with binary and ternary mixtures of the system Nitric Acid - Sulfuric Acid - Water for comparison of the VLE with experimental data. When attempting the pure molecules' simultations in GOMC using the GEMC_NVT ensemble (with only the CPU), I get great results when comparing to Towhee's simulation time (19 min - GOMC; 1h21min - Towhee) for a simulation of 100 molecules of sulfuric acid and 1,000 MC cycles (100,000 MC steps for GOMC). However, when I attempt the binary mixture HNO3 (nitric acid) and H2O (water) in the GEMC_NpT ensemble, 1,000 MC cycles for 200 molecules (100 HNO3/100 H2O), I get better results with Towhee (1h3min - Towhee; 8h00 - GOMC). Input files The input files used for the binary simulation in GOMC are attached in the compact folder HNO3_H2O. HNO3_H2O.zip<https://github.com/GOMC-WSU/GOMC/files/5960915/HNO3_H2O.zip> Please complete the following information: * OS: Cray Linux Environment * Ensemble: GEMC * Code version: 2.70 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#323>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC7C7WKILBETA26BFGS5DLTS6LL6BANCNFSM4XNQXLHQ>.