Why does interactive LAMMPS become slower?

[samwaseda]

I just realised that interactive LAMMPS gets slower and slower. For this, I created the following short example:

pr = Project('RUNTIME')
import time

structure = pr.create.structure.bulk('Ni').repeat(4)
lmp = pr.create.job.Lammps('time')
lmp.potential = '1995--Angelo-J-E--Ni-Al-H--LAMMPS--ipr1'
lmp.interactive_open()

n_iter = 100
t_lst = np.zeros(n_iter)
t = time.process_time()
for i in range(n_iter):
    for j in range(10):
        lmp.structure = structure.copy()
        lmp.structure.positions += np.random.randn(len(lmp.structure), 3)/1000
        lmp.run()
    t_lst[i] = time.process_time() - t

And I observed the following evolution of time (of course I plotted np.diff(t_lst) and not t_lst). Why does interactive LAMMPS get slower and slower?

[pmrv]

Probably the best way to find out is to run the for loop under cProfile.

[samwaseda]

I just made sure that it doesn't happen with AtomisticExampleJob, so it must be either LAMMPS or the pyiron-implementation of LAMMPS. I'll try to see how cProfile works but I'm a total loss in this kind of stuff XD

[pmrv]

I did a quick profile and roughly ~45% of the time is spent in GenericInteractiveOutput._lst_from_cache, specifically this list comprehension

pyiron_atomistics/pyiron_atomistics/atomistics/job/interactive.py

Line 407 in 3035031

return [np.array(out) for out in lst]

I suppose it makes sense, as the cache gets longer during the calculation, more list -> array conversions are done.

Another ~15% are talking to the database, the rest is writing/reading HDF files.

[samwaseda]

AAAAAAHA! So it means there could be a huge gain by making it more efficient.