Builder updates (topology)

[mkhorton]

Topology builders should work now, including with Nil's OPs. I need to submit a quick bugfix to pymatgen too, but this won't change the builder.

I did run into a few issues:

• When trying to update_targets with only a few documents, items is padded with None values to meet the chunk size (which is fine), but the Store update method can't handle these None values... can submit a quick fix to maggma for this unless something else is planned?

• For a reason I couldn't figure out, I had to manually .connect() to a Store before the Runner would work. Error message wasn't too helpful here, .query() just returned None, instead of warning the Store wasn't accessible?

• I'm not sure why, but many errors/exceptions seemed to not display to stdout unless I was running my build script interactively... is there a logging setting to change this somewhere? Would help with debugging.

[dwinston]

@mkhorton wrt logging, I added a logging utility to maggma.lava.util before @shyamd nuked the lava module (which is fine, but perhaps we can re-add logging utils somewhere). Usage for me was

import logging
logstreamhandle(runner, level=logging.DEBUG)
runner.run()

Perhaps you can re-add this util somewhere?

[mkhorton]

Ok, thanks.

Regarding the first issue I mentioned I added a quick:

# filter out None values
docs = [doc for doc in docs if doc]

to Store.update in my maggma PR materialsproject/maggma#15 unless there's a better solution to this

[shyamd]

Can you ensure the codacy issues are worked out? Ignore the unused variable class. I turned off that rule just now.

1.) I'll fix the first issue in maggma runner.
2.) Yup, problem with the new maggma update. I'm fixing that right now.
3.) This is done somewhat intentionally, so that the root logger can be set by w/e script is running it. You can also save the Runner as a json and mrun that file. I'm currently thinking of how to make an MPI enabled logger, so which will be added with a basic default logging hook that we can change with mrun.

[shyamd]

Thanks!