Questions Regarding Accuracy Evaluation After Fine-Tuning with GGA+U Data

[mugeunson]

Hi there,
I find CHGNet to be incredibly interesting, and I’m enjoying using it as it fits well with my research.

Recently, I encountered some questions while evaluating the accuracy of the CHGNet model.

In my case, I applied the U = 3.9 eV correction only to Mn, consistent with the MaterialsProject2020Compatibility guidelines. To evaluate the completeness of the training, I compared the results using two approaches:

(1) Comparing the fine-tuned CHGNet model (with GGA+U compatibility applied) to the VASP raw energy.
(2) Comparing the fine-tuned CHGNet model (without GGA+U compatibility applied) to the VASP raw energy.

Unexpectedly, the accuracy in case (1) was worse than in case (2).
This led me to wonder whether my comparison method might be incorrect.

As far as I know, the MP dataset applies corrections for compatibility between GGA functionals and other functionals (e.g., SCAN, GGA+U, etc.) by adjusting non-GGA functional results.
Therefore, I assume that the results calculated using CHGNet (e.g., energy) are also corrected energies.
If that is the case, would it be incorrect to directly compare CHGNet results with raw DFT energy from VASP to assess the fine-tuning?
Should I instead compare the VASP data corrected using pymatgen MaterialsProject2020Compatibility with CHGNet’s data?

Alternatively, should GGA+U VASP data be used with the composition model enabled during training?