[BUG] Inconsistency in ultimates and full triangle

[jbogaardt]

Describe the bug
This shows up when using IncrementalAdditive with BornhuetterFerguson

To Reproduce
Steps to reproduce the behavior. Code should be self-contained and runnable against publicly available data. For example:

import chainladder as cl
tri = cl.load_sample("ia_sample")
pipe = cl.Pipeline(steps=[
    ('dev', cl.IncrementalAdditive()),
    ('model', cl.BornhuetterFerguson(apriori=.6))]
)
test = pipe.fit(X=tri['loss'], sample_weight=tri['exposure'].latest_diagonal)

assert test.named_steps.model.full_triangle_.iloc[..., -1] == test.named_steps.model.ultimate_

Expected behavior
The assert should not fail

Desktop (please complete the following information):

• Chainladder==0.8.13

[henrydingliu]

To clarify, the assert above isn't supposed to be true since the bf a priori is set to .6. my original example was regarding a couple odd values in the bf.full_triangle.

this issue appears to have been fixed in 0.8.14. Not sure what did it. I don't think it was any of my changes in 7de2cd7.

however, the above assert with apriori = 1 is still failing in 0.8.14

[jbogaardt]

Oh, you're right, it looks fixed. But for me I can get it to work with the latest release. So I don't see any issue here anymore. Screenshot from google colab:

[henrydingliu]

the odd values in bf.full_triangle_ was fixed in #330

here is the testing code that shows equality between the three methods. i was incorrect expecting that the same sample_weight should be used for incr and bf.

tri = cl.load_sample("ia_sample")
incr_est = cl.IncrementalAdditive().fit(tri['loss'], sample_weight=tri['exposure'].latest_diagonal)
incr_trans = incr_est.transform(tri['loss'])
incr_tri = incr_est.incremental_.incr_to_cum()
incr_ult = incr_tri[incr_tri.development == incr_tri.development.max()]
cl_est = cl.Chainladder().fit(incr_trans)
bf_est = cl.BornhuetterFerguson(apriori=1).fit(incr_trans, sample_weight=incr_ult)
print(incr_est.incremental_)
print(cl_est.full_triangle_.cum_to_incr())
print(bf_est.full_triangle_.cum_to_incr())

Even with equality achieved, I think this can create unexpected results when I want to use different denominators between incr pattern estimation and actual IBNR calculation. For example, if I'm doing an incremental ALAE to ultimate loss analysis, and I want to use countrywide incremental % to apply to individual states ultimate loss. In that example, Chainladder would not produce the right results; and BF would be cumbersome to find the right sample_weight. But this is probably more suitable to address in #377