Possible bugs?

[crazylyf]

Hello,
When I running ocropus-ltrain, it will occasionally warning: "FloatingPointError: overflow encountered in exp", and the program seems to restart from the nearest saved state. The problem occurs mainly in the "ffunc" function in lstm.py, which defines the softmax function using: 1.0/(1.0+exp(-x)). Same problem also occurs in the "sigmoid" function. I think this may be caused by large values in x. In the CLSTM source code, the values x is clipped to 20 for positive values, and -20 for negtive values. After clipping like this, the program goes well without warning.

Another problem is that the "backward" method in class "Parallel" returns None. This is correct for 1-layer BLSTM system, but for multiple layers BLSTM configuration which stacking paralleled BLSTM one over another, this will lead to error, as the deltas of subsequent layer is assigned as the current deltas. So, maybe the method should return deltas.

Best,

[tmbdev]

I'll keep the bug open, but for LSTM training, at this point, you're probably better off using the CLSTM implementation. It is almost a drop-in replacement and considerably faster.

If you want to submit patches for a "safe" sigmoid and for the BLSTM, please do.

[zuphilip]

Fixed in #201.

[zuphilip]

Second part may still be open:

Another problem is that the "backward" method in class "Parallel" returns None. This is correct for 1-layer BLSTM system, but for multiple layers BLSTM configuration which stacking paralleled BLSTM one over another, this will lead to error, as the deltas of subsequent layer is assigned as the current deltas. So, maybe the method should return deltas.