BTEPFS0

BGB Entropy Post-Filter Stream

Idea:

• Attempt to derive some additional compression from already entropy-coded data.
• Try to be faster than a normal bitwise range coder being used as a post filter.

Premise:

• Use an Order-2 PPM with 4-bit symbols.
  • This will need 4kB for the main model, and possibly ~ 1kB for housekeeping data.
• Use a simplistic adaptive entropy scheme for the backend.
  • Will work similar in concept to AdRice.
  • Limit symbols to 8 bits to allow an 8-bit lookup.

Entropy Tables (k):

• k0: 0=0(k0), 1=10(k0), 2=110(k1), 3=1110(k1), 4..15=1111xxxx (k2)
• k1: 0/1=0x(k0), 2/3=10x(k1), 4/5=110x(k2), 6/7=1110x(k2), 8/9=11110x(k3), 10..15=11111xxx(k3)
• k2: 0..3=0xx(k1), 4..7=10xx(k2), 8..15=11xxx(k3)
• k3: 0..7: 0xxx(k2), 8..15: 1xxx(k3)

Each of the 256 possible contexts will have a k value starting at 3.

Each context will be 16 bytes (8 bytes=values, 8 bytes=index). Update will be done STF style.

• idx=(model_idx[ctx][sym>>1]>>((sym&1)*3))&15;
• swap values and update model index.
• emit bits and update k.