Handle utf-16-encoded annotations (#463)

[jsvine]

Thanks to @tungph for the fix proposal.

[codecov]

Codecov Report

Merging #519 (c3be777) into develop (4b61c38) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head c3be777 differs from pull request most recent head df98f9c. Consider uploading reports for the commit df98f9c to get more accurate results

@@           Coverage Diff            @@
##           develop     #519   +/-   ##
========================================
  Coverage    98.30%   98.30%           
========================================
  Files           10       10           
  Lines         1236     1239    +3     
========================================
+ Hits          1215     1218    +3     
  Misses          21       21           

Impacted Files	Coverage Δ
pdfplumber/page.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4b61c38...df98f9c. Read the comment docs.

[samkit-jain]

Approving the PR but this feels more like a hack. There could be a possibility in the future that we might have to add more encodings as defined here. A more reliable solution would be a logic that automatically identifies the codec to be used (which I don't think is even possible to do with 100% accuracy). Or perhaps, adding a parameter for the users to pass in the encoding.

[jsvine]

Thanks, and I agree: Not the world's most robust fix, but workable for now. I think the parameterizing is an intriguing idea, though I wonder: Are PDFs guaranteed to maintain encoding consistency throughout their annotations (or other content)? I suppose even if they do not, a user with a particularly tricky PDF could then try whatever combination of encodings suited them.