Overhaul TokenStream ICloseable/IDisposable Patterns, #271

[paulirwin]

• You've read the Contributor Guide and Code of Conduct.
• You've included unit or integration tests for your change, where applicable.
• You've included inline docs for your change, where applicable.
• There's an open issue for the PR that you are making. If you'd like to propose a change, please open an issue to discuss the change or find an existing issue.

Overhaul TokenStream ICloseable/IDisposable Patterns

Fixes #271

Description

This pull request focuses on refactoring the codebase by replacing the Dispose method with the new ICloseable interface's Close method in all provided TokenStream implementations to make them reusable. None of the implementations in our codebase need a non-reusable cleanup method, however, in discussion in #271, it was determined that there are possible use cases in which a custom TokenStream implementation might need to have their instance Disposed when the owning Analyzer (with stored values set by a ReuseStrategy) is disposed, so we have retained the IDisposable pattern as well. This PR also ensures that those values are disposed by way of DisposableThreadLocal and TokenStreamComponents disposing of their values.

[paulirwin]

I feel confident enough in this now that it is ready for review.

This PR achieves the original goals of #271, of not only ensuring that the original issue is fixed, but that we have implemented a design that allows for TokenStreams to be "closed" in a reusable way (via this new ICloseable interface to mirror Java) while also still allowing for an extensibility point for them to be "disposed" in a non-reusable way, such as if your TokenStream implementation needs to dispose of native resources or file handles.

Additionally, the code in BaseTokenStreamTestCase has been reviewed several times and is now much closer to the original implementation. I've left the try/finally calls, since they probably should have been in the original Java anyways, and it seems to possibly help cascading failures (although I haven't been able to reproduce those, possibly as a result of other previous fixes).

With all tests passing reliably, this design change implemented, and all concerns addressed in the original issue, I think it's time to wrap this up and call this 5+ year old issue complete.