Preprocessor line reader

[opoudjis]

In https://github.com/metanorma/metanorma-standoc/blob/main/lib/metanorma/standoc/macros.rb , there is code to determine the status of a line during preprocessing of Asciidoctor, pass_status(). That code is used in the LinkProtectPreprocessor, so that lines that are passthrough or comments are not processed by that preprocessor. It needs to be made into a class, and brought into utils, so it can be shared among all line-based preprocessors; that includes metanorma-plugin-preprocessor, per metanorma/metanorma-plugin-glossarist#27

[opoudjis]

The line processing loop in preprocessors is now exemplified in the following:

    class LinkProtectPreprocessor < Asciidoctor::Extensions::Preprocessor
      def process(document, reader)
        p = Metanorma::Utils::LineStatus.new
        lines = reader.lines.map do |t|
          p.process(t)
          !p.pass && t.include?(":") and t = inlinelinkmacro(inlinelink(t))
          t
        end
        ::Asciidoctor::PreprocessorReader.new document, lines
      end

p.pass is whether the current line is to be passed through, as a preformatted line, or not. In most preprocessors, if the current line is a passthrough line, it needs to be ignored; that includes source code, passthrough, literals...

... and I've just realised: it also needs to include block comments.

[opoudjis]

I'm adding this to all Preprocessors under Metanorma. @HassanAkbar That includes metanorma-plugin-glossarist

[opoudjis]

Actually, no: datastruct and lutaml do need to be able to populate sourcecode and even comment text. I'm not going to insert it, at least for now, outside where I already have, the embed and link preprocessors. If we need to change that, we know where the method is.

[opoudjis]

This did debug some of the detecting of passthrough status, so it was worthwhile.