Indentation aware readtables implementation

[kristianmandrup]

I imagine it could be done using an indentation stack. Would require adding a few extra API methods.

readtable.identationStack = [];

if (readtable.newIndent()) {
  // add special chars such as '{|' and '|}' around block, which can be matched by macros 
}
// valid existing indent, just read as usual
if (readtable.indent()) {
 ...
} else {
  // bad indent
}

// at any time we can check
if (!readtable.isValidIndentation()) {
  throw Error("Invalid indentation: line #" + readtable.lineNumber);
}

What do you think?

[vendethiel]

That's probably fit for another repo to propose.

[kristianmandrup]

Could perhaps be done something like this:
Here to treat indentation start/end as { } blocks ;)
Should vary depending on context

// this.identationStack = [];

// onIndent function handles indentation, ie. 
// - checks if indentation lv is valid with respect to indentation stack
// - call `error` function if bad indentation
// - call `begin` if valid new indentation
// - call `end` if valid end of indentation

this.onIndent(reader, {
  // will automatically push to indentation stack
  begin: function(reader, continue) {
    // reads '{' into token stream as if it was in source :)
    this.readPunc('{');     
    continue(); // continue parsing
  },  
  // will automatically pop from indentation stack and continue
  end: function(reader) {
    // reads '}' into token stream as if it was in source :)
    this.readPunc('}');
  },
  // called on any indentation error
  error: function(err, reader) {
    throw Error("Indentation error at:" + reader.index + ' ' + reader.source.slice(20));    
  }
})

[DavidSouther]

Someone needs to keep track of indentation level, and correctly assign multiple dedents for a single new line:

function foo(a)
  return function barCalc(b)
    return a * b

foo(3)(4)

And the generated code needs to know that the first } has two leading whitespaces.

I've played around with the idea at https://github.com/DavidSouther/americano-script, and of course python and Coffee do the same.

[jlongster]

It's tempting to put a lot of stuff into the reader API, but I agree, it should stay a hard-to-use low-level API that other libraries can build on if they want. Readers really shouldn't typically be used, imho. They area hard to work with and easy to introduce bugs. (and don't compose nearly as well as macros)

[DavidSouther]

@jlongster No arguments there. Is there another way to make an indentation aware parser? EG add Python/Coffee-esque significant whitespace to denote blocking without this? On the one hand, I think not; on the other, if it's provided once and well that should allow anyone else to use a hypothetical indent-block step, and then stick to using macros after.

[disnet]

I'm mass closing issues due to the release of 1.0 obviating lots of bugs. If you feel this issue is something we still need to address feel free to reopen.