Convenience: method to get current token from iterator

[inexorabletash]

In every example I've written using this (like this one) I end up with this pattern:

const words = [];
const iterator = Intl.Segmenter(locale, {type: 'word'}).segment(text);
let pos = iterator.index();
for (let {index, breakType} of iterator) {
  if (breakType !== 'none')
    words.push(text.slice(pos, index));
  pos = index;
}

i.e. I need to maintain pos tracking the previous index and slice the string myself. How about:

%SegmentIterator%.prototype.segment

Return a substring of the input string from last index to the current index.

...so you can just write:

const words = [];
const iterator = Intl.Segmenter(locale, {type: 'word'}).segment(text);
for (let {index, breakType} of iterator) {
  if (breakType !== 'none')
    words.push(iterator.segment());
}

[littledan]

Maybe the iterator should always be of the string slices, with the understanding that the iterator is an extra convenience thing, on top of the lower-level imperative API based on advance which gives you the index. Thoughts?

[inexorabletash]

The strings aren't useful without the type so you'd need a (substring, type) tuple of some sort.

On Oct 18, 2016, at 1:13 PM, littledan [email protected] wrote:

Maybe the iterator should always be of the string slices, with the understanding that the iterator is an extra convenience thing, on top of the lower-level imperative API based on advance which gives you the index. Thoughts?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[littledan]

The next method does this.