-
Notifications
You must be signed in to change notification settings - Fork 184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial MessageFormat 2 #2272
base: main
Are you sure you want to change the base?
Initial MessageFormat 2 #2272
Conversation
Notice: the branch changed across the force-push!
~ Your Friendly Jira-GitHub PR Checker Bot |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't try to run the code, but noticed a few mismatches with the spec. If intending to fix them later, might be good to add comments about that to the code?
|
||
#[inline] | ||
fn skip_ws(&mut self) { | ||
while get_current_byte!(self) == Some(&b' ') { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should also account for the other valid whitespace chars:
WhiteSpace ::= #x9 | #xD | #xA | #x20
pub fn parse(mut self) -> ParserResult<ast::Message<S>> { | ||
let mut declarations = SmallVec::new(); | ||
|
||
loop { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would probably be easier to skip whitespace before rather than after. Technically, I'm pretty sure that leading whitespace even at the very start is valid.
} | ||
} | ||
|
||
fn parse_declaration(&mut self) -> ParserResult<ast::Declaration<S>> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should throw if a pattern or a select has already been parsed.
fn parse_declaration(&mut self) -> ParserResult<ast::Declaration<S>> { | ||
assert_eq!(self.next(), Some(&b'e')); | ||
assert_eq!(self.next(), Some(&b't')); | ||
self.skip_ws(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whitespace should be mandatory here, so that let$foo
is not allowed.
assert_eq!(self.next(), Some(&b'n')); | ||
let mut key = SmallVec::new(); | ||
|
||
self.skip_ws(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whitespace should be mandatory here, so that whenfoo
is not allowed.
self.skip_ws(); | ||
|
||
if self.next_if(b'*') { | ||
key.push(ast::VariantKey::Asterisk); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whitespace should be mandatory here, so that when *foo
is not allowed.
Also, it looks like parsing for when foo
and when (foo)
is missing?
let options = SmallVec::new(); | ||
ast::Placeholder::Markup { name, options } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I've understood it, according to our blackboxing plan a MarkupStart with options should not throw?
fn parse_name(&mut self) -> ParserResult<S> { | ||
let start = self.ptr; | ||
while let Some(b) = get_current_byte!(self) { | ||
if b.is_ascii_alphabetic() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The actual Name
spec is a bit different: https://github.com/unicode-org/message-format-wg/blob/develop/spec/syntax.md#names
} | ||
} | ||
|
||
fn parse_literal(&mut self) -> ParserResult<ast::Literal<S>> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't handle escapes for literals.
let mut start = self.ptr; | ||
let mut body = SmallVec::new(); | ||
while let Some(b) = self.next() { | ||
match b { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't handle escapes in Text
correctly.
Is this something that someone else could (or should) take on and move forward? |
This is an initial draft branch for MessageFormat component based on MF2.0 spec.