Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify parse_keyword_apis more #1626

Merged
merged 1 commit into from
Dec 29, 2024
Merged

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Dec 28, 2024

Rationale:

@davisp made some great improvements to performance, but now the API for managing tokens is a bit unwieldy. Before we release a new version I am working to slim down the APIs (ideally so we can migrate more code to avoid cloning)

Inspired by @iffyo's comments on #1618

Not sure I followed the intent of the comment, did it mean potentially replacing this api with something similar called advance or would that have a different behavior?

I spent some time playing around with the API and I found these APIs are not necessary, so I propose removing them to simplify the parser interface

Changes:

  1. Remove parse_keyword_token and parse_keyword_token_ref in favor of get_current_token

/// If the current token is the `expected` keyword, consume it and returns a reference to the next token.
///
#[must_use]
pub fn parse_keyword_token_ref(&mut self, expected: Keyword) -> Option<&TokenWithSpan> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as discussed on #1618, the fact this takes &mut self makes avoiding clones challenging (because as long as &TokenWithSpan) is around, rust treats the parser as borrowed mutably (and you can't call other APIs)

pub fn expect_keyword(&mut self, expected: Keyword) -> Result<TokenWithSpan, ParserError> {
if let Some(token) = self.parse_keyword_token_ref(expected) {
Ok(token.clone())
if self.parse_keyword(expected) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually think using parse_keyword and get_current_token is clearer than the original formulation of paser_keyword_token_ref and checking for Some

Copy link
Contributor

@iffyio iffyio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@iffyio iffyio merged commit fe36020 into apache:main Dec 29, 2024
8 checks passed
@alamb alamb deleted the alamb/faster_parsing_more branch December 29, 2024 14:44
@alamb
Copy link
Contributor Author

alamb commented Dec 29, 2024

Thanks @iffyio

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants