Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strongly-typed parsed data structures, no longer stringly-typed #26

Draft
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

jdtatz
Copy link

@jdtatz jdtatz commented Sep 20, 2022

Major work-in-progress, but the goal is to parse everything (attributes and text) into strongly-typed data structures where the only strings left are names & comments. A goal is to make it so no downstream crate will need to parse any C expressions/types/defines. Also I wanted to ensure no information was lost.

  • Bike-shedding of all of the new types
  • Actual error-handling and not just panicking with my unhelpful messages
  • Possibly remove backward-compat workarounds
  • Possibly handle parsing the platform-specific typedef edge-case code that uses Objective-C
  • Need to rebase/merge
  • Remove extraneous clippy fixes
  • Move to using nom for the string parsing if it's ok to add as a dependency

I started working on this as I needed more information for extending the ash code-gen for formats, and neither vk-parse or vkxml include what I needed. I'm creating this early pull request to get some feedback on naming, backwards compatibility policy, and the usage of nom.

jdtatz added 12 commits August 20, 2022 01:10
parse_name_with_type still has some leftovers from early iterations
only preprocessor commands are un-parsed
If possible using nom would make this much better
TODO convert stringly-typed attributes
FIXME still not a bijective parsing
removed `TypeCode`, but `TypeCodeMarkup` still needs to go
FIXME finish error messages/handling
TODO strongly type more stringly typed attributes
TODO Add full c-expr/c-type/c-preproc parsing with nom?
Remove parse_name_with_type's  handle_extra param
Which was a leftover of my old method, but unused now
I only added paste b/c only after copying & updating the patch version,
did I realize I forgot the test name.
But at least it makes maintenance easier
Preparing to break it into a smaller enum without any unused fields
Also stop blindly generating the code string with invalid C,
it didn't handle the const ptr switch that offical python generator does
The `queues` attribute should always be used instead per offical specs
See the vulkan 1.2.180  changelog
New 'stride' attribute for <param>
remove unused info in TypeBitmask::Definition
@MarijnS95
Copy link
Contributor

@jdtatz That's great to see, as this (parsing types) is one of the major blockers to get ash off of vkxml entirely and purely use vk-parse, as the former isn't updated to add new fields which are needed in ash. I have various branches that migrate parts of ash's generator to vk_parse, but I believe you have such a thing to prototype this change as well?

For the rest, let's start with:

  1. Yes please, move as many clippy fixes and strictly code cleanup to a separate PR - those can go in relatively quickly (will automagically disappear from this PR after a rebase) and won't clobber this PR;
  2. I'd be fine using nom which seems quite ergonomic to use in at least ash.

Then, as more of a loose question: is anything holding us back from finally removing vkxml support from vk-parse, when ash ceases to use it?

Used in the altlen attribute parsing & the #define type tag, mostly
The prototype parses is complete enough that it can parse the
C subset found in in the spec
can't share rules across peg grammars, so all in one
Fixed Expression::Comma
No Vec in Expression allowed
Add `opticalflow` support for CommandQueue
NameWithType types can be const (arrays) & not ptrs, so
seperate constness out from PointerKind to its own field
Fix some minor type errors w/ NonZero & Vec
@krolli
Copy link
Owner

krolli commented Oct 6, 2022

Ooof, to be honest, I saw your fork a few weeks ago (probably just before you opened up the PR) and I didn't really like the direction it was taking. I didn't comment on it because, well, there was no PR or ticket related to it. I'm also not particularly fond of a PR that basically decided to refactor everything without asking before doing so. Even if those changes are good, I am looking at a few thousands lines of diff and I don't have time to review it to see if I agree with this direction or not. And if I don't like these changes and refuse to accept them, then all that work has been wasted.

Things I didn't like when I skimmed through it:

  • Use of nom or any addition of other crates without consulting first. I am trying to keep the dependency tree as small and stable as possible, so unless it is something that dictates a lot of the implementation (eg. XML parsing), I would rather not have it there. So far, I'm not convinced that we need a full-blown C parser, but if you can make a strong case for it, I will consider it.
  • Introduction of strongly typed enums for just about everything instead of simple strings. vk.xml changes a lot, with related RNC being very loose, and I don't want to have to release new version of vk-parse more than necessary. It just adds unnecessary maintenance for IMO no clear benefit (users can just match strings directly).
  • Clippy fixes. Really want to see that as separate PR so it doesn't complicate this one.

I did like the use of paste to get rid of repetition in CI (though I wish this was simple enough that another crate was not needed) and that would be another change I'm open to getting in as a PR of its own.

One more thing might be formatting. I thought I saw some weird formatting in some commits, but I wasn't sure if it was because of some differences in rust-fmt settings (I'm trying to stick to defaults) or whether you didn't format the code before commit. Would be good to get that lined up so it doesn't cause unnecessary noise in diffs.

Ultimately, I think there are two options here:

  • fork and take over the crate, going in whatever direction you think would be better, at which point, you won't need to deal with grumpy-old-me doing the review
  • deal with grumpy-old-me and possibly make a lot of changes before I accept the PR (in parts)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants