Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cache parsed results across multiple runs #667

Merged
merged 2 commits into from
Aug 13, 2020
Merged

cache parsed results across multiple runs #667

merged 2 commits into from
Aug 13, 2020

Conversation

rzpt
Copy link
Contributor

@rzpt rzpt commented Aug 11, 2020

Before this PR

Parsed results were not cached between runs, so certain common files could be parsed many times in a single run.

After this PR

==COMMIT_MSG==
Each conjure file will only be parsed once
==COMMIT_MSG==

Possible downsides?

The code is no longer thread safe due to sharing of a static HashMap across threads, but the cli is single-threaded.

@changelog-app
Copy link

changelog-app bot commented Aug 11, 2020

Generate changelog in changelog/@unreleased

Type

  • Feature
  • Improvement
  • Fix
  • Break
  • Deprecation
  • Manual task
  • Migration

Description

Each conjure file will only be parsed once

Check the box to generate changelog(s)

  • Generate changelog entry

@rzpt rzpt requested a review from iamdanfox August 11, 2020 00:13
@ferozco
Copy link
Contributor

ferozco commented Aug 11, 2020

@rzpt mind giving a bit more details about how you're using Conjure? Given that it is typically used an executable, I don't believe the change you have made will have any impact across "run"s

@rzpt
Copy link
Contributor Author

rzpt commented Aug 11, 2020

Sorry, multiple "runs" is not the right word. I meant the loop here: https://github.com/palantir/conjure/blob/master/conjure-core/src/main/java/com/palantir/conjure/defs/Conjure.java#L38

In a certain internal repo, we have many conjure yml files in a single project (117), and certain files are being parsed many times while invoking the gradle conjure-ir task. Many of these files were being parsed several times, with one of them being parsed 40+ times.

@rzpt
Copy link
Contributor Author

rzpt commented Aug 11, 2020

It seems like I can't write the changelog by using changelog-app? Can you do that for me?

@ferozco
Copy link
Contributor

ferozco commented Aug 11, 2020

Ok, rather than introducing static state could we refactor the code to allow us to re-use the Recursive parser?

@rzpt
Copy link
Contributor Author

rzpt commented Aug 11, 2020

Done.

@rzpt
Copy link
Contributor Author

rzpt commented Aug 13, 2020

@ferozco Can this be merged? I can't write a changelog with the changelog-app. Could you try that?

@ferozco
Copy link
Contributor

ferozco commented Aug 13, 2020

👍

@svc-autorelease
Copy link
Collaborator

Failed to load project - please reach out to #dev-foundry-infra or check Aries to debug

@ferozco ferozco deleted the cache-results branch August 13, 2020 19:15
@carterkozak
Copy link
Contributor

I've fixed up the changelog so we can cut a release with this fix. #669

bulldozer-bot bot pushed a commit that referenced this pull request Aug 14, 2020
Fix changelog from #667
CRogers added a commit that referenced this pull request Aug 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants