Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance of vocabulary validation #287

Closed
ledsoft opened this issue Aug 15, 2024 · 3 comments · Fixed by #295
Closed

Performance of vocabulary validation #287

ledsoft opened this issue Aug 15, 2024 · 3 comments · Fixed by #295
Assignees
Labels
performance Performance issue

Comments

@ledsoft
Copy link
Contributor

ledsoft commented Aug 15, 2024

When validating a moderately-sized vocabulary (ca 400 terms) with flat structure validation often times out on the proxy after 60 seconds. This is unacceptable.

@ledsoft ledsoft added the performance Performance issue label Aug 15, 2024
@ledsoft
Copy link
Contributor Author

ledsoft commented Aug 15, 2024

Update: the validated vocabulary imports another vocabulary containing ca 660 terms, so the total number of terms validated is around 1000.

@ledsoft
Copy link
Contributor Author

ledsoft commented Aug 15, 2024

TTL export of the relevant vocabularies (which is what the Validator uses - it exports the vocabulary contexts into a byte stream and imports it to a Jena in-memory model) has approximately 1.4MB.

@ledsoft
Copy link
Contributor Author

ledsoft commented Aug 15, 2024

Did a quick comparison with termit-dev, which does not run in Docker and is thus able to utilize all the CPUs on the kbss server. It was able to validate a vocabulary with 1800 terms in 22 seconds (note that GraphDB it uses is on a different server).
Will need to try testing it locally in Docker so that CPU availability can be tuned for the test.

lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 5, 2024
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 5, 2024
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 5, 2024
…tion event, fix event names, move business vocabulary service test
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 5, 2024
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 5, 2024
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 6, 2024
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 9, 2024
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 9, 2024
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 9, 2024
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 9, 2024
lukaskabc added a commit to lukaskabc/termit that referenced this issue Sep 9, 2024
@ledsoft ledsoft linked a pull request Sep 12, 2024 that will close this issue
ledsoft pushed a commit that referenced this issue Sep 13, 2024
…t, fix event names, move business vocabulary service test
ledsoft pushed a commit that referenced this issue Sep 13, 2024
ledsoft pushed a commit that referenced this issue Sep 13, 2024
ledsoft pushed a commit that referenced this issue Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Performance issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants