-
Notifications
You must be signed in to change notification settings - Fork 179
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
openjpeg: update to 2.3.1 #5079
Conversation
Perhaps we don't need any of them? At least Solaris does not and we may be very close to that: https://github.com/oracle/solaris-userland/tree/master/components/desktop/openjpeg. From other suggestions:
Are you going to rebuild it's dependencies? I found |
Thanks for the comments and review, Michal. It's appreciated. I didn't know whether there were any copyright or licensing issues that would keep me from looking at Oracle's work, so up to this point, I've never looked in solaris-userland. I do sometimes look at what one or more Linux distributions include for patches or configure/packaging options, since their patches and build recipes are generally very liberally licensed. If comparing solaris-userland to hipster is OK to do in all cases, I'll make a point of looking at solaris-userland when I run into more issues like these. Our existing component doesn't run the test suite, but I did try the tests when I build openjpeg without the mct.c/SSE patch. The first 27 tests pass, and then next 1200+ tests fail. I didn't look closely at those tests, but they all seemed to be doing md5sums on resulting files. I expect there are very minor differences in the files between our test results and the expected, but of course they're enough to fail an md5sum, but that's just a guess. I would have to look more closely at the tests. I did notice that SONAME has not changed since 2.1.0, but to be safe I had been planning on doing dependency rebuilds (once I had some direction on what to do about the SSE patch). |
@Mno-hime @timmooney I created the patch partly based on PDFium but their patch was incorrect so I modified it. Later on a similar patch was picked by Fedora but they made a mistake, other distributions like Ubuntu did not have this patch despite bug reports. If the alignment is enforced now outside of this function call then it is not useful anymore but I would suggest to verify that. |
Thanks much for that background information @alarcher , it's very useful to know. Looking at openjpeg issue 625 and openjpeg issue 624 , my interpretation of the comments is that they've addressed this by ensuring the correct alignment -- it's not super clear, though. |
I believe it's OK to look at and take from solaris-userland. Their userland is under the same CDDL license as ours. Just make sure Oracle copyright is brought over to oi-userland, if something is copied. The other good source of patches & knowledge is pkgsrc (https://github.com/NetBSD/pkgsrc/).
It's a good idea to pursue this further. Once you are able to run the test suite via
I believe this is OK, unless the major |
I've made progress with the test suite for With the conformance data in place, when building 32-bit, there are 9 test failures:
However, when building 64-bit there are only 2 test failures:
How would you like to proceed? |
To give us a baseline for comparison, I went back to
So at least for the test suite, our current shipping version would have fewer failures if we didn't apply the SSE patch. The detailed results for each case are:
|
@timmooney This is totally normal that there are more test failures with the patch, it does not mean that the code is broken. Bottom line is: if the memory is guaranteed to be aligned now then simply remove the patch, otherwise not as it will break reading some pictures like before. Thanks for your work :) |
@timmooney Did you decided one way or the other? You may consider asking on some upstream mailing list for guidance. |
Good suggestion @Mno-hime , I've posted to the OpenJPEG Google group asking for clarification. I should know how to proceed once there's been some kind of response there. |
@timmooney We can check on one of the offending PDFs submitted to the ML if the patch is still necessary. If not, just push it as-is. |
Thanks, @alarcher . There's been no response to my question on the openjpeg googlegroup. I don't know where to get the offending PDFs you're talking about. If you can provide a bit more direction I would be willing to do more testing. |
Could you check the links at #1648 ? |
Thanks for the pointer, I found the multi-part 7zip of a PDF that triggered the problem in illumos issue #6463 . I'll piece that together and test with it. |
After testing, I think it's safe to push my changes without the patch to mct.c. Should I move I tested with the DrayTek_UG-Vigor2860_V3.1.pdf from illumos issue 6463. The baseline for my tests:
After that baseline, I installed
I'm not currently planning on triggering rebuilds of any dependencies, since the SONAME didn't change. |
Thanks for the testing, we are good to go then :) the patch will stay in the git history, it should be fine.
https://abi-laboratory.pro/index.php?view=timeline&l=openjpeg seems to agree with you :) |
2eae504
to
507bfef
Compare
My updated changes were auto-merged because of Michal's previous approval, which was a little unnerving. :-) For reference, this fixes all of the OpenJPEG CVEs that mention "before 2.2.0" or "in 2.1.2" in their description. |
@timmooney I am afraid we don't understand what you are saying. What you mean by "those changes being auto-merged"? As far as I know this wasn't merged. I am sure we are still interested in our contribution. |
@Mno-hime I mean that the moment I force-pushed my updated changes, github appears to have both automatically merged the changes and automatically closed this issue. I didn't do anything to close this; it closed automatically as soon as my changes were pushed. Also, the message I see says "Pull request sucessfully merged and closed". This is the first time I've ever seen that happen, and I had assumed it was because you had previously approved an older change-set. It's of course possible that something else is going on and I'm misunderstanding what github has done. |
@timmooney There are no commits on this branch, are you sure you pushed the updated changes to the right branch? |
You're correct, I was not on the right branch. Sorry for the trouble. I have fixed the issue in my local git repo and pushed the changes from the local repo to timmooney:openjpeg-update on github. When I look at my repositories, I can see that the openjpeg-update branch changed just a few minutes ago with the changes I pushed. However, I expected the changes to automatically appear in this pull request, and they are not. Any thoughts on what I should do to get the changes from my branch to appear in this pull? |
@timmooney I see 55b3ada in "Commits". Do you miss any other commits? Whenever I see something wrong with my PR, I rebase against |
@Mno-hime That is the single commit that's needed. It's actually the rebase that gives me the most trouble. There are thousands of articles on git rebase, but there seems to be many different methods or approaches. I have yet to find one that I'm fully comfortable with. Since I only have time to contribute on an infrequent and unpredictable basis, I'm not getting as comfortable with git as quickly as I would if I could contribute every day. When I can create my branch, make my changes, push to github, and then open a PR all before a bunch of stuff changes on master, then I usually don't have any problems. The problems start when a lot of time goes by after creating my branch, so lots of stuff has changed on master that isn't reflected in my branch. Thanks for all the patient help from you and @alarcher . |
This is what I do: |
Anyway, I think we are good here. If you agree @timmooney, remove the "WiP" tag in the PR. |
Thanks Aurélien! Sorry for the extra work you had to do on this one. I'll try avoid causing merge issues like this in the future! |
This broke the ffmpeg build. Just a note to remind us to check it next time. |
This is currently a work in progress -- I need some direction regarding what to do about one of the outdated patches. The latest version includes security fixes, and is needed for other packages I would like to rebuild and update.
The patch
patches/openjpeg-02-mct-sse.patch
does not apply. It looks like the section of code it relates to is still present. However, it looks like the patch we're using may have come from "PDFium", and it appears they later removed that patch: PDFium bugI can't build with the existing patch, so it either needs to be removed or it needs to be updated.