Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory mapped file support #677

Closed
wants to merge 1 commit into from
Closed

Memory mapped file support #677

wants to merge 1 commit into from

Conversation

boxerab
Copy link
Contributor

@boxerab boxerab commented Dec 30, 2015

I've added support for memory mapped files, windows and unix..
This will be useful for large images that do not fit into memory, but where one wants to decode a tile.
Currently there is an artificial limit on file size in OpenJPEG, this may help to remove this limit.

Also, with these enhancements, there is potential for dramatic memory savings: for decode, instead
of allocating all of the code block buffers in advance, the block can store a virtual memory pointer from the mapping for all code block segments. Only when the code block is about to be decoded do we need to allocate contiguous memory for the block.

Also for decode, there is no need to keep a separate memory buffer in the stream, and copy into this buffer. The OS will handle buffering.

}


#ifdef WIN32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest using _WIN32 (as in the rest of the file).

@boxerab
Copy link
Contributor Author

boxerab commented Dec 30, 2015

Thanks, Stefan. I've tried to address the issues you mentioned.
Do you have time to test this on a unix system?

@malaterre
Copy link
Collaborator

@boxerab travis-ci is setup to test your branch on *nix system

@boxerab
Copy link
Contributor Author

boxerab commented Dec 30, 2015

@malaterre thanks, yes there are errors. should be easy to fix when I have time

@boxerab
Copy link
Contributor Author

boxerab commented Dec 31, 2015

*nix should now compile with latest changes.

@stweil
Copy link
Contributor

stweil commented Dec 31, 2015

@boxerab, yes, compilation on Debian now works (with some warnings which are addressed in pull request #648).

@mayeut
Copy link
Collaborator

mayeut commented Dec 31, 2015

@boxerab,

x64 builds (linux, windows vc14, macos) are failing under ci because of regressions:

NR-DEC-issue254.jp2-65-decode
NR-DEC-issue254.jp2-65-decode-md5

x86 builds segfaults on many tests
windows vc10 does not build: http://my.cdash.org/viewBuildError.php?buildid=883028, related to those warnings: http://my.cdash.org/viewBuildError.php?type=1&buildid=882996

You can find all tests results on CDash in the Experimental section.

@boxerab
Copy link
Contributor Author

boxerab commented Dec 31, 2015

@malaterre how do the cool kids deal with failed tests? How to find the error ?

@malaterre
Copy link
Collaborator

I would suggest having a local VM on your windows host. Eg: http://ruby-journal.com/debug-your-failed-test-in-travis-ci/

@boxerab
Copy link
Contributor Author

boxerab commented Dec 31, 2015

Thanks. Test fixed.

@boxerab
Copy link
Contributor Author

boxerab commented Jan 3, 2016

Closing this pull request, as my Zero Copy PR includes these changes, plus more improvements.

@boxerab boxerab closed this Jan 3, 2016
@boxerab boxerab deleted the memory_mapped branch January 3, 2016 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants