Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve compression of PNG images embedded in generated SVG files #8620

Merged

Conversation

Rob--W
Copy link
Member

@Rob--W Rob--W commented Jul 6, 2017

Fixes issue #8560 for Node.js.

The first four commits are refactorings, the last one contains the actual fix (read its commit message for the motivation for the fix, test steps and suggestions for future improvements).

@Rob--W Rob--W requested a review from yurydelendik July 6, 2017 13:36
@Rob--W Rob--W changed the title Issue 8560 improve png compression Improve compression of PNG images embedded in generated SVG files Jul 6, 2017
@yurydelendik
Copy link
Contributor

/botio test

@pdfjsbot
Copy link

pdfjsbot commented Jul 7, 2017

From: Bot.io (Linux m4)


Received

Command cmd_test from @yurydelendik received. Current queue size: 1

Live output at: http://54.67.70.0:8877/e1723c2f4aabe8d/output.txt

@pdfjsbot
Copy link

pdfjsbot commented Jul 7, 2017

From: Bot.io (Windows)


Received

Command cmd_test from @yurydelendik received. Current queue size: 1

Live output at: http://54.215.176.217:8877/6904598f8d26383/output.txt

Copy link
Contributor

@yurydelendik yurydelendik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the patch.

@pdfjsbot
Copy link

pdfjsbot commented Jul 7, 2017

From: Bot.io (Linux m4)


Success

Full output at http://54.67.70.0:8877/e1723c2f4aabe8d/output.txt

Total script time: 16.64 mins

  • Font tests: Passed
  • Unit tests: Passed
  • Regression tests: Passed

@pdfjsbot
Copy link

pdfjsbot commented Jul 7, 2017

From: Bot.io (Windows)


Failed

Full output at http://54.215.176.217:8877/6904598f8d26383/output.txt

Total script time: 28.91 mins

  • Font tests: Passed
  • Unit tests: Passed
  • Regression tests: FAILED

Image differences available at: http://54.215.176.217:8877/6904598f8d26383/reftest-analyzer.html#web=eq.log

Rob--W added 5 commits July 10, 2017 18:45
Move the DEFLATE logic in convertImgDataToPng to a separate function.
A later commit will introduce a more efficient deflate algorithm,
and fall back to the existing, naive algorithm if needed.
Do not directly export to global. Instead, export all stubs in domstubs.js and
add a method setStubs to assign all exported stubs to a namespace. Then replace
the import domstubs with an explicit call to this setStubs method.  Also added
unsetStubs for undoing the changes. This is done to allow unit testing of the
SVG backend without namespace pollution.
btoa is already defined by src/shared/compatibility.js,
which is unconditionally imported by src/shared/util.js.
Use the environment's zlib implementation if available to get
reasonably-sized SVG files when an XObject image is converted to PNG.
The generated PNG is not optimal because we do not use a PNG predictor.
Futher, when our SVG backend is run in a browser, the generated PNG
images will still be unnecessarily large (though the use of blob:-URLs
when available should reduce the impact on memory usage). If we want to
optimize PNG images in browsers too, we can either try to use a DEFLATE
library such as pako, or re-use our XObject image painting logic in
src/display/canvas.js. This potential improvement is not implemented by
this commit

Tested with:

- Node.js 8.1.3 (uses zlib)
- Node.js 0.11.12 (uses zlib)
- Node.js 0.10.48 (falls back to inferior existing implementation).
- Chrome 59.0.3071.86
- Firefox 54.0

Tests:

Unit test on Node.js:

```
$ gulp lib
$ JASMINE_CONFIG_PATH=test/unit/clitests.json node ./node_modules/.bin/jasmine --filter=SVG
```

Unit test in browser: Run `gulp server` and open
http://localhost:8888/test/unit/unit_test.html?spec=SVGGraphics

To verify that the patch works as desired,

```
$ node examples/node/pdf2svg.js test/pdfs/xobject-image.pdf
$ du -b svgdump/xobject-image-1.svg
 # ^ Calculates the file size. Confirm that the size is small
 #   (784 instead of 80664 bytes).
```
@Rob--W Rob--W force-pushed the issue-8560-improve-png-compression branch from 7d07dfb to 01f03fe Compare July 10, 2017 16:58
@Rob--W
Copy link
Member Author

Rob--W commented Jul 14, 2017

FYI, I rebased the branch to account for the change from #8630.
@yurydelendik You've approved the changes. Is it good to merge?

@yurydelendik
Copy link
Contributor

You've approved the changes. Is it good to merge?

Yes. (it was kind of r+)

@yurydelendik yurydelendik merged commit ca3c08f into mozilla:master Jul 14, 2017
@timvandermeij
Copy link
Contributor

Awesome work!

@Snuffleupagus
Copy link
Collaborator

Snuffleupagus commented Jul 14, 2017

When running the unit-tests locally (in Firefox), either with gulp unittest or directly in the browser with http://localhost:8888/test/unit/unit_test.html, one of the newly added unit-tests isn't working as expected with SPEC HAS NO EXPECTATIONS should produce a svg:image even if zlib is unavailable displayed in the results.

My first guess was that it's simply a problem with the unit-tests being asynchronous, but missing the required done calls to make sure that we wait for each unit-test to complete properly; i.e. like so:

diff --git a/test/unit/display_svg_spec.js b/test/unit/display_svg_spec.js
index 515e59fe..ccd89f79 100644
--- a/test/unit/display_svg_spec.js
+++ b/test/unit/display_svg_spec.js
@@ -101,7 +101,7 @@ describe('SVGGraphics', function () {
       });
     }
 
-    it('should produce a reasonably small svg:image', function() {
+    it('should produce a reasonably small svg:image', function(done) {
       if (!isNodeJS()) {
         pending('zlib.deflateSync is not supported in non-Node environments.');
       }
@@ -117,10 +117,12 @@ describe('SVGGraphics', function () {
         // Without zlib (uncompressed), the size of the data URL was excessive
         // (80247).
         expect(imgUrl.length).toBeLessThan(367);
-      });
+        done();
+      }).catch(done.fail);
     });
 
-    it('should produce a svg:image even if zlib is unavailable', function() {
+    it('should produce a svg:image even if zlib is unavailable',
+        function(done) {
       withZlib(false, getSVGImage).then(function(svgImg) {
         expect(svgImg.nodeName).toBe('svg:image');
         expect(svgImg.getAttribute('width')).toBe('200px');
@@ -129,7 +131,8 @@ describe('SVGGraphics', function () {
         expect(imgUrl).toMatch(/^data:image\/png;base64,/);
         // The size of our naively generated PNG file is excessive :(
         expect(imgUrl.length).toBe(80247);
-      });
+        done();
+      }).catch(done.fail);
     });
   });
 });

Having made the above changes locally, which are necessary to ensure that the tests work correctly, the same test now fail consistently with Failed: Requesting object that isn't resolved yet [object Object].

@Rob--W I don't really have time to delve deeper into this right now, so I'd appreciate if you could take a look!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants