JpegImagePlugin - while save file pointer write method is not called

[darkowic]

I found strange bug with saving jpeg files. PngImagePlugin in _save method calls file pointer (fp) write method whereas in JpegImagePlugin's _save method it is not used anywhere.

What did you do?

I use django-storages to save files on s3 storage. Here is how I generate thumbnails (this is old code)

img = Image.open(default_storage.open(instance.uploaded.name, 'r'))
img.thumbnail(meta_data['size'], Image.ANTIALIAS)
f = default_storage.open(str('%s' + meta_data['prefix'] + '%s') % (file_path, filename,), 'wb')
img.save(f, ext, quality=settings.IMAGE_QUALITY)
f.close()

What did you expect to happen?

Files will be saved on storage.

What actually happened?

With png works ok.
Jpg thumbnailes are not saved. write method wasn't call.

django-storages returns in _get_file method SpooledTemporaryFile. With older django-storages version (1.1.8) where instead of SpooledTemporaryFile was StringIO it worked ok.

What versions of Pillow and Python are you using?

2.9.0 and 3.2.0

Please confirm that is Pillow bug.

[wiredfool]

Encoders are complicated, and most of the writes happen in ImageFile._save after taking a trip through the C encoder layer. Pngs are a special case where the metadata is written in python, and the image chunks in the C layer.

There's not enough info in the test case for me to actually run it. And if it's a Pillow bug, it should be possible to produce a test case that doesn't involve the django stack.

For example, this test case works:

from PIL import Image
im = Image.open('Tests/images/hopper.jpg')
im.thumbnail((16,16), Image.ANTIALIAS)
im.save('thumbnail.jpg')

Questions I have are:

1. Are you getting an exception? That might tell you something.
2. Are you sure that you're getting a jpeg encoder when going through the default_storage and save?
3. What is the settings.IMAGE_QUALITY value? Is it necessary?

[darkowic]

1. I'm not getting any errors. Simply file is not saved
2. Im sure that the JpegImagePlugin's _save is called (I tested it). File has jpg extension.
3. IMAGE_QUALITY is number 90

The write method from django-storages has logic uploading file to s3 (I don't know exactly how).

Here I found the difference where write is called

    try:
        fh = fp.fileno()
        fp.flush()
    except (AttributeError, io.UnsupportedOperation):
        # compress to Python file-compatible object
        for e, b, o, a in tile:
            e = Image._getencoder(im.mode, e, a, im.encoderconfig)
            if o > 0:
                fp.seek(o, 0)
            e.setimage(im.im, b)
            while True:
                l, s, d = e.encode(bufsize)
                fp.write(d)
                if s:
                    break
            if s < 0:
                raise IOError("encoder error %d when writing image file" % s)
            e.cleanup()
    else:
        # slight speedup: compress to real file object
        for e, b, o, a in tile:
            e = Image._getencoder(im.mode, e, a, im.encoderconfig)
            if o > 0:
                fp.seek(o, 0)
            e.setimage(im.im, b)
            s = e.encode_to_file(fh, bufsize)
            if s < 0:
                raise IOError("encoder error %d when writing image file" % s)
            e.cleanup()

[darkowic]

Exactly^

In png case there is an except called

(Pdb) fp.fileno()
*** AttributeError: '_idat' object has no attribute 'fileno'

[wiredfool]

If you run something like:

from PIL import Image
thumbnail_size=(16,16)  # or whatever you're actually using
im = Image.open('[failing image here]')
im.thumbnail(thumbnail_size, Image.ANTIALIAS)
im.save('thumbnail.jpg', 'jpg', quality=90)

Does it succeed?

[darkowic]

This works. Thumbnail is saved where the path points locally

[wiredfool]

So, it's not the Jpeg, nor the quality.

I'd suggest looking at exactly how django-stores gets the data to S3, either by tempfile or StringIO buffer and enable as much logging as you can around that, including http requests if you can't figure it out any other way.

[wiredfool]

FWIW, the writes to file based objects happen within encode_to_file