Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check if transparent color is really used when examining frame transparency #401

Closed
stanmots opened this issue Apr 6, 2017 · 9 comments
Closed

Comments

@stanmots
Copy link
Contributor

stanmots commented Apr 6, 2017

Hi,

I've been trying to increase the GifDrawable seeking performance because, as you may know, it is currently very poor (especially when seeking backwards). And I encountered the following issue: for a lot of GIFs ImageMagick reports Transparent color: none for all frames. You can check it with this command:

identify -verbose GIF_IMAGE.gif | grep -C 1 Transparent

However, here controlBlock.TransparentColor returns different strange values like 127, 98, 0 etc. You can verify this with such code:


__android_log_print(ANDROID_LOG_DEBUG, "LOG_TAG", "TransparentColor is %u", controlBlock.TransparentColor);

It seems that the control block returns the incorrect transparent color. And if it is really incorrect than it's a great performance hit because break cannot be reached and a lot of unneeded frames must be rendered. BTW, the same GIFs can be sought very quickly in my iOS app.

I created a new branch in my forked version to experiment with the seeking functionality. I added there:

  1. An example with the media player + the GIF which has Transparent color: none for all frames
  2. An option to set an upper-bound for frames that are rendered during seeking operations. This allows to prevent the freezes during seeking. For example, if you set maxFramesToRenderWhenSeeking to 1 it is guaranteed that no more than one frame will be rendered when seeking. -1 means the default unbounded behavior. And if you try to quickly drag the seek bar thumb forwards and backwards with the default behavior the sample is often stuck with

screen shot 2017-04-06 at 5 14 05 pm

Hope all this will be helpful.

@koral--
Copy link
Owner

koral-- commented Apr 6, 2017

If there is no transparency flag set controlBlock.TransparentColor should be set to NO_TRANSPARENT_COLOR which is -1 as it is stated here: https://github.com/koral--/android-gif-drawable/blob/master/android-gif-drawable/src/main/c/giflib/dgif_lib.c#L406
If values are random for frames without that flag it is a bug.
Thanks for detailed description. Will investigate it ASAP.

@koral-- koral-- added the bug label Apr 6, 2017
@koral-- koral-- added this to the 1.2.7 milestone Apr 6, 2017
@stanmots
Copy link
Contributor Author

stanmots commented Apr 6, 2017

Thank you for the reply.

I added a log statement at this line:

___android_log_print(ANDROID_LOG_DEBUG, "LOG_TAG", "DGifExtensionToGCB: TransparentColor is %d", GCB->TransparentColor);

and also in the seek method.

Here is the logcat snippet after dragging the seek bar (as you can see, DGifExtensionToGCB method is getting called only four times):

04-07 01:41:03.328 24378-24604/pl.droidsonroids.gif.sample D/LOG_TAG: DGifExtensionToGCB: TransparentColor is 0
04-07 01:41:03.328 24378-24604/pl.droidsonroids.gif.sample D/LOG_TAG: DGifExtensionToGCB: TransparentColor is 0
04-07 01:41:03.328 24378-24604/pl.droidsonroids.gif.sample D/LOG_TAG: DGifExtensionToGCB: TransparentColor is 0
04-07 01:41:03.328 24378-24604/pl.droidsonroids.gif.sample D/LOG_TAG: DGifExtensionToGCB: TransparentColor is 0
04-07 01:41:06.819 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 26
04-07 01:41:06.973 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 31
04-07 01:41:06.993 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 30
04-07 01:41:07.038 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 97 at index 32
04-07 01:41:07.077 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 98 at index 34
04-07 01:41:07.117 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 96 at index 37
04-07 01:41:07.151 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 95 at index 39
04-07 01:41:07.230 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 97 at index 44
04-07 01:41:07.254 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 97 at index 44
04-07 01:41:07.335 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 99 at index 46
04-07 01:41:07.346 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 101 at index 45
04-07 01:41:07.429 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 95 at index 51
04-07 01:41:07.449 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 96 at index 53
04-07 01:41:07.475 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 96 at index 53
04-07 01:41:07.489 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 91 at index 55
04-07 01:41:07.511 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 91 at index 55
04-07 01:41:07.590 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 91 at index 55
04-07 01:41:07.656 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 91 at index 55
04-07 01:41:07.740 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 91 at index 55
04-07 01:41:07.756 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 93 at index 54
04-07 01:41:07.831 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 95 at index 52
04-07 01:41:07.853 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 91 at index 50
04-07 01:41:07.880 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 101 at index 45
04-07 01:41:07.922 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 94 at index 40
04-07 01:41:07.955 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 96 at index 37
04-07 01:41:07.997 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 98 at index 35
04-07 01:41:08.034 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 31
04-07 01:41:08.070 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 30
04-07 01:41:08.135 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 26
04-07 01:41:08.181 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 111 at index 20
04-07 01:41:08.236 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 13
04-07 01:41:08.269 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 8
04-07 01:41:08.315 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 109 at index 5
04-07 01:41:08.359 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 116 at index 1
04-07 01:41:08.874 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 4
04-07 01:41:08.890 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 17
04-07 01:41:08.908 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 30
04-07 01:41:08.936 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 95 at index 39
04-07 01:41:08.972 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 96 at index 53
04-07 01:41:09.023 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 62
04-07 01:41:09.064 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 105 at index 71
04-07 01:41:09.090 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 86
04-07 01:41:09.151 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 100 at index 96
04-07 01:41:09.186 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 100 at index 100
04-07 01:41:09.233 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 102 at index 103
04-07 01:41:09.287 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 107
04-07 01:41:09.341 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 97 at index 115
04-07 01:41:31.714 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 101 at index 112
04-07 01:41:31.747 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 106 at index 109
04-07 01:41:31.780 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 105 at index 106
04-07 01:41:31.822 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 103 at index 99
04-07 01:41:31.896 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 93 at index 54
04-07 01:41:31.922 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 101 at index 33
04-07 01:41:31.962 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 31
04-07 01:41:32.325 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 97 at index 32
04-07 01:41:32.523 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 31
04-07 01:41:32.606 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 30
04-07 01:41:32.723 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 106 at index 29
04-07 01:41:32.755 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 97 at index 32
04-07 01:41:32.786 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 31
04-07 01:41:32.808 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 30
04-07 01:41:32.843 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 106 at index 29
04-07 01:41:32.890 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 28
04-07 01:41:32.908 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 27
04-07 01:41:32.943 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 104 at index 26
04-07 01:41:32.978 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 25
04-07 01:41:32.991 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 105 at index 24
04-07 01:41:33.027 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 106 at index 23
04-07 01:41:33.043 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 111 at index 22
04-07 01:41:33.060 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 106 at index 21
04-07 01:41:33.093 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 111 at index 20
04-07 01:41:33.142 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 109 at index 19
04-07 01:41:33.159 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 106 at index 18
04-07 01:41:33.192 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 17
04-07 01:41:33.225 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 109 at index 16
04-07 01:41:33.243 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 109 at index 15
04-07 01:41:33.274 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 14
04-07 01:41:33.306 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 14
04-07 01:41:33.339 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 107 at index 13
04-07 01:41:33.375 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 106 at index 12
04-07 01:41:33.410 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 10
04-07 01:41:33.441 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 8
04-07 01:41:33.476 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 8
04-07 01:41:33.511 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 109 at index 5
04-07 01:41:33.540 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 116 at index 1
04-07 01:41:33.574 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 116 at index 1
04-07 01:41:33.890 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 116 at index 1
04-07 01:41:33.926 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 108 at index 4
04-07 01:41:34.011 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 106 at index 12
04-07 01:41:34.071 24378-24413/pl.droidsonroids.gif.sample D/LOG_TAG: Seek: TransparentColor is 109 at index 16

@koral--
Copy link
Owner

koral-- commented Apr 17, 2017

I've opened that GIF with a cat in hex editor, and it seems that values read by DGifExtensionToGCB function are correct. They match the content of GIF file.
E.g. 2nd frame at offset 0000:5D50 there is 00 21 F9 04 05 14 00 74.
The LSB in 05 is 1 so transparency flag is set and 74 is a transparent color index according to GIFLib documentation.
gifsicle -II <file.gif> also reports the same values.

I'm not sure what Transparent means in ImageMagick's identify does. Maybe it checks if color marked as transparent is actually used in raster?

So transparent color index reading seems to be correct.

Maybe condition here can be refactored to better find key frames.
Naive solution is to check if transparent color index is really used however it requires iteration over all pixels of the frame. E.g. here are some thoughts.

the same GIFs can be sought very quickly in my iOS app
Maybe all the frames are buffered so seeking there consists of only moving some pointer.

What do you think @storix ?

@koral-- koral-- removed the bug label Apr 17, 2017
@stanmots
Copy link
Contributor Author

@koral-- Thank you very much for verifying this. Indeed it seems that identify utility somehow checks whether the transparent color is really used by the current GIF frame. I tested it with different GIFs and if a GIF clearly has transparent areas it reports a correct transparent color value.

What's important here is the fact that the transparency flag doesn't guarantee that a frame will use transparency color.

Naive solution is to check if transparent color index is really used however it requires iteration over all pixels of the frame

I think, it is not necessary to iterate over the entire pixels array. Every pixel in a GIF frame is represented by an index in a color table (regardless, global or local). During pixels composition we can save all theses indices in a set data structure. This set will give us possibility to verify in a constant time O(1) whether the given transparent color is presented.

It also shouldn't increase the memory usage too much because:

  1. sets prevent duplicates
  2. a set is associated with a frame and is cleared when the corresponding frame is destroyed
  3. in the worst case it will contain as much values as the color table

I have a lot of GIFs where the transparent colors seem to be not used at all. Unfortunately, I currently have very limited time to implement and test this.

@koral--
Copy link
Owner

koral-- commented Apr 20, 2017

During pixels composition we can save all theses indices

Yeah, but pixels composition does not happen in all cases. E.g. if seek just created drawable to frames which weren't rendered yet. Of course, we can assume that frame is not transparent in such cases.

I currently have very limited time to implement and test this.

So have I.

I'll keep this issue open and if you (or someone else) provide PR I'll review immediately but, I can't promise when and if at all I'll implement this feature myself.

@koral-- koral-- removed this from the 1.2.7 milestone Apr 20, 2017
@koral-- koral-- changed the title Strange TransparentColor values (also related to seeking performance) Check if transparent color is really used when examining frame transparency Apr 20, 2017
@stanmots
Copy link
Contributor Author

Yeah, but pixels composition does not happen in all cases. E.g. if seek just created drawable to frames which weren't rendered yet.

Is it possible to render the requested frame excluding the pixels with transparent colors and without analyzing all the previous frames disposal methods? We then could just compare the resulting pixel number to the gif bounds pixel number. It they are equal then the frame is full and no other actions are required. If not - fallback to current implementation. However, in this case when rendering the previous frames the already rendered pixels of this frame must be taken into account.

@koral--
Copy link
Owner

koral-- commented Apr 26, 2017

Is it possible to render the requested frame excluding the pixels with transparent colors

Technically drawing a transparent pixel is just skipping of draw operation. It happens here: https://github.com/koral--/android-gif-drawable/blob/master/android-gif-drawable/src/main/c/drawing.c#L54

But the code in that loop is only executed when draw is needed.
Except for case from this issue (transparency flag set but transparent pixels not exist), it seems to be more efficient to rely on metadata.

Maybe it will be better to solve this issue in another way. Introduce additional, let's say mode when entire frames are buffered into memory. So will have intant seeking in any direction (like sample from iOS app you mentioned) at the cost of higher memory usage.

@stanmots
Copy link
Contributor Author

Introduce additional, let's say mode when entire frames are buffered into memory.

OMG, I thought for some reason that the lib is already caching the frames. It is clear now why there is so big difference in seeking performance between Android and iOS app.

On iOS I'm using this lib. As you can see here, it caches up to 50 frames. Maybe it's better to implement an adaptive buffer size depending on the available memory using something like Glide or Picasso.

Taking into account new considerations, I think, you can close this issue. I can open another one when I will have something to bring to the table regarding the caching approach.

@koral--
Copy link
Owner

koral-- commented Apr 26, 2017

OK, feel free to send PRs.

@koral-- koral-- closed this as completed Apr 26, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants