Warn about timeout error in bigger images & better error handling session in readme #34
Comments
|
I'm happy to take PRs that add or clarify the docs. If you update jsdocs in the code, realize you'll need to If you think there's a code change is warranted, again, PRs are welcome. I'm surprised that the file size would affect latency, though--most all files I've seen (even > 1GB) don't take a long time to return, because I use the (default) "-fast" option that returns as soon as the metadata headers are fully read. What kind of file was it choking on? Can you post it someplace so I can see? |
|
Hey @mceachen, I don't think I can share the file because the one I'm talking about is from my production database. I'll ask my CTO if that's okay and get back to you. What I can tell is that it is a 2.6mb image, It has some different colours and contrasts, so it may be the reason? I don't think any code changes will be needed. My fix was only add the |
|
That's not a large image--my Google Pixel regularly takes images that large. If you don't want to share the image, can you share the results of If you're getting timeouts, I'd bet it's from IO timeouts from disks being overtaxed. |
|
Removed the image description but that's the output with metadata (Maybe it can help?). The image was saved on my HD (aws batch runs in machines with HD). After I moved it to my SSD, that's the difference: |
That's crazy, I've never seen exiftool take that long just to extract metadata. And even on an SSD! I'd submit this file to the ExifTool team, this smells like the file's format is so arcane it's getting ET's knickers in a bunch. In any event, I set the timeout to 10s because that's 10x longer than I've ever seen ET take (even on 15 year old hardware). I guess I'll bump the default to 20s, but you can do this immediately in your own code if you instantiate your own ExifTool singleton. |
|
I will se what I can do to submit the image. About instantiate the singleton, I'm already doing it. Opened this issue just to let you know about the problem in this use case. PS: This problem also happened with other images in our base. Increasing the taskTimeout solved it and the script ran gracefully. |
Is your feature request related to a problem? Please describe.
First of all, congratulations! I'm running this project on a migration script for my company and it's helping us to extract exifData from 300k+ images.
Problem: It's running on a aws batch script. It started to just stop, exiting with an 0 code and I though that it might be related to the cpu (I was only using 2 vcpus), memory and stuff related to machine limitation. Spent some time improving the script and running it on a docker container with limited resources to replicate the environment and nothing changed.
In the end, the problem was that the image was too big and would take more than 10 seconds to extract its exifData.
Describe the solution you'd like
Took a deeper dive on your readme today and saw the error handling session... I think it could have a code sample and some kind of highlight to warn the users to use it to get proper error logs. In my case, I had to throw the error on the node script so the aws batch could set it as failed instead of success. Also had to increase the timeout with the
taskTimeoutMillisparam.I think this project should have some kind of warning to tell users that are dealing with bigger images that they probably would like to increase the task timeout. Also, warning them about the error logging and creating a code sample showing how to handle errors would be good.
I can open a PR if you guys are too busy.
The text was updated successfully, but these errors were encountered: