Will the Sony model work for dual camera iPhone? #94
Comments
|
When using the Sony model, I have had good/equivalent results when using iPhone 6s and Xs images. No real differences in the results. |
|
@gaseosaluz i have tried with RAW images from my iphone X and changed the black level but got terrible results. Here are the details: Do you have any idea what might the issue be? |
|
@mevinDhun I looked at the details in #75 and I can't off the top of my head figure out what the problem you have could be. I have not worked on this in a while, but I still have my code. Maybe you can find a way to send me the RAW image and I can try to find time run it in my setup? |
|
@gaseosaluz here is a link to the dng sample: In my tests, i have calculated the black level rather than hardcoding it: im = raw.raw_image_visible.astype(np.float32)
black_level = raw.black_level_per_channel[0]
im = np.maximum(im - black_level, 0) / (16383 - black_level) # subtract the black levelIn case you can't find time to try, please share your modified code which could be helpful. |
|
@mevinDhun I will try to get to this as soon as I can. It may be the weekend. Unfortunately I don’t have my code in a public repo, but if I need to share it I will see what I can do about it. |
|
@mevinDhun I found a few minutes to run your image through my code. Unfortunately I did not get good results either. At this point I don't think that the problem is related to the black level. I will keep your picture handy and when I have some time I will try to run the code through a debugger and see if I find anything. A quick question - when you took the underexposed image, did you make sure that you held the camera steady (or had it in a tripod or something holding it steady?). I ask because I noticed that my corrected image appeared to show signs of the camera being 'shaken' when the picture was taken. I have had problems with these type of images before. I don't know if this is the problem but I thought I would mention it. In any case, sorry I was not able to help. If I do figure something out I will post here. |
|
@gaseosaluz That picture was taken handheld, so it might have some camera shake. Let's assume it is a faulty picture. But i have tried with another one where i place the phone against a wall to ensure minimal shake. Here is the DNG taken from Halide camera app: Running it through the model outputs this (which looks very bad): At this point, i'm very confused since i could not get any decent results but the paper stated it should work on iphone camera sensors. |
|
@mevinDhun Ok, I will try to run this new picture tonight. I am also puzzled because I have gotten decent results from pictures form an iPhone 6S and iPhone Xs. I took the pictures with the VSCO app, but I am not sure it is app dependent. Again, if I have better news, I will post here |
Can I test the pictures taken by my camera or mobile phone directly? How should I do? |
|
I think that theoretically it is possible take pictures in your phone. I have done some work on this for an iPhone and this is what I did to test it (but not in an iPhone application) the idea:
Converted he TF model to a CoreML Model. I was succefull in converting the model from TF to CoreML.
I crated a quick Swift Program (following an Apple example) to load the CoreML model into Xcode. I did this to make sure the model would be callable from Swift and useable inside Xcode.
I tested the converted CoreML model in a Jupyter Notebook. This was probably the hardest part for me. Since I don’t know Swift I tried to emulate in Python the things that I needed to do to manipulate the image per the origianl See in the Dark code. One of the big reasons that I did this is that I needed the NumPy manipulations form the original See in the Dark Repo to do the image slicing for the model. I didd not and until today I do not now how to do that in Swift.
Once I had the image sliced appropriately in Python in a Jupyter Notebook (I used the same steps as in the original code) I fed it to tthe CoreML model using a Python API that Apple described in this WWDC presentation: https://developer.apple.com/videos/play/wwdc2018/719/ <https://developer.apple.com/videos/play/wwdc2018/719/>. Unfortunately Apple pulled the Python library and as far as I know it is no longer available.
Once I got a proof of concept for this, I stopped this work because I don’t know enough Swift to replicate the NumPy functionality in Swift (and AFAIK there is no NumPy equivalent in Swift).
I did not put this in a repository because I am not really 100% if did this right and I am not sure if anybody would find this useful.
I hope this helps.
Lalo Silva
… On Mar 7, 2020, at 6:12 AM, littleqing0914 ***@***.***> wrote:
When using the Sony model, I have had good/equivalent results when using iPhone 6s and Xs images. No real differences in the results.
Can I test the pictures taken by my camera or mobile phone directly? How should I do?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#94?email_source=notifications&email_token=ALS3Q62BMWKHZSXOHBBFPALRGI23HA5CNFSM4I3RNEUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEODX2FY#issuecomment-596081943>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALS3Q66VW67JGRZTDDYIIKDRGI23HANCNFSM4I3RNEUA>.
|
|
@gaseosaluz it would be really useful if you could provide the coreml model. I am an ios developer and i would like to try to run it on mobile. Probably i will have to use Accelerate framework to replicate numpy matrix operations in swift. |
|
That is a good idea. Maybe you can complete th work that I could not :-) … let me see how I can find a way to share the model. Just be aware that I only tested it with a couple of pictures and only in Python. There are likely to be other problems
… On Mar 17, 2020, at 2:37 PM, Mevin ***@***.***> wrote:
@gaseosaluz <https://github.com/gaseosaluz> it would be really useful if you could provide the coreml model. I am an ios developer and i would like to try to run it on mobile. Probably i will have to use Accelerate framework to replicate numpy matrix operations in swift.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <#94 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABN6VVB3EZ5SF3QVXNWA7ULRH7GPZANCNFSM4I3RNEUA>.
|
|
Hi @gaseosaluz |
|
@l0stpenguin if you resolved the issue, can you share the code for removing black layer in swift. I also want to port the model into iPhone. |
|
@gaseosaluz how did test the coreML model. Any script for that? |
I don't have a iPhone 6s handy, but I heard some people are getting good results with iPhone 6s. If I try Sony model with raw images taken with iPhone with two or three cameras, (iPhone 8 or iPhone 11), will the result be roughly the same as with iPhone 6s? Thanks!
The text was updated successfully, but these errors were encountered: