Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Not getting correct outputs #6

Open
ricshaw opened this issue Aug 17, 2018 · 46 comments
Open

Not getting correct outputs #6

ricshaw opened this issue Aug 17, 2018 · 46 comments

Comments

@ricshaw
Copy link

ricshaw commented Aug 17, 2018

Hi, thank you for the code. I followed your instructions and managed to train a model using the vanilla network. However, my test results are not correct. The room classification label is mostly correct, but the layout is almost always wrong. See the examples below...
0de2e3da9aea62b94141039401b6205c08800063_pred_4
01fe1b8ceed678b27fd45aca2bde6b8c7f0cad40_pred_5
See my training loss curves below. The classification loss goes down to almost zero but the layout loss is still quite high. Do you know why this might be?
graphg
@CesarWu410
Copy link

Hi! @ricshaw
Can u run this program dicrectly successfully? Did u not encounter this error call "IOError: [Errno 2] No such file or directory: 'rcnn/test/2.npz'"

@ricshaw
Copy link
Author

ricshaw commented Aug 29, 2018

Hey, I encountered that error too. The author hasn't provided that file, and anyway I think it is just some test code. To make it run I had to comment out the lines from "out=np.load('rcnn/test/2.npz')" up to "os.makedirs(outpath)" in the file get_res.py. I also had to comment the lines at the bottom of get_res.py from "for i in range(20)" to the end of the file.

@CesarWu410
Copy link

@ricshaw
Thanks very mach
I'll have a try

@DRAhmadFaraz
Copy link

@CesarWu410 Have you run this code successfully.?

@CesarWu410
Copy link

The run did not succeed although annotated some code according to @ricshaw hints and the following error was prompted.
screenshot from 2018-09-14 11-53-15

@CesarWu410
Copy link

@faraz6313

@ricshaw
Copy link
Author

ricshaw commented Sep 14, 2018

Not sure what this error means. It looks like you're running out of memory? How much ram do you have on your GPU? Perhaps try reducing the batch size (you'll have to change it in multiple files). Are you running vanilla net? I couldn't get resnet to work.

@sairajab
Copy link

sairajab commented Oct 8, 2018

I have same issue. Results are not correct even i tried with RCNN.

@DRAhmadFaraz
Copy link

@CesarWu410 any more updates about the installation of this code.?

@DRAhmadFaraz
Copy link

@ricshaw Can you properly guide us which steps you did till that results.?
i mean which instructions of author you follow and how you train this model using the vanilla network.?

@DRAhmadFaraz
Copy link

These are my predicted results

3894c4ff50e24ede4196a38b83a1fa34d132e3ef_pred
4c07ebb141b726243b7dd9346054b693eeeea22a_pred
4fe8887c6204a0bb59116996b2100af2e1e6d317_pred
36b5f1b90ae2e4fbd103342079ac07e49c41dbbc_pred
38b27f746dcc25542e7a5cadf3334ab5256a643c_pred
3734d236acdccd707ad76cf7808b0ecbaf581126_pred

@SlimeQ
Copy link

SlimeQ commented Dec 27, 2018 •
edited
Loading

I'm also getting these types of outputs when running --test after ~30000 training epochs. My loss numbers are pretty much the same as @ricshaw. Seems very interesting to me that 2 of the 6 points are always very close to another 2 points, makes me think maybe some indices are getting swapped during the training process. I've encountered several other simple bugs in the code so this would be unsurprising to me.

EDIT: perhaps an issue with prepare_data.py? I personally used this script from the nestarz fork to grab and prepare the data.

@thbupt
Copy link

thbupt commented Jan 21, 2019

@ricshaw Hi, I have the similar results with you. I think maybe there are bugs in the code. Have you fixed it?

@aunaik
Copy link

aunaik commented Feb 12, 2019 •
edited
Loading

@SlimeQ, I think your suspicion is right about issue with 'prepare_data.py' file. Its mentioned in the data section of Readme that during data preparation the images are horizontally flip for which 'cv2.flip()' function is used with flipCode = 1 which means vertical flip.

The correction would be using 'cv2.flip()' with flipCode=0 instead of 1.

Please correct me if I'm wrong here.

Note: flipCode argument in 'cv2.flip()' is not same as axis argument in 'np.flip()'

Modification: Apologies for the confusion. cv2.flip() function mentioned in the code is correct. I referred a site which mentioned it incorrectly. @thbupt thanks for pointing it out

@thbupt
Copy link

thbupt commented Feb 12, 2019

@aunaik But flipCode=1 means horizontal flippoing in cv2.

@wenc13
Copy link

wenc13 commented Mar 25, 2019

@ricshaw Hi, I have the similar results with you. Have you fixed it now?

@ricshaw
Copy link
Author

ricshaw commented Apr 19, 2019

@wenc13 @thbupt @DRAhmadFaraz I'm going to have another look at this. Does anyone have the data? They seem to have taken it down.

@wenc13
Copy link

wenc13 commented Apr 19, 2019

I have this data @ricshaw , it is Mr Gabriele Cimato who gives me the link.

@wenc13
Copy link

wenc13 commented Apr 19, 2019

And I think I already have some ideas about this problem @ricshaw

@ricshaw
Copy link
Author

ricshaw commented Apr 19, 2019

@wenc13 The link doesn't work anymore. Could you put the data online somewhere? Like Dropbox for example?

@wenc13
Copy link

wenc13 commented Apr 19, 2019

OK, Could you please give me your email address? @ricshaw

@ricshaw
Copy link
Author

ricshaw commented Apr 19, 2019

@wenc13 sure, my email is: [email protected]

@xiaochengfuhuo
Copy link

@wenc13 @ricshaw The link doesn't work anymore. Could you put the data online somewhere? Like Dropbox for example? thank you very mach it's my email :[email protected]

@Wap8CN
Copy link

Wap8CN commented Jun 13, 2019

@ricshaw @wenc13 @xiaochengfuhuo I met the same problem. The link did not work now. Could you send me the data such as image.zip, *.mat, ground truth , my email: [email protected]

@kimukook
Copy link

kimukook commented Aug 5, 2019

@wenc13 @ricshaw @xiaochengfuhuo @Wap8CN Hi guys, I am also working on this code and met with the same problem. The link for the LSUN data set is not working anymore. Could you please share me with the data? The *.mat and ground truth files would suffice. My email: [email protected]. Thank you so much!

@xiaochengfuhuo
Copy link

xiaochengfuhuo commented Aug 6, 2019 via email

@kimukook
Copy link

@xiaochengfuhuo Thanks for the sharing! Eventually, I am switching to another room layout detection code provided in this repo. In my own opinion, the result he has is better than this repo. But thank you anyways.

@run-me
Copy link

run-me commented Aug 19, 2019 •
edited
Loading

@kimukook thanks for sharing an alternative model. Much appreciated :)

@kimukook
Copy link

@run-me No problem at all. Just a reminder that the repo I linked before just provide you a general position of the room layout. You have to write your own code to determine the exact key points from the image generated from his repo. Regarding this, you could take a look at the Physic Inspired Optimization part in this paper to extract the key points.

@xiaochengfuhuo
Copy link

@kimukook thank you very much

@lakshmankanakala
Copy link

Hi @kimukook @DRAhmadFaraz @xiaochengfuhuo , How to run inference for the roomnet repo for single image. can you please help me out.

Thank you.

@lakshmankanakala
Copy link

@run-me No problem at all. Just a reminder that the repo I linked before just provide you a general position of the room layout. You have to write your own code to determine the exact key points from the image generated from his repo. Regarding this, you could take a look at the Physic Inspired Optimization part in this paper to extract the key points.

@kimukook if you dont mind, can you explain more. which key points I need to consider? If you have any code snippet please share it.

Thanks

@kimukook
Copy link

@lakshmankanakala Sorry I can not share my code since this is part of my internship work under NDA. The key points are those 11 types defined in this paper. So for each image, you have to optimize for the best key points position for each room type of those 11 type of rooms. And retrieve the best room layout based on your objective function. You can further refer this part to the Physic Inspired Optimization part of the paper shared in my last comment. Hope it helps :-)

@lakshmankanakala
Copy link

thanks for your quick reply @kimukook , I have seen the paper. I have doubt i need to create those 11 types key points functions and based on the test image i need to correlate between the those 11 types. and then for optimization.
Am I correct ? please tell me if I am wrong.

Thanks

@kimukook
Copy link

@lakshmankanakala No you don't need to create functions for those 11 types. You just need to create one optimization solver which iteratively optimizes the key points for each of those 11 types, e.g. optimize room type 0 and get the best 8 key points from it, then optimize room type 1 then get the best 6 points. For each type you have to initialize the keypoints, iteratively move the keypoint and optimize the overlapping area of wall, ceiling, ground. You might consider the type of edges as well in your optimization.

@lakshmankanakala
Copy link

Thanks @kimukook , I will try to do that optimization. if somewhere I facing the problem then I will ask you.

@run-me
Copy link

run-me commented Aug 23, 2019

Thanks @kimukook for providing us with the insight. @lakshmankanakala let me know if you want to do a colab on cracking this keypoint optimization. I will be forking this repo and will be working on solving it over the weekend. Thanks guys :)

@qzj19951231
Copy link

@ricshaw Sorry to bother you. i meeting the issue
image can you help me to resolve it? thank you!

@xiaochengfuhuo
Copy link

xiaochengfuhuo commented Dec 3, 2019 via email

@qzj19951231
Copy link

qzj19951231 commented Dec 3, 2019 via email

@xiaochengfuhuo
Copy link

xiaochengfuhuo commented Dec 3, 2019 via email

@qzj19951231
Copy link

qzj19951231 commented Dec 3, 2019 via email

@xiaochengfuhuo
Copy link

xiaochengfuhuo commented Dec 3, 2019 via email

@qzj19951231
Copy link

qzj19951231 commented Dec 3, 2019 via email

@entrpn
Copy link

entrpn commented Mar 30, 2021 •
edited
Loading

I'm late to the party but I think the prepare_data.py has a bug in line 73. For flipped images is the same equation:

layout[:,:,l_list[ltype]+flip_idx[ltype][i]-1]=guassian_2d(x_mean, y_mean)

I think it should be (but not 100% sure):

layout[:,:,l_list[flip_idx[ltype][i]]=guassian_2d(x_mean, y_mean)

That would skew the results like explained by OP.

@jkstyle2
Copy link

@entrpn Can you elaborate what you meant? I also guess there's an issue on the script, but can't figure out how to deal with it.
Anyone has an idea?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests