Make visiond more HW portable - open source it all except DNN part

[kozuch]

Hi there,

this thread is aimed primarily at comma.ai staff (@geohot etc. - others wecome to comment too though!). You guys want openpilot to be successful and have as many people as possible using it, right? I see a big obstacle here towards that path in the visiond binary being locked down a single high-end $700 smartphone model. Let's make the whole project more accessible by changing this. Seems like the project is still in experimental stage and you need loads of field testing to validate your code and to push it closer to production. You may be getting many more testers when the HW needed costs (just guessing) $100 instead of $700.

I see your business model in keeping your deep neural network closed source and I am perfectly fine with it. But would you want to open-source the most of visiond except your DNN blob so that you keep your intellectual property while allowing for a much wider variety of HW that can be used? Let's open source all visiond (the camera stuff etc.) except your DNN blob and also make that blob as portable as possible (I guess there are ways to making DNN portable?). You will have much better reach and could test performance on various HW and the project may gain more popularity like this. I know this may hurt by having to support various CPU/GPU architectures but I think the benefits may outweigh the drawbacks.

Hope you will be hearing to my post! :)

[geohot]

This is a duplicate issue, see #4

One issue is even if you ported it, performance of the network won't be good with other cameras. And you really wouldn't be able to bring costs down much, since you need an 820. Currently we have no plans to open source visiond. If I saw substantial pull requests and contributions to other parts of the code I'd be more open to it.

[kozuch]

You are killing the community by insisting on (rather) expensive and limited set of hardware. This is not the way community projects work, but I guess you know this and you do not need to grow probably.

[geohot]

It won't work on (much) cheaper hardware. Show me a $100 phone with an 820 equivalent GPU, and we'll happily port it to that. We didn't choose the OnePlus 3 because we want people to have to spend needless money, it's the cheapest open 820 phone available.

If you are talking about a PC port, the ACC portion will run on a PC without visiond. Show me that you have that working first. Then you can connect your own camera to a PC and use whatever DL library you'd like.

[kozuch]

I am not accusing you of pushing people to spend endless money and I dont have anything working but I am trying to say that a successful community project needs to have the barriers to entry as low as possible...

Even though I do not have experience with DNN performance I do have some with classical computer vision algos - often they perform well on low resolution input images. I did some research with stereo vision and found that performance did not really scale with resolution (16x more pixels had only 4x less vision error). In other words one was able to downscale the image to speed things up without hurting the performance (in terms of errors) much. Maybe something similar could be done with DNNs.

What input resolution do you use for your DNN? Have you tried to lower the resolution to speed things up and see how it performs (in terms of both speed and errors)?

[johnnwvs]

@kozuch

I was just at a symposium where similar topics were discussed. I did hear some experts saying that higher resolution tended to have diminishing returns with respect to neural nets, pattern recognition, etc.

Also, if there were an interest in opening up more of the code, or just a general need for a port, I will offer my time to help.

I would like to have this running to test on my car (which has LKAS, etc) but I do not want to spend $700 to do the testing up front. I might spend that much once I am convinced it will work though.

[geohot]

Just out of curiosity, what would you spend?

[johnnwvs]

In order to differentiate, I would be willing to spend anywhere from $750-1000 for a fully working system that is guaranteed to work with my vehicle. IE: a production level device.

For testing a system I am thinking around $150-$300.

[kozuch]

Bump here. The visiond was open sourced lately. I created #562 for new discussion.