Hello Umar #2
Comments
|
Hi touchhope, Wow, I didn't know this much has happened since I did that weekend hack on Ractiv. Thank you for such a good breakdown on the situation of things. I can't promise anything but I will be compiling the code that Lai has done and play around with it in the coming days to see what could be done. Will keep you guys posted as I have progress on http://gharbi.me/ractiv/ |
|
Ractiv/touch_plus_source_code#37 "to make a try to run something on windows, to see only 6 little windows with your fingers moving in realtime for now, just copy turbojpeg.dll file to build folder, after compiling in VS Community 2015, then launch daemon_plus.exe directly from build folder, without modifying any code in fact as soon as your path is without any blank spaces (C:\touch_plus_source_code\build), see : Ractiv/touch_plus_source_code#36" |
|
Hi Umar To me, Lai must have had build a kind of debugging console mode that must be disabled somewhere in code to get output on windows interface with win_cursor_plus to get the famous pointing green circle as seen on v0.55b version. Maybe some true or false to change somewhere or part of the code has to be commented or uncommented somewhere. As you seems to be cleaver on many things Umar, could you explain us, to Matthew and me :
Thanks in advance, Umar, if you find a little time and answers to us. Kind regards, Christian |
|
I forgot to tell you that Dr. Andrea Tagliasacchi from Graphics Laboratory @ University of Victoria, Canada http://gfx.uvic.ca/people/ataiya/ is ok, in response to my Hello Anastasia (https://github.com/OpenGP/htrack/issues/17), to try to test his https://github.com/OpenGP/htrack on a unit we would send him. So I've contacted Lachlan Pollock at [email protected] to see if he could find a unit for him, because I've made a publicity, sending about 218 emails to members, to get some for Lacklan's programmer friend on OpenCV, and so he already have contacts. |
|
Hi touchhope, I am sorry for not responding earlier on this. I did go through the code for a couple of hours a few weeks back and in its current state it has been left in a debug state where the dev was midway through adding some functionality and has lots of testing code splattered across the code. Secondly the code has very few comments which makes it very hard to understand functionality throughout. I think for it to be fully functional and usable it would require a lot of time to work on it.
This part of the code is commented in the code which I was able to run but still wouldn't move the cursor based on my movements. I will see if I can get this working over the weekend so that even with buggy movement end users can have a controllable cursor.
UDP is a networking protocol and the touch plus is built with multiple different executable (daemon, gesture recognizer, cursor control) all components talk to each other via UDP.
I am guessing this would be for controlling the cursor on windows 8 and above as then the application would need to have a Microsoft certificate to be able to work as an input device.
NWJS is a framework to run JavaScript apps on desktops using chrome. I am not sure if the menu application is made using NWJS or directly via flash but it just communicates via UDP with the daemon process which is made in c++ and is part of the project. |
|
See on http://gharbi.me/ractiv/ " Matthew H. Fogle Backer 57 minutes ago Ok, I think I see the issue, the release around 7/25/2015 has some code in main.cpp which clearly sends the points over to the cursor app. In the current branch that never happens. Instead code is called into the scopa section which clearly gets the points but never sends them over the udp channel. I'll tinker with adding back in some hybrid routines and see if I get anywhere... |
|
See on : https://www.kickstarter.com/projects/haptix/haptix-multitouch-reinvented/comments "Matthew Fogle Il y a 29 minutes You know the really really interesting part? This code base in Github has stubs for pretty much everything they demonstrated in their videos. If I didn't know better I'd assume it looks like they had a working project but then had to change something vastly. Which leads me to the assumption that the etron chip disparity map might have been used in the prototype... Why else would they completely change everything? Even the unity game engine code is here..." |
|
Here it is on my dropbox the 07/25/2015 version, if it can be useful to you Umar as Matthew has suggested, to have a look at main.cpp : On github on fork, you can find old branches too : master and depth-experiment, updated 7 months ago : But Windows Environment Variables have to be set and it was not supposed to be for compiling under VS Community 2015 but under Visual Studio 2013 without Update 4. |
|
It seems there were radical changes during July, here is state of code on 07/08/2015 : |
|
Hi Umar, Robert Sweetman (http://www.robertsweetman.co.uk/) says he will send next week a unit to Dr. Andrea Tagliasacchi. I've just send him this email in response to his question about Kickstarter Haptix project and also on http://faceshift.com/studio/2015.2/introduction.html#prerequisites : Hi Andrea Here's initial Kickstarter's project, called Haptix : https://www.kickstarter.com/projects/haptix/haptix-multitouch-reinvented/comments Jerry Bocci posted on 23th February : Don't forget, they also had assistance from: I guess [email protected] has technical details on eSP870 Etron 3D Depth-Map Camera Controller : https://plus.google.com/107414035407354334983 Maybe let's also have a look at here : https://www.leopardimaging.com/uploads/LI-ESP870-STEREO-M031_datasheet.pdf http://www.etron.com/en/pr/latestnews2.php?ID=341 The initial Ractiv's roadmap was : Me and my ACER Aspire R7 with Ezel Hinge are particularly interest with "Table mode (manual calibration on untextured surfaces)" : I dream of controling my pc directly from the flat surface of my white physical desktop. But is it possible if the sensor has no landmarks to determine 3D positions of my fingers on a virtual untextured surface to detect hard or soft pressure of fingerprint for example ? Indeed, the 07/25/2015 version I put to my dropbox for Umar (#2) must have been used to compile v0.55b (https://www.youtube.com/watch?v=ulFTmjiwmAA), and used a disparity mapping system with two sorts of cambered chessboards images (0.jpg and 1.jpg in C:\touch_plus_source_code\build\userdata\0101007491) and references to 4 keys on querty keyboard on calibration step. I particularly intrigued by the 565 .nrocinunerrad files in C:\touch_plus_source_code\build\database and \labels (rocknroll1 to 19, rest0 to 350, poit1(all alone) and point0 to 206) which seems to generate all sorts of numbers calculations, see : Ractiv/touch_plus_source_code#36 I know nothing about math and geometry, I'm full of paradox because I've studied letters and languages and however I love computer engineering and technology in general. The last 12/07/2015 version, very confused with unity game engine inside, send nothing to udp channel for win_cursor_plus, but Windows 10 is so buggy even with the last builds, I'm fed up with Microsoft, often my PC just don't load the new interface and to shutdown I have to push Windows key + R to run shutdown /s ! My favorite linux distro is Salix from Slackware Darkstar. By the way, do you think the device could also have been badly designed electronically and so not giving full access to all of the eSP870 possibilities ? Cheers, Touchhope Message du : 12/03/2016 21:49 Hello there, I am always clearly interested in looking at new sensors. As I mentioned in my earlier messages is how does it compare to commercial software. Recently Intel announced its linux drivers, thus finally allowing me to perform research on up-to-date hardware. So, how does your device compare to what’s already out there? My colleagues at FaceShift had a comparison (somewhere on this site) visually analyzing the performance of various hardware with their product. And how does your technology compares internally? (i.e. what are you doing differently that gives you better results). Have a great day |
|
I forgot to tell you Lokki (Lachlan Pollock) and Vic Jang, student at UC Davis, California will provide Maz with a unit (Lokki's programmer friend on OpenCV) in California too (and close to Vic), so that he and his team can have a look at the hardware as well as test software. |
|
Hi Umar, I know there's the Cashmere question between Pakistan and India, so I hope you won't mind that I did ask for help an Indian from Bangalore : Jay Rambhia ;-) see on http://gharbi.me/ractiv/ : "I've just found the very interesting simple work on OpenCV for beginners on Jay Rambhia, Bangalore, India's personal website, see https://github.com/jayrambhia/Vision and http://www.jayrambhia.com/computervision/ ! Yesterday I sent him an asking for his help email to see if he would by be to try to help us. Indeed, Dr. Andrea Tagliasacchi isn't sure anymore if he's finally willing or not to receive a unit and if it would really worth his time. He wants technical specs of Etron's eSP870 embeded in Touchplus device to compare with Intel Realsense SR300 he is using in his lab : "comparison in terms of variance of z-noise, precision of silhouette images, range of operation and frame rate". Unfortunately I don't have these informations to send him. However, Robert Sweetman from UK was ready to send him his unit." And then Matthew H. Fogle posted : "touchhope, interestingly that's the key point I'm making. There are NO specs that matter on the etron chip! Because ractiv failed to follow the reference design the etron chip cannot be used for anything but USB communications and alignment/control of the stereo webcams. Our entire solution is software based." By the way, here are Lai's commits on github : https://github.com/Ractiv/touch_plus_source_code/commits/master?page=7 And for beginners, there's also web ressources, such as : http://docs.opencv.org/2.4/modules/core/doc/basic_structures.html Lachlan Pollock told me yesterday his friend Maz and his team will soon get Vic Jang's unit to start testing. Thanks in advance for your eventual contribution to this "sinking project" as Dr. Andrea Tagliasacchi said. Cheers, Christian |
|
Hey touchhope, haha Kashmir is a political issue, I believe we are all children of the internet and wish the world didn't have borders or conflict. I think I won't be able to do much for some time on the touchplus project now as my only windows laptop at home has died. I might be replacing it but that decision is a few months down the road. Its great that you are keeping track of all the progress as this allows this issue post to have the complete low down for anyone ever planning to help on the project. Cheers! |
|
Hi everyone ! It's again Touchhope from http://gharbi.me/ractiv/ ! Well, with my slow old brain, I've only just understand that it was possible to get ancient states of touchplus source code from github, clicking on "307 commits", then on the update that you want and then on "Browse files" on the right, then on the green button 'Clone or download", for example at https://github.com/Ractiv/touch_plus_source_code/commits/master?page=9 from the very beginning. After a lot of search on internet forums, to solve all compiling errors under vs2015 community from vs2013 code, (adding compiling options in properties under vs2015, adding several includes on top of a lot of the cpp files such as #include "stdafx.h", adding file in C:\External Storage\Dropbox\projects\touch_plus_source_code\track_plus_core\track_plus\imu.cpp, making all includes pointing to right absolute path dependencies, using Find And Replace Tool (https://findandreplace.codeplex.com/), I have zipped an ancient state of code from early 2015 July with all functionnal calibration steps and an udp channel opened for win_cursor_plus process which is launch in console mode unfortunately, in file win10vs2015touch_plus_source_code.zip, available on dropbox at https://www.dropbox.com/s/7oh7defgiwpe42z/win10vs2015touch_plus_source_code.zip?dl=0 To get it work on your win10, first install Touch+ Software.exe to get path C:\Program Files (x86)\Ractiv\Touch+ Software on your pc, then overwrite all its content with "build" folder content. Don't forget to set your keyboard to English QWERTY instead of AZERTY to perform calibration steps ! Be patient while moving your fingers, and hold finger long time enough to let the software detect the key pressed on the keyboard ! Then you could make some code modifications under Visual Studio 2015 Community you'll find here : https://www.visualstudio.com/en-us/products/vs-2015-product-editions.aspx, loading track_plus.sln project file in C:\touch_plus_source_code\track_plus_visual_studio. I hope it will help someone (Matthew H. Fogle maybe, or Umar Nizamani when he would be able to buy a new win10 pc, or Maz and his team on OpenCV in California, or someone else with better knowledge than me, a pure self-educated c++ dabbler and beginner) to produce some hybrid version under win10 and vs2015 (of old and last state of code from 1st of 2015 December). Now the technical challenge is maybe to pick up some portions of code in the different states of code during 2015 year and to mix it adding it maybe to this basic version on win10 and vs2015, instead of vs2013 and win7 or win8 ? I'd like to end with these words from backer Robert Sweetman one year ago : "As someone once said... No man is an island. Thus no great things come from someone who cannot communicate with their fellows as they remain forever doomed as legend in their own mind only. An inability to deliver on anything, promises, product etc. leads to further isolation. Seek first to understand before trying to be understood. Be of service to others ahead of expecting rewards for empty words. From the short interview text below you could drive a bus through the perceptual issues/potential roadblocks associated with this insular approach. I now formally abandon all hope of seeing anything from this venture. The smartest people in the world are usually sunk by their inability to communicate." You're so right Robert ! Indeed what about Albert Einstein ? Was he able to communicate enough ? And Nicolas Tesla ? And Rudolf Diesel ? And Xue Lai (or Lai Schwe ?) And Darren Lim ? Surely, maybe too much complicated thoughts to get explain to other people ! Could someone tell me (Corey Manders maybe at IEEE or Dr. Andrea Tagliasacchi ?) why this win_cursor_plus process isn't launched properly to show its big green pointing circle ? Indeed if you'll make a try under vs2015 with this old state of code from vs 2013, you'll need to first install "vs2015 nuget package manager", you'll find here : https://visualstudiogallery.msdn.microsoft.com/5d345edc-2e2d-4a9c-b73b-d53956dc458d and then, once installed in vs2015, going in the nuget manager in tools menu, click on browse thumb to search for missing "TCD.System.TouchInjection. 1.0.0" ! Cheers everyone and thanks in advance for any eventual help ! Touchhope settings file loaded a: 10.8047 b: -3.57327 c: 116.99 d: 37.003 message received open udp channel win_cursor_plus0 |
|
OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::Mat::Mat, file C:\builds\2_4_PackSlave-win32-vc12-shared\opencv\modules\core\src\matrix.cpp, line 323 |
|
Regular update commit number d44e966 on 5 Aug 2015 is much more functional than commit number 65abaf6 on 8 Jul 2015 that I previously toughly solved on compiling errors under vs2015 community : absolutely no compiling errors under vs2015 community apparently very fast gesture recognition and 4 keys press detection during the 3 calibration steps also udp channel messages sent and received between track_plus and win_cursor_plus processes good base I think to try to understand what wrong in subsystem console instead of subsystem windows launch maybe (https://msdn.microsoft.com/en-us/library/f9t8842e.aspx) I'm definitely not gifted enough on C++ coding language ! However I still try to change different parameters and still search on internet forums. I'm quite obstinated when I start to solve something ! |
|
Hi Jerry ! Pleased to finally get in touch(plus ;-) ) with the famous Jerry Bocci Jr, also a kind of "touchhope" at https://www.kickstarter.com/projects/haptix/haptix-multitouch-reinvented/comments ! It was funny the other day when I've sent an email to australian backer Lachlan Pollock (friend of Maz), asking him to post you on Kickstarter a request of technical explanations on how you finally get it to work on your pc ! Indeed, as the frenchman that I am, I didn't understood on first read it was a joke ! In the minute I've sent another email to Lokki to say him I realize you were joking. He said you're probably a sarcastic man and of course you were ironic on that post ! Lai must be laughing at us seeing how stupid we are on not understanding his few commented code. Or maybe he is in a depressive phase so he is psychologically washed out after so big coding effort last year. Fortunately a band of stupid people can become intelligent if they communicate with each other. That's what we need : a good brain storming ! Lai is surely a brillant young engineer but, as he will grow old, he will be learning to better communicate with other people. No man is an island and what is more exciting in human life is surely what we can learn one from each other to go further together. You know like monkeys and in 2016 we are in the chinese year of monkey, aren't we ? I invite Lai to reestablish dialog with us, people who still believe in your genious idea and project ! Come on Lai, thanks in advance man, you're the best :D ! You're mustn't be a dog and us neither, we're human beings ! I guess what is preventing win_cursor_plus process from being launched properly must be a silly thing such as "true" or "false", or "1" instead of "null" in some function. As a csharp expert like Matthew H. Fogle, maybe you could explain us how a csharp exe is controlled ? Is there such thing as the manifest file for example ? (https://msdn.microsoft.com/en-us/library/1w45z383%28v=vs.110%29.aspx) What about C:\touch_plus_source_code\track_plus_visual_studio\win_cursor_plus\ cs files, also in TUIO folder ? Maybe some examples to examine at http://www.tuio.org/?software and http://blog.mosthege.net/2012/08/19/windows-8-touchinjection-with-c/ and http://touchinjector.codeplex.com/ where you suspect windows firewall to get implicated on port blocking (blue message from last commit in December) ? To me there could also be a kind of debugging console mode that Lai has established to develop his code and which is preventing win_cursor_plus to get launched in windows mode : some lines somewhere to get commented or uncommented maybe ? Just time to find ! I'm sure one day we'll all, even Lai, be renamed Tom, Jerry, ultimate mouse killers ! I'll be glad to read you also on http://gharbi.me/ractiv/ soon Jerry ! Keep being confident in finding a solution together ! Cheers, Touchhope (or Christian or The Hopin'Frog or The Artist or Prince or the unbelivable ramblings spammer at http://gharbi.me/ractiv/ !) |
|
Hi all the seekers (and researchers) for functional solution for the touchplus device's software ! Well in the past few days, I didn't found much things : If you don't want console to get shown, you just have to comment "// #define SHOW_CONSOLE" before compiling in file track_plus_core/daemon_plus/globals.h ! Then the 4 processes (daemon_plus.exe, menu_plus.exe, track_plus.exe and win_cursor_plus.exe) still get launched but without showing any console, like in the commit 950e348 on 6 Oct 2015 (with pretty circle jauge instead of 4 calibration steps with 3 gestures and 4 keys pressed), which indeed shows no console while running, but unfortunately with inert launched process "win_cursor_plus.exe", maybe not receiving any udp port message from track_plus.exe ! Well it would be cool to finally understand how and where in code udp messages are sent from track_plus process to win_cursor_plus process and its subprocess "MainWindow" as seen on taskmgr. If it could help, for the specialists on csharp, here is the debugging errors tracking while testing a commit but I don't know which anymore, I've obtained the following errors from launched win_cursor_plus that just crashed : "Unhandled exception: System.Windows.Markup.XamlParseException: The constructor invocation on the type 'win_cursor_plus.MainWindow' that matches the specified binding constraints threw an exception. ---> System.IO.DirectoryNotFoundException: Could not find a part of the path 'C: \ touch_plus_source_code \ build \ ipc'. |
|
Martin Kaltenbrunner have just answered my demand of help : mkalten commented 12 hours ago • edited "hi @touchhope, honestly I never heard of this device before ... looks interesting though. you could try to use one of the available TUIO client demos for any platform or language and see if you can receive anything on their UDP port (which seems to be different from the standard TUIO UDP port 3333) you could also try to update the TUIO CSHARP code, which would also be the right place for this discussion, since this here is the C++ version ... https://github.com/mkalten/TUIO11_NET if you want to provide me with such a device, I can have a closer look of course edit: it also seems that they modified/extended the TUIO protocol, which probably results in making it incompatible with standard TUIO software. what's exactly the function of the win_cursor_plus component?" |
|
http://reactivision.sourceforge.net/ Absolutely interesting ! |
|
A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction : |
|
Hi the touchhopers ! After testing a lot of commits, the most interesting to me is Regular update d679834 on 24 Jul 2015 Ractiv/touch_plus_source_code@d679834 because it shows us how win_cursor_plus process is supposed to control TUIO cursor. Indeed, when you compile this commit (a lot of dependencies are missing in its dependencies folder and must be completed with other dependencies comming from other later commits) you can get a medium window called "image_visualization_pointer_mapper" showing us the supposed behavior of TUIO cursor as seen on Alexandru Bauer's Youtube on v0.55b release without full available source code on github. 2 circles get superimposed as finger is coming to a centered small pointing area and distance to get in contact with virtual surface on keyboard is detected by pointer mapper system which is called from main.cpp in track_plus in this commit. Unfortunately this commit isn't stable enough and you get crashing of deamon_plus and also track_plus process in few minutes. But no TUIO cursor is launched through win_cursor_plus process and indeed nothing can be listened using https://github.com/gregharding/TUIOListener ! However I've set TUIO port to standard 3333 port instead of random TUIO port. How could we send to TUIO cursor tracking data sent to window called "image_visualization_pointer_mapper" ? How could it had worked on Alexandru Bauer pc as seen on Youtube : https://www.youtube.com/watch?v=ulFTmjiwmAA ? To get a very stable commit without the tasklist command problem in software path, you can try the last commit before "processes.cpp stability fixes" commit 6b7e36e on 2 Nov 2015, "In progress: new algorithm for finger segmentation" Ractiv/touch_plus_source_code@266d5e6. The problem is that 4 points keyboard keys location calibration system seems to have been abandoned in the last commits (new circle initialization calibrating jauge) and we have to analyze how we can reconstruct logical pointer mapper system. Indeed the notion of "palm point" seems to be important to inject tracking data to the win_cursor_plus process under csharp TUIO protocol. So "Improved location of palm_point" on commit 2c0207a on 4 Nov 2015, post "processes.cpp stability fixes" commit must also be examined to try to understand how pointing circles get controled in "image_visualization_pointer_mapper" window Ractiv/touch_plus_source_code@2c0207a. At worst, we could restart from the beginning from Umar Nizamani work and programming the proposed mathematical solution "A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction" by Qunqun Xie, Guoyuan Liang, Cheng Tang, and Xinyu Wu, and maybe mixing it with Matthew H. Fogle csharp proposal to get interaction the TUIO cursor ! So we need enraged newbies programmers to get it working with the minimal cursor function because it seems coding experts won't lose their precious time on that damn project ! What a pity that Lai Schwe isn't kind enough to share minimal information so that we must made a sort of retro-engineering to try to understand something making lot of tests ! Cheers ! Your desperate Touchhope still trying to find something interesting... until the hope will definitely died ! |
Hello Umar,
On our long way to try to get a software for the Ractiv's Touch Plus device, as a buyer member of our community of backers and buyers provided by Wassim Gharbi (http://gharbi.me/ractiv/), i think we should all thank you so much for your "Code to access video stream for Ractiv Touch+" (https://github.com/umarniz/TouchPlusLib/tree/master).
Indeed, thanks to your code, a backer, Matthew H. Fogle, had tried to build a csharp "Touch+ Camera Application" (https://github.com/shadowmite/TouchPlusCMDR).
Then, Lai Schwe from Ractiv had finally publish his own source code for Ractiv Touch+ Software (https://github.com/Ractiv/touch_plus_source_code/tree/master).
Before the fork version on Github, a 0.55b version (http://pan.baidu.com/s/1pJ0zNfl) was released by Ractiv and a backer Alexandru Bauer did run the software on his computer (see : https://www.youtube.com/watch?v=ulFTmjiwmAA) but dll files (msvcr110.dll and msvcp110.dll or sometimes msvcr120.dll and msvcp120.dll) are missing on main systems (win32 and win64) and not included in Ractiv's installer.
Then we did made some debugging with Lai during few weeks this summer and he made a journey to Singapore to meet Corey Manders at the Institute for Infocomm Research to get help on OSX code.
After announcing in november, that almost every members had been gone at Ractiv, Lai was talking about high pressure and a soon software release because he made great improvments on main algorithm.
Then seeing nothing coming from Lai, ultim8k asked him a full list of: "hardware, lib dependencies, what has been done, what is missing" but got no answer.
So i was thinking of you, to maybe get, if you have a little time to help us, your expert look at actual Lai's code and simply telling us your technical opinion with your knowledge in software development.
Some backers are willing to put money in court action against Ractiv, a company that maybe doesn't exist anymore.
I wonder if for example $30000 would be enough to get a c++ engineer for two or three months to finish the job.
But maybe, there's too much to do at this point to get something running on win32, win64, osx and linux in an so short period.
The initial Ractiv's roadmap was :
(OSX support)
Linux support
SDK (will give you access to images, hand segmentation, and finger coordinates)
Multitouch gestures (pinch to zoom, two-finger scroll)
Table mode (manual calibration on untextured surfaces)
Improved tracking performance
Faster performance (including SSE and GPU acceleration) and lower CPU usage
Gesture learner (so you can teach Touch+ to recognize different gestures to activate different functions)
3D hand model
We saw interesting things on the net (https://github.com/nickgillian/grt
), (https://github.com/OpenGP/htrack), asking ourselves if it could be usefull to get mixed with the Touch+ code.
Well, if you unfortunately got no time for us, it doesn't matter and again thank you for your initial code.
Happy New Year 2016 to you, your family and your friends, Umar !
Sorry for my bad english (i'm a french man).
Best Regards,
Christian LERICHE
also known as "touchhope" and "The Forg" on Wassim's website
The text was updated successfully, but these errors were encountered: