Is Markerless AR possible with AR.js?

[MP619212]

Apple ARKit and Google ARCore allow apps to generate AR Content anchored to the real world using Plane Detection instead of Markers.

Is this possible with AR.js on iOS and Android devices?

GOAL
To view AR Content on a Mobile Web Browser without using a Marker.

Thanks!

[jarodium]

https://en.wikipedia.org/wiki/LLA_Marker
Different approach from what you want but I think it could be a great addon for this lib

[ajsutrave]

Hey has anyone made any progress on this? If not, I'd like to volunteer

[MP619212]

Please contact me at: [email protected] and let's discuss via Skype.

[larsgaardsoe]

I'm also very interested in this. How can I follow og maybe contribute?

[MP619212]

To be honest with you, I don’t come from a technical background; I just had an idea for a business that would involve this. I was just asking to see if this is possible given existing technology. So I’m not exactly sure where we would start... Matt Powell +1(619) 791-6159 LinkedIn.com/in/mattpowell1985

…

On Sep 29, 2017, at 3:13 AM, larsgaardsoe ***@***.***> wrote: I'm also very interested in this. How can I follow og maybe contribute? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#190 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AekW4-oHfLk3yPQYTZktBbDOesvwBzznks5snJikgaJpZM4PacoQ>.

[ajsutrave]

OK so I took a little bit of a look. This might be involved change because the Daqri ARKit package that is compiled by ASM.js is what is supplying the movement information to AR.js. We'd have to replace this or add more functionality to make it marker less. We could add something like OpenCV to do the feature point detection.

TLD;DR Difficult but maybe possible

[1d10t]

hi2all!

i wrote three basic aframe components for gps- and compass-based markerless ar
(see code at https://github.com/1d10t/test/blob/16b9a59ec9fcacfaa7b5c485622f21c7f4191fbb/a-frame-webcam-dae.html)

'gps-place' component is for object entity
'gps-position' and 'compass-rotation' components is for camera

in result is not too smooth, cause of compass and gps sensor is laggy. maybe need some algorithmic workaround

[jeromeetienne]

hi @1d10t

i love what you did there. and im finnaly got the time to integrate it :) unfortunatly i can't make your example to work. i run it but nothing appears on the screen beside the webcam... any hint on how to make it work ? :)

What i am doing wrong ?

[jeromeetienne]

@1d10t i know what i was doing wrong, i wasnt updating the longitude/lattitude :)

[1d10t]

there was a bug in calculation of move direction, latest commits of this file is better.
also u can test simplified example with sphere primitives https://1d10t.github.io/test/phills-sphere.html

[1d10t]

i expect that we can use some sort of LLA-markers to stabilize position in highly noisy locations, where GPS like a crazy bunny

[jeromeetienne]

@1d10t i would love to talk about how to improve the accuracy

[jeromeetienne]

@1d10t i noticed you dont handle altitude in your code. is that just small shortcut or is there an issue with handling altitude ?

[1d10t]

just didnt try

[raynt]

I am very interested in testing this out. From the example:
https://github.com/1d10t/test/blob/16b9a59ec9fcacfaa7b5c485622f21c7f4191fbb/a-frame-webcam-dae.html

What would need to be edited in this file to test? I adjusted the latitude and longitude but don't see any AR object appearing at these coordinates. Would be very thankful for a quick run through how to get this example running.

[AndreaMonzini]

Hello, i just tested it and it works on iPad Air® with iOs® 11 !

@1d10t great work! are your components under MIT license like AR.js?

@raynt 3D .dae object are commented, just delete comments ;)

Testing: http://amvr.it/test/agps.html

[raynt]

Thank you for the reply Andrea. When I delete the comments I just get a white screen.

`		<a-asset-item id="fence_50_31_0040507_9_asset" src="fence_50_31_0040507_9.dae"></a-asset-item>
		<a-asset-item id="fence_50_31_0040507_6_asset" src="fence_50_31_0040507_6.dae"></a-asset-item>
		<a-asset-item id="fence_50_31_0040507_16_asset" src="fence_50_31_0040507_16.dae"></a-asset-item>
		<a-asset-item id="fence_50_31_0040507_25_asset" src="fence_50_31_0040507_25.dae"></a-asset-item>

`

If I set a long/lat coordinate for across the street will I be able to see the 3d object looking across from the opposite side of the street or does the mobile device need to be at that specific long/lat for the 3d object to appear?

[AndreaMonzini]

@raynt i keep testing on Android® and GPS this time and it works ( a bit laggy but ok).
Have you changed Longitude and latitude coordinates near to your actual position?
The object appears on the gps-place="longitude: xx.xxxxxxx; latitude: xx.xxxxxxx" from the perspective of the mobile device coordinates.
Remember to delete comments from <a-entity> too.

[1d10t]

example with reduced code
https://1d10t.github.io/test/phills-sphere.html
@AndreaMonzini MIT

[AndreaMonzini]

@1d10t Thank you :)

I'm testing it with different 3D Object, trying to move around the position and it's working but the position is quite trembling probably for the jitter of the sensor data.

[raynt]

Thanks @AndreaMonzini and @1d10t I got it working as well with your assistance. It was very unstable but I was testing indoors so maybe the GPS signal is weak. Interesting to see if it works better outside.

[AndreaMonzini]

Hello i found this very interesting project by @andygup , with Apache License Version 2.0, and it could be useful to reduce the jitter:

https://github.com/Esri/html5-geolocation-tool-js

In particular:

https://github.com/Esri/html5-geolocation-tool-js/blob/master/js/GeolocationHelper.js

[andygup]

@AndreaMonzini Thanks for the mention! There are three ways to reduce location fluctuations on the current generation of smartphones: location averaging (GelocationHelper.js is one example), have user turn off all data access to Cellular and Wifi (not always possible or ideal) to force the device to use GPS-only, and/or use a Cordova plugin that separates NETWORK locations from GPS locations.

Good news is that high-accuracy GNSS-enabled phones are coming (hopefully) next year.

[javismiles]

Im super interested in this as well, thank you @1d10t @AndreaMonzini MIT for your great work, question, I tried your example https://1d10t.github.io/test/phills-sphere.html , on android, and I can see the white sphere right on the place where the mobile phone is. Now, if I wanted to put the sphere x meters in front of me, what and where would I have to change the code? I see that at the moment as I move the phone, the sphere follows the position of the phone, I change the gps-place coordinates, but object remains in same place always, how can I station the object so that its always either in the same global coordinates independently of the mobile phone position or better in the same relative position x distance away in relation to the mobile phone position?

Update: I wasn't testing right yesterday, now I have put the right latitude and longitude coordinates and now the testing works well, I can see and move around the entity but of course no precision, it jumps all over the place, but I can see the potential indeed

thanks a lot again ;)

[raynt]

@javismiles this example requires Unity and Kudan SDK ( https://www.youtube.com/watch?v=7-SiR_qJwaU ) but I think it helps explain what you are looking for (scroll to end of video). Basically the 3d model stays in an anchored position and you can walk closer, further away or around the object. This would be an amazing addition to AR.JS

[1d10t]

@javismiles try to go further and you will see, that spheres moves with large jumps by several meters. if you've ever played in ingress (by niantic-google), you might have noticed similar accuracy.

[javismiles]

@raynt, @1d10t thank you for your messages, so @raynt yes that's exactly what I mean, and this is something that Jerome showed recently in his first experiment combining Google ArCore and ar.js https://www.youtube.com/watch?v=I1foLzF8slc , I wonder how these ar.js - arcore experiments tie in with this other markerless gps based approach, in any case I can see that the final effect is similar except that here we have at the moment a lack of precision, the positioned object jumps all over the place because of the gps accuracy issues I guess,

and @1d10t so you mean that if i move around a lot the sphere won't stay in the center but will jump all around, ok so you are right yes, I just made new tests, I wasn't testing correctly yesterday, now I get it all, I input my latitude and longitude and now yes, I can see the box entity I added to the scene and I can move around it but yes it jumps all around and it's not precise at all, I get you all now yes, so it works but it's not precise at all, we can see the huge potential of it, but it has a total lack of precision right now

I wonder if, as some have suggested above, if LLA markers used indoors would help to make this experiment more precise?

btw, speaking of markerless AR and a bit off topic, another type of markerless ar is feature based detection and last week totally by chance i discovered these people:
https://github.com/Tastenkunst/brfv4_javascript_examples
https://tastenkunst.github.io/brfv4_javascript_examples/
I don't know if you are familiar with them, but it's pretty amazing...

thank u everybody, this is all fascinating stuff, Ive been playing with ar.js and aframe for a while now and really enjoying all of this

[AndreaMonzini]

@andygup thank you for the explanation!
@javismiles of course the system works better outdoor and only if you do not need precise position accuracy, i'm implementing a project in large squares for example.

[raynt]

Thanks for sharing those links @javismiles.
@jeromeetienne will you be releasing your implementation of ar.js/arcore?

[1d10t]

hi ppl
now u can test 'gps-place' and 'gps-position' components on another example with your PC

1. open https://1d10t.github.io/test/a-frame-dae-gmap-ground.html

2. run code in console
   camera.components['gps-position'].zeroCrd = {latitude: -33.0519424, longitude: -71.3836586};
   camera.components['gps-position'].crd = {latitude: -33.0519424, longitude: -71.3836586};
   camera.setAttribute('position', camera.getAttribute('position'));
   document.querySelector('a-scene').append(ce('a-sphere', {'gps-place': "longitude: -71.3836586; latitude: -33.0519424"}));

3. sphere will appears under your virtual location

4. use wasd controls to move camera

[jucall]

@1d10t great work !!! Thanks.
Let's try it @jlcasasg

[evaristoc]

Hi,

I confirm I saw this example on Samsung Galaxy S7, Android Oreo: https://1d10t.github.io/test/a-frame-dae-gmap-ground.html

This one could see updating parameters but no sphere: https://1d10t.github.io/test/phills-sphere.html. The size of the scene didn't fit the size of screen.

Both tested indoors.

[evaristoc]

Is this part of the project developed somewhere I could watch to see if I could help/test? It seems you are using some ML for image stabilization? I probably could help.

[evaristoc]

@1d10t @jeromeetienne and all:

Have you decided which technologies / solutions you would like to work for the markerless example?

• LLA (something like what @1d10t has been working so far)?
• SLAM?
• other(s)?

[evaristoc]

@1d10t for the many (excellent) examples you made using mobile location sensors, argon.js appears to do more or less what your code does.

If I understand correctly what argon.js tries to do differently is making you the 0,0 coordinate and re-calculating all the coordinates to any point in the world from you as reference. In that way you would not be mapped incorrectly if the precision fails: no matter your GPS coordinates, you are always 0,0.

The problem in both cases resides in the lack of precision and the variability of the GPS data. In your case the positioning is as accurate and precise as the GPS data. In the case of argon.js the positioning is more accurate but still unstable, ranging between 2 and 100mts away from my actual position. If I understand correctly the argon.js approach, I think its problem is the speed at which the GPS data changes versus the time it takes to recalculate the reference points. It is possible that by the time I am setup as 0,0 the GPS data changed so all the relations become wrongly estimated.

I actually don't know.

Here an explanation of how argon.js works:
https://docs.argonjs.io/tutorial/part1/

They recommend the following (which now make sense...):

Since the local reference frame may change at any time, a programmer should not save and use the values in this frame for more than a single update and render step. If the values need to be saved and used over multiple frames, it is possible to be notified when the local frame of reference changes.

The argon.js repository is now deprecated: its authors are putting their efforts in what it seems the main focus of several organizations nowadays: XR browsers - I think @jeromeetienne is also involved in one of those initiatives? Despite of being deprecated I would recommend you to check it out! They have been using another library (cesium.js) to estimate positioning.

[droid001]

I'm guessing one needs a tango phone to run the markerless tracking method in ar.js which Jerome was writing about in this Medium article?

[FangerZero]

I found this but can't seem to get any WebAR to work on my Pixel XL regardless of what I try... I can't even get the Google Samples to work...
https://medium.com/@urish/web-powered-augmented-reality-a-hands-on-tutorial-9e6a882e323e

[jonnyscholes]

RE Google's samples the spec is very unstable at the moment and the samples lag behind what's implemented. My understanding is that they only work on Chrome Canary <71. More details here: https://bugs.chromium.org/p/chromium/issues/detail?id=918375&q=component%3ABlink%3EWebXR%3EAR%20&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified

On Thu, Feb 7, 2019, 7:50 AM Fanger Zero ***@***.***> wrote: I found this but can't seem to get any WebAR to work on my Pixel XL regardless of what I try... I can't even get the Google Samples to work... ***@***.***/web-powered-augmented-reality-a-hands-on-tutorial-9e6a882e323e — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#190 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABM3ZzfTxjLR_GitHZzpK6vM_V6Qm38Nks5vK0ARgaJpZM4PacoQ> .

-- *Jonny Scholes* *Web Developer, Photographer & Designer* Takeflight | Web Developer 42° | TotesProfesh www.jonnyscholes.com

[commentatorboy]

I found this but can't seem to get any WebAR to work on my Pixel XL regardless of what I try... I can't even get the Google Samples to work...
https://medium.com/@urish/web-powered-augmented-reality-a-hands-on-tutorial-9e6a882e323e

It is because you have to download the chrome-like browser for it work.
https://github.com/google-ar/WebARonARCore.

I'm guessing one needs a tango phone to run the markerless tracking method in ar.js which Jerome was writing about in this Medium article?

Yes you do.

So it seems like it is not possible to do markerless AR in ar.js in common browsers.
And just to notify people. Tango is now called ARCore. So I do not know if Tango is supported in any way.

So for anything to work you have to install:
https://play.google.com/store/apps/details?id=com.google.ar.core&e=-EnableAppDetailsPageRedesign

Also you can see here how it should be done.
https://developers.google.com/web/updates/2018/06/ar-for-the-web

[tomasdev]

For those needing plane detection I highly recommend 8thwall.com

[droid001]

A comment about 8thwall, I think they don't have true SLAM detection but rather use the accelerometers and other sensors to place a model in 3D space. It works on close flat surfaces but if for instance you've placed the model on the floor and then move backwards where there's table surface then the model isn't obscured by the table top. Expensive tech for something which doesn't do the job 100% in my opinion.

[a7medo778]

hey guys
did anyone see this yet seekxr.com
i want to use it sooo bad but its kinda out of my price range 😅

[droid001]

Guess you can entertain yourself with a version running on Chrome Canary in the meanwhile: https://doc.babylonjs.com/how_to/webxr

[pavan4]

@a7medo778 they are not doing anything special. You can use it for free using Apple's Quick Look

[a7medo778]

@pavan4
apple's implementation isnt for web though

[nicolocarpignoli]

Apple Quick Look IS for the web, for safari 13 (only iOS 13) for now.
Very limited, only loading and placing objects with good plane detection

[commentatorboy]

Actually it says iOS 12 also: https://developer.apple.com/augmented-reality/quick-look/
But is not open source, and it uses ARkit right? And does not mention about support GLTF or similar files. Only that it supports USDZ.

[nicolocarpignoli]

@commentatorboy Apple said that but I personally verified and they confirmed that depends on hardware, so having iOS 12 is not enough.

https://twitter.com/jamesmccrary/status/1178766367614492674

But is not open source, and it uses ARkit right? And does not mention about support GLTF or similar files. Only that it supports USDZ.

Exactly. For now they are using proprietary file. Read on tweet above for more details

[nicolocarpignoli]

closed, see related PRs.