Just another AR library?
Its most useful features are:
- Performance, ~60 FPS on two year old phones (!)
- Cross Browsing, works on every phone browser (and obviously desktop) supporting webgl and webrtc (so basically every Android phone and iPhones above iOS 11)
- Simplicity, it’s a wrapper of different frameworks that makes web AR developing very trivial. It is built on top of a-frame and three.js.
So we don’t need Hololens, Cardboard or expensive phones; we just have to reach an AR.js based webapp on the web and then we can experience ar.
How is that possible? Well, turns out AR.js works with markers (not only, but if we wanna be cross-browser, we have to use markers).
Markers are a sort of simplified qr-codes. On AR.js we defined specific 3D scenes for specific markers, so when the camera recognize a marker, the web-app will show the 3D model on top of it (or near, it can be customized). Also, markers can contain qr-codes.
An important feature of AR.js is the possibility to use custom markers, which their default form is the ‘pattern’ one.
I search a lot for limitations of pattern markers, their size and shape. Turns out that:
- They can have a maximum of 16×16 px of resolution
- They have to be squared
- They cannot have white/transparent part, only light grey parts (something like F0F0F0 color)
- They cannot contain colors, only black and light grey
- They have to contain simple text, better if only one letter or number, or a symbol.
Custom markers can be generated using this tool. If you analyze the “.patt” output file, you will see that patterns describes the given image as a bunch of characters. So, for good results, it’s better to avoid inputs like similar images for different markers, too complex images and images showing words.
Markers can also be barcodes. They represent a number as a symbol, created with calculus on a matrix. It is better to choose barcodes based on a matrix that has an high level of hamming distance (see this table), so the camera can recognize them better. Barcodes markers, based on their matrix, have a specified maximum number of possible generated markers (for example the “AR_MATRIX_CODE_3x3_PARITY65” matrix can generate 32 different barcode markers).
Keep in mind to avoid, if possible, white/transparent color also in barcode markers (this is not confirmed on barcode markers but it’s based on empirics).
This is the online generator of barcode markers used for the following example.
I will show you now some basic code from AR.js documentation and blog, with some workarounds, as currently there are some open issues regarding custom markers.
In this simple HTML, we
- Import libraries (it’s important to use 1.5.5 version of AR.js)
- Tell AR.js to recognize barcode patterns that are generated with a 3×3 matrix
- Define different
that shows different 3D boxes according to their pattern. Blue and red patterns are very similar, and they will confuse the camera — later I will show you why
- Define another
, this time based on barcode
- Define a static camera entity.
To avoid problems, please use specific tag and attributes as showed, as for now (April 2018) this example works and it’s based on the new released version of AR.js (1.5.5). Also it’s better to use remote urls for pattern resources, and not relative ones.
Patterns in detail
‘Blue image’ and ‘red image’ generated patterns are very similar. If you check the following images, you can see that the pattern generated from the image is a bunch of numeric values representing the symbols from the given image. If the image is complex, the pattern will be unrecognizable. Also, the more the given images are similar for different patterns, the more a camera will fail on recognizing one or another.
Keeping in mind these simple rules for create barcode/pattern markers makes a very trivial job for the phone camera to recognize them, so we can build stable and reliable ar apps.
So, it’s AR.js worth it?
I will answer “absolutely”. It’s semplicity is stunning. Marked based nature is not a real limitation but rather a paradigm that suites well for an incredible range of use cases. Markers can be showed on mobile and desktop displays or printed and hanged up in walls or boards.
As AR.js shows us, it’s not necessary to have another device over a mobile phone to experience ar, nor download a third party app, that a user will surely uninstall after the experience, or that probably he will never download. Delivering augmented reality on the web, and make it easily suitable for everyone (literally!) is the real key to make this technology pervasive