*your mileage may vary
In this tutorial, I’ll walk through how to start building an Augmented Reality Android application using Google’s new Sceneform SDK. First off, I’ll cover some quick concepts and then jump into project setup and some code. If you just want to dive in straight away, you can check out my GitHub repo below!
The screenshot below shows what we’ll build today!
ARCore is Google’s platform for building Augmented Reality experiences. It uses your mobile device for motion tracking, environmental understanding, and light estimation. With these three capabilities, ARCore can build an understanding of the real world around the device. With the mobile device’s position, we as developers can then augment the real world with information, 3D objects, or anything else!
Sceneform is a new SDK built to empower developers to work with ARCore without having to learn the intricacies of 3D graphics programming and OpenGL. ARCore was initially released with OpenGL samples, and as someone with very basic graphics programming knowledge, I found the learning curve to be super steep. With Sceneform, you can easily write in Java (or Kotlin) just like you would when building a normal Android app. The official samples for ARCore and Sceneform further illustrate this.
To get started, you’ll need Android Studio 3.1 or higher. We want to create a new project with Kotlin support targeting API 24: Android 7.0, so let’s start with an empty activity for now.
app/build.gradle file, we can add the dependency for Sceneform— remember to click “Sync Now” to update the project. Although we’ll be writing this tutorial in Kotlin, Sceneform uses some language constructs from Java 8, so we’ll need to explicitly add support since our minimum API is less than 26.
Since this is an augmented reality app, we’re going to have to ask the user for access to the camera. We also need an AR permission, which designates that this app requires ARCore. Finally, there’s a specific AR meta tag used by the Play Store to filter out applications for users who do not have an ARCore-enabled device.
To make life easier, and to allow us to import our own 3D assets, we’ll also include the Sceneform plugin for Android Studio. You can install this by selecting preferences, plugins, browse repositories, and search for Google Sceneform Tools (Beta)
The first thing we want to do is get a 3D model to use—if you don’t want to make your own, you can head on over to Poly and grab one for free (just remember to credit the creator!). The Sceneform plugin supports OBJ, FBX and glTF file formats. Then, it’s just a case of creating a
sampledata/ directory within your app and adding your object file.
Note that the purpose of this folder is for design time only, and these resources won’t be added to the final APK. The next step is right clicking your downloaded object and selecting “Import Sceneform Assert”. For this app, I’m going to pick the Earth asset created by Google.
After this gradle task has finished, you’ll find a reference to your object in the app level gradle file and a
.sfb file inside the assets directory. This Sceneform Binary asset is what we’ll use inside our app. The Sceneform Android Studio plugin lets us view a realtime preview of our newly created asset.
The next step is to add the Sceneform fragment to our layout. You can just replace the default textview with this element. We’ll also add a Floating Action Button (FAB) to enable us to add our model to the real word.
We want our users to be able to add a model only when ARCore has detected a scene. In order to do this, we’ll only display the FAB when a suitable plane has been detected. In the official Google Codelab, they achieve something similar by drawing a custom Pointer on the screen when a model can be added to the scene.
The structure of our main activity is as follows:
- Detect a suitable plane using ARCore
- Show the user the Floating Action Button (FAB)
- On click of the button load in our model
- Use the
.sfbfile to create a renderable object
- Attach our renderable to the scene and display to the user
And that’s it! In less than 150 lines of code, we have our very first augmented reality app. I’ve added comments where applicable in the MainActivity.kt. Please feel free to get in touch with me if you have any questions whatsoever. Hopefully you can use this as a starting point for your AR experience on Android. Thanks for reading!
If this quick tutorial has piqued your interest, I’d recommend reading the below links to find out more about Android and AR.