Rapid Game Development With Tiltbrush Toolkit
TiltBrush Toolkit enables exporting of 3D assets created in Google's TiltBrush to FBX files, which can then be imported easily into Blender for rigging and animation, or directly into Unity. This provides solo game developers with the means to create high-quality fully 3D game worlds and characters incredibly fast, which is extremely useful for fast-iteration prototyping or indeed final asset creation.
In this beginner's guide I'll run through my experience of getting a sketched idea to a basic playable demo like this one in a couple of afternoons:
What You'll Need
- A HTC Vive
- Google's TiltBrush: Available on Steam for ~$30
- Tiltbrush Toolkit: Available here for free
- Unity: Available here for free
- Blender: Available here for free
Painting
The first step in this process was to paint the models and environments needed for my demos. The design is entirely up to you, and I personally found it easy to draw from imagination in TiltBrush. However, if you prefer to work from a prepared sketch or image, that can be done using the Media Library option shown here:
I won't go into too much detail about the painting itself - the best thing about this process is you do not need to be an extremely talented or experienced 3D artist to produce usable models. However, one tip is to use a different brush type for surfaces which you want to be collidable in your game (e.g. floors, walls). This isn't essential but makes the Importing to Unity step a little faster.
Once you have a model you're happy with, simply export it using the option in the Labs menu:
Make sure to also save your sketch, and that's it! You're done with the VR portion of the process. The assets you've created can now be imported straight into Unity, or be passed through Blender if they need to be animated.
Animating in Blender
Note: This is the slowest and most manual part of the process. For the first time I would recommend using models that don't require animation to look good - a very good reason to use robot characters. Keep an eye out for a future animation tool that makes use of VR - it's only a matter of time!
When exporting from TiltBrush, an fbx file should have been created in a folder like Documents\Tilt Brush\Exports\Untitled_0. This file format can be easily imported to Blender by going to File->Import->FBX:
Details on how to rig and animate the imported model are outside the scope of this guide (and my area of expertise), so to start I'd recommend this excellent tutorial by Mark Masters at Pluralsight.
Tip: You may notice your model's colours are missing once you've exported into Blender. This is just based on the render mode you're using - the colours are stored correctly. However, once you export to a .fbx or .blend file, and import to Unity, there may be an extra step required to get the colours back onto the model. This is covered in more detail below.
Importing to Unity
Thanks to TiltBrush Toolkit, this is an easy step. In a fresh (or existing) Unity Project, make sure to first import the package downloaded at the link above. This is done by going to Assets->Import Package...->Custom Package..., then selecting the downloaded .unitypackage and importing everything.
Next, simply click-and-drag your FBX file produced by TiltBrush, like so (click to fullscreen):
Your stationary model is now in Unity as a GameObject, with all the required meshes and textures. Note, if you exported an animated model from Blender, you may notice your textures are missing. I was able to fix this by updating the Shader to match that of the corresponding mesh on the TiltBrush export. In the case below, I changed Shader from Standard to Brush/Standard.
Left: Imported from fbx produced by TiltBrush. Right: Imported from fbx produced by Blender
Fixed by changing the Shader to match the TiltBrush output
Models imported from Blender will have all their animations included in the prefab, which Unity will recognise. I recommend taking a look at Unity's official Animation tutorials if you haven't already.
Collisions and Scripting
Finally, I'll go through a very fast process I put together for getting your character controllable, and colliding with other assets you may have imported (preferably a floor). For my purposes, I've placed the robot in a space station he can walk around:
First, we need to make sure that the floor is collideable. This simply requires that you select the parts of the environment you want to collide with, and add a MeshCollider. That's it! What makes this even simpler is if you separated your collidable and non-collideables in TiltBrush by using different brush types. If you have issues where some meshes are being selected when you don't want them to be, you can try separating them by colour, which is an option under Tilt Brush -> Labs in the main Unity menu bar.
Next, you need to add a CharacterController, which is a nice in-built Unity component which will allow your character to fall down onto a collidable mesh, and move along it. Once added to your model, the only thing you'll need to do is adjust the location and shape of the shape it uses for collision detection so that it roughly overlaps your character.
Character Controller after adjustment of Center, Radius and Height.
The only remaining thing to do is hook up some controls to your character. I've prepared a basic C# script which will take default mouse and keyboard input to control a character. This can be found in a zip file here. (Note: this is a variation on a commonly-used script, also seen for example here).
Simply add this script to your character, and if you'd like to have your main camera track the player, drag it from the scene into the Main Camera slot in the script. You should hopefully end up with something like this when running the game:
And that's it! This should be all you need to get a very basic prototype game working, with full 3D assets. Please feel free to build on the methods above - this is just what I figured out in a few evenings of playing around with the technology. If you happen to find any more tips and tricks, please pass them on to me and I'd be happy to update the guide.
Thank You
Thanks for taking the time to read this guide! And thanks to the cool devs in GoogleVR, Blender and Unity for enabling all this! If you'd like to hear more from me and be informed about future guides, please follow me on twitter :)