Offworld Droid Engineer

 OUR OFF-WORLD DROIDS NEED YOU!

Off-World Droid Engineer (O.D.E) is an augmented reality application that helps users read and understand code by having them assume the role of a futuristic robot engineer. Turn dry code into FLY code as you tweak variables and observe the result as your robot explores a variety of simulated environments.

Description

O.D.E loads up a simulation by detecting a test environment image with your webcam. Tweak your droid inside the 3D rendered environment using the on-screen buttons and code display, and watch as the behaviour of the droid changes in real time.

Why do this? Coding is difficult – it is abstract and unintuitive. Coding interfaces are typically stark text boxes to be edited by mouse and keyboard, and don't engage with the human capacity and desire for visuo-spatial interaction. The abstract nature of coding, as well as its fundamentally dry user interface, limits the ability for human imagination to spur the learning process onward.

In O.D.E, the novel hook of using your device for droid training simulations encourages the user to let their imagination into the learning process. The interface is designed to be more human-friendly by focusing on the visual simulation of code and the relationship between code and the droid.

O.D.E uses Augmented Reality due to its synergy with the theme of the app. Because the user is role playing as an engineer – laden in sci-fi media with compact tools and holographic displays– it is appropriate that the user would do the same. The contextual and perceptual shift from ‘editing my code in an editor’ to ‘experimenting with a simulation overlaid over the real world’ adds believability and inspires the imagination.

Interaction Design

The on-screen user interface is the only means of interaction. In the prototype the available buttons are code display/hiding, and modifiers for droid speed, rotation, and movement. When a user presses these buttons, any displayed code will update and the droid will react accordingly. The user can smoothly move between experimenting with inputs and reading the denser source code. An on-screen UI provides ease of access and unobtrusiveness; during development I experimented with using real objects for some variables, but the effect of slowing down the process of changing variables and seeing an output harmed the user experience. On-screen dials to smoothly are particularly suited to floating point numbers, and will be implemented in a future release.

Storyboard

The learner finds themself in class, struggling with a text editor that they can't make sense of.

Is this really what they want to do? All this code just to print output to the console...

The teacher has a surprise: today, they'll be taking on the role of an off-world droid engineer! The teacher hands out sheets of paper to each student. Each piece of paper launches a simulation through their webcam. Fun!

The simulation comes to life! They see that they can control the droid with the buttons on the right. They start testing them out.

The learner decides to press the display code button. The code changes when they press the buttons. Ah! These buttons relate to floats and boolean values stored in variables.

Before long, the learner feels like they have a good grasp on how the code works. They're having fun and have started to imagine ways to test the droid even more. They're thinking like a programmer!

Technical Details

This application was built in Unity (2022.1.7f1) using the Vuforia Engine (10.8). The Simulation_Target image (see screenshots) must be detected by your webcam while the application is running. Upon detection, the scene and on screen user interface will appear in front of the image. 

3D Models

All the models in action:

The droid! Change its code and observe the result as it moves about the simulation.

Cave crystals of several different colours add unobtrusive lighting  and reinforce the 'alien cave' theme of the simulation.

Rocks help sell the idea of the cave simulation, and allow the user to experiment with navigation and collision.

References

Glowing crystal models were sourced from : https://assetstore.unity.com/packages/3d/environments/fantasy/translucent-crystals-106274

The 3D robot model, pre-made animations, and its animation transition source code (which I simplified to suit the requirements of the app), were from: https://assetstore.unity.com/packages/3d/characters/robots/robot-sphere-136226

Rock models in the scene were sourced from: https://assetstore.unity.com/packages/3d/environments/lowpoly-rocks-137970

User interface fonts were sourced from: https://assetstore.unity.com/packages/2d/fonts/fatality-fps-gaming-font-216954