If you want to load an ONNX Deep Learning
model in Unity for your own game in this series, you will learn how! Keep watching! Hi there!, I’m Manuel Gutierrez from dlighthouse.co
and in this quick tips series, I will share snackable videos focusing on just one feature
of AEM, docker, TensorFlow, unity, javascript and more. So, if you are new here hit the subscribe
and the bell notification buttons below so that you don’t miss a thing! So let’s start!!! In a previous video, I showed you how to transform
a simple Keras model into ONNX In this video, I will create a simple Unity
game and show you how to load the model and use it
For the complete application, we will have two projects,
one using Python for transforming the current Keras model
and another one using C# for loading the model in Unity Let’s create a new Unity project for this
Create some folders for the model, scenes, and scripts
Copy the converted model into the assets/Keras/Model folder
You will need barracuda’s package installed, so, first, go to the package manager and look
for barracuda If like me, you were not able to find it,
open the project in your code editor inside the Packages folder
look for the manifest.json file then at the end, add "com.unity.barracuda":
"1.0.4" this will cause unity to download and import
the package If you select your ONNX model it will allow
you to configure some properties. You should check that there are no errors
or warnings being shown, for this simple model there shouldn’t be any. Let’s create a simple UI with
a text input that will be used for providing a single input value to the model
and a simple text for showing the returned value from the model
On the hierarchy window, create a game object for the canvas, and add
a panel with An input textfield from text mesh pro to enter
a number Which will prompt us to install TextMeshPro
essentials and also examples if you like Rename the new game object to input value
Create another game object of type text for the Predicted Value label
And a button to trigger the prediction and show its value
Finally, organize the UI Let’s create a MonoBehaviour that will take
care of controlling the UI, loading the ONNX model, and using it for making predictions
Create a MonoBehaviour and edit it in VSCode Add a using for TMPro for Text Mesh Pro
And also one for barracuda Add editor fields for the input text, the
text output, and the ONNX model Create variables for the runtime model, the
worker which is the one that will execute the predictions against the model and the
name for the output layer On the Start method,
Load the model from the assets Create the worker
And get the name for the output layer from the runtime model
Let’s create the method for performing the predictions using the model
try to parse the inputValue from the UI as an integer and if successful assign the value
into our number variable With the using statement, declare a new Tensor
that will be automatically disposed Assign the number we’ve just parsed to it
Start the prediction process And get the output from the model
Finally, assign it to the output prediction text
Inside the OnDestroy event function, dispose of the worker object
Go to unity, drag and drop the new KerasModel MonoBehaviour into the canvas, scroll down
on the inspector until you see it And drag and drop the Input Value Game Object
into the Input Value field The Predict Value into the Output Prediction
field And then from the model folder, drag the simple-model
asset into the Keras model field On the Predict button add a listener for the
OnClick event by clicking on the plus button drag the canvas object into it and from the
KerasModel Monobehaviour select the Predict method
Click on play, type a couple of numbers into the input field and click the Button to get
their respective outputs from the model Awesome! Congratulations and thanks for watching! You can join the free mini course associated
with this video, by clicking on the link in the description below
Share any comments or suggestions about the series or future topics
And if you liked the video also hit the like button below!
