TensorFlow Lite on Android

Today, I am going to walk through my process of moving my Keras trained model from my desktop onto my Android phone.

Why?

There are certain processes that I want to run against my model while I am away from my computer that I can’t do from the web. These range from some data collection to some larger data processing tasks. For example, I don’t want my web site to scrape all the spreads every time someone goes to it.

How I Got Here

After I created my deep learning model to predict NCAA basketball scores in Google Colab I decided I needed to deploy it a few more places. My first change was using TensorFlow.JS to deploy on the web. After that, I pulled the model down to my desktop and wrapped it in a larger .NET application. Today, I decided that I wanted to create an Android application that would allow me to have my model on my local device instead of calling out to the web. It will also allow me to run a more robust set of commands that I couldn’t do on the web.

TensorFlow Lite

Part of being a GDE allows me to get access to certain groups and people at Google. One of the groups that I am part of is the ML on Mobile group that works on TensorFlow Lite (link). We have had a few meetings and that was the final push I needed to carve out some time and do this project.

This library is fantastic. Since all of my stuff was done in TFv2 and Keras it would be a simple conversion and then learning the Java API calls.

Steps

First, I had to convert my Keras model to a TF Lite model. This was as easy as the following commands:

#Convert the model
converter = tf.lite.TFLiteConverter.from_keras_model(restored_model)
tflite_model = converter.convert()

#Save the TF Lite model.
with tf.io.gfile.GFile('model.tflite', 'wb') as f:
  f.write(tflite_model)

Second, I had to add that model to my ‘Assets’ folder in Android Studio.
Third, within Android Studio, I had to add code to the Gradle file to ensure the model doesn’t get compressed and to add the TensorFlow Lite libraries.

android{
...
    aaptOptions {
        noCompress "tflite"
    }

dependencies {
...
    implementation 'org.tensorflow:tensorflow-lite:2.2.0'

Now, I needed to actually create the code that will interact with my model.

Android Calling Code

The key class here is the Interpreter class. This is the class that takes in your model and runs all of the predictions. In my case, I fought this like crazy. It started with trying to figure out how to turn the stream I got from the Assets folder into a File object (I had to write it to local storage, fyi). Then, I had to figure out what the input and output objects were going to be.

I have a simple model in that it takes in 6 decimals and outputs a single number. So, my input was a simple float[] and my output was float[][]. Here is the code:

File mdl = CreateFile(); //Creates the TF Lite model if it doesn't exist
Interpreter intp = new Interpreter(mdl);
//Create a 6 element float array. NOTE: I needed to do some normalization.
float[] inputs = BuildInputArray( 72.1,63.8, -5.1, 65.1, 70.8, -2.3);
//Create the output array that returns a Tensor
float[][] out = new float[1][1];
//Run the prediction
intp.run(inputs, out);
//Close the model
intp.close();
//Get the results
float results = out[0][0];

Summary

Well, there are my simple steps to get your trained model onto a mobile device. In the future, I will be adding a bunch of features so that I can do all the work from my phone and not have to go to my desktop each day during the season to get my gambling picks.

Introducing FishButler!

At the start of the summer I wanted to do something that would keep my nephew (13) and daughter (11) productive and off of YouTube or Fortnite. My plan was to create a fairly simple Android application that they could help me with and get published before they went back to school. FishButler is that application.

I threw them into the deep end with full test coverage (unit and user) and source control. For testing, I used the built in testing frameworks in Android Studio. I had them create branches and tags and pull requests in GitHub to make sure they got to experience source control. I wanted them to get the full development life-cycle.

To get started, I took a day off of work and had them sit around a table with their computers and we had a mini hackathon. After that they went on their ways and handled the GitHub tasks I assigned them.

My nephew had done some intro programming but nothing at this level. My daughter had done nothing more than watching me. To help, I created 10 documents that covered coding standards as well as Git commands. For their tasks I did a fairly detailed write-up that they could see a high level overview of the step and then the commands to do them.

In the future, I would like to add some image recognition software and add in some other external items like Maps and reports. I will also have them do some more open ended research tasks to get a better feel of Android as a whole.