I was inspired by Google’s Quick Draw AI experiment (https://quickdraw.withgoogle.com) to design the first neural network handheld game console. This console runs a neural network on the chip, and there is no internet connectivity required. I have tried to have the game rules as close as possible to Google’s Quick Draw game.
Upon starting, the console shows a random keyword out of 100 categories and the user has 5 seconds to think and 20 seconds to draw the keyword on the screen. After 20 seconds system converts the user’s drawing to an image and feeds it to the neural network. If the drawing is recognized as one of the top five output probabilities, then the system considers it as a correct drawing for the keyword. STM32F429 ARM microcontroller is used as the main processor. It has 2 Mbytes of Flash memory and 256 Kbytes of RAM. The neural network only uses 1.42MB of flash and needs 103KBytes of RAM for inference.
I trained a Convolutional Neural Network with 100 categories from Google`s doodle drawings dataset using TensoFlow 2.0. The network accuracy for TOP5 category was 95%.
Using STM32CUBEMX.AI, the network is converted to a C code to be used as a part of the code for STM32. The code including AI and drivers is compiled using SystemWorkbench IDE and flashed to STM32F429I-DISC1 .
Python code for training: https://github.com/nimaaghli/quickdrawTF2
C code for STM32F429I-DISC1: https://github.com/nimaaghli/STM32AI_QuickDraw