Finn

Fairly Intelligent Non-humaN is a personal assistant that is
voice commanded through your Android phone.

Project Description

The Problem

Do you ever find yourself:

  • Wanting to know the temperature but don’t want to move?

  • Wanting to control your room/house with just a voice command?

  • Wanting to customize the things you are able to control in your house?

    “I wished I could customize the hardware in my Alexa/Bixby/Google assistant...”

As an engineering student, I have ran into these problems regularly and have based my project on solving a real world problem.

The Proposal

The solutions to all the problems - Finn:

  • Bluetooth connection for remote access

  • Extremely robust, easily customizable and trainable gives the option to include specific hardware to tackle specific tasks

  • Recognizes different phrases for the same intent, no matter the accent or phrasing thanks for Microsoft's Language Understanding AI.

  • Simple, small, and powerful

The Process

Finn was created using an Arduino Mega 2560 and C++. The companion app that goes with Finn was created using Android Studio, Java and Microsoft's Language Understanding AI.

My Role

Everything! I created the companion app using Java and Android Studio. I created the library to handle the different commands on Arduino, I programmed the bluetooth behaviour between the Arduino and companion app, I integrated Microsoft's language understanding AI (via querying and parsing JSON response), and I trained the Microsoft's language understanding AI.

Components

Opening Screen

The opening screen has a list of your phones paired bluetooth device. To access Finn, you'll have to tap "HC-06". That is the bluetooth module that connects the Arduino with the companion app.

Main Screen

The main screen allows users to interact with that Arduino via voice commands. This uses Google's Speech to text. The application will then use the text to query a response from Microsofts Language understanding. Depending on the response, different commands will be send to the Arduino via bluetooth.

Arduino Circuit

The following is the circuit setup for Finn. This setup currently supports LEDs, Temperature sensors, Servo Motors and HC-06 Bluetooth Module.

Currently Supported Features

Questions?

I'd love to hear your thoughts!