Timeline
Apr 2022 (Presented at VRTO 2022)
Overview
I attempted to learn French through various methods when I arrived in Canada. I used popular apps like Duolingo, completed French grammar exercises, and watched French movies. Despite picking up some grammar and vocabulary, I found these methods inefficient. I discovered that traditional technological tools for language learning lacked real context. I believed immersion could dramatically speed up the learning process. So, I decided to create an app that could make language learning enjoyable and highly efficient for myself and other learners.
Tools used
Android Studio, Tensorflow Lite (Object Recognition, Translation, and Text-to-Speech Conversion).
Design
I sketched a few design concepts to visualize the user and how they could interact with the environment to learn languages.
Development
Things is an app built for Android smart glasses with a camera and a microphone. The app features real-time intelligence to help you learn languages in an immersive context. When users wearing smart glasses look at any object around them, they see what’s that object called in the target language on the HUD(heads up display) screen. It currently supports 3 languages (French, Italian, and Japanese) and allows the users can switch between languages by tapping on the capacitive touch panel on the glasses. The smart glasses display the translation of the object’s name and speak out the pronunciation as well.
Language learning tools integrated in the WIS Open source project
Things project on the Vuzix Blade smartglasses
Things project exhibited at VRTO 2022
