Home
mnemorai is inspired by the method detailed in the paper SmartPhone: Exploring Keyword Mnemonic with Auto-generated Verbal and Visual Cues by Jaewook Lee and Andrew Lan. The aim is to recreate their approach using accessible, open-source models. The pipeline they propose, as shown below, serves as the blueprint for our project. It illustrates the process of automating language learning, blending cutting-edge AI techniques with proven language learning methodology.
The SmartPhone paper uses the ideas from the 2014 paper TransPhoner: Automated Mnemonic Keyword Generation. This paper is the foundation of the methods they use in SmartPhone but does not use AI for the creation of mnemonics. In the image below you can see how it creates phonetically similar mnemonics for a given word, such as ratatouille.
mnemorai is meant for people who want a more efficient way of learning vocabulary when learning languages.
In this section, we will describe the technical process of how mnemorai creates the mnemonics and the images.
We start by asking the LLM for candidates, given the foreign word. This foreign word is also split up on syllables so that we can have more candidates. The candidates based on the syllables are then combined and ranked according to their frequency in English.
To determine the best mnemonic candidate, we ask the LLM to choose the most fitting one, given a list of candidates.
Just like the previously mentioned papers, the best mnemonic word is passed to an LLM that creates the input for the text-to-image model. The output of this LLM is then passed to the text-to-image model which then generates the image.