Skip to content

This is a sample program that recognizes hand signs and finger gestures with a simple MLP using the detected key points. Handpose is estimated using MediaPipe.

Notifications You must be signed in to change notification settings

mfabdullrahma/ECS-170-Group-Project

 
 

Repository files navigation

Where to download dataset

https://www.kaggle.com/datasets/grassknoted/asl-alphabet

Set up Virtual Environment

Ensure to set up your virtual environment and install requirements

How to train models

Step 1 - Preprocess Data

python training/preprocess.py

Step 2 - Train Models

python training/train_mlp.py
python training/train_lstm.py --augment-only

Step 3 - Export to TensorFlow.js

python training/export_models.py

Step 4 - Copy to frontend

cp -r models/tfjs/* frontend/public/models/

Running Frontend

pnpm install
pnpm run dev

About

This is a sample program that recognizes hand signs and finger gestures with a simple MLP using the detected key points. Handpose is estimated using MediaPipe.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 63.7%
  • Python 18.8%
  • TypeScript 17.1%
  • Other 0.4%