NanoMind
7B LLM @ 30+ TOK/s on Jetson Orin Nano
Overview
NanoMind is my personal edge AI project: running a 7-billion parameter LLM entirely on an NVIDIA Jetson Orin Nano with zero cloud dependency.
Tech Stack
Llama-3, Mistral, Phi-3
Jetson Orin Nano (8GB)
llama.cpp, CUDA, TensorRT-LLM
FastAPI + Web UI