Sparky

Your own AI, on a USB stick.

Sparky runs open-weight models from a USB stick on Windows, macOS and Linux. There is nothing to install and no account, and once the stick is set up it needs no internet. You choose models by what you want them for, and a setup wizard picks ones that fit the stick and the computers you will use it on.

Get it on GitHub How it works

Free and open source under the MIT licence. Nothing you type leaves the computer.

Answered on the computer itself, from the stick. Then Ctrl+T moves to the next model on the stick; the toolbar shows which one answers next.

Who it is for

Anyone who wants an AI model they can carry, without a graphics card, a subscription or a fast connection, and without learning how local models work first.

You answer a few plain questions when you set up the stick. After that it is one file to run on whichever computer is in front of you.

No GPU, no model know-how
The wizard asks how much memory your computers have and picks models that will run on them. You never have to compare model sizes yourself.
Travel and offline work
On a plane or somewhere with no signal it works the same, because nothing is fetched while you use it.
Private documents
A medical letter or a contract you would not paste into a website stays on the computer you are using, and the conversation is saved only on the stick.
Classrooms and labs
One stick works on every computer in the room, whatever it runs, and nobody needs an account. Copy the stick to give a class the same models.
Coders on locked-down machines
Where you cannot install anything, code mode is a local coding agent that reads and edits files in a folder and runs commands after asking you.

Models small enough to carry are less capable than the large cloud services. They are slower on most laptops and make more mistakes, so check anything that matters.

Set up a stick

You need a USB stick and an internet connection once, for the downloads. After that the stick works offline.

  1. Format a USB stick as exFAT and name it Sparky. FAT32 cannot hold files over 4 GB, and most model files are bigger than that.
  2. Download the repository (the green Code button, then Download ZIP, or git clone) and run setup.bat on Windows or ./setup.sh on macOS and Linux. Both fetch a portable Python first if the computer has none.
  3. The wizard asks where to install, what you want it for, and how much memory your computers have (8, 16 or 32 GB). It recommends models that fit, shows their sizes, and downloads them to the stick. By default the stick also gets the runtimes for Windows, macOS (Apple Silicon and Intel) and Linux.
  4. On any computer: open the stick and run sparky.cmd. On Windows you can double-click it. 
# once, from the downloaded repository
setup.bat           # Windows
./setup.sh          # macOS and Linux

# then on any computer, from the stick
sparky.cmd          # Windows
./sparky.cmd        # macOS and Linux

# later
sparky.cmd models add <name>
sparky.cmd doctor   # check this computer
  • Use a USB 3 stick. A model is read from the stick each time it loads, and a 3 GB model takes well over a minute over USB 2.
  • Size. A small everyday model is about 3 GB and the large coding model about 19 GB, plus a runtime for each kind of computer. The wizard tells you what fits before it downloads anything.
  • Run it again to change things. sparky.cmd setup asks the questions again, and sparky.cmd models remove frees space.

Pick by purpose, not by model name

The wizard and sparky.cmd models recommend sort models by what they are good at. These are the current picks for each purpose; if your computers have less memory, you are offered a smaller model for the same purpose.

  • Everyday

    qwen3.5:4b

    Questions, explanations, emails and summaries. The one most people start with.

  • Code

    qwen3-coder:30b

    A coding agent for a folder of code. It needs about 20 GB of memory, so smaller computers get a smaller coder.

  • Write

    gemma4:12b

    Drafting and editing longer text, such as a letter or a report.

  • My documents

    qwen3.5:9b

    Answers from the files you put in the stick's context folder, and says which file each answer came from.

  • Images

    qwen3-vl:4b

    Reads screenshots, photos and scanned pages that you paste or attach.

  • Translate

    translategemma:4b

    Translation between many languages, from a model trained for that job.

  • Reason

    gpt-oss:20b

    Works through maths and logic step by step before it answers. Slower, needs 16 GB of memory, and /think shows its working.

  • Old computers

    qwen3.5:0.8b

    Not a purpose of its own: answer 8 GB to the memory question and you are offered smaller models, down to this one. Quick, and it makes more mistakes.

You are not limited to the list. Any model in the Ollama library, or a GGUF file on Hugging Face, can go on the stick, and every model is stored once for all the computers it runs on. 

sparky.cmd models add llama3.2:3b
sparky.cmd models add hf.co/<user>/<repo>
sparky.cmd models recommend --for code

Four ways to use it

Each is one command on the stick. On macOS and Linux, write it as ./sparky.cmd.

Chat in the terminal

The default. Replies stream in as they are written, formatted as Markdown, with the model and mode in the toolbar.

sparky.cmd

Chat in the browser

Opens a chat page in your browser, served from the stick to this computer only. The same models and modes, for anyone who would rather not use a terminal.

sparky.cmd web

One answer

Prints a single answer and exits. Text piped in is added to the question, so it works in scripts.

sparky.cmd ask "Summarise" < notes.txt

A model server for other apps

Starts only the model server and prints an OpenAI-compatible address that an editor plugin or your own script can use.

sparky.cmd serve
# http://127.0.0.1:11500/v1

In a chat

The same keys and commands work in the terminal and the browser.

Ctrl+T
The next model on the stick, for the next reply.
Ctrl+C
Stop the reply in progress. The conversation stays.
/model
Switch to a model by name. /models lists the ones on the stick.
/mode
Chat, code, write or study. Code mode edits files and runs commands, and asks before each command.
/pull
Download another model, while you are online.
/think
Show or hide the model's reasoning, for models that have it.
/paste
Attach an image from the clipboard for a vision model. /img takes a file path.
/resume
Carry on from the last conversation. /sessions lists the others.

How it runs

sparky.cmd
  shell script and batch file in one
 
portable Python + bundled Ollama
  this OS and chip, from runtime/
 
model weights in runtime/ollama/models
  one copy for every OS on the stick
 
terminal chat or browser chat
  or ask, or serve; all on 127.0.0.1
Windows, macOS and Linux.
Windows on x86_64, macOS on Intel and Apple Silicon, Linux on x86_64 and ARM. The stick gets all of them by default, and the launcher works out which one it is on.
Nothing is installed.
Python, the model server and the models all run from the stick. Some managed computers block programs on USB drives, and Sparky cannot get around that.
Settings stay on the stick.
HOME, configuration and caches are pointed at the stick while it runs, so nothing is left in your user folder.
Models are shared.
Each model is stored once. A model downloaded on Windows is the same file a Mac or a Linux machine loads; each OS only adds its own runtime.
Speed depends on the computer.
The model server uses a supported GPU when it finds one, and the processor otherwise. Small models are the ones that feel quick on an ordinary laptop.
Offline after setup.
Once the models and runtimes are on the stick, no connection is needed. The model server listens on 127.0.0.1 only.

Where your data goes

Nowhere. The model runs on the computer you plugged into, and the chat page and model server only answer on that computer. There is no account and no telemetry.

Sparky downloads in two cases only: during setup, and when you add a model or a runtime, from Ollama or Hugging Face.

Conversations are saved on the stick in data/sessions, as files you can read or delete. Anyone holding the stick can read them too, so look after it like any drive with private files on it.