LLM Grader for Anki

An Anki desktop add-on that lets you answer flashcards by typing, speaking, or handwriting, and has an LLM of your choice grade the answer as correct, partly wrong, or totally wrong. It can also explain what you got wrong. It works for any subject (languages, biology, chemistry, IT, law…), because the grader reads the card itself rather than relying on exact string matching.

Answer under the question…	…and get graded

Requirements:

• Anki desktop 25.09 or newer on Windows, macOS or Linux. It's tested on Anki 26.09.
• One of: a free Google account, a local model through Ollama, or an API key from an AI provider. The setup wizard walks you through it.

Privacy: your answer and the card's text are sent to the AI provider you choose. Nothing is sent anywhere else. With Ollama, nothing leaves your computer. Voice is always transcribed locally. Google's free Gemini tier may use requests to improve Google's products, so use Ollama or a paid key for private material.

Features

• Providers: Anthropic (Claude), OpenAI, and any OpenAI-compatible server: Ollama, LM Studio, vLLM, llama.cpp, OpenRouter, or Google Gemini's OpenAI endpoint. On-prem models work offline.
• Answer modes
  • Type: Enter submits, Shift+Enter starts a new line.
  • Speak: records from the microphone you choose in Settings → Voice; that tab has a Test button with a level meter. Recording stops automatically when you stop talking. If the microphone delivers silence (muted, unplugged or a virtual device), you get a clear message with a Choose microphone… button instead of a made-up transcript. The audio is transcribed by the built-in Whisper, which runs offline on your computer and is free. You can edit the transcript before grading. Cloud speech-to-text (OpenAI) or your own Whisper server are optional.
  • Draw / handwrite: a pressure-sensitive canvas for a Wacom or other pen tablet, Surface Pen, or iPad + Apple Pencil through Sidecar/Duet. It has eraser support (including the pen's eraser end), undo, and palm rejection. The drawing is sent to a vision-capable model, which transcribes it and grades it. This works for kanji, chemical structures, formulas, and diagrams.
• Grading: you can set strictness (lenient, normal or strict), whether explanations are given (never, only when wrong, or always), the feedback language, the subject hint, and add extra instructions. Each of these can be overridden per deck.
• Read aloud: the verdict, the feedback and, optionally, the explanation are spoken after each graded answer. Playback is queued after the card's own audio, and 🔊 Listen replays it. It uses Anki's built-in TTS with your system's voices (Windows/macOS), so there's no download or account. It speaks in the feedback language when that voice is installed and falls back to English otherwise. Turn it off or change the speed in Settings → Input & review.
• Scheduling: each verdict maps to an Anki button (by default Correct→Good, Partly wrong→Hard, Totally wrong→Again). Space/Enter accepts the LLM's choice, and you can still press any other button. You can turn on auto-answer for correct cards.
• Structured output: uses JSON-schema output where the provider supports it. For servers without it, the add-on falls back to tolerant parsing.

Install

• From AnkiWeb: in Anki, open Tools → Add-ons → Get Add-ons…, paste the add-on's code, and restart Anki.
• From a file: download llm_grader.ankiaddon from the releases, double-click it (or use Tools → Add-ons → Install from file…), and restart Anki.
• From source: run python tools/build.py, which writes dist/llm_grader.ankiaddon.

For development (live-edit the source), link the add-on folder into Anki's add-ons directory:

# Windows, from the repository root
New-Item -ItemType Junction -Path "$env:APPDATA\Anki2\addons21\llm_grader" -Target "$PWD\llm_grader"

# macOS (Linux: ~/.local/share/Anki2/addons21)
ln -s "$PWD/llm_grader" ~/Library/Application\ Support/Anki2/addons21/llm_grader

Configure

On first start, a setup wizard opens (also available any time from Tools → LLM Grader → Setup wizard…). It offers three paths, and each one grades a test answer before it saves anything:

• Free, with your Google account. Opens Google AI Studio, where you create a key and paste it in. No credit card is needed. The wizard picks the newest free Flash model that works for your key and skips models that aren't on the free tier. Free-tier limits apply, and Google may use free-tier requests to improve its products; the wizard says so.
• Private, on this computer. Detects Ollama and offers to start or install it. It then downloads a handwriting-capable model sized to your RAM (qwen3-vl 8B or 4B instruct), with live progress.
• I have an API key. Claude, OpenAI, Mistral, DeepSeek, GLM or OpenRouter, with a "Get a key…" link for each.

Grading through the consumer ChatGPT or Claude.ai websites is deliberately not supported. Their terms forbid automated access outside the API, and it would also mean storing users' login sessions.

For everything else, open Tools → LLM Grader → Settings…:

Click Test connection to confirm it works.

Voice works without any setup. The first time you press Record, the add-on offers a one-time download. After that, everything runs locally and offline:

• Pinned downloads. The files come from the official whisper.cpp GitHub release (v1.9.4) and Hugging Face. They're pinned to exact versions and checked against SHA-256 before use.
• Resuming. Interrupted downloads continue where they stopped, including after an Anki restart.
• Where files go. They're stored in the add-on's user_files/whisper folder, which survives add-on updates. Delete downloaded files in Settings → Voice removes them.
• Background server. The add-on runs the Whisper server on 127.0.0.1 only. It starts loading the model while you're still speaking, and stops after 10 idle minutes and when Anki closes.
• Set the deck language. stt_language per deck (e.g. es) makes short answers about twice as fast and avoids wrong-language guesses on one-word answers.
• Platforms. Windows x64 and Linux x64 are fully automatic. On macOS, run brew install whisper-cpp once; the add-on finds whisper-server and downloads the model itself.

Per-deck profiles (Settings → Decks):

[
  {"deck": "Spanish*", "subject": "Spanish vocabulary", "strictness": "strict", "stt_language": "es"},
  {"deck": "Medicine::*", "subject": "Pharmacology", "explanations": "always"}
]

How it works

llm_grader/
  __init__.py          entry point (only runs inside Anki)
  reviewer_ui.py       hooks: injects the panel, handles pycmd messages, grades in background
  settings_dialog.py   Qt settings UI
  setup_wizard.py      first-run wizard (Gemini free tier / Ollama / API key)
  voice_recorder.py    microphone choice, level meter, auto-stop recording
  voice_local.py       built-in Whisper inside Anki: download prompt/progress, server lifetime
  web/grader.js|css    answer panel: textarea, recorder button, pen canvas
  core/                Anki-independent, unit-tested
    config.py          defaults, deck matching, per-deck profiles
    text.py            card HTML → text (strips FrontSide, marks cloze deletions)
    prompts.py         grading rubric + JSON schema
    providers.py       Anthropic / OpenAI / OpenAI-compatible over plain HTTP
    grading.py         call + robust verdict parsing → GradeResult
    stt.py             /audio/transcriptions client
    whisper_local.py   built-in Whisper: pinned downloads, resume, whisper-server process
    audio.py           WAV conversion, silence detection, auto-stop
    speech_out.py      what to read aloud, in which language
    setup_helpers.py   Gemini model choice, Ollama API, "is anything configured?"

• Hooks used: card_will_show appends the panel or the verdict to the card. webview_did_receive_js_message carries submissions. webview_will_set_content loads the JS/CSS. Reviewer.onEnterKey and _defaultEase are wrapped, so Space/Enter submits on the question side and accepts the grade on the answer side.
• HTTP: requests go through requests, which ships with Anki, so the add-on has no third-party dependencies.
• Untrusted input: the student's answer is treated as data. The prompt tells the model to ignore any grading instructions inside it.

Development

python -m unittest discover tests     # core tests, no Anki needed
python tools/build.py                 # package

End-to-end test in real Anki: tests/e2e/run_e2e.ps1 starts a throwaway Anki profile next to your normal one, along with a mock LLM server. It then drives a review session (typing, real key presses, pen drawing, voice, an error and retry, skip) and the settings dialog. With -WhisperMirror <folder of the pinned files>, it also runs real speech (Windows text-to-speech) through the built-in Whisper on CPU and GPU, including the first-use download. It writes report.json, screenshots, and the received drawings to %TEMP%\llmg-e2e\out.

Open tools/preview.html in a browser to try the panel and pen canvas without Anki.

Limitations

• Desktop Anki only. AnkiMobile (iOS) and AnkiDroid don't run Python add-ons. To use an Apple Pencil, run Anki on a Mac and use the iPad as a drawing tablet (Sidecar), or use a Windows pen tablet.
• Grading pronunciation isn't supported. Voice answers are graded on the transcript, and Whisper tends to write the word you meant even if you mispronounced it.
• Images on the cards themselves aren't sent to the model yet; only their text is. Image-occlusion and diagram-only cards therefore get a weaker reference answer.
• Each graded answer costs one API call. Local models are free but slower, and handwriting needs a vision model.
• API keys entered in the dialog are stored in plain text in the add-on's meta.json. Use environment variables if that matters to you.

Credits

• whisper.cpp (MIT) and OpenAI's Whisper models (MIT) handle speech recognition. They're downloaded from their official sources on first use and aren't bundled.
• Local grading models are provided by Ollama under their own licenses.
• Built for Anki (AGPL-3.0).

License

Copyright © 2026 the LLM Grader contributors.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License v3.0 as published by the Free Software Foundation. See LICENSE.