info: some of the tutorial videos use version of programs which were in german at that time, but everything is set to english in the updated final version here

info: the videos are without voice, you should think of them as scrollable gifs and a visual guide where the informaton is in text (pause the video appropriately and try to follow along and redo on your own machine the steps if there is a need for it)

<details> <summary>tutorial videos in table with names and unique identifiers (overview for reference, dont watch this, these are just all the videos as an overview which are then used in the individual guides)</summary>

</details>

Main Content

Cards and selfhosted Browser Addons Demo

<details>

<summary>miningsimple Note</summary>

miningsimple Note: https://www.youtube.com/watch?v=evXtRlJ-Yww

<details><summary>see video table with github links</summary>

</details> </details>

<details>

<summary>Grammar Note</summary>

Grammar Note: https://www.youtube.com/watch?v=9j2wlW7fYvs

<details><summary>see video table with github links</summary>

</details> </details>

<details>

<summary>GrammarBrowserAddon</summary>

GrammarBrowserAddon: https://www.youtube.com/watch?v=k_bjdNjxrg4

<details><summary>see video table with github links</summary>

</details> </details>

<details>

<summary>MinedWordTextMarkerAddon</summary>

MinedWordTextMarkerAddon: https://www.youtube.com/watch?v=v6W_xkRYVak

<details><summary>see video table with github links</summary>

</details> </details>

Overview and Main Idea

The main idea of the system is to create cards for Anki with a good format for learning Japanese which were mined from either Lightnovels or Anime AND also at the same time to create subtitlefiles where the mined words are highlighted and translated, so you can watch the anime with this special subtitle file again (in case of lightnovels the "video" will be the audiobook played back and the subtitles the lightnovel text alligned to the audio). We prepared 2 main addons that can be used during the mining process, a grammar parser that marks selected grammar patterns and a vocabulary marker that marks all the words in the text that have been mined before at least once. In order to learn grammar more efficiently also a seperate notetype has been integrated in the system. The system has 5 main components:

Set Up:

In this part we will install the system on your computer

CardCreation:

We will demonstrate how to create cards, this step actualy consists of 3 subparts, in the first part the cards are mined from the source, in the second part we will use pyhton scripts in order to connect the mined vocabulary with the source: this means on the one hand that we will extract media from the source into our cards and on the the other hand that we will create a special subtitlefile so that the mined vocabulary with definition and pronounciation is inserted into the source and can be played with it, in the last part we will review the final product and explain some learning strategies.

MediaExtaction:

This is actualy the second part of the card creation, we will explain in detail how to use the Phtyhon Scripts in order to extract the media, i.e. inject the mined definitons into the source. We will also explain how we actualy obtain the source material and prepare it properly. We split this process in a Anime branch and a Lightnovel branch

Operating the Browser Addons:

We will explain in detail the basic capabilities and possibilites of the browseraddons we created ourselves which we are using during the minning process

Configuring the Grammar Addon:

This is an optional extension to the system and looks even deeper into the capabilities of the Grammar Addon and the connection with the Anki backend and provides different possibilites for further personalization.

Set Up

<details>

<summary>Set Up</summary>

The outline of the setup is as follows

1. Installation of external programs
2. cloning repository
3. Python Set Up
4. Anki and Addons Set Up

<details><summary>1. Installation of external programs</summary>

You have to download and install the following programs: necessary: 7Zip,Git,Anki,mpv Player,Chrome recommended:Deskpins, VisualStudioCode, Firefox

<details>

<summary>Installation of supplementary programs</summary>

<details> <summary>AnkiDownload</summary>

AnkiDownload

Short name ID of videos in playlist: A2

AnkiDownload: https://www.youtube.com/watch?v=al5T_EroVAg

<details><summary>see video table with github links</summary>

</details> </details>

<details> <summary>7zSetUp</summary>

7zSetUp

Short name ID of videos in playlist: A33

7zSetUp: https://www.youtube.com/watch?v=1bBnArzaJ5A

<details><summary>see video table with github links</summary>

</details> </details>

<details> <summary>DeskpinSetUp</summary>

DeskpinSetUp

Short name ID of videos in playlist: D2 D1

DeskpinSetUp: https://www.youtube.com/watch?v=oqho4W6O2VM

<details><summary>see video table with github links</summary>

</details> </details>

<details> <summary>GitSetup</summary>

GitSetup

Short name ID of videos in playlist: G14

GitSetup: https://www.youtube.com/watch?v=S82NWdfHD-8

<details><summary>see video table with github links</summary>

</details> </details>

<details> <summary>mpvinstallation</summary>

mpvinstallation

Short name ID of videos in playlist: M3 M2 M1

Important Note: sub-auto=exact (mpv config)

mpvinstallation: https://www.youtube.com/watch?v=HgIYkSLYEh4

<details><summary>see video table with github links</summary>

</details> </details>

<details> <summary>VisualStudioCodeSetUP</summary>

VisualStudioCodeSetUP

Short name ID of videos in playlist: V1

VisualStudioCodeSetUP: https://www.youtube.com/watch?v=AyQzpnptYJs

<details><summary>see video table with github links</summary>

</details> </details> </details> </details> <details><summary>2. cloning repository</summary>

After that you have to clone the repository with git, use the following command in your shell

Windows:

git clone --filter=blob:none --no-checkout https://github.com/nihonmiah71/miningsimple-system-overview.git

cd miningsimple-system-overview

git sparse-checkout set --no-cone "/*" "!/README.md"

git checkout

Linux / macOS / Git Bash:

git clone --filter=blob:none --no-checkout https://github.com/nihonmiah71/miningsimple-system-overview.git

cd miningsimple-system-overview

git sparse-checkout set --no-cone '/*' '!/README.md'

git checkout

<details>

<summary>Repository Cloning</summary>

Repository Cloning (and Anki Addons Installation)

You can skip the beginning of the video if you have already installed GIT and anki

Repository Cloning: https://www.youtube.com/watch?v=3se8GMVOHUY

<details><summary>see video table with github links</summary>

</details> </details> </details>

<details><summary>3. Python Set Up</summary>

Then you have to set up python, the following guide can be used as a reference, also an alternative playlist was created in case the guide did not work, in case the guide did not work we recommend to run the python scripts each one time and debug with AI the error message if the code did not work and install the necessary packages.

<details>

<summary>Python Installation Guide</summary>

<details>

<summary>Example Installation for Windows</summary>

Python Example Installation Windows: https://www.youtube.com/watch?v=KFX2cINnzR8

<details><summary>see video table with github links</summary>

</details> </details>

<details>

<summary>Notes</summary>

🐍 Global Python & System Dependencies Setup (No Virtual Environments)

This guide installs Python 3.12, FFmpeg, and all required machine learning and data packages globally on your system.

Because Anki addons and background tools invoke subprocess.Popen(["python", script_path]) directly from the OS environment, we do not use virtual environments (venv). Installing globally guarantees that scripts, Anki integrations, and terminal commands find all dependencies everywhere without silent missing-module failures.

⚠️ Important: After cloning the repository, ensure you navigate into the cloned folder (cd miningsimple-system-overview) before installing the requirements.

📦 Requirements File (requirements.txt)

Ensure requirements.txt exists in your project root with the following content:

torch
torchaudio
whisperx
pydub
pysrt
pykakasi
opencv-python
pandas

🪟 1. Windows (Windows 10 / 11)

Open PowerShell (Run as Administrator) and execute the following steps:

Step A: Install Python 3.12 & FFmpeg

1. Download and open the official Python 3.12 installer

(We launch the interactive installer to avoid silent background installation issues and ensure proper component selection)

Invoke-WebRequest -Uri "https://www.python.org/ftp/python/3.12.10/python-3.12.10-amd64.exe" -OutFile "$env:TEMP\python-3.12.10-amd64.exe"
Start-Process "$env:TEMP\python-3.12.10-amd64.exe" -Wait

In the Python Setup Window that appears:

1. ✅ Check the box: Add python.exe to PATH (at the very bottom).
2. ✅ Check the box: Use admin privileges when installing py.exe (if visible).
3. Click "Customize installation" (or "Modify" if already installed):
   • Ensure pip, tcl/tk, and py launcher are checked.
4. Click Next $\rightarrow$ Check "Install Python 3.12 for all users" $\rightarrow$ Click Install.
5. When finished, if you see "Disable path length limit", click it, then click Close.

2. Install FFmpeg

winget install Gyan.FFmpeg

3. Permanently Register Python to System-Wide PATH

(This ensures Python works across ALL user accounts, CMD prompts, and background tools like Anki)

$pyDir = "$env:LOCALAPPDATA\Programs\Python\Python312"
if (-not (Test-Path "$pyDir\python.exe")) { $pyDir = "C:\Program Files\Python312" }
if (-not (Test-Path "$pyDir\python.exe")) {
    $pyDir = (Get-ChildItem -Path "$env:LOCALAPPDATA\Programs\Python", "C:\Program Files\Python*", "C:\Users\*\AppData\Local\Programs\Python\Python312" -Filter "python.exe" -Recurse -ErrorAction SilentlyContinue | Select-Object -First 1).DirectoryName
}

if ($pyDir) {
    $machinePath = [System.Environment]::GetEnvironmentVariable("Path", "Machine")
    if ($machinePath -notlike "*$pyDir*") {
        [System.Environment]::SetEnvironmentVariable("Path", "$pyDir;$pyDir\Scripts;" + $machinePath, "Machine")
    }
    $env:Path = "$pyDir;$pyDir\Scripts;" + $env:Path
    Write-Host "✅ Python successfully registered system-wide: $pyDir" -ForegroundColor Green
    python --version
}

Step B: Install All Packages Globally

1. Ensure pip is active and updated

python -m ensurepip --upgrade
python -m pip install --upgrade pip setuptools wheel

2. Install PyTorch Globally (for CPU)

python -m pip install torch torchaudio

(OR for NVIDIA GPUs, run this instead):

<details> <summary>⚡ Click here for NVIDIA GPU CUDA command</summary>

python -m pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu121

</details>

3. Install all project requirements globally into the system Python

(Make sure your PowerShell is in your cloned project folder with cd)

python -m pip install -r requirements.txt

🛠️ Windows Troubleshooting

<details> <summary>🔍 Issue 1: <code>'python' is not recognized as an internal or external command</code> in CMD</summary>

If a new cmd window cannot find python, the PATH variable is missing from the global machine registry. Run this in an Administrator PowerShell:

$pyDir = (Get-ChildItem -Path "$env:LOCALAPPDATA\Programs\Python", "C:\Program Files\Python*", "C:\Users\*\AppData\Local\Programs\Python\Python312" -Filter "python.exe" -Recurse -ErrorAction SilentlyContinue | Select-Object -First 1).DirectoryName

if ($pyDir) {
    $machinePath = [System.Environment]::GetEnvironmentVariable("Path", "Machine")
    if ($machinePath -notlike "*$pyDir*") {
        [System.Environment]::SetEnvironmentVariable("Path", "$pyDir;$pyDir\Scripts;" + $machinePath, "Machine")
    }
    Write-Host "✅ Python added to Machine PATH: $pyDir" -ForegroundColor Green
}

Restart all open CMD/PowerShell windows after running this. </details>

<details> <summary>📦 Issue 2: <code>No module named pip</code></summary>

If python --version works but pip is missing or uninstalled, restore it using the built-in bootstrap:

python -m ensurepip --upgrade
python -m pip install --upgrade pip setuptools wheel

(Alternative manual bootstrap if ensurepip fails):

Invoke-WebRequest -Uri "https://bootstrap.pypa.io/get-pip.py" -OutFile "$env:TEMP\get-pip.py"
python "$env:TEMP\get-pip.py"

</details>

<details> <summary>🛑 Issue 3: Windows opens the Microsoft Store when running <code>python</code></summary>

Windows has dummy app aliases enabled by default. Disable them:

1. Open Windows Settings (Win + I).
2. Go to Apps $\rightarrow$ Advanced app settings $\rightarrow$ App execution aliases.
3. Toggle OFF both "App Installer (python.exe)" and "App Installer (python3.exe)". </details>

<details> <summary>🧹 Issue 4: Clean Uninstall / Reset (if previous installations are corrupted)</summary>

To cleanly wipe broken Python installations and start fresh:

# 1. Remove residual Python folders
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\Programs\Python\Python312" -ErrorAction SilentlyContinue
Remove-Item -Recurse -Force "C:\Program Files\Python312" -ErrorAction SilentlyContinue
Remove-Item -Recurse -Force "$env:APPDATA\Python" -ErrorAction SilentlyContinue
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\pip" -ErrorAction SilentlyContinue

# 2. Re-run installer in Step A1

</details>

🐧 2. Linux (Ubuntu / Debian / Pop!_OS / Linux Mint)

Note on Modern Linux (PEP 668): Modern distributions block global pip by default. We use --break-system-packages so all Anki addons and background daemons can access packages system-wide without venv.

Run the following in your terminal:

# 1. Update and install Python 3.12, Tkinter, FFmpeg, and OpenCV libraries
sudo apt update && sudo apt install -y software-properties-common
sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo apt update
sudo apt install -y python3.12 python3.12-dev python3.12-tk python3-pip \
                    ffmpeg libgl1 libglib2.0-0

# 2. Make 'python' and 'python3' resolve to Python 3.12 globally
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.12 1

# 3. Upgrade global pip
python3.12 -m pip install --upgrade pip setuptools wheel --break-system-packages

# 4. Install PyTorch Globally
# For NVIDIA GPU (CUDA 12.1):
python3.12 -m pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu121 --break-system-packages
# (OR for CPU only):
# python3.12 -m pip install torch torchaudio --break-system-packages

# 5. Install all requirements globally
python3.12 -m pip install -r requirements.txt --break-system-packages

🐧 3. Linux (Fedora / RHEL / Rocky)

# 1. Install Python 3.12, Tkinter, FFmpeg, and dependencies
sudo dnf install -y python3.12 python3.12-devel python3.12-tkinter python3-pip \
                    ffmpeg ffmpeg-devel mesa-libGL glib2

# 2. Make 'python' resolve to Python 3.12 globally
sudo ln -sf /usr/bin/python3.12 /usr/bin/python

# 3. Install Dependencies Globally
python3.12 -m pip install --upgrade pip setuptools wheel --break-system-packages
python3.12 -m pip install torch torchaudio --break-system-packages
python3.12 -m pip install -r requirements.txt --break-system-packages

🐧 4. Linux (Arch Linux / Manjaro)

# 1. Install system tools
sudo pacman -Syu --needed python python-pip tk ffmpeg mesa libglvnd

# 2. Install Dependencies Globally
python -m pip install --upgrade pip setuptools wheel --break-system-packages
python -m pip install torch torchaudio --break-system-packages
python -m pip install -r requirements.txt --break-system-packages

🍏 5. macOS (Apple Silicon M1/M2/M3/M4 & Intel)

Open Terminal and run via Homebrew:

# 1. Install Python 3.12, Tkinter, and FFmpeg
brew install python@3.12 python-tk@3.12 ffmpeg

# 2. Unlink old versions and force link 3.12
brew unlink python 2>/dev/null
brew link --overwrite --force python@3.12

# 3. Ensure 'python' and 'pip' point to Homebrew's 3.12 permanently
echo 'export PATH="$(brew --prefix python@3.12)/libexec/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

# 4. Install PyTorch and Requirements Globally
python3.12 -m pip install --upgrade pip setuptools wheel --break-system-packages
python3.12 -m pip install torch torchaudio --break-system-packages
python3.12 -m pip install -r requirements.txt --break-system-packages

✅ Verification Check

To confirm that the global system Python has all modules loaded and ready for Anki / command line execution, run this one-liner anywhere in your terminal or command prompt (without activating any environment):

python -c "import sys, cv2, pydub, pysrt, pykakasi, pandas, tkinter, whisperx; print(f'SUCCESS: Python {sys.version.split()[0]} has all packages installed globally!')"

Expected Output:

SUCCESS: Python 3.12.x has all packages installed globally!

🛡️ Why this fixes all edge cases across OSes:

1. Windows System-Wide Machine PATH: Writing Python to the Machine environment variable guarantees that every user account, standard CMD prompt, and background subprocess (e.g., Anki) finds python.exe and pip globally without missing path errors.
2. Interactive Visual Setup for Windows: Bypasses silent winget installer failures and ensures essential components (pip, tcl/tk, and py launcher) are cleanly registered.
3. Linux / macOS PEP 668 Fix (--break-system-packages): Modern versions of Debian/Ubuntu, Arch, Fedora, and Homebrew intentionally reject pip install outside a virtual environment with the error externally-managed-environment. Adding --break-system-packages forces pip to install the modules into the global system site-packages so Anki and subprocesses can access them globally from anywhere.
4. python -m pip instead of pip: Invoking python -m pip ensures that packages are placed into the exact Python version that python points to, eliminating discrepancies where pip points to an old Python version while python points to 3.12.

</details>

</details>

<details>

<summary>Alternative Playlist for Reference</summary>

You dont have to watch it if you already were succesful before in the python installation

Python3.12SetUp

Short name ID of videos in playlist: P3 P2 P1

Python3 12SetUp Alternative: https://www.youtube.com/watch?v=du50koQ6HqQ

<details><summary>see video table with github links</summary>