Watch the TTSizer Demo & See It In Action: (The demo above showcases the AnimeVox Character TTS Corpus, a dataset created using TTSizer.)

TTSizer automates the tedious process of creating high-quality Text-To-Speech datasets from raw media. Input a video or audio file, and get back perfectly aligned audio-text pairs for each speaker.

🎯 End-to-End Automation: From raw media files to cleaned, TTS-ready datasets
🗣️ Advanced Multi-Speaker Diarization: Handles complex audio with multiple speakers
🤖 State-of-the-Art Models - MelBandRoformer, Gemini, CTC-Aligner, Wespeaker
🧐 Quality Control: Automatic outlier detection and flagging
⚙️ Fully Configurable: Control every aspect via config.yaml

graph LR
    A[🎬 Raw Media] --> B[🎤 Extract Audio]
    B --> C[🔇 Vocal Separation]  
    C --> D[🔊 Normalize Volume]
    D --> E[✍️ Speaker Diarization]
    E --> F[⏱️ Forced Alignment]
    F --> G[🧐 Outlier Detection]
    G --> H[🚩 ASR Validation]
    H --> I[✅ TTS Dataset]

git clone https://github.com/taresh18/TTSizer.git
cd TTSizer
pip install -r requirements.txt

• Download pre-trained models (see Setup Guide)
• Add GEMINI_API_KEY to .env file in the project root:

GEMINI_API_KEY="YOUR_API_KEY_HERE"

Edit configs/config.yaml:

project_setup:
  video_input_base_dir: "/path/to/your/videos"
  output_base_dir: "/path/to/output"
  target_speaker_labels: ["Speaker1", "Speaker2"]

python -m ttsizer.main

pipeline_control:
  run_only_stage: "ctc_align"      # Run specific stage only
  start_stage: "llm_diarize"       # Start from specific stage  
  end_stage: "outlier_detect"      # Stop at specific stage

The project is organized as follows:

TTSizer/
├── configs/
│   └── config.yaml                 # Pipeline & model configurations
├── ttsizer/
│   ├── __init__.py
│   ├── main.py                     # Main script to run the pipeline
│   │── core/                       # Core components of the pipeline
│   ├── models/                     # Vocal removal models
│   └── utils/                      # Utility programs
├── .env                            # For API keys
├── README.md                       # This file
├── requirements.txt                # Python package dependencies
└── weights/                        # For storing downloaded model weights (gitignored)

This project is released under the Apache License 2.0. See the LICENSE file for details.

• Vocals Extraction pcunwa/Kim-Mel-Band-Roformer-FT by Unwa
• Forced Alignment: ctc-forced-aligner by MahmoudAshraf97
• ASR: NVIDIA NeMo Parakeet
• Speaker Embeddings: Wespeaker/wespeaker-voxceleb-resnet293-LM from Wespeaker