Intelligent Adaptive Batch Processing (Coming in v1.2.5)

[cyberofficial]

Hello everyone!

I'm incredibly excited to share a preview of a major new feature coming in version 1.2.5: Intelligent Adaptive Batch Processing.
As a solo developer, I'm always looking for ways to make Synthalingua more powerful and efficient for you, and this update is a huge step forward in that direction.

Let's talk about the biggest bottleneck when using Synthalingua: waiting.
If you've ever generated subtitles for a long video, you know that it can take a while. Your powerful GPU might be working hard, but often, your CPU is sitting by, mostly idle. That's a lot of wasted processing power.
This new feature is designed to fix that and dramatically cut down your waiting time.

---

The Problem: Wasted Power and Long Waits

Until now, when you generated subtitles, the transcription jobs were processed one after another, primarily on your GPU.
While fast, it wasn't optimal. It was like having a professional chef (your GPU) cooking a multi-course meal alone, while a capable assistant (your CPU) just watched.

This could lead to long processing times, especially for multi-hour videos, and sometimes even memory issues on very large files.
I knew there had to be a smarter way to use all the resources your computer has to offer.

---

The Solution: Adaptive Batch Processing

The new Adaptive Batch system is a smart scheduler that turns your entire computer into a subtitle-generating powerhouse.
It intelligently divides the work between your GPU and CPU, making sure both are working together to get the job done as fast as possible.

See It in Action: Side-by-Side Comparison

But don't just take my word for it. See the difference for yourself.
The video below shows a side-by-side comparison of the same file being processed with the old method versus the new Adaptive Batch mode.

Synthalingua.mp4

Here's what you're seeing:

• On the left: The old "Single Mode" method, processing jobs one by one on a single GPU.
• On the right: The new "Adaptive Batch" mode, running two jobs on the GPU and three jobs on the CPU, all at the same time.

The results speak for themselves:

• Single Mode Finish Time: 3 minutes and 25.6 seconds
• Adaptive Batch Finish Time: 2 minutes and 36.6 seconds

By using the hardware that was previously sitting idle, the new adaptive mode finished the job nearly a full minute faster — that’s a speed increase of almost 24%.

---

How It Works

The system is designed to be smart and automatic:

1. It Analyzes the Workload
   First, it scans your audio or video file and identifies all the segments that actually contain speech, using the existing Silence Detection feature.

2. It Learns Your System
   It runs a few small test jobs to figure out how fast your GPU and CPU are.
   It learns your unique hardware's performance characteristics.

3. It Distributes Jobs Intelligently
   This is where the magic happens. The system sorts all the speech segments and assigns them to the best processor for the job:

   • Long, difficult segments are sent to your powerful GPU.
   • Short, easy segments are sent to your CPU to process in parallel.

---

The "Endgame Strategy" – A Smart Finish

To make sure the process doesn't get stuck waiting for a slow CPU job at the very end, the system includes an endgame strategy.
Once the transcription is about 80% complete, it stops sending new jobs to the CPU.
Your faster GPU then takes over to finish the remaining tasks, ensuring a smooth and predictable completion time.

---

Why This Matters: More Productivity, Less Waiting

• Massive Speed Improvements:
  As you saw in the video, using your GPU and CPU together can drastically reduce processing times.

• Maximized Efficiency:
  You're getting the most out of the computer you paid for — no more idle hardware.

• Greater Stability:
  By breaking large files into smaller, managed chunks, there's less risk of running out of memory, which means fewer crashes when working with multi-hour videos.

• It’s Automatic:
  You don't need to be a hardware expert. The system automatically detects your VRAM and CPU cores to figure out the best way to split the work.
  You'll just have to input a few variables first, which are explained in the GUI.

---

I'm really proud of this update and I believe it will make a huge difference in your workflow.
Thank you so much for your continued support — it allows me to work on exciting new features like this.