Building an Automated YouTube Video Summarization System with Whisper, Claude API, and Python

Artificial Intelligence tutorial - IT technology blog
Artificial Intelligence tutorial - IT technology blog

Last week I spent nearly 3 hours watching and taking notes on a 15-video YouTube playlist about AI. Looking back at that messy pile of notes, I asked myself: why not write a Python script to do this automatically? After two evenings of work, I had a system that could transcribe videos, summarize the content, and even write it up into a complete blog post.

In this article I’m sharing exactly what I did — from installation to getting the first result.

Up and Running in 5 Minutes — Let’s Go

You’ll need Python 3.9+ and an Anthropic API key. Try running it first, understand it later.

Step 1: Install Dependencies

pip install openai-whisper yt-dlp anthropic
  • openai-whisper: OpenAI’s speech-to-text model, runs completely locally, free to use
  • yt-dlp: tool for downloading audio from YouTube (an actively maintained fork of youtube-dl)
  • anthropic: Python SDK for calling the Claude API

Step 2: Complete Script to Try Out

import subprocess
import whisper
import anthropic
import sys

def download_audio(url: str, output: str = "audio") -> str:
    subprocess.run([
        "yt-dlp", "-x", "--audio-format", "mp3",
        "-o", f"{output}.%(ext)s", url
    ], check=True)
    return f"{output}.mp3"

def transcribe(audio_path: str) -> str:
    model = whisper.load_model("base")  # base: fast; small/medium: more accurate
    result = model.transcribe(audio_path)
    return result["text"]

def summarize_to_article(transcript: str, topic: str) -> str:
    client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from env
    message = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=2048,
        messages=[{
            "role": "user",
            "content": f"""Based on the transcript from a YouTube video about "{topic}",
write a complete, clear blog post with H2/H3 structure,
including code examples where applicable. Get straight to the point — no lengthy intro.

TRANSCRIPT:
{transcript[:8000]}"""
        }]
    )
    return message.content[0].text

if __name__ == "__main__":
    url = sys.argv[1]
    topic = sys.argv[2] if len(sys.argv) > 2 else "technology"

    print("Downloading audio...")
    audio = download_audio(url)

    print("Transcribing (1-2 minutes for a 10-15 minute video)...")
    transcript = transcribe(audio)

    print("Writing article with Claude...")
    article = summarize_to_article(transcript, topic)

    with open("output.md", "w") as f:
        f.write(article)
    print("Done! Article saved to output.md")

Try It Out

export ANTHROPIC_API_KEY="sk-ant-..."
python main.py "https://youtube.com/watch?v=VIDEO_ID" "Docker container"

If you see the output.md file appear with blog content inside, you’ve made it past the first hurdle.

Deep Dive — Why Use Whisper Instead of YouTube Subtitles?

This is a question I get a lot. YouTube’s auto-generated subtitles are often riddled with typos, lack punctuation, and many technical videos simply don’t have subtitles in certain languages. Whisper processes audio directly, making it significantly more accurate — especially with regional accents and specialized IT terminology.

Choosing the Right Whisper Model

Whisper comes in 5 sizes. I typically use small for videos under 30 minutes:

  • tiny: ~39M params — fastest, least accurate
  • base: ~74M params — good balance for demos and testing
  • small: ~244M params — the sweet spot (my daily driver)
  • medium: ~769M params — high accuracy, needs a decent GPU
  • large: ~1.5B params — best quality but slow on CPU

If your machine has an NVIDIA GPU, Whisper will auto-detect CUDA and run 5–10x faster. It still works on CPU — just takes longer.

Context Limit Issues with Long Videos

A one-hour video can produce 15,000–20,000 words of transcript. Claude Sonnet supports up to 200K tokens, but to keep costs down I usually chunk the transcript into 6,000-word segments, summarize each segment individually, then synthesize them into a final article.

Advanced — Handling Long Videos and Playlists

Smart Transcript Chunking

def chunk_transcript(text: str, chunk_size: int = 6000) -> list[str]:
    words = text.split()
    return [" ".join(words[i:i + chunk_size]) for i in range(0, len(words), chunk_size)]

def summarize_long_video(transcript: str, topic: str) -> str:
    client = anthropic.Anthropic()
    chunks = chunk_transcript(transcript)
    summaries = []

    for i, chunk in enumerate(chunks):
        print(f"Summarizing part {i+1}/{len(chunks)}...")
        # Use Haiku for intermediate steps — 10x cheaper than Sonnet
        resp = client.messages.create(
            model="claude-haiku-4-5-20251001",
            max_tokens=1024,
            messages=[{"role": "user", "content": f"Summarize this section briefly (100-200 words):\n\n{chunk}"}]
        )
        summaries.append(resp.content[0].text)

    # Synthesize into a complete article using Sonnet
    combined = "\n\n".join(summaries)
    final = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=3000,
        messages=[{"role": "user", "content": f"Write a complete blog post from these summaries about '{topic}':\n\n{combined}"}]
    )
    return final.content[0].text

Processing an Entire Playlist

# Get all URLs from the playlist
yt-dlp --flat-playlist --print url "https://youtube.com/playlist?list=PLAYLIST_ID" > urls.txt
with open("urls.txt") as f:
    urls = f.read().splitlines()

for url in urls:
    print(f"\nProcessing: {url}")
    audio = download_audio(url, output=f"audio_{abs(hash(url))}")
    transcript = transcribe(audio)
    article = summarize_to_article(transcript, "AI and Machine Learning")
    with open(f"article_{abs(hash(url))}.md", "w") as f:
        f.write(article)

Practical Tips from Daily Use

In practice, this has become one of the most valuable skills I’ve built into my workflow — not because it sounds impressive, but because it genuinely saves several hours every week when I need to research video-heavy material.

1. Cache Transcripts to Avoid Re-transcribing

Whisper can take a while on long videos. Save the transcript to a file so you can reuse it:

import os

def get_or_transcribe(audio_path: str) -> str:
    cache_file = audio_path.replace(".mp3", "_transcript.txt")
    if os.path.exists(cache_file):
        with open(cache_file) as f:
            return f.read()
    transcript = transcribe(audio_path)
    with open(cache_file, "w") as f:
        f.write(transcript)
    return transcript

2. Reduce API Costs

Use Haiku for intermediate summarization steps and reserve Sonnet for the final article-writing pass. This cuts costs by around 60–70% with virtually no impact on final output quality. When you’re processing 50–100 videos a month, that adds up fast.

3. Force Language When Needed

result = model.transcribe(audio_path, language="vi")  # Force Vietnamese
result = model.transcribe(audio_path, language="ja")  # Force Japanese

4. Prompts for Specific Use Cases

PROMPTS = {
    "blog": "Write a complete technical blog post with H2/H3 headings and practical code examples...",
    "notes": "Summarize into bullet points, highlighting key takeaways and action items...",
    "linkedin": "Write a 200-300 word LinkedIn post, professional tone with real-world insights...",
    "quiz": "Create 5 multiple-choice questions to test comprehension of this video's content...",
}

5. Legal Considerations

This system is best suited for content you have the right to use: your own videos, conference talks with a Creative Commons license, or courses you’ve purchased. Don’t use it to copy commercial content — most platforms have ToS that explicitly restrict automated downloading.

This system doesn’t replace reading the original source — it gives you a draft that’s about 80% there, and you’ll still need to polish it before publishing. But instead of staring at a blank page every time, you have a solid starting point. For me, that’s the difference between “writing 3–4 articles a week” and “never finding the time to write.”

Share: