YouTube Transcripts for RAG
I wanted to ask Andrej Karpathy's YouTube channel questions and get back the exact moment in the video where he answers them. Not a summary. A link that starts the video at the right second.
His three most recent videos together are almost 10 hours long. This post shows how I turned them into a searchable index in about 60 lines of Python: 15,285 caption segments, 598 one-minute chunks, and a search that answers "what is flash attention" with &t=7241s.
Full disclosure: I built the transcript scraper used here and sell it on the Apify Store. Everything below is a real run from 20 October 2026, with the real output.
For one video, you can open the transcript panel and copy it. For a whole channel, the usual paths are:
The input takes a channel handle. maxVideos limits how many of its newest videos you get:
{ "channelUrls": ["@AndrejKarpathy"], "maxVideos": 3, "transcriptFormat": "both" } Enter fullscreen mode Exit fullscreen mode transcriptFormat: "both" returns the timestamped segments and the joined plain text. For RAG you want the segments, because they carry the start time of each line. You can also pass videoUrls, playlistUrls, a dateFrom and dateTo window, and a language code. If a video has no captions in that language, YouTube translates them, and wasTranslated tells you so.
{ "videoId": "l8pRSuU81PU", "url": "https://www.youtube.com/watch?v=l8pRSuU81PU", "title": "Let's reproduce GPT-2 (124M)", "channelName": "Andrej Karpathy", "publishedAt": "2024-06-09T16:31:35-07:00", "durationSeconds": 14486, "viewCount": 1212773, "language": "en", "availableLanguages": ["en"], "isAutoGenerated": true, "wasTranslated": false, "status": "success", "transcript": [ { "start": 0.16, "duration": 4.52, "end": 4.68, "text": "hi everyone so today we are going to be" }, { "start": 2.36, "duration": 3.64, "end": 6, "text": "continuing our Zero to Hero series and" } ], "transcriptText": "hi everyone so today we are going to be continuing our Zero to Hero series and ..." } Enter fullscreen mode Exit fullscreen mode That one video has 6,049 segments and 223,000 characters of text. Each row also includes the description (with the chapter list, when the creator wrote one), likeCount, commentCount, keywords and the thumbnail.
Notice isAutoGenerated: true. All three videos use YouTube's automatic captions, so the text has no punctuation and has some misheard words ("chpd" for ChatGPT). Embeddings handle that well. A keyword search like the one below is more sensitive to it.
Most RAG tutorials split text every N tokens. For video, split by time instead. Each chunk then keeps the start time of its first segment, and that start time becomes the citation:
Here is the full script. It runs the scraper, chunks every video, and ranks the chunks with BM25, the classic keyword ranking that Lucene and Elasticsearch use by default:
Once you have the top chunks, give their text to an LLM with the instruction "answer only from these excerpts and cite the links", and you have a channel Q&A bot with clickable sources.
{ "mcpServers": { "youtube-transcripts": { "command": "npx", "args": ["-y", "mcp-remote", "https://mcp.apify.com/?tools=harshmaur/youtube-transcript-scraper"] } } } Enter fullscreen mode Exit fullscreen mode The first time your client starts the server, a browser window opens to connect your Apify account.
$5 per 1,000 transcripts on the free plan, and less on paid plans. You pay only for transcripts that come back. A private, deleted or caption-less video returns a row with status: "failed", and that row is free. There is no start fee.
My three-video run cost 1.4 cents. Indexing a channel with 200 videos costs about $1, once. For more than about 20 videos, do not use the run-and-wait endpoint, which waits at most 5 minutes. Start the run with POST /v2/acts/{ACTOR}/runs, wait for it to finish, then read /v2/datasets/{defaultDatasetId}/items. After that, a weekly run with dateFrom costs a few cents.
Can I download all transcripts from a YouTube channel at once? Yes. Put the channel handle in channelUrls and set maxVideos to the number of videos you want. Playlists work the same way with playlistUrls.
Do I need a YouTube API key? No. You need only an Apify API token, which comes free with an Apify account.
What happens when a video has no transcript? You get a row with status: "failed" and no charge. The row has only videoId, url, errorCode and error, with no transcript fields, so filter on status == "success" as the script does.
Can I get transcripts in another language? Set language to a code like es or de. If the video has captions in that language, you get them. If not, you get a machine translation and wasTranslated is true. If no translation is available, you get the original captions, and language shows which language you got.
Do I get SRT or VTT files? Yes. Set transcriptFormat to srt or vtt, and each row has an srt or vtt field that holds the subtitle file as text. If you already have an SRT or VTT file from somewhere else and only want the words, this free SRT and VTT to text converter removes the timestamps and the repeated lines that YouTube's automatic captions often have.
If you build a channel Q&A bot with it, tell me in the comments what you indexed and which retrieval worked best for you.
Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink.
For further actions, you may consider blocking this person and/or reporting abuse
DEV Community — A space to discuss and keep up software development and manage your software career
Built on Forem — the open source software that powers DEV and other inclusive communities.
We're a place where coders share, stay up-to-date and grow their careers.
Original Source
This content was distilled for a focused reading experience. All rights belong to DEV Community.
Read original publication