Almost everyone who needs YouTube captions programmatically follows the same path. They find the YouTube Data API v3, see captions.list and captions.downloadsitting right there in the reference, wire up a Google Cloud project, and build a day or two of code on top of it. Then they point it at a video from someone else's channel and get a 403 that no amount of quota, scopes, or billing will fix.
This post is the thing I wish had existed at that moment. It covers why the official endpoint cannot do what you want, what the working alternatives are, and — the part that costs people the most time — the two failure modes in the community tooling that look like something else entirely when you try to debug them.
The endpoint that looks like the answer
captions.download is real, documented, and completely useless for third-party video. The reason is not rate limiting or regional restriction: the endpoint is authorised against the channel that owns the caption track. It requires an OAuth 2.0 access token, and that token has to belong to the account that uploaded the video. An API key alone will not authenticate it. A service account will not either, unless that service account has been delegated ownership of the channel.
Here is the call, written the way you would actually write it, so you can confirm the behaviour yourself rather than take my word for it:
import google_auth_oauthlib.flow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
SCOPES = ["https://www.googleapis.com/auth/youtube.force-ssl"]
flow = google_auth_oauthlib.flow.InstalledAppFlow.from_client_secrets_file(
"client_secret.json", SCOPES
)
creds = flow.run_local_server(port=0)
yt = build("youtube", "v3", credentials=creds)
VIDEO_ID = "dQw4w9WgXcQ" # not your video
# captions.list works fine for anyone. 50 quota units.
tracks = yt.captions().list(part="snippet", videoId=VIDEO_ID).execute()
for item in tracks["items"]:
print(item["id"], item["snippet"]["language"], item["snippet"]["trackKind"])
caption_id = tracks["items"][0]["id"]
# captions.download does not. 200 quota units you will never spend.
try:
body = yt.captions().download(id=caption_id, tfmt="srt").execute()
print(body[:200])
except HttpError as e:
print(e.resp.status, e.error_details)
# 403 [{'reason': 'forbidden',
# 'message': 'The permissions associated with the request are not
# sufficient to download the caption track. The request
# might not be properly authorized, or the video order
# might not have enabled third-party contributions for
# this caption.'}]Note the asymmetry, because it is what makes the trap effective: captions.list succeeds. You get back caption IDs, languages, whether the track is ASR or manually uploaded. Everything looks like it is working. The metadata is public; the payload is not. People build a whole pipeline on the strength of that first successful call.
captions.list tells you a transcript exists. captions.download tells you it is none of your business.
There is no flag, no allowlist, no elevated quota tier that changes this. If you are enumerating videos across channels you do not control — competitor research, media monitoring, dataset building, search over a niche — the official API is a dead end for caption text specifically. It remains genuinely good for everything else, which is worth saying: videos.list, playlistItems.list and channels.list are cheap, stable, and the right tool for metadata. Keep them. Replace only the caption half. If you are unpicking an existing Data API integration, the migration guide maps the endpoint-for-endpoint equivalents.
Quota, for context
Even in a hypothetical world where ownership were not the blocker, the quota maths would still stop you. A default Google Cloud project gets 10,000 units per day, and the endpoints are not priced evenly:
| Endpoint | Units per call | Calls in a 10,000-unit day |
|---|---|---|
captions.download | 200 | 50 |
search.list | 100 | 100 |
captions.list | 50 | 200 |
videos.list | 1 | 10,000 |
playlistItems.list | 1 | 10,000 |
channels.list | 1 | 10,000 |
Fifty transcripts a day, for your own videos, before you are locked out until midnight Pacific. A single mid-sized channel backfill would take a month. The quota increase form exists, and the honest summary of its success rate for this use case is that you should not plan around it.
Notice too that search.listat 100 units means discovery is expensive independently of captions. If your job is "find every video in this niche and read it", you burn the entire daily allowance on 100 searches and have nothing left. Cheap enumeration goes through playlistItems.listagainst the channel's uploads playlist instead — 1 unit per page of 50.
So everybody ends up on the community route
The practical answer, and it has been the practical answer for years, is to read the same data the browser reads. Two tools dominate: the youtube-transcript-api Python library, and yt-dlp with --write-auto-subs --skip-download. Both work on any public video regardless of who owns it, because they are consuming the public watch page and the timed-text endpoint behind it rather than an authorised API surface.
# pip install youtube-transcript-api
from youtube_transcript_api import YouTubeTranscriptApi
api = YouTubeTranscriptApi()
fetched = api.fetch("dQw4w9WgXcQ", languages=["en"])
for snippet in fetched:
print(f"[{snippet.start:7.2f}] {snippet.text}")
# Listing what exists, including translatable tracks:
for t in api.list("dQw4w9WgXcQ"):
print(t.language_code, "generated" if t.is_generated else "manual")On your machine this runs in under a second and you conclude the problem is solved. It is not solved. It is solved from your residential IP address, which is a materially different thing, and you will find that out roughly four minutes after your first deploy.
Then you deploy it
YouTube blocks datacenter IP ranges for this traffic. AWS, GCP, Azure, Hetzner, DigitalOcean, Fly, Render — the ranges are well known and the block is aggressive. What comes back is an IpBlocked or RequestBlocked exception, sometimes immediately on the first request from a fresh instance, with no warm-up period during which things work.
This is not a rate limit you can back off from. Adding sleeps between requests does not help, because the block is on the address, not on the frequency. Neither does rotating your user agent, adding browser-shaped headers, or reusing a session cookie — all things you will try, in that order, over about half a day.
The trap: curl says everything is fine
This is the part that costs people the most time, and it is the reason this post exists.
When you get IpBlocked in production, the natural debugging move is to reach for curl and check whether the proxy itself is working. So you shell into the box, curl the watch page through the same proxy, and get a clean HTTP 200. Then you grep the HTML for captionTracks, and it is there, fully populated, with base URLs and language codes and everything you expect. Proxy healthy. Page fetching. Captions visibly present.
The obvious conclusion is that the proxy is fine and your code is broken. The obvious conclusion is wrong.
Through a datacenter proxy, the watch page returns HTTP 200 with a populated captionTracks field, while the transcript endpoint the library actually calls is blocked. The two are gated separately. A successful curl of the watch page tells you nothing about whether transcript fetching will work, and it will send you hunting for a bug in your own code that does not exist.
We measured this on 18 August 2026 against ten free Webshare datacenter IPs plus their rotating endpoint. Every one of those addresses served the watch page without complaint. Every single transcript fetch through them failed.
IpBlocked — while the same IPs returned the watch page with a populated captionTracks field. Measured 18 August 2026.If you take one operational rule from this post, take this one: never validate a proxy for transcript work by fetching the watch page. Validate it by running an actual transcript fetch through it. A three-line health check that calls the library and catches IpBlocked is worth more than any amount of curl.
Residential proxies, and only residential proxies
The fix is a rotating residential pool. Not datacenter. Not "static residential" or ISP proxies, which are cheaper and which fail the same way datacenter does for this traffic — the library's own documentation is explicit that you want rotating residential specifically, and our measurements agree with it.
youtube-transcript-api ships a first-class config for Webshare, which is the path of least resistance. The two parameters that matter are the ones people leave off:
import os
from youtube_transcript_api import YouTubeTranscriptApi
from youtube_transcript_api.proxies import WebshareProxyConfig
api = YouTubeTranscriptApi(
proxy_config=WebshareProxyConfig(
proxy_username=os.environ["WEBSHARE_USER"],
proxy_password=os.environ["WEBSHARE_PASS"],
# Pin the exit region. Mixed-region pools give you EU nodes,
# and EU nodes give you the cookie-consent wall.
filter_ip_locations=["us"],
# Each retry rotates to a different exit IP, which is the whole
# point -- a retry on the same address would be pointless.
# The library defaults to 10. On a metered plan that is expensive:
# every retry re-downloads the watch page, so a single request can
# spend ~3.8 MB before giving up. We run 3.
retries_when_blocked=3,
)
)
fetched = api.fetch("dQw4w9WgXcQ", languages=["en", "de"])
print(" ".join(s.text for s in fetched))If you are on a generic residential provider rather than Webshare, the equivalent is GenericProxyConfig with an HTTP and HTTPS URL, and you implement the retry loop yourself — the important detail being that the retry must go out through a different exit node, so your provider needs per-request rotation or a rotating gateway hostname.
The EU consent wall
The second failure mode, and the reason filter_ip_locationsis in the snippet above: EU residential exit nodes get served YouTube's cookie-consent interstitial instead of the watch page. It surfaces as a redirect to a google.com consent URL that the library cannot follow, so the error you see has nothing obviously to do with geography. If your pool is mixed-region, you get intermittent, unreproducible failures on a fraction of requests that varies with whatever the pool handed you that minute.
Pin the pool to a single non-EU region and let retries_when_blocked rotate you out of a bad node. That combination is what turns this from a flaky pipeline into a boring one.
Bandwidth: the number that wrecks cost models
Residential proxies are billed by the gigabyte, so your cost per transcript is entirely a function of bytes on the wire. This is where the second measurement error lives.
One transcript fetch is not one request. It is the full watch page, plus an InnerTube player API call, plus the timed-text XML. If you measure it the obvious way — summing len(response.content) across those requests — you get about 1.55 MB and you build a budget around that. But response.content is the decompressed body. Your proxy provider bills the gzipped bytes that actually crossed the connection.
Measured through a byte-counting CONNECT proxy across four videos ranging from 19 seconds to 61 minutes, the real figure is 377 KB average, in a 351–408 KB band:
len(response.content) reports.The length-independence surprises people. A 61-minute talk costs about the same to fetch as a 19-second short, because the timed-text payload is a rounding error next to the watch page HTML. That is good news for cost predictability and bad news for anyone hoping to optimise by filtering to short videos.
Run the numbers for your own volume before committing: 377 KB per fetch means roughly 2,650 transcripts per gigabyte of residential proxy traffic. At typical residential pricing that is a real line item once you are past a few thousand videos a day, and it is a metered one — it does not amortise.
The maintenance nobody quotes for
Proxies solve the access problem. They do not solve the parsing problem, which is the one that wakes you up.
None of this is an API with a contract. You are parsing an application's internal response shapes, and YouTube changes those without notice or versioning. A recent example: channel video listings moved from videoRenderer objects to lockupViewModel. Nothing threw. No status code changed. Parsers that looked for videoRenderersimply started returning empty lists, which every downstream system happily interpreted as "this channel has no new videos". That class of failure is silent by construction, and the only defence is an alarm on the shape of your output rather than on your error rate.
Concretely, if you self-host, budget for:
- A metered residential proxy, sized from the on-the-wire number above and not the decompressed one.
- Region pinning plus retry-with-rotation, or you will chase phantom EU consent-wall failures.
- Health checks that perform a real transcript fetch, because a
200on the watch page is not evidence of anything. - A canary on non-empty results per channel, to catch the silent parser drift that status codes will never surface.
- Whatever your own rate discipline is, since a blocked residential node still costs you the bytes it burned before failing.
When to stop building this
All of the above is genuinely doable. The code in this post works; people run it at scale. The decision is not technical feasibility, it is whether keeping a scraper honest against an adversarial, unversioned surface is a thing your team wants to own indefinitely alongside the product you are actually building.
If the answer is no, that is what we built TranscriptAPI for: one authenticated GET /transcript call, no proxy account, no parser to babysit, and the shape changes absorbed on our side before they reach your pipeline. If you are porting existing Data API code, the migration guide is the shortest path, and the cookbook has the batching, channel-backfill and translation recipes that sit on top of the snippets above.
Either way, the one thing worth carrying out of here is the curl trap. Whichever route you take, test the thing you are actually going to call.