May 11, 20266 min read
What Automating Humble Bundle Taught Me About Playwright
May 11, 2026 · 13 min read
I got one of those “your key is expiring” emails from Humble Bundle and realized I had no idea how many unclaimed games were sitting in my library. That was the start. What followed was a crash course in everything Playwright throws at you when you’re trying to automate a real, bot-averse, React-rendered storefront.
I’ve bought Humble Bundles on and off for years. A few dollars here, a charity bundle there. At some point I had a library big enough that I couldn’t reasonably check it manually — dozens of orders, Humble Choice months I’d subscribed to and then forgotten, and the occasional email informing me that a key I owned was about to expire. That last one is what finally pushed me to sit down and write humble-bundle-keys.
The goal was simple: walk every order in my Humble library, reveal any unrevealed keys, claim any unclaimed Choice games, and dump everything into a CSV I could actually sort and search. The implementation was not simple. This post is about the Playwright side of that journey — the parts where the browser fought back.
The starting point: two scrapers, one problem
The tool has two separate paths for getting keys out of Humble. The first is an API path: Humble has private JSON endpoints at /api/v1/order/{gamekey} that return structured order data, and you can call Humble’s /humbler/redeemkey endpoint to reveal a key without ever touching the DOM. That path is fast — a warm run through a large library takes about 10 seconds with the order cache in play.
The second path is browser-driven. Humble Choice subscription games — the “pick N games from this month’s lineup” mechanic — don’t always appear in your order JSON until they’ve been claimed. For legacy “pick N of M” months, the only way to get the key is to navigate to the /membership/<slug> page, find the game card, click “GET GAME ON STEAM,” wait for Humble to allocate the key, and extract it from the DOM. That’s where Playwright comes in.
I started with what seemed like the obvious approach: use Playwright’s APIRequestContext for the POST calls and a headless browser for the Choice pages. That worked fine in my head. It did not work fine in practice.
Cloudflare doesn’t care that your GETs work fine
The first real wall was Cloudflare. Early runs were producing 0 revealed keys and 167 errors. Every single reveal attempt was returning a Cloudflare challenge page — the <html class="no-js ie6 oldie"> template that Cloudflare serves when it decides your traffic looks like a bot.
The confusing part was that GETs worked fine. Fetching order JSON with APIRequestContext went through without issues. It was specifically the state-changing POSTs — /humbler/redeemkey and /humbler/choosecontent — that triggered the challenge. Cloudflare’s bot management fingerprints incoming HTTP/2 and TLS handshakes, and Playwright’s APIRequestContext has a distinct enough fingerprint on POST requests that it was getting flagged even when the session cookies were valid.
The fix was to stop using APIRequestContext for POSTs entirely and route them through the actual browser instead:
Python
# Instead of page.request.post(...), we inject a fetch() call into the
# browser's own JS context. Cloudflare sees a real Chrome request from
# the same origin with normal cookies and a clean TLS fingerprint.
result = await page.evaluate("""
async ({ url, body }) => {
const resp = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'X-Requested-With': 'XMLHttpRequest',
'Accept': 'application/json, text/javascript, */*; q=0.01',
},
body: new URLSearchParams(body).toString(),
});
return { status: resp.status, text: await resp.text() };
}
""", {"url": url, "body": body})This became humble_bundle_keys._browser_fetch.post_form_in_browser() — a shared helper used by both the API scraper and the Choice claimer. GETs still use APIRequestContext since they don’t trigger the challenge; only the mutations go through page.evaluate.
There’s a secondary piece here that bit me before I figured out the Cloudflare issue: missing headers. Humble’s backend uses X-Requested-With: XMLHttpRequest as a soft anti-CSRF check. Without it, reveal calls returned 2xx responses but no key — the server was silently rejecting the request. Getting both the TLS fingerprint and the headers right was what finally got reveals working consistently.
Headless vs. headed: it matters for POSTs specifically
Related to the above: I verified empirically that --headless produces 0 reveals and consistent Cloudflare errors, while --no-headless resolves them. Once I moved POSTs into page.evaluate, headless mode started working again — but the experience taught me something worth keeping: Cloudflare’s detection is sensitive enough to distinguish headed vs. headless Chromium on state-changing requests, at least in this case.
The tool now auto-flips to headed mode whenever a run will mutate state (any run without --dry-run). If you really want headless for a mutation run, you can force it, but the default is deliberately conservative.
The DOM is not what you think it is
The browser-driven Choice claim path requires clicking a “GET GAME ON STEAM” button on each game card, waiting for a modal, and extracting the key from the modal’s DOM. This sounds straightforward. It wasn’t.
The first version used *:has-text('GET GAME ON STEAM') as the click target. That selector matched the modal wrapper itself, not the button inside it. Clicking the wrapper hit the backdrop and closed the modal. 0 keys, no errors — just silent failure.
The real element is a <div class="js-keyfield keyfield enabled"> styled to look like a button. Not a <button> element, not a link — a div with a CSS class that makes it interactive. The post-claim key appears in a child .keyfield-value div, and the parent gets a .redeemed class added after the key allocates. That’s the signal you wait for.
The selector chain I ended up with, from most-specific to fallback:
Python
KEYFIELD_SELECTORS = [
"div.js-keyfield.keyfield.enabled",
"div.keyfield.enabled",
"button.button-v2",
"button",
"a.button-v2",
]There’s also a pre-click check for .expired keyfields. Subscription trial entries like IGN Plus or Boot.dev have class="js-keyfield keyfield expired" and a “This key has expired” placeholder. Clicking them does nothing — no modal, no error, just a page that doesn’t respond. Checking for the expired class before clicking and skipping cleanly saved a lot of wasted timeout budget.
The diagnose subcommand (humble-bundle-keys diagnose -v) was essential for figuring all of this out. It captures a sanitised zip of the page interaction — DOM snapshots, XHR captures with PII stripped — that you can inspect without having to instrument a live run.
Waiting is not the same as waiting correctly
The Choice claim flow has a timing problem that isn’t obvious until you see it fail. After you click the keyfield, Humble shows a “claimed” success banner almost immediately — within about a second. But Humble’s backend takes another 5-15 seconds to actually write the key. If you read .keyfield-value as soon as the banner appears, you get the placeholder string (“Get game on Steam”) instead of the real key.
The fix was to never use the “claimed” banner as the completion signal. Instead, poll the keyfield directly until either a real key pattern matches or a timeout expires:
Python
KEY_PATTERN = re.compile(r"[A-Z0-9]{5}-[A-Z0-9]{5}-[A-Z0-9]{5}")
async def _wait_for_key(self, modal, timeout_ms: int = 30_000) -> str | None:
deadline = asyncio.get_event_loop().time() + timeout_ms / 1000
while asyncio.get_event_loop().time() < deadline:
el = modal.locator(".keyfield-value")
text = (await el.text_content() or "").strip()
if KEY_PATTERN.match(text):
return text
await asyncio.sleep(0.5)
return NoneThe default timeout is 30 seconds, bumped up from 20 after seeing some top-tier titles take longer than expected on Humble’s backend. The pattern match is strict: _extract_key explicitly rejects placeholder strings, so if the timeout fires and you never got a real key, you get None rather than “Get game on Steam” saved to your CSV.
A related issue: scoping the wait to the just-opened modal rather than the page. The membership pages show multiple game cards, each with its own keyfield. If you wait for .keyfield-value on the page, you’ll pick up already-claimed cards from previous runs and short-circuit immediately with someone else’s previously-extracted key. Scoping the locator to the current modal handles this.
Retry logic and the false positive problem
Cloudflare 403s on reveal POSTs aren’t always permanent. In a large run, you’ll occasionally get a transient 403 that would succeed on a retry. The naive fix — retry N times on any 4xx — is wrong because some 4xx responses are permanent (Humble returning a real rejection reason, key already redeemed, etc.).
The approach I landed on: retry specifically on 403 with exponential backoff (1s, then 2s, up to 3 attempts), and surface the response body on any non-retriable 4xx. Humble’s server-provided rejection messages in the body are actually useful — things like “key already redeemed” or “no keys available” — and logging up to 200 characters of the body makes debugging a lot faster than just seeing a status code.
The exhausted key pool case deserved its own handling. When Humble’s key pool for a game runs dry, the modal shows “no more keys available at this time.” Without specific detection, this burns the full 30-second timeout per entry before failing. With a selector check for that specific message, detection drops to about 1 second. Exhausted entries are now separated in the run summary so they don’t inflate the generic failure count.
Choice months vs. bundle keys: two completely different page flows
One thing that took a while to fully understand: Humble Choice games and bundle games are structurally different in ways that affect every layer of the tool.
Bundle keys are simple. They appear in your order JSON as tpks with a keyindex field. If keyindex is not null, the key has been allocated and you just need to call /humbler/redeemkey to reveal it. The API path handles this entirely.
Choice games are a two-step flow. First you call POST /humbler/choosecontent to select the game from your subscription month. Then you call POST /humbler/redeemkey to reveal the key. The choosecontent body needs the membership slug, the tpk machine name, and a CSRF token extracted from the page. If you skip the choosecontent step, the redeemkey call silently returns 2xx with no key — the “silent no-key” response that took a while to diagnose.
Legacy “pick N of M” months are worse: the choosecontent endpoint doesn’t work for them at all. They require browser-driven navigation to the actual membership page, clicking each game card through the modal UI, and waiting for the key to appear. That’s what --browser-claim is for.
There’s also a keytype classification problem. Humble’s API returns tpks with machine names like road96_europe_hoice_steam (yes, with a missing ‘c’ — that’s real), dccomicsfreetrial_november2023choice_coupon, somegame_naeu_choice_steam, and so on. Categorizing these correctly into choice, voucher, keyless, softwarebundle, etc. determines whether you attempt a reveal, skip pre-claim, or surface them for manual investigation. Getting the regex right — and keeping it up to date as Humble’s naming conventions drift — is ongoing work.
The execution context destruction problem
One error I kept seeing in early runs: Page.evaluate: Execution context was destroyed, most likely because of a navigation. This happens when you hold a reference to a page and then something navigates it out from under you — an auth refresh, a Cloudflare interstitial, a redirect on session expiry.
The pattern I settled on: both the API scraper and the Choice claimer keep an “anchor page” navigated to humblebundle.com that’s used for page.evaluate POST calls. On each POST, the anchor page’s URL is checked before the call. If it’s drifted off-origin, re-navigate back to humblebundle.com. If re-navigation fails, open a fresh page. It’s a small amount of defensive boilerplate but it makes long runs reliable where they’d otherwise fail unpredictably on entry N of a 200-item library.
What I’d tell someone starting a similar project
A few things I’d do differently from the start:
Route state-changing POSTs through page.evaluate from day one. Don’t assume APIRequestContext will work just because your GETs work. Cloudflare specifically targets the TLS fingerprint difference on mutation requests. The page.evaluate(fetch(...)) pattern isn’t elegant but it’s reliable.
Build a diagnose tool early. The humble-bundle-keys diagnose subcommand — which captures sanitised DOM snapshots and XHR captures — saved me hours of live debugging. Having a reproducible artifact you can inspect offline is worth the initial investment.
Don’t trust visual signals as completion indicators. “Claimed” banners, loading spinners, and modal transitions all fire before the backend has finished writing. Always poll for the actual data you need, with a real timeout and a pattern match.
Pre-skip before you click. Expired keyfields, structurally unclaimed entries, vouchers, keyless entries — anything that can’t succeed should be identified and skipped before you attempt any browser interaction. A 30-second timeout per failed entry adds up fast in a large library.
Scope your locators to the modal, not the page. This one tripped me up multiple times. Once you open a modal, every locator should be scoped to that modal’s container. Page-level locators will match previously-claimed elements on the same page and produce silent wrong results.
The tool is at github.com/gfargo/humble-bundle-keys and on PyPI. If you’ve got a Humble library you’ve never fully inventoried, start with humble-bundle-keys --dry-run -v — it’s read-only and tells you exactly what’s there before anything mutates. The project page on griffen.codes has the full feature overview: humble-bundle-keys project page.
If you run into selector breakage after a Humble UI update, humble-bundle-keys diagnose -v generates a safe-to-share.zip you can attach to a GitHub issue. I’m curious whether anyone hits the legacy Choice month path in practice — those are the cases that are hardest to test without a real account with old subscription history.
Keep Reading
Discussion
Have thoughts? Drop them in.
Comments are powered by Disqus. Sign in once, comment anywhere.
Loading comments…