The 350-Object Problem

Generation was never the hard part. Here’s the lockfile we built to track what we actually paid for, and the three bugs it took a real, messy account to shake loose.

Before we had a name for any of this, we had a folder of 65 achievement badges and no idea how we’d gotten them.

We’d been driving PixelLab.ai, an AI image-generation API built specifically for pixel art, conversationally: one prompt at a time, through chat, picking whatever candidate looked closest to right and moving on. It worked, in the sense that we ended up with 65 files that shipped. But generating that set had produced 350 separate objects on our PixelLab account, and when we went looking for a record connecting any of those 350 objects to the 65 files we actually used, there wasn’t one. Two piles that used to be one thing, with nothing left mapping them together.

That’s the moment this project started. Not “AI art is hard” – it isn’t, particularly – but “we have no idea what we paid for or where it went.”

The reframe: generation is cheap, supervision is expensive

PixelLab‘s own pricing hints at the right way to use it. A single API call returns a batch of candidates for a fixed price – anywhere from 4 to 64, depending on the generator – so the economical move was always to generate broadly and pick fast. We weren’t doing that. We were generating narrowly, one bespoke prompt at a time, and paying the same either way.

What we were missing wasn’t a better prompt or a better model. It was bookkeeping. The same problem package-lock.json and Pipfile.lock solve for dependencies: a durable, machine-written record of what you actually have, kept separate from the human-authored file describing what you want.

Three things were broken, and none of them were about image quality:

• Generation was cheap; supervision was expensive, and nothing amortized the cost of choosing well.
• Nothing tracked provenance. Objects on the account and files in the repo were two unrelated piles.
• No style contract existed. Dozens of hand-written prompts drift from each other over time, with nothing enforcing consistency.

What we built

Pixelkiln is a manifest-driven CLI that treats bulk pixel-art generation the way a package manager treats dependencies. Two files do the work:

• pixelkiln.manifest.json – hand-authored, committed. Your asset dictionary: styles (the visual contract – a prompt template plus reference images plus a generator) and assets (just a subject and a short prompt each).
• pixelkiln.lock.json – machine-written, also committed. Maps every style-and-asset pair to the exact PixelLab object that produced it and the exact file it became, with hashes on both ends.

There’s no LLM anywhere in that loop, and that’s a deliberate positioning, not a caveat. PixelLab exposes a plain REST API. Submitting a job, polling it, downloading the result, and filing it away are ordinary scripted mechanics: deterministic, free, no model call. The only step that actually needs a human is choosing among candidates, and that happens in a small local “contact sheet” web page you scan in seconds, not a chat transcript you scroll through. Even the pre-ranking of candidates is plain RGB palette-distance math against a style’s reference images – arithmetic, not a model, so it’s free and it’s reproducible.

Candidate selection is a palette-distance calculation. Deterministic, no model, no inference cost. The contact sheet exists so a human can override it, not so a model can decide.

(Full command reference, cost breakdown, and setup steps live on the Pixelkiln project page.)

The real test

Pixelkiln got built, and we used it successfully for a while. The real test came a few weeks later, when we pointed it at a live, shared PixelLab account used by two separate production projects – 798 total generated objects sitting on one account, with no obvious separation between them.

The command that matters here, because it’s genuinely the point of the tool:

pixelkiln salvage --dry-run --claims ../other-project/pixelkiln.lock.json

pixelkiln salvage --dry-run --claims ../other-project/pixelkiln.lock.json

salvage is a recovery tool, not a cleanup tool, for the inevitable pile of objects that get generated, paid for, and never land anywhere. Point it at every project’s lockfile and it tells you what’s actually unclaimed. Running it against that shared account: 798 objects total, 627 already accounted for by the two projects’ lockfiles, and 341 genuinely unclaimed – work that had been generated and paid for and simply never made it into either repo.

The twist: using it for real found bugs in it

This is the part that makes this worth writing up instead of just announcing a feature. Three honest, specific incidents, all found the same way: by actually running the tool against messy, real, shared data instead of trusting it in the abstract.

The account was messier than a glance suggested

A first look at those 341 unclaimed objects – sorted newest-first, which is what a quick CLI preview shows by default – looked like a clean, single-project set: rejected alternate takes of one project’s achievement-badge art. Pulling the full list instead of the truncated preview told a different story. 54% of those 341 objects, 183 of them, were a completely different project’s terrain and character art: isometric trees, fences, tee signs, character portraits, all sitting on the same account. A quick glance would have led straight to importing someone else’s game assets mislabeled as achievement badges.

A literal oops, I clicked the buttons moment

During what was supposed to be a deliberately inert connectivity check – just confirming a review page loaded, nothing more – someone watching the opened browser tab went ahead and clicked through it for real, submitting three genuine import decisions. Nobody’s fault, an honest mix-up mid-task. But it surfaced something real: applying even one import had been silently rewriting the entire project manifest file on every save, not just the new entry. A three-item change produced a 239-line diff, because every pre-existing asset got quietly rewritten with its in-memory normalized form. We caught it immediately with a git diff before committing anything, but it’s exactly the kind of bug that would otherwise survive by looking like a no-op every single time. Nothing was visibly wrong; the diff was just needlessly enormous.

A landmine sitting in the tool’s own cache

Digging further while fixing the first bug, we found a second one in the tool’s internal hash cache: it derived its own cache file path from the lockfile’s name using a pattern match that only worked for the default lockfile name. Any project using a custom lockfile name – and one of the two real projects sharing this exact account already does, for a variant workflow – would have had the cache silently write itself directly on top of the real lockfile, using a completely different internal format. The first run would look fine, a coincidence. The next slightly-differently-named lockfile would be catastrophic, silent data loss. Zero test coverage had ever touched that code path.

All three were root-caused, fixed, and covered with regression tests the same day. The fixes and the tests that prove them are what shipped, not just an apology.

What changed as a result

• salvage now automatically groups a mixed account’s unclaimed objects by which project’s style they actually match, opening one correctly-scoped review session per style instead of one undifferentiated pile that defaults every import to whichever project happens to be listed first.
• A project can now recognize a sibling project’s art even when it has no style pattern of its own to check against – the exact shape of the smaller of the two real projects here, which previously had no way to tell “not mine” from “mine by default.”
• Every review tab now visibly names the style it’s scoped to and exactly where an import will land, so several tabs open at once are no longer indistinguishable.
• The internal cache bug is fixed for any lockfile name, not just the default, and now has test coverage where it previously had none.

The meta-lesson

Build tooling around the exact failure mode you’re trying to prevent – in our case, generated work that nobody can account for – and the only way to actually trust it is to run it against real, messy, shared data before you rely on it. That’s not a failure of the design. Running it for real is what the design was for. The bugs it found were bugs the tool’s own philosophy predicted would exist somewhere; they just hadn’t been exercised yet.

There’s a small postscript that echoes the same lesson one level up. For weeks, our own internal naming notes claimed “Pixelkiln” couldn’t be the real name because it collided with an established npm package. Checking that claim directly against the npm registry instead of trusting the note that said so: the name had never actually been published by anyone. The real collision our research had found was two different, real packages, spritesmith and its engine pixelsmith, that got conflated into a name that was never actually taken. Same habit, applied to a five-minute naming decision instead of a production account: verify the specific claim, not the confidence it was stated with. Pixelkiln is the real name.

The provider layer was deliberately built to be swappable from day one – the manifest, lockfile, diffing, and review UI don’t know anything about PixelLab specifically; only one seam does. We looked at Midjourney and ruled it out: it has no public developer API, and every “Midjourney API” wrapper out there automates the Discord client against Midjourney’s own terms of service, which risks getting an account banned. Retro Diffusion is the real next candidate whenever we get to a second provider.

The repo’s up on GitHub now if you want to look under the hood, and if this is a problem you’ve run into too, I’d like to hear about it.