A practical guide to AI image generation, from diffusion to agents
This is my general guide to AI image generation. It covers how the models work at a high level, the different types, a
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
This is my general guide to AI image generation. It covers how the models work at a high level, the different types, a bit of history, what you actually get from ChatGPT and how to build image generation into your own code and agents.
It’s based on my own experiments calling a handful of image providers from Python. I’ve generated pixel art sprites, photorealistic card table backgrounds and a series of restaurant posters, trying to make one that doesn’t look AI generated. Every image here came from those experiments.
For a few years the easiest way to spot an AI image was to count the fingers or try to read the sign in the background. Both problems are now mostly fixed, and why they happened says a lot about how these models work.
Hands are complicated, flexible and usually partly hidden in photos. Behind a cup, in a pocket, half out of frame. The models learned what hands look like as a texture, not how many fingers they have. Midjourney v5 in March 2023 was widely reported as the turning point. Experts suspected better curated training data with clearer pictures of hands, plus bigger models and higher resolutions. It wasn’t perfect. People in the background still had alien anatomy for a while.
Text had a more specific cause. Most models read your prompt through a tokeniser that splits it into chunks of words rather than letters. The model never sees that “chocolate” is spelled c-h-o-c-o-l-a-t-e, so it can’t draw it letter by letter. A 2022 paper, Character-Aware Models Improve Visual Text Rendering, showed that giving the model access to individual characters fixed much of the problem. Ideogram, whose team included former Google Imagen researchers, made text a selling point in 2023. By 2025 the multimodal models from OpenAI and Google, which understand language as well as any chatbot, had made readable text normal.
The fix arrived in stages, and the old problem leaked into the real world in between. The Willy’s Chocolate Experience in Glasgow in February 2024 promised “a pasadise of sweet teats” on its AI-generated posters. Long text is still not completely solved. In December 2025 Nano Banana Pro turned “Lived experience” into Liver experience on an otherwise good infographic.
You don’t need the maths to use these models well. You do need a rough picture of what’s going on, because it explains most of the surprises.
Most image models are diffusion models. During training they learn to remove noise from images. Take a photo, add a bit of static and teach the model to predict what the clean version looked like. Repeat billions of times at every level of noise.
To generate, you run that in reverse. Start from pure static and remove a little noise at a time over a number of steps. A text encoder turns your prompt into numbers that steer each step towards an image that matches it.
The newer approach puts image generation inside a model that also understands language. Because the same model reads your prompt and makes the picture, these are much better at following instructions, rendering text and editing an image you give them. GPT Image, Gemini’s image models and Meta’s Muse work this way. Some reason about the layout before they draw, which makes a real difference when the prompt is about where things go.
Often partly, and the line is blurrier than the marketing suggests.
OpenAI’s launch post for GPT-4o image generation hints at a hybrid. A language model plans the image as a sequence of tokens, the same way it writes a sentence, and a diffusion-style decoder paints the final pixels. Google hasn’t published how its Gemini image models work. Going the other way, modern “diffusion” models like Flux use transformer blocks that process text and image together, which is the same building block as a chat model.
So the useful distinction isn’t the maths. It’s how deeply the image model understands language. That’s what gives you instruction following, readable text and editing.
The random static a diffusion model starts from is set by a seed. Same seed, same prompt and same model gives the same image. That’s what makes batches repeatable. Many hosted models ignore or reject seeds, so don’t build anything that depends on them without checking.
“Image generation” covers several different jobs. Most providers support some of them, not all.
Tileable textures are worth a special mention. Plain models won’t reliably produce a seamless tile. You need a model or LoRA trained for it or a tiling option in the tool.
For my experiments I deliberately avoided subscriptions. Everything was pay per image through an API or was run locally.
Most people’s first experience of AI images is typing a request into ChatGPT. It’s worth knowing what you’re getting.
ChatGPT uses OpenAI’s own GPT Image models, which have been built into the chat since March 2025.
Is DALL-E still a thing? Not really. DALL-E 3 was ChatGPT’s image generator from October 2023 until the switch to GPT-4o image generation in March 2025. OpenAI then removed DALL-E 2 and 3 from its API on 12 May 2026. The name lives on mostly as a generic term for AI images.
Provenance. OpenAI adds both C2PA metadata and a SynthID watermark to its images, according to its help centre. Google adds SynthID to Gemini images. Metadata is easily stripped. Watermarks survive screenshots better, but neither is something to rely on if you’re trying to prove an image is or isn’t AI.
ChatGPT or the API? The same family of models is available through OpenAI’s API, billed per image. The API gives you sizes from a fixed list, quality tiers, batching from code and, on some models, transparent backgrounds. If you need more than a few images or the same kind of image repeatedly, use the API. The Gemini app versus the Gemini API is the same story.
The most useful mental model I’ve found is a painter who only owns large canvases, charges per canvas and paints every square inch with the same care. A flat blue sky gets as much effort as a face.
Newer flagship models cost more per image, not less. The cheap end of the market is open-weight models and the “flash”, “mini” and “lite” tiers. OpenRouter’s model list is the easiest place to compare current prices.
Price per image is misleading on its own.
cost per usable image = (price per image x attempts needed) + your time
A free local model that needs ten tries can lose to a $0.039 model that gets it right first time. In my experiments a model costing a tenth of a cent more saved several retries.
I wanted card table backgrounds for a game. That meant a table seen from directly above, with a few props around the edges and the middle kept clear for the cards.
Asking for “seen from directly above” isn’t enough. Models learn from photos, and most photos of tables are taken at an angle. My first attempts came back as nice pictures of a table with a wall and a window behind it, and every prop bunched down one side.
Here’s the same prompt for a pine kitchen table, with both changes, run on two different models.
Both are now top-down. Flux leaves the props whole and pushes them into the corners. Muse, a multimodal model that plans before it draws, put one item on each edge, cropped each one and left the middle clear.
Posters, flyers and infographics are one of the most common uses for AI images. They’re also where it most often looks cheap.
So I set myself a challenge. Could I get a restaurant poster that wouldn’t immediately read as AI?
My first attempt was a screenprinted trattoria poster. Rather than describe a look, I described a real print process. Three flat spot inks on uncoated paper, slight misregistration, a scan rather than a photo, an off-centre composition and a named design tradition.
The ink texture is convincing. The bottom line looks like clean digital type but the fold crease may be too much.
The harder test was a 90s burger poster with food that looks real. AI food tends to be too perfect, too glossy and too heroic. I asked for 1996 medium-format film photography and deliberately imperfect food, like a lopsided bun and pale winter tomatoes.
The first version had good food but read as a “vintage” filter, with comic lettering and fake wear. A real 1996 poster would have been clean and corporate.
So I made it boring. A pristine scan, a flat red background with an early-Photoshop gradient, a cut-out product photo with a slightly too-even drop shadow and plain heavy italic type. I even asked for it to look “competent and slightly bland”. That one would pass a casual glance. It isn’t quite 1996, because the shake is in a dome-lid cup you’d see in the 2010s.
For balance, I asked for the worst version I could think of. Unreal Engine, volumetric god rays, floating ingredients and a galaxy. It delivered a dozen patties with no bottom bun, a shake with no glass and fries raining down from space. Unlike AI posters of old (2 years ago), the spelling is correct these days. The giveaway now isn’t the spelling. It’s that every element is turned up to the maximum.
Pixel art and game sprites look like the easiest possible job for an image model. Small, simple, few colours. They turn out to be a great example of how AI image generation is harder than you would expect.
A pixel art knight on Gemini 2.5 Flash Image took 7.5 seconds and cost about $0.039. It’s clean and convincingly retro.
But it’s a 1024 x 1024 picture of a sprite, not a sprite. The “pixels” wobble in size, don’t sit on a true grid and use far more colours than a real palette. The “solid” magenta background was a slightly varying pink. To make it usable I still had to remove the background, snap it to a grid with nearest-neighbour downscaling and reduce the palette, all in ordinary code.
If you pay per canvas and a sprite only needs a fraction of it, the obvious move is to ask for many sprites in one image. It’s cheaper and more consistent, because the model draws everything at once with the same palette and lighting.
For a set of different objects it works brilliantly. I asked for 100 fantasy inventory icons and got 156, about 140 of them distinct, for one $0.039 call. That’s about $0.00025 per icon. Ordinary code then slices them by shape, removes the background and repacks them on a clean grid.
For one character in many poses it falls apart quickly. A 4 x 4 grid gave me about 13 distinct poses of the same knight. Asking for 10 x 10 gave me the same knight stamped 120 times.
Here is what I learned from seven calls and about $0.27.
What I did was a basic experiment with general models. If you need pixel art (a very specific type of image generation), there are better routes.
PixelLab is built for game developers. It draws real sprites at their actual size rather than a large picture that looks like pixel art. Palette, outline, camera angle and transparency are settings rather than prompt wording, and it can animate and rotate a character. It’s a subscription, so I didn’t use it here. Retro Diffusion is a similar commercial option.
To do it yourself, use a model add-on trained for pixel art such as Pixel Art XL, then shrink the result onto a clean grid and fix the palette in code. PixelOE can also turn any image into pixel art without AI. That works well for still images. Consistent animation is where the commercial tools are still ahead.
Vectors are another format that looks easier than it is. An LLM can write SVG code cheaply, but characters come out crude because it’s describing a picture without seeing it. Dedicated text-to-vector models like Recraft look much better but cost more than raster images.
The general lesson from pixel art is that AI is good at “what should this look like?” and bad at “put exactly this in exactly this cell”. What matters isn’t how simple the output looks. It’s whether the hard part is creative or structural. Use the model for the creative part and code for the structure.
Every API is different. OpenAI has an images endpoint. Gemini returns images inside a chat-style response. OpenR
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。