Detecting AI-Generated Profile Photos
3 min read
CVPR Workshop research on detecting GAN-generated profile photos from compact embeddings: a lightweight approach to catching synthetic faces at scale.
Fake accounts increasingly rely on AI-generated profile photos: a synthetic face is unique and photorealistic, and it leaves none of the reverse-image-search traces that stolen photos do. Detecting these images automatically, at platform scale where false positives are expensive, is a hard, high-stakes problem for any platform that fights fake accounts.
Together with Professor Hany Farid at UC Berkeley, my team developed a detector that is small, cheap to run, and extremely accurate against the family of AI-generated profile photos most common in fake accounts at the time. We published the details at the Workshop on Media Forensics at CVPR 2023.
The Tell: Synthetic Faces Are Too Regular
The core observation behind the paper is simple to see and surprisingly powerful: GAN-generated profile photos share a rigid facial geometry. Average 400 StyleGAN2 faces and you get a sharp, recognizable face. Eyes, nose, and mouth land in nearly the same place every time. Average 400 real profile photos and you get a blur.
Averaging 400 StyleGAN2 faces (left) produces a sharp composite; averaging 400 real profile photos (right) produces a blur. The bottom row visualizes reconstruction behavior from a compact embedding learned on synthetic faces. (Figure from the paper.)
A Compact Embedding Instead of a Heavy Classifier
That structural regularity means synthetic faces live in a much smaller space than real photos, small enough to capture with a 128-dimensional linear embedding learned from a few thousand synthetic faces. Synthetic images reconstruct from that embedding with low error; real profile photos, with all their natural variation, do not.
Reconstruction error from the learned linear embedding cleanly separates StyleGAN faces (blue) from real profile photos (orange) across StyleGAN1, StyleGAN2, and StyleGAN3. (Figure from the paper.)
A threshold on reconstruction error alone already makes a usable classifier. Going one step further and fitting a simple logistic regression to the embedding coordinates caught 99.6% of synthetic photos at a 1% false positive rate, outperforming much larger CNN-based forensic classifiers. A learned autoencoder embedding behaves similarly.
That efficiency is the point. At platform scale, false positives are costly and inference budgets are real. A detector this small can be trained on modest data, audited easily, and deployed cheaply, which is what makes it practical as one layer of automated anti-abuse defenses.
The Generator Landscape
Representative synthetic faces from five generation engines: StyleGAN1, StyleGAN2, StyleGAN3, Generated.photos, and Stable Diffusion. (Figure from the paper.)
The paper focuses on the StyleGAN family, which dominated fake-profile imagery at the time. The bottom rows of the figure hint at what came next: diffusion models without the same rigid alignment. Detecting those in real-world conditions became the subject of our follow-up work, Finding AI-Generated Faces in the Wild.