How Virtual Try-On Works: What AI Does With Your Photo

Most clothing try-on today doesn't build a 3D model of you. It reads your photo and redraws it with the new garment in place, which makes it good for judging a look and poor at judging fit.

A smartphone on a small tripod in a bright, empty room, set up for a full-length photo

Key takeaways

  • There are three kinds of virtual try-on: live camera overlays, 3D body models and AI image try-on.
  • AI image try-on works from a photo of you and photos of the clothes, with no body scan or measurements.
  • Newer try-on models redraw the garment on your photo instead of stretching a cut-out, which is why folds and shadows look more natural.
  • Research shows current image try-on tends to make clothes look well fitted whatever the actual size, so it shows how an outfit looks, not whether it fits.

You upload a photo of yourself, pick a jacket, and a few seconds later you're "wearing" it. It looks like a trick, and it raises fair questions. Did the app measure your body? Is it scanning your face? How virtual try-on works is simpler than either. Most clothing try-on today doesn't build a 3D model of you at all. It reads your photo, then redraws it with the new garment in place.

Here's what happens in those few seconds, what the result can tell you, and what it can't.

A person taking a full-length photo of themselves in a mirror with a phone
One clear photo is all image try-on starts from.AI-generated

Three kinds of virtual try-on

"Virtual try-on" covers three quite different technologies, and most explanations blur them together.

Type What it needs from you What it's good for
Live camera (AR) Your phone camera, in real time Glasses, makeup, hats
3D body model A body scan or measurements, plus a 3D version of each garment Simulating how a size drapes
AI image try-on One photo of you and photos of the clothes Seeing how an outfit looks on you

Live camera try-on finds points on your face in every video frame and pins the virtual object to them. That works well for rigid things like glasses. 3D try-on builds a model of a body and drapes digital fabric over it with physics simulation. It can say more about fit, but it asks for much more: measurements or scans, and a 3D version of every garment.

The third kind is what most clothing apps use now, and it's the one this post is about. No scan, no tape measure: just pictures.

How virtual try-on works, step by step

AI image try-on does three jobs. A Google Research system called TryOnDiffusion, presented in 2023, is one of the best-documented examples, and the general pattern holds for most current VTO technology.

Inside a virtual try-on

  1. Read the photo

    The model finds your pose as points at shoulders, elbows and hips, and labels which pixels are hair, skin, arms and clothing.

  2. Clear the space

    It masks the area where the new garment will go and keeps what should stay the same, like your face and hands.

  3. Draw the new garment

    A generative model fills that space with the chosen garment, matching your pose and adding the folds and shadows the fabric would make.

Notice what isn't on that list: no measuring and no 3D scan. The system works with what's visible in the picture. That's also why the photo you start from matters so much.

Why newer try-on looks more natural

Earlier systems took the flat product photo of a garment, cut it out and stretched it over the person's outline. As Google's researchers put it when introducing their newer approach, the results never quite looked right: folds landed in odd places and clothes didn't follow the body.

Newer models use diffusion, a technique that starts from visual noise and refines it, step by step, into an image. Instead of bending a picture of the shirt, the model draws the shirt again, on you, guided by both photos. Because the garment is redrawn rather than stretched, the model can add a crease at the elbow, a shadow under the collar or a fold where fabric gathers at the waist.

For you, that means the result can look convincingly like a real photo. It's worth remembering that it is a new image, not a measurement.

Close-up of soft fabric falling in natural folds with shadows between them
Folds and shadows are what older try-on got wrong and newer models redraw.AI-generated

What the result can and can't tell you

The biggest limit is fit. A 2026 study from the University of Washington and Google Research found that current try-on models tend to show clothes fitting well regardless of the actual size of the garment or the person. An oversized coat and a fitted one can come out looking much the same. Showing true fit would need precise body and garment measurements, which standard image try-on doesn't use.

Fine details are the second limit. The TryOnDiffusion researchers reported cases where tattoos and accessories disappeared in the redrawn area, and noted that their models were trained mostly on photos with clean, plain backgrounds. The makers of one widely used image model list fine details and longer text among the things they're still improving. A small logo or an intricate print may come out simplified.

Useful for Not reliable for
Colour combinations Whether a size fits
How pieces work together as a look Fabric weight, stretch or texture
A quick yes or no before getting dressed Exact logos, lettering and fine prints

Read the result the way you'd read a stylist's sketch: good for the idea, not a guarantee of the details.

Does try-on need your face or body data?

It needs your photo, and a photo of you is personal data. But under EU law, a photo only counts as biometric data when it's processed with technical means designed to identify or authenticate a specific person, and the UK's data regulator says the same. Image try-on redraws clothing on the picture itself; it doesn't need to recognise who you are.

Whether a particular app does anything more with your photo is a question for its privacy policy. Five questions to check before uploading covers what to look for.

Try it with what you own

The easiest way to understand try-on is to test it against something you already know well: a piece from your own wardrobe.

  1. Take a clear base photo

    Full length, head to toe, against a plain wall in even daylight. A tidy background gives the model less to guess.

  2. Stand simply

    Face the camera with your arms slightly away from your body and hands visible. Crossed arms and odd angles make results less reliable.

  3. Wear fitted, plain clothes

    Baggy layers hide your outline, so the model has to guess where your body is.

  4. Try a piece you know

    Pick something you own whose fit you already know, then compare the try-on with what you see in the mirror.

  5. Judge the look, not the fit

    Use the result to compare colours and combinations. Check details like prints against the real garment.

Where Kashe fits

Kashe uses this kind of image generation for one job: showing the day's outfit, built from clothes you already own, on your own digital twin photo. Because those clothes are already in your wardrobe, you know how they fit; what the try-on adds is a look at how the pieces work together before you get dressed. Kashe asks for your separate consent before you upload that photo, and uses it only to show clothes on you, not to verify your identity or for biometric recognition.

Sources

  1. TryOnDiffusion: A Tale of Two UNets — Zhu et al., CVPR 2023
  2. How AI makes virtual try-on more realistic — Google, The Keyword (2023)
  3. FIT: A Large-Scale Dataset for Fit-Aware Virtual Try-On — Karras et al., SIGGRAPH 2026
  4. MediaPipe Face Mesh — Google
  5. Introducing Gemini 2.5 Flash Image — Google Developers Blog (2025)
  6. Recital 51 — General Data Protection Regulation (EU) 2016/679
  7. What is special category data? — UK Information Commissioner's Office

Frequently asked questions

How does virtual try-on work?

AI image try-on reads your photo for pose and outline, clears the area where the new garment goes and generates a new image of you wearing it. Live-camera and 3D try-on work differently.

Is virtual try-on accurate?

It's useful for seeing how colours and pieces look together. Research shows current image try-on tends to make clothes look well fitted whatever the size, so it isn't a fit check.

Does virtual try-on need a body scan?

Image-based try-on doesn't; it works from a photo. 3D try-on is the type that relies on body scans or measurements.

Why does my virtual try-on always look good?

Current models tend to draw clothes as if they fit well, regardless of the real sizes involved. Treat the result as a preview of the look, not of the fit.

Does virtual try-on use face recognition?

It doesn't need to, because it redraws clothing on the image itself. What a specific app does with your photo beyond that is set out in its privacy policy.

Keep reading