The Preview and the Photo Come Out of the Camera Facing Different Ways
On an iPhone 17 Pro, AVFoundation's video data output and photo output hand you buffers that sit 180° apart, the front camera ignores the rotation angle you set, and copying EXIF orientation after rotating the pixels yourself tips the photo over again. What I measured, and the 8×8 luma trick that settled it.
Yunseon Hong4 min read
I’m building Thirdstop, an iOS camera app that draws its own preview with Metal and renders its own photos instead of saving whatever the camera hands back. That means touching raw buffers on both paths. The first thing raw buffers teach you is that “up” is not a property of the pixels.
This is the first of six posts of notes from that build. They are about how the platform behaved, not how the app works. Every measurement comes from one phone, an iPhone 17 Pro on iOS 26.6.1. Other models may well behave differently, and I have not checked any.
Two outputs, two orientations
A capture session with a live preview and a shutter usually carries two outputs: AVCaptureVideoDataOutput for the frames you draw, and AVCapturePhotoOutput for the shot. Each hangs off its own AVCaptureConnection. The connections do not agree on which way is up.
On this phone, photo buffers arrive in sensor orientation, 4032×3024, tagged EXIF orientation 6 on both the front and the back camera. Orientation 6 means the image has to be turned 90° clockwise to display. The preview buffer sits 180° away from that. So on the front camera the preview had to be turned counterclockwise and the photo clockwise, from the same sensor, in the same session.
I got this wrong twice by reasoning about it. Each fix looked right in the case I happened to be testing and broke another. The third time I stopped reasoning and measured.
Measuring rotation with an 8×8 luma grid
The measurement takes a few minutes to write and ends the argument. Point the camera at a still scene. Grab one preview frame and one photo. Shrink both to an 8×8 grid of average luma. Rotate one grid by 0°, 90°, 180° and 270°, and compute a normalized difference against the other at each angle.
The correct angle is the one with the smallest difference, and it was not close. Preview against photo, 180° scored 0.151. The runner-up scored 0.668.
The grid is coarse on purpose. At 8×8 it no longer cares that one side is a preview frame and the other is 12 MP, or that the two are framed slightly differently. It is also faster than squinting at two thumbnails, and it gives the same answer every time you run it.
The front camera ignores videoRotationAngle = 90
Since iOS 17 the rotation you want goes in AVCaptureConnection.videoRotationAngle. I set 90 on the video data connection. Front camera frames kept arriving at 1608×1206, wider than they are tall, as if nothing had been set. No error, no console line.
The angle from AVCaptureDevice.RotationCoordinator did not help with this either. It tells you what the angle ought to be. It does not tell you whether the connection applied it.
What fixed it was to stop trusting the setting and read the buffer. If width is greater than height, the frame still needs turning. Then crop, rotation and mirroring all went into a single uv transform matrix in the shader. The same code now draws correctly whether the connection honored the angle or ignored it, and nothing upstream has to know which happened.
The camera position travels with the frame
Switching between front and back produced a few frames drawn with the wrong mirroring and rotation. Going back to the rear camera left a brief upside-down ghost. The cause was reading position on the main thread. By the time the main thread learned about the switch, the capture queue was already delivering frames from the other camera.
Passing the front or back flag from the capture queue together with each frame fixed it. Anything that describes a frame should arrive with the frame. Looking it up later from shared state is how the ghost got in.
Front buffers are mirrored whatever EXIF says
Front camera photos came with metadata saying they were not mirrored. The buffer was mirrored anyway. If you render front camera pixels yourself, decide mirroring from which camera produced the frame. On this point the metadata is wrong.
After you rotate the pixels, orientation is 1
A photo buffer arrives in sensor orientation, and the only record of which way is up is a metadata tag. Re-render the buffer and that link is gone, so I turn the pixels upright myself.
Then I copied the original EXIF across, orientation 6 included. Every viewer rotated the already upright image another 90°, and the photo lay on its side. Once the pixels are upright, the orientation tag has to say 1. The rest of the metadata should come along, and part three is about how easily it doesn’t, but not that one field.
“If it’s wide, stand it up” only holds for sensor buffers
The width-over-height rule that rescued the front camera is right for buffers straight off the sensor, which always come in sideways. I later ran a horizontal photo imported from the library through the same function, and it tipped the photo over. The function needs to know where its input came from, not only its shape.
Every orientation bug in this list had the same fix underneath: print the buffer’s width and height next to the decision that uses them. Two numbers in the console would have caught both of my wrong guesses.
Filed under Build log