On the other hand, the new APIs make it much easier to integrate non-photo-taking uses of the camera into an app. I could imagine using the iPhone 4′s front camera for non-touch gesture controls or facial expression recognition. Makes me wish I knew something about real time image processing!

I am using UIImagePickerController to let the user choose an image from their photo library, and allowing image editing (move and scale).

When the user is done moving and scaling, the OS calls my UIImagePickerControllerDelegate‘s didFinishPicking method with information about the original image and the cropping rect in editingInfo.

One might think that the cropping rectangle returned by the UIImagePickerController would be in the coordinate system of the original UIImage, since that would make the most sense. One would be wrong, however. Based on my own experiments on my iPhone and various threads on Apple message boards, it seems that sometimes the cropping rectangle is based on a 640×480 image (that is: the original photo is first resized to 640×480 pixels, and the cropRect is in the resized image’s coordinate system).

But not always! Screenshots taken on iPhone (using power button + home button), which are 320×480, have a cropRect based on the original image size.

So that’s easy to deal with then, right? Just compare the original image’s size to see if it’s 320×480 or not…

Not so fast. That does not always work. I have on my iPhone some 320×480 images that, when picked with the image picker, return cropRects that are in some weird coordinate system. An unzoomed (1:1) crop returns a width and height of 432×433! That would imply the image is scaled to 432×648 before cropping. Odd!

(Where did these images come from? Both of them were created by other iPhone apps, saving a 320×480 image to the photo library. UPDATE: it seems to happen with JPEGs of certain sizes/proportions. See my comments for sample images.)

Unfortunately, so far I have been unable to distinguish between these two types of 320×480 images in code. Without being able to get reliable info about the coordinate system of the cropping rect, the cropping rect is basically useless. (And no, you can’t just compare the cropRect to see if it’s 432 pixels wide :P Once you scale that picture in the image picker, the width could be anything, still in that weird coordinate system.)

If anyone has any code that reliably deals with all images (including images of arbitrary dimension [e.g 527 x 325] that are saved to the iPhone), at all zoom levels, please let me know :)