Thoughts on iOS 4 camera APIs: privacy issues, new UI possibilities?

August 17th, 2010 § Comments Off on Thoughts on iOS 4 camera APIs: privacy issues, new UI possibilities? § permalink

While playing with the new AVFoundation APIs, it occurred to me that in iOS 4, apps can now easily access the camera with no feedback to the user. Before, apps had to use UIImagePickerController, which shows the iris-opening animation before recording starts, even if you hide the preview image using cameraViewTransform. With AVFoundation’s AVCaptureSession, there is no indication to the user at all that the camera is in use unless the app provides its own. There is no permission alert, nor any LED indicator like a webcam. An app could secretly be recording your face with the iPhone 4’s front-facing camera and sending it to who knows where. I wonder if Apple’s app review team checks for this in some way?

On the other hand, the new APIs make it much easier to integrate non-photo-taking uses of the camera into an app. I could imagine using the iPhone 4’s front camera for non-touch gesture controls or facial expression recognition. Makes me wish I knew something about real time image processing!

Turn your iPhone into a vampire with AVFoundation and iOS 4

August 15th, 2010 § 7 comments § permalink

iOS 4 added a lot to AVFoundation, including classes and APIs that give you much more control over the iPhone camera. One of the things you can now do with the camera is read the video frame data in real time.

In this post, I’ve created a simple demo that simulates a Twilight-style vampire. In the Twilight series, vampires aren’t hurt by daylight; instead, they sparkle. Yes, sparkle.

Here are a couple of screenshots from the app:

And here’s a low-quality video of the vampire simulator in action.

The app detects the amount of light shining on the phone by doing very simple image analysis of the incoming video frames from the camera. The brighter the image seen by the camera, the more sparkles it draws on the vampire.

So how does this all work?
» Read the rest of this entry «

iDevBlogADay

August 8th, 2010 § Comments Off on iDevBlogADay § permalink

Like many people with barely-updated blogs, I want to blog more often. I’m not much of a writer but that can only change with practice, right? Plus, I’m a big believer in community and the sharing of knowledge, and I wanted to contribute to that more. But where would I find the writing discipline?

I’ve also been a fan of personal writing challenges for some time. For example, I’ve participated in National Novel Writing Month (NaNoWriMo) for many years, where the goal is to write a 50,000-word novel in 30 days (in November). The concrete and public goal, combined with the camaraderie of the others involved, makes for a fun creative exercise and certainly helps with motivation.

When I heard about #iDevBlogADay on Twitter, I had to know more about it. It apparently all started when independent iPhone developer @MysteryCoconut wanted the impetus to blog more often, a sentiment that I can definitely relate to myself. What began as an offhand tweet has ballooned into what could become a bonafide movement.

Here’s how it works: Each day of the week is assigned to two indie iOS developers. They must post a blog post on their assigned day. If they miss a day, they’re out and sent to the end of the current waiting list. The next blogger in the waiting list now takes that person’s place.

What’s cool about this is that this kind of motivation helps everyone. The explosion of shared knowledge and inspiration pouring forth from these blogs has been pretty awesome.

It’s now my turn to take a spot on the Sunday roster. I have to admit I’m intimidated, as the quality of the blog posts has been high! I can only hope I can live up to the standards that have been set. (And I apologize for the “fluffy” nature of this post—I had a “crunchier” blog idea that I had apparently been sitting on for far too long, since it was rendered obsolete by recent versions of the iPhone OS. That’ll teach me!)

Hats off to MysteryCoconut for starting a fun “game” that helps and fosters the entire iDevelopment community :)

Loading an image mask from a file

July 23rd, 2010 § 1 comment § permalink

illustration of an image being masked
Core Graphics image masks are handy, but if you want to load the mask image from a file, things don’t always work the way you expect.

The function CGImageCreateWithMask() can take either a mask or an image as the second parameter, but it turns out that Core Graphics (at least on iOS) is pretty picky about what is an acceptable image for the mask.

I’ve seen this snippet of code suggested in a few places:

CGImageRef mask = CGImageMaskCreate(CGImageGetWidth(image),
CGImageGetHeight(image), CGImageGetBitsPerComponent(image),
CGImageGetBitsPerPixel(image), CGImageGetBytesPerRow(image),
CGImageGetDataProvider(image), NULL, false);

the idea being that you create a mask with the pixels that are in the loaded image, but it turns out that this code is not 100% reliable either.

The truth of the matter is that CGImage is an incredibly versatile object. The bits that represent the image can be in a variety of formats, bit depths, and colour space. When you load an image from a file, you are not guaranteed what format those bits are going to be in—for example, there are reports online of how people can get image masks to work if they save it in one way from an image editing program, but not if they save it a different way (e.g. http://stackoverflow.com/questions/1133248/any-idea-why-this-image-masking-code-does-not-work )

Thus, I’ve found that the best and most reliable way to generate an image mask from an arbitrary image is to do this:

  1. Create a bitmap graphics context that is in an acceptable format for image masks
  2. Draw your image into this bitmap graphics context
  3. Create the image mask from the bits of the bitmap graphics context.

The following function has worked well for me so far:

CGImageRef createMaskWithImage(CGImageRef image)
{
    int maskWidth               = CGImageGetWidth(image);
    int maskHeight              = CGImageGetHeight(image);
    //  round bytesPerRow to the nearest 16 bytes, for performance's sake
    int bytesPerRow             = (maskWidth + 15) & 0xfffffff0;
    int bufferSize              = bytesPerRow * maskHeight;
 
    //  we use CFData instead of malloc(), because the memory has to stick around
    //  for the lifetime of the mask. if we used malloc(), we'd have to
    //  tell the CGDataProvider how to dispose of the memory when done. using
    //  CFData is just easier and cleaner.
 
    CFMutableDataRef dataBuffer = CFDataCreateMutable(kCFAllocatorDefault, 0);
    CFDataSetLength(dataBuffer, bufferSize);
 
    //  the data will be 8 bits per pixel, no alpha
    CGColorSpaceRef colourSpace = CGColorSpaceCreateDeviceGray();
    CGContextRef ctx            = CGBitmapContextCreate(CFDataGetMutableBytePtr(dataBuffer),
                                                        maskWidth, maskHeight,
                                                        8, bytesPerRow, colourSpace, kCGImageAlphaNone);
    //  drawing into this context will draw into the dataBuffer.
    CGContextDrawImage(ctx, CGRectMake(0, 0, maskWidth, maskHeight), image);
    CGContextRelease(ctx);
 
    //  now make a mask from the data.
    CGDataProviderRef dataProvider  = CGDataProviderCreateWithCFData(dataBuffer);
    CGImageRef mask                 = CGImageMaskCreate(maskWidth, maskHeight, 8, 8, bytesPerRow,
                                                        dataProvider, NULL, FALSE);
 
    CGDataProviderRelease(dataProvider);
    CGColorSpaceRelease(colourSpace);
    CFRelease(dataBuffer);
 
    return mask;
}

Example of use:

UIImage *maskSource = [UIImage imageNamed:@"mask.png"];
CGImageRef mask = createMaskWithImage(maskSource.CGImage);

Then use the mask as you wish, for example in the aforementioned CGImageCreateWithMask() or CGContextClipToMask()

And don’t forget to dispose of the mask when you’re done. createMaskWithImage() returns the mask with a retain count of 1, and expects the caller to take ownership.

CGImageRelease(mask);

Uncanny resemblance

June 7th, 2010 § Comments Off on Uncanny resemblance § permalink

Planet iDev tweak

March 4th, 2010 § Comments Off on Planet iDev tweak § permalink

Some of the articles included by Planet iDev were getting pretty long, so I’ve decided to display just excerpts of the posts on the website rather than the entire articles. The RSS/Atom feeds should still have the full text.

Some of the excerpts seem to be showing up empty. I’m not 100% sure of what is causing that, but I’ll look into it (I made some changes to FeedWordPress, so it’s likely something I did :)

Thanks!

Planet iDev

January 31st, 2010 § Comments Off on Planet iDev § permalink

When I was doing more Flash development, one of the most valuable web resources I used was Adobe Feeds. It aggregates many, many Adobe-oriented developer blogs from all over the web. Not only is it a great place to keep an eye on the pulse of the Flash development world, but it’s where smaller Flash developers can get more exposure to more of the community. Thanks to my own blog’s inclusion in Adobe Feeds, I was able to get some answers to some Flash questions more quickly than I probably would have otherwise, all while sharing what knowledge I had to as broad an audience as possible.

When I started doing iPhone development, I looked for something similar for iPhone dev blogs. While there are a lot of iPhone developers out there blogging, I could not find a good aggregator out there. I did find Planet iPhone SDK, but it does not seem to be active (I tried contacting the owner, but did not receive a response). Planet Cocoa is good and quite active, but iPhone development is just part of the coverage there (the rest is Mac desktop development, which I don’t do myself).

Thus, I am starting Planet iDev. Armed with FeedWordPress and a free WordPress theme, I have cobbled together a simple blog aggregator. I should note that many of the blogs I am aggregating I found on this tremendously helpful post by Travis Dunn.

Having never run a blog aggregator before, I am unsure of the ethics and etiquette of Planet-style sites. I have no intention of financially profiting from other people’s hard work, nor do I want to negatively impact others’ endeavours. With that in mind:

  • There will never, ever be ads on Planet iDev
  • All authors are credited, and all articles link back to their original sources (as do all of the items in the Planet iDev feed).
  • I have set up the WordPress install to not be indexed by search engines.

Regardless, if any blog owner wants to be removed from this aggregation, I will be more than happy to do so, and I offer my sincerest apologies.

On the other, if any iPhone developer out there wants their blog to be added to this Planet, simply email me at planetidev@gmail.com (or comment on this post) with your blog’s information, and it will be considered for inclusion. Preference will be given to blogs that focus on the development and production process, rather than blogs that are mostly for product promotion.

I still have some questions regarding Planet etiquette. While most Planet sites seem to include the entire contents of the aggregated blogs’ posts, I wonder if it would be more polite to only display an excerpt (but include the entire article in the feed). Please share any thoughts you have on this matter, or on Planet iDev in general.

I hope others out there find this resource valuable and useful, and I hope this helps encourage a sense of community. Thanks!

Visit Planet iDev.

iPhone offline web applications: tips and gotchas

January 24th, 2010 § Comments Off on iPhone offline web applications: tips and gotchas § permalink

I spent this evening updating my “iPhone VR” Javascript/CSS demo to work with iPhone OS 3.0 (it had stopped working 100% correctly since the OS update). I also decided to spend some time making it work as an offline-capable web application.

The basic process of making your web app cacheable offline is, in theory, fairly straightforward, and generally well-documented at Apple’s website. I ran into some interesting headaches though.

Gotchas:

  • First thing to note is that your web server must serve your cache manifest file with a MIME type of text/cache-manifest. This may mean editing mime.types to add a line that looks like this:
    text/cache-manifest   manifest

    or perhaps a line in httpd.conf that looks like this:

    AddType text/cache-manifest  .manifest
  • The next thing that had me stumped for a while: while your web server may know how to take the URL http://example.com/webapp/ and automatically and invisibly serve up the file http://example.com/webapp/index.html, your offline web app knows nothing of this mapping of webapp/ to webapp/index.html. If the URL bar reads http://example.com/webapp/ and you save it locally using a home screen bookmark, it will fail to launch correctly if the device is offline, even if index.html file has been cached. The device simply does not know to look for webapp/index.html instead of webapp/. Thus, you must ensure that the URL bar reads webapp/index.html before the user makes a home screen bookmark.
  • At first, I thought I would enforce the above with a tiny bit of JavaScript that simply reads window.location and redirects to window.location + "index.html" if the URL ends in a slash. This worked while online, but it broke my app when offline, even when the redirection was not taken. Why? It seems that any reference to window.location in your script is treated as network access. Since the device is offline, it generates an error alert.

    Instead, I made my app live at webapp/main.html, and created a small webapp/index.html file that simply redirects to main.html. (I could have used a server redirect inside of an .htaccess file instead, but I chose not to, for no particular reason.)

Tips:

  • You can specify a custom icon for the home screen bookmark using a <link> element, like so:
    <link rel="apple-touch-icon" href="custom_icon.png" />

    If you don’t want iPhone to automatically apply the “shine” effect on your icon, use the following instead:

    <link rel="apple-touch-icon-precomposed" href="custom_icon.png" />
  • Since iPhone OS 3.0, offline web apps can have custom splash screens, just a like a native app! Simply add a link element to your web page:
    <link rel="apple-touch-startup-image" href="/splash.png" />

If you want to see the results of all this, go to http://bunnyherolabs.com/iphone/xform/ on your iPhone or iPod Touch, then click the “+” (add bookmark) button and choose “Add to Home Screen.” Now you can click on the “Panorama” icon on your home screen to see the demo at any time, even when not connected to the internet.

Easter egg in your iPhone app? Don’t hide it from Apple

October 21st, 2009 § Comments Off on Easter egg in your iPhone app? Don’t hide it from Apple § permalink

Yesterday, when I logged into the iPhone Dev Centre, I was greeted with a “please agree to the new developer agreement” alert. As usual, I copied the text and diffed it with the previous version. The bulk of the changes had to do with allowing in-app purchases in free apps.

But the following change caught my eye: in section 12.2, “Termination”:

12.2 Termination
This Agreement and all rights and licenses granted by Apple hereunder and any services provided hereunder will terminate, effective immediately upon notice from Apple:

(f) if You engage, or encourage others to engage, in any misleading, fraudulent, improper, unlawful or dishonest act relating to this Agreement, including, but not limited to, misrepresenting the nature of Your submitted Application (e.g., hiding or trying to hide functionality from Apple’s review).

(emphasis added)

So be careful, hidden-unlockable-feature-mongers. Apple could terminate your licence!

Genesis^H^H^H^H^H^H^H Droid Does

October 18th, 2009 § Comments Off on Genesis^H^H^H^H^H^H^H Droid Does § permalink

Verizon’s hard-hitting “Droid Does” anti-iPhone ad (via TechCrunch):

I love my iPhone (and developing for it) but I definitely would like to see more competition in this space. I think this ad hits home for those of us who are frustrated with the iPhone’s shortcomings. On the other hand, the style of the end of the video makes me think of trailers for horror movies. I’m not sure that’s such a good association to make!

This ad campaign also reminds me of the old Sega Genesis commercials back in the day:

Where Am I?

You are currently browsing the iPhone category at bunnyhero dev.