You see, the datastore limits entities to 1 Mb. I’m trying to store XML data in there, and sometimes that can exceed the 1 Mb limit.

XML being the verbose creature that it is compresses very nicely, so it occurred to me that if I selectively compress the larger blocks, I should be able to quite easily squeeze in underneath the limit. Sure enough, a 1.6 Mb XML block compressed into about 200 Kb.

App Engine makes it very easy to define custom properties on data models, so I’ve written a CompressibleTextProperty class that automatically compresses/decompresses properties above a certain size. This means that there’s no performance loss for entities that are small enough to fit easily, but still enables the storage of bigger blocks of content.

The alternative was to break entities up into several different database entities, but this sounded like much more work, and sounded much less elegant.

So here’s what I came up with — it’s used the same way the other Property types are used.Download it here: compressible_text_property.py

#!/usr/bin/env python
# encoding: utf-8
"""
compressible_text_property.py
 
A string property that will automatically be stored compressed if larger than a given length threshold
 
Created by Michael Tyson on 2011-01-07.
Copyright (c) 2011 A Tasty Pixel. All rights reserved.
 
BSD LICENSE
"""
 
from google.appengine.ext import db
from google.appengine.api import datastore_types
import zlib
Text = datastore_types.Text
 
LENGTH_THRESHOLD            = 500000 # Bytes
EXPECTED_ZLIB_HEADER        = u"x\x9c"
 
class CompressibleTextProperty(db.TextProperty):
    """A string property that will automatically be stored compressed if larger than a given length threshold
 
    This is designed to be used with textual properties that may exceed App Engine's 1MB entity size limit.
    Note that, if compressed, property will not be searchable.
    """
 
    def validate(self, value):
      """Validate text property; Nicked verbatim from TextProperty.
 
      Returns:
        A valid value.
 
      Raises:
        BadValueError if property is not instance of 'Text'.
      """
      if value is not None and not isinstance(value, Text):
        try:
          value = db.Text(value)
        except TypeError, err:
          raise BadValueError('Property %s must be convertible '
                              'to a Text instance (%s)' % (self.name, err))
      value = super(db.TextProperty, self).validate(value)
      if value is not None and not isinstance(value, Text):
        raise BadValueError('Property %s must be a Text instance' % self.name)
      return value
 
    def get_value_for_datastore(self, model_instance):
        """For writing to the datastore: Performs compression if length is greater than the threshold"""
        value = super(CompressibleTextProperty, self).get_value_for_datastore(model_instance)
        if len(value) > LENGTH_THRESHOLD and not value.startswith(EXPECTED_ZLIB_HEADER):
            value = unicode(zlib.compress(value), 'ISO-8859-1')
        return Text(value)
 
    def make_value_from_datastore(self, value):
        """For reading from the datastore: Decompresses if compressed data detected"""
        if value is None:
            return None
        if value.startswith(EXPECTED_ZLIB_HEADER):
            value = zlib.decompress(value.encode('ISO-8859-1'))
        return value
 
    data_type = Text

I came across a great XCode project wrapper for OpenSSL yesterday, by Stephen Lombardo. This is an XCode project file that contains a target to build OpenSSL from source, and works with both Mac and iOS projects. I made some modifications to it, in order to make it work by just dropping in the OpenSSL source tarball, without having to dirty up your source tree with the extracted OpenSSL distribution.

Here’s how to use it:

1. Download the OpenSSL source.
2. Put the downloaded OpenSSL source tar.gz into the same folder as openssl.xcodeproj (I put it in Library/openssl within my project tree).
3. Drag the openssl.xcodeproj file into your main project tree in XCode.
4. Right-click on your project target, and add openssl.xcodeproj under “Direct Dependencies” on the General tab.

5. On the Build tab for your project’s target, find the “Header Search Paths” option, and add the path:

   $(SRCROOT)/Library/openssl/build/openssl.build/openssl/include

   (Assuming you’ve put openssl.xcodeproj at the path Library/openssl — adjust as necessary).

6. Expand your target’s “Link Binary With Libraries” build stage, and drag libcrypto.a from the openssl.xcodeproj group.

Then, you can just import and use as normal (#import <openssl/dsa.h>, etc).

Download it here

That’s all well and good, but if you have more than one object in your app that may do things with the network simultaneously, you’re going to clobber yourself.

A nice and easy solution: Maintain an activity counter and create a category on UIApplication to maintain it, and show or hide the indicator as appropriate. Then, whenever you start doing something with the network:

[[UIApplication sharedApplication] showNetworkActivityIndicator];

…And when you’re done:

[[UIApplication sharedApplication] hideNetworkActivityIndicator];

Here’s a category that’ll do it:

@interface UIApplication (TPAdditions)
- (void)showNetworkActivityIndicator;
- (void)hideNetworkActivityIndicator;
@end
 
 
static NSInteger __activityCount = 0;
 
@implementation UIApplication (TPAdditions)
- (void)showNetworkActivityIndicator {
    if ( __activityCount == 0 ) {
        [self setNetworkActivityIndicatorVisible:YES];
    }
    __activityCount++;
}
- (void)hideNetworkActivityIndicator {
    __activityCount--;
    if ( __activityCount == 0 ) {
        [self setNetworkActivityIndicatorVisible:NO];
    }    
}
@end

Any app that makes use of the Remote IO audio unit framework (which is generally necessary for the kind of responsiveness required in a realtime musical app) provides audio to the hardware via a callback, which is periodically called when the hardware is ready for more.

The trick here is to provide the right chunk of samples in this callback for the current time position.

Loopy achieves this by:

1. Keeping track of where in the timeline we are at the time the callback is called

This is easily accomplished by keeping a record of the time the clock was started, subtracting this from the current time, and possibly performing a modulus with the tempo. For example:

• (now - startTime) % timePerBar gives the number of time units into the current bar (lets call it timeIntoBar).
• timeIntoBar / (timePerBar/beatsPerBar) gives the number of beats into the current bar, and
• timeIntoBar % (timePerBar/beatsPerBar) gives us the time into the current beat.

2. Determining first if we should be playing audio at this time, and if so, which samples should be playing

This involves first converting our time units from step 1 into samples. For instance, you can convert microseconds to samples by dividing your time by 1000000/yourSampleRate. Aside: Of course, you can convert back from samples to time by multiplying instead of dividing.

Next, in the case of Loopy’s metronome, for example, we test for whether samplesIntoBeat < sound.lengthInSamples. If so, that means we should be playing audio. If the sound was a loop, of course, we could be always playing.

The offset into the sound, in samples, is just samplesIntoBeat, in the case of the simple metronome. In the case of a loop, you probably will be more interested in the number of samples into your loop — so instead of determining (now - startTime) % timePerBar, you may be interested in (now - startTime) % timePerLoop.

So, we want to return the requested number of samples starting from this offset into the sample array representing our audio.

3. Returning smooth audio in time

Note that if you just go returning any old set of samples, willy-nilly, you’re going to get nasty clicks and pops from discontinuities you get by not matching the start of your next buffer to the last one.

To ensure smoothness, Loopy keeps track of the offset of the last samples we returned, and just return the immediately following bunch of samples — unless we’re more than some threshold number of samples out of time, in which case we’ll suffer the pop in order to stay synchronised. Actually, you can even generally avoid the pop if you smoothly blend buffers over a short time, removing any discontinuity.

Final words

The example above was a relatively trivial one, for a metronome sound. For longer audio that may span multiple bars, you’ll probably want to perform a modulus by the length of your audio clip, possibly quantised to your time signature, and possibly using a per-loop time base, so you can start the loop at any point in your timeline and have it begin from the start. This is something Loopy doesn’t currently do — Loopy will keep your loops synchronised so when you start a loop playing, it’ll play whatever part corresponds to the current timeline, not from the start of the loop. Maybe it’ll be an option in the future?

I wrote a little about the timing of loops in my second article on Loopy’s implementation.

This can be done by adding -[NSException raise] and objc_exception_throw to your breakpoints list.

Once an exception happens, you can then check out the exception itself to see what went wrong. The approach varies between platforms. If you’re in the simulator (or any Mac OS X app running on Intel), the exception will be stored in the $eax register. Take a look by typing:

po $eax

If you’re on the iPhone, it’ll be $r0, so:

po $r0

So, I wrote a class that provides similar functionality to CABasicAnimation, but works on any object. I thought I’d share it.

Features:

• From/to value settings (currently only supports NSNumber and scalar numeric types, but easily extendable)
• Duration, delay settings
• Timing functions: Linear, ease out, ease in, and ease in/ease out
• Animation chaining (specify another configured animation for chainedAnimation, and it’ll be fired once the first animation completes)
• Delegate notification of animation completion
• Uses a single timer for smooth lock-step animation
• Uses CADisplayLink if available, to update in sync with screen updates

Use it like this:

- (void)startMyAnimation {
  TPPropertyAnimation *animation = [TPPropertyAnimation propertyAnimationWithKeyPath:@"state"];
  animation.toValue = [NSNumber numberWithFloat:1.0]; // fromValue is taken from current value if not specified
  animation.duration = 1.0;
  animation.timing = TPPropertyAnimationTimingEaseIn;
  [animation beginWithTarget:self];
}

Make sure you also include the QuartzCore framework, used to access CADisplayLink, if it’s available.

It’s BSD-licensed.

Grab it here: TPPropertyAnimation.zip

• cufón – fonts for the people A very impressive framework that embeds any font into a website, via javascript and the canvas element. Great cross-browser support.
• mikeash.com: Method Replacement for Fun and Profit Method replacement and method swizzling in Objective-C.
• Core Data Tutorial: How To Use NSFetchedResultsController | Ray Wenderlich
• TwitThis – Use Multiple Twitter Clients on your iPhone Application The class TwitterClientManager loads a list list of supported Twitter clients is loaded from a plist file, which can be extended to support more clients in the future;
  Each Twitter client is represented by an instance of the TwitterClient class;
  The user can choose his preferred Twitter client at any time, and launch the application by a simple touch; the TwitterClientManager class stores the selected value in the user settings.

Why we don’t have common image format with both transparency and lossy compression is beyond me, but there’s a relatively easy alternative: Using a JPEG and masking it with another JPEG.

Based on Rodney Aiglstorfer’s solution on how to mask an image, I derived a category on UIImage which would apply a mask to an image. The method required a little tweaking to work with JPEG images — the CGImageCreateWithMask function won’t work correctly on source images that don’t have an alpha channel, so one has to create one first, from the original. Jean Regisser figured out the solution which he presents in a comment on the above article, but it needs one more addition: A check on line 37 for kCGImageAlphaNoneSkipLast. Update: Oh, and one more – kCGImageAlphaNoneSkipFirst

So, the complete category for applying a mask to a JPEG image, to achieve the same result as using a PNG but with less download time for your users:

// Header
 
@interface UIImage (TPAdditions)
- (UIImage*)imageByMaskingUsingImage:(UIImage *)maskImage;
@end
 
 
// Implementation
 
CGImageRef CopyImageAndAddAlphaChannel(CGImageRef sourceImage) {
    CGImageRef retVal = NULL;
 
    size_t width = CGImageGetWidth(sourceImage);
    size_t height = CGImageGetHeight(sourceImage);
 
    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
 
    CGContextRef offscreenContext = CGBitmapContextCreate(NULL, width, height, 
                                                          8, 0, colorSpace, kCGImageAlphaPremultipliedFirst);
 
    if (offscreenContext != NULL) {
        CGContextDrawImage(offscreenContext, CGRectMake(0, 0, width, height), sourceImage);
 
        retVal = CGBitmapContextCreateImage(offscreenContext);
        CGContextRelease(offscreenContext);
    }
 
    CGColorSpaceRelease(colorSpace);
 
    return retVal;
}
 
 
@implementation UIImage (TPAdditions)
 
 
- (UIImage*)imageByMaskingUsingImage:(UIImage *)maskImage {
 
    CGImageRef maskRef = maskImage.CGImage; 
    CGImageRef mask = CGImageMaskCreate(CGImageGetWidth(maskRef),
                                        CGImageGetHeight(maskRef),
                                        CGImageGetBitsPerComponent(maskRef),
                                        CGImageGetBitsPerPixel(maskRef),
                                        CGImageGetBytesPerRow(maskRef),
                                        CGImageGetDataProvider(maskRef), NULL, false);
 
    CGImageRef source = [self CGImage];
 
    NSInteger alphaInfo = CGImageGetAlphaInfo(source);
    if ( alphaInfo == kCGImageAlphaNone || alphaInfo == kCGImageAlphaNoneSkipLast || alphaInfo == kCGImageAlphaNoneSkipFirst ) {
        source = CopyImageAndAddAlphaChannel(source);
    }
 
    CGImageRef masked = CGImageCreateWithMask(source, mask);
    CGImageRelease(mask);
 
    if ( source != [self CGImage] ) {
        CGImageRelease(source);
    }
 
    UIImage *result;
    if ( [UIImage respondsToSelector:@selector(imageWithCGImage:scale:orientation:)] ) {
        result = [UIImage imageWithCGImage:masked scale:self.scale orientation:self.imageOrientation];
    } else {
        result = [UIImage imageWithCGImage:masked];
    }
 
    CGImageRelease(masked);
 
    return result;
}
 
@end

Note that the image mask should be another JPEG (or PNG, if you really like), without transparency, and greyscale, where black represents full opacity, and white represents full transparency.

Reginald is a kindly old gentleman devoted to assisting you with those tricky regular expressions.

Provide some sample input, and your regular expression, and Reginald will provide you with colour-coded output and a list of all your matches and the corresponding capture groups for your exploration. Select a match or capture group in the list to the right, and the corresponding text will be selected in the panel to the left.

Reginald is built on RegexKitLite, and so uses the ICU syntax.

It will run on Mac OS X 10.6 and above.

Download Reginald here, or access the source on GitHub.

The solution I came up with for this was to implement a category on UINavigationBar and UIToolbar, and overriding drawRect: with a method that does absolutely nothing. Then I can place my own textures behind the bar, and they’ll be seen, instead of the default bar background.

@interface UINavigationBar (TransparentAdditions)
@end
@implementation UINavigationBar (TransparentAdditions)
- (void)drawRect:(CGRect)rect {
    // Do nothing!
}
@end

For UIToolBar, if you’re using it within a UINavigationController, you’ll want to also override drawLayer:inContext:, as this appears to be used instead of drawRect: when used within a navigation controller, for some weird reason.

Note that this method will affect all bars in your app. If you only want some bars to be transparent, you’ll need to do a little objc-hocus-pocus. Thanks to Mike Ash for this solution on method replacement (read that article for the whys and hows). This technique replaces the default methods as before, but keeps track of the defaults. If you now set the tintColor of the bar to [UIColor clearColor], the bar will have a transparent background. Otherwise, it’ll just look the same as usual.

For UIToolbar (same principle for UINavigationBar):

#import <objc/runtime.h>
 
// Keep track of default implementation
static void (*_origDrawRect)(id, SEL, CGRect);
static void (*_origDrawLayerInContext)(id, SEL, CALayer*, CGContextRef);
 
// Override for drawRect:
static void OverrideDrawRect(UIToolbar *self, SEL _cmd, CGRect r) {
    if ( [[self tintColor] isEqual:[UIColor clearColor]] ) {
        // Do nothing
    } else {
        // Call default method
        _origDrawRect(self, _cmd, r);
    }
}
 
// Override for drawLayer:inContext:
static void OverrideDrawLayerInContext(UIToolbar *self, SEL _cmd, CALayer *layer, CGContextRef context) {
    if ( [[self tintColor] isEqual:[UIColor clearColor]] ) {
        // Do nothing
    } else {
        // Call default method
        _origDrawLayerInContext(self, _cmd, layer, context);
    }
}
 
 
@implementation UIToolbar (TransparentAdditions)
 
+ (void)load {
    // Replace methods, keeping originals
    Method origMethod = class_getInstanceMethod(self, @selector(drawRect:));
    _origDrawRect = (void *)method_getImplementation(origMethod);
 
    if(!class_addMethod(self, @selector(drawRect:), (IMP)OverrideDrawRect, method_getTypeEncoding(origMethod)))
        method_setImplementation(origMethod, (IMP)OverrideDrawRect);
 
    origMethod = class_getInstanceMethod(self, @selector(drawLayer:inContext:));
    _origDrawLayerInContext = (void *)method_getImplementation(origMethod);
 
    if(!class_addMethod(self, @selector(drawLayer:inContext:), (IMP)OverrideDrawLayerInContext, method_getTypeEncoding(origMethod)))
        method_setImplementation(origMethod, (IMP)OverrideDrawLayerInContext);
}
 
@end

You can now add background texture to the bars in a number of ways. These are two I’ve used:

• By adding a CALayer to [bar layer] — but note that UINavigationBar will try to add elements at index 0, underneath your background. To make this work, I provided a subclassed CALayer (and overrode UINavigationBar’s +layer method) which only lets you insert layers at index 0, via a custom method, and override insertLayer:atIndex: method, setting index to 1 if it’s 0. UIToolbar doesn’t require this.
• Or, by adding a CALayer to your view layer. Note that the view’s bounds do not cover the UINavigationBar; I had to offset the layer by the height of the bar in question (navigationBarLayer.frame = CGRectMake(0, -self.navigationController.navigationBar.frame.size.height, [barImage size].width, [barImage size].height);, for example), and set self.view.clipsToBounds = NO to allow the layer to be seen.

Of course, you can also draw the texture in drawRect:, instead. It’s entirely up to you. The advantage in using a CALayer is that it can overlap the view boundary, for effects like drop shadows.

• Implementing iBooks page curling using a conical deformation algorithm Excellent summary of how to implement a convincing page turn animation in OpenGL
• Multiplottr.com — Plot, save and share multiple locations on your own customized maps. Batch plot multiple addresses
• gmaps.kaeding.name :: Plot multiple locations on Google Maps Enter addresses, one per line, to plot all entries on a map at once
• Free iPhone App Marketing — Online and iPhone Marketing Includes a useful list of review sites, blogs, etc.

Talkie is my newest product, a Walkie Talkie for iPhone and Mac.

In Part 1 of this series, I wrote about basic broadcasting. This works fine with one network device, but it’s worth discussing how to send through all devices, so you can communicate with others connected via, say, Ethernet and WiFi simultaneously.

So, in Part 2 I’ll write about the approach I took in Talkie for broadcasting from all network devices (a.k.a. network interfaces), so that one can communicate with others connected via WiFi, Ethernet (on a Mac), and any other network devices simultaneously.

Bind them

From Part 1, we have a working broadcast mechanism, but it will only send through the default interface — whatever you’re connected to the network via. This is often sufficient, but if you have more than one device that you communicate through, like Ethernet and WiFi, then you will find that it only works with one.

In order to send through all your connected network interfaces, we need to create one socket for each interface, and bind the socket to its corresponding interface.

Here’s how:

First, we need to obtain a list of all network interfaces with getifaddrs.

#include <ifaddrs.h>
...
struct ifaddrs *addrs;
int result = getifaddrs(&addrs);
if ( result < 0 ) {
  // Error occurred
  return 0;
}

Now, addrs is a list of interfaces that we can iterate over. We now do so, picking out those devices that support broadcasting, and that aren’t loopback or point-to-point devices — loopback is an internal interface that is provided for your computer’s inner dialogue, and point-to-point (ppp) devices include dialup interfaces, 3G modems and the like. We can exclude those guys.

const struct ifaddrs *cursor = addrs;
while ( cursor != NULL ) {
  if ( cursor->ifa_addr->sa_family == AF_INET 
          && !(cursor->ifa_flags & IFF_LOOPBACK) 
          && !(cursor->ifa_flags & IFF_POINTOPOINT) 
          &&  (cursor->ifa_flags & IFF_BROADCAST) ) {
 
    // We will do some stuff in here
 
  }
  cursor = cursor->ifa_next;
}

Now, for each interface that meets our criteria, we create a socket (which we covered in Part 1), then bind the socket to the network interface, to force transmission from that particular device. Finally, as we did in Part 1, we enable broadcasting using setsockopt with SO_BROADCAST.

We want to store the sockets we create in an array, so we can access them later. If we assume a maximum number of interfaces we will support (lets call it kMaxSockets), we can just use an array of that length. So, putting it together:

#define kMaxSockets 16
...
int sock_fds[kMaxSockets];
int number_sockets = 0;
 
while ( cursor != NULL && number_sockets < kMaxSockets ) {
  if ( cursor->ifa_addr->sa_family == AF_INET 
          && !(cursor->ifa_flags & IFF_LOOPBACK) 
          && !(cursor->ifa_flags & IFF_POINTOPOINT) 
          &&  (cursor->ifa_flags & IFF_BROADCAST) ) {
 
    // Create socket
    sock_fds[number_sockets] = socket(AF_INET, SOCK_DGRAM, 0);
    if ( sock_fds[number_sockets] == -1 ) {
      // Error occurred
      return 0;
    }
 
    // Create address from which we want to send, and bind it
    struct sockaddr_in addr;
    memset(&addr, 0, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_addr = ((struct sockaddr_in *)cursor->ifa_addr)->sin_addr;
    addr.sin_port = htons(0);
 
    int result = bind(sock_fds[number_sockets], (struct sockaddr*)&addr, sizeof(addr));
    if ( result < 0 ) {
      // Error occurred
      return 0;
    }
 
    // Enable broadcast
    int flag = 1;
    result = setsockopt(sock_fds[number_sockets], SOL_SOCKET, SO_BROADCAST, &flag, sizeof(flag));
    if ( result != 0 ) {
      // Error occurred
      return 0;
    }
 
    number_sockets++;
  }
  cursor = cursor->ifa_next;
}

Finally, as before, we can setup a broadcast address to send to, and use sendto to broadcast, this time for each socket we created:

// Initialise broadcast address
struct sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = INADDR_BROADCAST;
addr.sin_port = htons(kPortNumber);
 
// Send through each interface
int i;
for ( i=0; i<number_sockets; i++ ) {
  int result = sendto(sock_fds[i], data, length, 0, (struct sockaddr*)&addr, sizeof(addr));
  if ( result < 0 ) {
    // Error occurred
    return 0;
  }
}

Note that the receive routine only needs a single socket, as we can receive on any interface when we use INADDR_ANY. So, the receive routine needs no changes from the single-interface version we saw in Part 1.

Here’s the test app with the above modification: broadcast_sample_all_interfaces.c

Again, compile by opening Terminal, and typing make broadcast_sample_all_interfaces or cc -o broadcast_sample_all_interfaces broadcast_sample_all_interfaces.c, then run it with ./broadcast_sample_all_interfaces "Message to send" to send, or just ./broadcast_sample_all_interfaces with no arguments to listen.

You may notice that multiple messages may be received: These have probably arrived via multiple network interfaces, virtual or otherwise. It’s usually a good idea to check for duplicate messages, if this is an issue for program operation, by including a sequence number into the message — this will be discussed in Part 3.

It also may be a good idea to ignore your own messages, which may find their way back to you. One way to accomplish this is to examine the source address (addr in the example above) and compare it with your local interface addresses (stored in addrs, above). If you get a match, the message came from you, and you can drop it.

Multicast

Broadcast is fine when everyone on the local network is interested in what you have to say. If this isn’t the case, though (lets face it, those Chuck Norris jokes aren’t for everyone), effort is wasted delivering to those who aren’t particularly interested.

Multicast works by using a specific address that one ‘subscribes’ to in order to receive messages sent to that address. So, it’s opt-in, allowing for better efficiency and one day, Internet-wide support for ‘to-many’ communications.

Well, in theory. Actually, multicast is still quite new, and for the most part — from what I understand — it behaves pretty much like broadcast on a local area network. However, support can only increase, and given that many services already use it — Multicast DNS (mDNS), also known as Bonjour, being one of the most well-known examples — it seems a good idea to follow their lead. Note also that IPv6, the successor to IP as we know it, and our saviour-to-be from our little Internet overpopulation problem (among other things), doesn’t even have broadcast provisions — the future is all multicast.

So, for these reasons, Talkie speaks multicast, instead of plain ol’ broadcast.

Making use of multicast is relatively straightforward: To receive, you join a multicast group using setsockopt with IP_ADD_MEMBERSHIP, and the address of the multicast group, which is in the range 224.0.0.0-239.255.255.255 (for IPv4, of course). To send, you just use sendto to transmit data to a multicast group address.

Using multicast on all network interfaces is just a little more complicated. Here’s how it’s done with Talkie:

Sending

The send routine is very similar to the one above, using broadcast. However, instead of using bind to specify the outgoing network interface and enabling broadcast, we assign a multicast interface using setsockopt with IP_MULTICAST_IF. And, instead of transmitting to the broadcast address, we transmit to the multicast group address.

Again, we loop through all network interfaces. This time, we pick out those that support multicast (ifa_flags & IFF_MULTICAST):

const struct ifaddrs *cursor = addrs;
while ( cursor != NULL ) {
  if ( cursor->ifa_addr->sa_family == AF_INET 
          && !(cursor->ifa_flags & IFF_LOOPBACK) 
          && !(cursor->ifa_flags & IFF_POINTOPOINT) 
          &&  (cursor->ifa_flags & IFF_MULTICAST) ) {
 
    // We will do some stuff in here
 
  }
  cursor = cursor->ifa_next;
}

And, after creating the socket, we assign the interface:

if ( setsockopt(sock_fds[number_sockets], IPPROTO_IP, IP_MULTICAST_IF, &((struct sockaddr_in *)cursor->ifa_addr)->sin_addr, sizeof(struct in_addr)) != 0  ) {
  // Error occurred
  return 0;
}

Finally, as a nicety, we can disable loopback so that we don’t receive our own messages. This isn’t 100% reliable, as certain network conditions can result in the local machine still receiving its outgoing messages, but it can improve efficiency:

u_char loop = 0;
if ( setsockopt(sock_fds[number_sockets], IPPROTO_IP, IP_MULTICAST_LOOP, &loop, sizeof(loop)) != 0 ) {
  // Error occurred
  return 0;
}

Now that our sockets are set up, we can prepare to send to the multicast address:

#define kMulticastAddress "224.0.0.123"
...
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = inet_addr(kMulticastAddress);
addr.sin_port = htons(kPortNumber);

And, as before, we send through each of the sockets we created:

int i;
for ( i=0; i<number_sockets; i++ ) {
  if ( sendto(sock_fds[i], data, length, 0, (struct sockaddr*)&addr, sizeof(addr)) < 0 ) {
    // Error occurred
    return 0;
  }
}

Receiving

Receiving messages from a multicast group on several network interfaces is a little more involved than doing so with broadcast: We need to subscribe to the multicast group from each network interface, explicitly. If we were to just specify no device in particular, the system would choose a single interface for us, neglecting the others.

Joining the multicast group for each interface takes place in a now-familiar loop:

const struct ifaddrs *cursor = addrs;
while ( cursor != NULL ) {
  if ( cursor->ifa_addr->sa_family == AF_INET 
          && !(cursor->ifa_flags & IFF_LOOPBACK) 
          && !(cursor->ifa_flags & IFF_POINTOPOINT) 
          &&  (cursor->ifa_flags & IFF_MULTICAST) ) {
 
    // We will do some stuff in here
 
  }
  cursor = cursor->ifa_next;
}

For each network device, we use the IP_ADD_MEMBERSHIP property with setsockopt to join — thereby subscribing to any messages sent to the multicast group address that reach that network interface.

First, we prepare the join request structure. This provides the multicast group address, and the network interface:

struct ip_mreq multicast_req;
memset(&multicast_req, 0, sizeof(multicast_req));
multicast_req.imr_multiaddr.s_addr = inet_addr(kMulticastAddress);
multicast_req.imr_interface = ((struct sockaddr_in *)cursor->ifa_addr)->sin_addr;

Now we use this structure to join the group:

if ( setsockopt(sock_fd, IPPROTO_IP, IP_ADD_MEMBERSHIP, &multicast_req, sizeof(multicast_req)) < 0 ) {
  // Error occurred
  return 0;
}

Now, a caveat: While it’s perfectly legal to join the same multicast group on more than one network interface, and up to 20 memberships may be added to the same socket (see ip(4)), for some reason, OS X spews ‘Address already in use’ errors when we actually attempt it.

As a workaround, we can ‘drop’ the membership first, which would normally have no effect, as we have not yet joined on this interface. However, it enables us to perform the subsequent join, without dropping prior memberships.

So, before we perform the above IP_ADD_MEMBERSHIP, we do a IP_DROP_MEMBERSHIP first:

setsockopt(sock_fd, IPPROTO_IP, IP_DROP_MEMBERSHIP, &multicast_req, sizeof(multicast_req));

This sets up our socket to receive messages sent to the multicast group that are received via any interface.

Here it is all put together: multicast_sample.c

Compile by opening Terminal, and typing make multicast_sample or cc -o multicast_sample multicast_sample .c, then run it with ./multicast_sample "Message to send" to send, or just ./multicast_sample with no arguments to listen.

Still to come

So, now we mostly have networking covered. There’s one obvious omission, though, for an iPhone app: Bluetooth. In Part 3, I’ll discuss how to perform communications over Bluetooth on the iPhone, in a way that’s compatible with the above generic network communications. I’ll also talk about how to connect to other devices automatically, without user intervention — This is one particularly popular feature of Talkie that allows it to behave more like a real walkie-talkie.

I promised in Part 1 that I’d talk about packet formats. We’ve covered a lot of ground in Part 2, however, so it shall be postponed to Part 3 — I’ll discuss how to ensure you get messages in the correct order by using sequence numbers, as well as providing for versioning and a few other bits and pieces.

I couldn’t figure out how to easily open up my databases, other than manually creating a managed object model, then a persistent store coordinator, then a managed object context on the console. I couldn’t find any existing tools, and I wanted a quick workflow for opening up my databases, so I put together a script that prompts for the application bundle or .xcdatamodel(d) data model file, then prompts for the XML (.xml), binary (.binary) or SQLite (.sql or anything else) database file, and opens up the inspector.

I wrote it as an Applescript that just calls upon F-Script to evaluate the script, and saved it in an application bundle so I can pull it up quickly.

Here it is:

Core Data Browser.app

It just needs the F-Script app to be available.

Upon opening, the managed object context is available on the console as “context“. So, aside from using F-Script’s object browser, you can also do things like:

> request := (NSFetchRequest alloc) init
> request setEntity:(NSEntityDescription entityForName:'MyEntity' inManagedObjectContext:context)
> request setPredicate:(NSPredicate predicateWithFormat:'type = 3')
> result := context executeFetchRequest:request error:nil
> result
_PFArray {<NSManagedObject: 0x2006cf740> (entity: MyEntity; id: 0x20064c9e0 <x-coredata://BAC82A67-CC41-48C2-8A96-693B67A501D6/MyEntity/p1> ; data: <fault>), 
<NSManagedObject: 0x2006bdc80> (entity: MyEntity; id: 0x20064c9c0 <x-coredata://BAC82A67-CC41-48C2-8A96-693B67A501D6/MyEntity/p2> ; data: <fault>), 
<NSManagedObject: 0x2006bc680> (entity: MyEntity; id: 0x200651180 <x-coredata://BAC82A67-CC41-48C2-8A96-693B67A501D6/MyEntity/p3> ; data: <fault>)
...

Update: Now has better error reporting, and the option to load classes from a bundle.

For those interested, here’s the original F-Script:

panel := NSOpenPanel openPanel.
panel setAllowedFileTypes:{'xcdatamodel', 'app', 'framework'};setTitle:'Data model';setMessage:'Select a data model file or bundle containing a data model'.
classesButton := (NSButton alloc) initWithFrame:(0<>0 extent:200<>30).
classesButton setTitle:'Load classes in bundle';setButtonType:NSSwitchButton.
panel setAccessoryView:classesButton.
model := nil.
[model == nil] whileTrue:[
	error := FSPointer objectPointer.
	(panel runModal = NSFileHandlingPanelCancelButton) ifTrue:[ (NSException exceptionWithName:'CancelException' reason:'Cancelled' userInfo:nil) raise ] .
	(((panel URL) absoluteString) hasSuffix:'xcdatamodel') ifTrue:[
		model := ((NSManagedObjectModel alloc) initWithContentsOfURL:(panel URL))
	] ifFalse:[
		bundle := (NSBundle bundleWithURL:(panel URL)).
		(bundle == nil) ifFalse:[
			model := (NSManagedObjectModel mergedModelFromBundles:{bundle}).
			(model ~~ nil & (FSBoolean booleanWithBool:(classesButton state) = NSOnState)) ifTrue:[
				(FSBoolean booleanWithBool:(bundle loadAndReturnError:error)) ifFalse:[
					model := nil.
					(NSAlert alertWithMessageText:'Could not load bundle' defaultButton:'OK' alternateButton:nil otherButton:nil informativeTextWithFormat:((error at:0) localizedDescription)) runModal
				]
			]
		]
	].
	(model == nil & ((error at:0) == nil)) ifTrue:[ (NSAlert alertWithMessageText:'Could not load model from file or bundle' defaultButton:'OK' alternateButton:nil otherButton:nil informativeTextWithFormat:'') runModal ]
].
panel setAccessoryView:nil.
store := ((NSPersistentStoreCoordinator alloc) initWithManagedObjectModel:model).
panel setAllowedFileTypes:nil; setTitle:'Database file';setMessage:'Select Core Data SQLite database'.
opened := nil.
[opened == nil] whileTrue:[
	(panel runModal = NSFileHandlingPanelCancelButton) ifTrue:[ (NSException exceptionWithName:'CancelException' reason:'Cancelled' userInfo:nil) raise ] .
	type := NSSQLiteStoreType.
	((((panel URL) absoluteString) lowercaseString) hasSuffix:'xml') ifTrue:[type := NSXMLStoreType].
	((((panel URL) absoluteString) lowercaseString) hasSuffix:'binary') ifTrue:[type := NSBinaryStoreType].
	error := FSPointer objectPointer.
	opened := (store addPersistentStoreWithType:type configuration:nil URL:(panel URL) options:nil error:error).
	(opened == nil) ifTrue:[ (NSAlert alertWithMessageText:'Could not load database' defaultButton:'OK' alternateButton:nil otherButton:nil informativeTextWithFormat:((error at:0) localizedDescription)) runModal ]
].
context := ((NSManagedObjectContext alloc) init).
context setPersistentStoreCoordinator:store.
context inspectWithSystem:sys

cell.imageView.layer.masksToBounds = YES;
cell.imageView.layer.cornerRadius = 5.0;

Set this up when you create the cell (make sure you #import <QuartzCore/QuartzCore.h> at the top, of course). It would appear the UIImageView control creates sublayers to display the actual image content, which is why we use the masksToBounds property to then clip any sublayers.

I noticed a lot of people are seeking answers to the silly behaviour of UITableView with the grouped (UITableViewStyleGrouped) style and images:

Images don’t get clipped to the rounded cell border, which looks nasty. This technique is one way to remedy that:

One caveat – due to the inexplicable way the image view within the table view cell scales image content, there’s not really a simple, sensible way to provide an inset margin from the table view cell boundary to complement this rounded border effect.

I tried setting the frame property of the UIImageView itself (cell.imageView.frame), as well as setting the frame of the image view’s layer. I also tried applying a scale transform to the layer, with strangely inconsistent results: Setting scale to, say, 50%, made the image view 40x40px, only a pixel or two smaller than the full size. This may be because another entity (the table view cell?) performs scaling of the content, instead of the actual image view; given that my original image was 80×80, a 50% scale would result in 40×40.

My solution was to steer clear of that nonsense and just provide appropriately scaled images straight to the image view. Here’s a simple category on UIImage to scale an image:

@interface UIImage (TPAdditions)
- (UIImage*)imageScaledToSize:(CGSize)size;
@end
 
@implementation UIImage (TPAdditions)
- (UIImage*)imageScaledToSize:(CGSize)size {
    UIGraphicsBeginImageContext(size);
    [self drawInRect:CGRectMake(0, 0, size.width, size.height)];
    UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();
    return image;
}
@end

So, then I just do something like this in the UITableView data provider:

UIImage *image = account.image;
if ( image ) {
   cell.image = [image imageScaledToSize:CGSizeMake(38, 38)];
}

Sorted.

Talkie is my newest product, the result of a collaboration with a good designer friend, Tim Churchward, who did the user interface.

Talkie is a little different from many of the other walkie talkie applications on the App Store (aside from the fact that much of it was written by me from our motorhome in Tunisia!), and I thought I’d write a little about some of the tech underpinning the app, and some of the choices we made. Along the way it may get a little tutorial-esque.

• This first part will introduce our initial motivations, and will talk about basic broadcast communications — the broadcast communications part may be very familiar to some, in which case it may be worth skipping to the next instalment.
• In the second part, I’ll continue the theme of networking, and will talk about what I ended up with for Talkie’s network code after addressing a couple of things, including switching to multicast.
• Finally, I’ll talk audio, dual platform development, and anything else I think of along the way (Actually, I’m aching to talk about one particular upcoming feature that had me jumping up and down when I first thought of it, but for now, mum’s the word on that one.)## Inspiration

Right from the start, we wanted a product that brought back the fun walkie talkie experience we remember from our childhoods. I’m talking colourful plastic, whip antennas and hiding in tree-houses. This was mostly Tim’s domain, so I shall leave it to him to discuss how he found that in the user interface.

It also meant stepping back from traditional voice chat, with manual call initialisation and termination and simple one-to-one calls. We wanted to mimic a radio, so that meant broadcasting as soon as you hit “Talk” — whereupon anyone in the neighbourhood would hear you.

Basically, we wanted to get as close to the real thing as was practical. This included the addition of a prominent ‘morse code’ button, of course, as well as a ‘squelch’ control for ‘fine tuning’, which simulated the static of bad reception.

Going dual-platform

Soon after I started developing Talkie, I realised I wanted a version for the Mac too.

Having Talkie both on the iPhone and on the Mac made sense, as we envisioned that a fairly common usage pattern would involve communication between a desktop and a handheld — say, someone wandering a campus with the iPhone in their pocket, staying in touch with friends at their desks.

We wanted to offer good value with Talkie, which is why we made Talkie for Mac available for free, when it’s used with Talkie for iPhone.

One of the very convenient things about developing for both iPhone and Mac is that the platforms are so similar, porting code is usually effortless. So, going dual-platform was an easy decision.

Finding common ground

The result of all of this was the need to develop or find a communication protocol and codec that would work across the iPhone and the Mac, over both Bluetooth for iPhone-iPhone communication, and Wifi, for communication between iPhones and iPhones, and iPhones and Macs.

Version 3.0 of the iPhone SDK introduced the GameKit framework and as part of it, GKVoiceChatService, which provides two person voice chat straight out of the box. Immediately it was clear that it wouldn’t serve our purposes — two person is not broadcast. I was also keen to provide a Mac version of Talkie, and given that GameKit is iPhone-only, it was time to go off the beaten track.

There are a variety of voice communication protocols out there, with varying licences and varying complexity and feature sets. I could’ve spent a week or two researching the options and evaluating them against our need for broadcast functionality, figuring out compilation and linking on the iPhone, resolving any compatibility or performance issues that arose, etc.

Or, I could spend a day putting together a few big building blocks provided by Apple (and the underlying Unix system) and have an easily tweakable working solution that precisely meets our needs and provides for future features.

It may not have been the most scientific approach, but the core of Talkie’s functionality was up and running on our home wi-fi network within about four hours, and it was working well.

Getting from a to b (and c, d, e, and f)

For those unfamiliar with the ins and outs of TCP/IP (the most common communication protocol — or rather, set of protocols — between computers, and the fundamental building block of the Internet), communication between computers can be connection-oriented, or connectionless.

Connection-oriented (a.k.a. ‘reliable’) communications are most common: They’re used when you open up a website, connect to iChat, or check your email. This is the TCP part of TCP/IP, and includes niceties like built-in guaranteed delivery via retransmission (providing your cat hasn’t eaten your network cable, of course) and packet ordering so you receive the messages in the right order. This only works between two computers, though – you and the server.

Connectionless (or ‘unreliable’) communications are much more open — it’s basically just a spray of messages: The data isn’t carefully ushered along, it’s just spat out and left to fend for itself. This is UDP, and is commonly used for time-sensitive applications like audio communication, network gaming, etc., where once a packet arrives late, it’s useless: No point mucking about re-sending lost data that’s going to be past its use-by date by the time it arrives. The other thing about UDP is that, because there’s no co-operation required from the destination (to acknowledge receipt, etc.), it lends itself to one-to-many communications — broadcast.

Just what we’re after.

So, we use connectionless communications (UDP) to send messages containing the audio, and anyone who receives them unpackages and plays the audio within.

The basic mechanics of this are fairly simple, once you get past the arguably cryptic C syntax.

Here’s how it works.

Sending

On the transmission end, on startup, you create a socket using the socket call:

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
...
int sock_fd = socket(AF_INET, SOCK_DGRAM, 0);
if ( sock_fd < 0 ) {
   // Error occurred
}

The socket, identified by sock_fd, will be created using the IPv4 domain (AF_INET), a ‘datagram’ type (SOCK_DFRAM: that’s connectionless communication — if we wanted connection-oriented, we could put in SOCK_STREAM here instead), using protocol 0, which is IP.

Then, we enable broadcasting:

int flag = 1;
int result = setsockopt(sock_fd, SOL_SOCKET, SO_BROADCAST, &flag, sizeof(flag));
if ( result < 0 ) {
   // Error occurred
}

That is, we set the SO_BROADCAST option at the SOL_SOCKET level to 1 (via the flag variable, which we pass in as a pointer), thereby requesting permission from the operating system kernel to broadcast.

Now we have the carrier pigeon coop at our disposal, we can start dispatching pigeons. First, we fill out the address:

#define kPortNumber 1234
...
struct sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = INADDR_BROADCAST;
addr.sin_port = htons(kPortNumber);

(Note, htons converts kPortNumber into a format that can be understood by any computer, regardless of the way it represents numbers internally.)

Now, send:

NSData *dataToSend;
result = sendto(sock_fd, [data bytes], [data length], 0, (struct sockaddr*)&addr, sizeof(addr));
if ( result < 0 ) {
   // Error occurred
}

Receiving

That takes care of outgoing messages. To receive them, we want another socket:

int sock_fd = socket(AF_INET, SOCK_DGRAM, 0);
if ( sock_fd < 0 ) {
   // Error occurred
}

Now we specify where we want to receive messages from, by ‘binding’ the socket:

struct sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = INADDR_ANY;
addr.sin_port = htons(kPortNumber);
result = bind(sock_fd, (struct sockaddr*)&addr, sizeof(addr));
if ( result < 0 ) {
  // Error occurred
}

And we pass a buffer to recvfrom to fill up with tasty morsels of data:

#define kBufferSize 1024
...
char buffer[kBufferSize];
int addr_len = sizeof(addr);
int bytes_received = recvfrom(sock_fd, buffer, kBufferSize, 0, (struck sockaddr*)&addr, &addr_len);
if ( bytes_received < 0 ) {
   // Error occurred
}

Voila, we have bytes_received bytes sitting in buffer for us to do something with. Note also that addr now also contains the address of the sender. Now we can loop and continue to receive data as it comes in, passing the data off to some other part of the application to deal with.

We can put it all together into a simple test app: broadcast_sample.c

Compile it by opening up Terminal, and typing make broadcast_sample (or cc -o broadcast_sample broadcast_sample.c, if you like), then run it with ./broadcast_sample "Message to send", or just ./broadcast_sample to listen.

$ ./broadcast_sample 
Listening...
Hello world!

$ ./broadcast_sample 'Hello world!'
"Hello world!" transmitted.

Coming next

This will work happily between computers with just one network interface. But, if you have more than one (say, wireless, and an Ethernet connection too), you’ll notice that it will only communicate through one of the interfaces. That’s because just the default interface is used. You have to explicitly attend to each interface, to broadcast out of each one, and listen on each one.

In the next part of this series, I’ll write about how I addressed that, and about multicast, which is used in things like Bonjour (MDNS). I’ll also write about designing packet formats.

Thanks for reading!