Blog

Using RemoteIO audio unit

I’ve had nasty old time trying to get some audio stuff going on the iPhone, no thanks to Apple’s lack of documentation. I have a feeling it’s all still quite hush-hush, so no details here, but if you’re an iPhone developer interested in getting RemoteIO/IO Remote/whatever it’s called working on the iPhone… Do I have good news for you.

Read on here, at the Developer forums

Drop me a line if you find this helpful.

Update: I’m told the new NDA is pretty much all-good with blog postings. So, read on for the goods.

Update 2: Thanks to Joel Reymont, we now have an explanation for the “CrashIfClientProvidedBogusAudioBufferList” iPhone simulator bug: The simulator doesn’t like mono audio. Thanks, Joel!

So, we need to obtain an instance of the RemoteIO audio unit, configure it, and hook it up to a recording callback, which is used to notify you that there is data ready to be grabbed, and where you pull the data from the audio unit.


Overview

  1. Identify the audio component (kAudioUnitType_Output/ kAudioUnitSubType_RemoteIO/ kAudioUnitManufacturerApple)
  2. Use AudioComponentFindNext(NULL, &descriptionOfAudioComponent) to obtain the AudioComponent, which is like the factory with which you obtain the audio unit
  3. Use AudioComponentInstanceNew(ourComponent, &audioUnit) to make an instance of the audio unit
  4. Enable IO for recording and possibly playback with AudioUnitSetProperty
  5. Describe the audio format in an AudioStreamBasicDescription structure, and apply the format using AudioUnitSetProperty
  6. Provide a callback for recording, and possibly playback, again using AudioUnitSetProperty
  7. Allocate some buffers
  8. Initialise the audio unit
  9. Start the audio unit
  10. Rejoice

Here’s my code: I’m using both recording and playback. Use what applies to you!

Initialisation

Initialisation looks like this. We have a member variable of type AudioComponentInstance which will contain our audio unit.

The audio format described below uses SInt16 for samples (i.e. signed, 16 bits per sample)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
#define kOutputBus 0
#define kInputBus 1
 
// ...
 
 
OSStatus status;
AudioComponentInstance audioUnit;
 
// Describe audio component
AudioComponentDescription desc;
desc.componentType = kAudioUnitType_Output;
desc.componentSubType = kAudioUnitSubType_RemoteIO;
desc.componentFlags = 0;
desc.componentFlagsMask = 0;
desc.componentManufacturer = kAudioUnitManufacturer_Apple;
 
// Get component
AudioComponent inputComponent = AudioComponentFindNext(NULL, &desc);
 
// Get audio units
status = AudioComponentInstanceNew(inputComponent, &audioUnit);
checkStatus(status);
 
// Enable IO for recording
UInt32 flag = 1;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioOutputUnitProperty_EnableIO, 
                              kAudioUnitScope_Input, 
                              kInputBus,
                              &flag, 
                              sizeof(flag));
checkStatus(status);
 
// Enable IO for playback
status = AudioUnitSetProperty(audioUnit, 
                              kAudioOutputUnitProperty_EnableIO, 
                              kAudioUnitScope_Output, 
                              kOutputBus,
                              &flag, 
                              sizeof(flag));
checkStatus(status);
 
// Describe format
audioFormat.mSampleRate			= 44100.00;
audioFormat.mFormatID			= kAudioFormatLinearPCM;
audioFormat.mFormatFlags		= kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
audioFormat.mFramesPerPacket	= 1;
audioFormat.mChannelsPerFrame	= 1;
audioFormat.mBitsPerChannel		= 16;
audioFormat.mBytesPerPacket		= 2;
audioFormat.mBytesPerFrame		= 2;
 
// Apply format
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_StreamFormat, 
                              kAudioUnitScope_Output, 
                              kInputBus, 
                              &audioFormat, 
                              sizeof(audioFormat));
checkStatus(status);
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_StreamFormat, 
                              kAudioUnitScope_Input, 
                              kOutputBus, 
                              &audioFormat, 
                              sizeof(audioFormat));
checkStatus(status);
 
 
// Set input callback
AURenderCallbackStruct callbackStruct;
callbackStruct.inputProc = recordingCallback;
callbackStruct.inputProcRefCon = self;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioOutputUnitProperty_SetInputCallback, 
                              kAudioUnitScope_Global, 
                              kInputBus, 
                              &callbackStruct, 
                              sizeof(callbackStruct));
checkStatus(status);
 
// Set output callback
callbackStruct.inputProc = playbackCallback;
callbackStruct.inputProcRefCon = self;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_SetRenderCallback, 
                              kAudioUnitScope_Global, 
                              kOutputBus,
                              &callbackStruct, 
                              sizeof(callbackStruct));
checkStatus(status);
 
// Disable buffer allocation for the recorder (optional - do this if we want to pass in our own)
flag = 0;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_ShouldAllocateBuffer,
                              kAudioUnitScope_Output, 
                              kInputBus,
                              &flag, 
                              sizeof(flag));
 
// TODO: Allocate our own buffers if we want
 
// Initialise
status = AudioUnitInitialize(audioUnit);
checkStatus(status);

Then, when you’re ready to start:

1
2
OSStatus status = AudioOutputUnitStart(audioUnit);
checkStatus(status);

And to stop:

1
2
OSStatus status = AudioOutputUnitStop(audioUnit);
checkStatus(status);

Then, when we’re finished:

1
AudioUnitUninitialize(audioUnit);

And now for our callbacks.

Recording

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
static OSStatus recordingCallback(void *inRefCon, 
                                  AudioUnitRenderActionFlags *ioActionFlags, 
                                  const AudioTimeStamp *inTimeStamp, 
                                  UInt32 inBusNumber, 
                                  UInt32 inNumberFrames, 
                                  AudioBufferList *ioData) {
 
    // TODO: Use inRefCon to access our interface object to do stuff
    // Then, use inNumberFrames to figure out how much data is available, and make
    // that much space available in buffers in an AudioBufferList.
 
    AudioBufferList *bufferList; // <- Fill this up with buffers (you will want to malloc it, as it's a dynamic-length list)
 
    // Then:
    // Obtain recorded samples
 
    OSStatus status;
 
    status = AudioUnitRender([audioInterface audioUnit], 
                             ioActionFlags, 
                             inTimeStamp, 
                             inBusNumber, 
                             inNumberFrames, 
                             bufferList);
    checkStatus(status);
 
    // Now, we have the samples we just read sitting in buffers in bufferList
    DoStuffWithTheRecordedAudio(bufferList);
    return noErr;
}

Playback

1
2
3
4
5
6
7
8
9
10
11
static OSStatus playbackCallback(void *inRefCon, 
                                  AudioUnitRenderActionFlags *ioActionFlags, 
                                  const AudioTimeStamp *inTimeStamp, 
                                  UInt32 inBusNumber, 
                                  UInt32 inNumberFrames, 
                                  AudioBufferList *ioData) {    
    // Notes: ioData contains buffers (may be more than one!)
    // Fill them up as much as you can. Remember to set the size value in each buffer to match how
    // much data is in the buffer.
    return noErr;
}

Finally, rejoice with me in this discovery ;)

Resources that helped

No thanks at all to Apple for their lack of accessible documentation on this topic – They really have a long way to go here! Also boo to them with their lack of search engine, and refusal to open up their docs to Google. It’s a jungle out there!

Update: You can adjust the latency of RemoteIO (and, in fact, any other audio framework) by setting the kAudioSessionProperty_PreferredHardwareIOBufferDuration property:

float aBufferLength = 0.005; // In seconds
AudioSessionSetProperty(kAudioSessionProperty_PreferredHardwareIOBufferDuration, 
                        sizeof(aBufferLength), &aBufferLength);

This adjusts the length of buffers that’re passed to you – if buffer length was originally, say, 1024 samples, then halving the number of samples halves the amount of time taken to process them.

Related posts

Tagged , , , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

107 Comments

  1. Posted September 11, 2009 at 9:12 pm | Permalink

    Nice work on putting together this example. Yours and Aran’s(http://sites.google.com/site/iphonecoreaudiodevelopment/remoteio-playback) hard work have helped greatly in getting a jumpstart with core audio on the that popular mobile device…

  2. Posted October 8, 2009 at 10:27 pm | Permalink

    Hi,

    Code incredibly helpful indeed, both yours and Aran’s, so many thanks for these. Quick question: I’m trying to lower the volume with a slider. Works fine when I go up (multiplying the Uint32 of the signal by integer) BUT fails completely (sound distorted, yet recognizable) when I divide. So obviously I’m doing it wrong, but I don’t know why. I’ve tried shifting instead of dividing, same thing. Clues welcome :)

    • Posted October 9, 2009 at 10:49 am | Permalink

      Hi Guillaume, Hmm, try casting to float/double first.. e.g. sample = (float)sample * volume;

      It may be doing integer division, which would result in distortion. Good luck!

  3. Posted October 10, 2009 at 12:27 am | Permalink

    You & Aran were right, it is a cast problem. Casting to signed int did the trick. Heartfelt thanks for the advice, and for your outstanding contribution!

  4. Toby
    Posted October 29, 2009 at 10:22 pm | Permalink

    Hi Michael,

    I have got an app working using your example above having the audio from the mic going straight through to playback callback. However I am having problems getting the audio to come out of the speaker rather than the earpiece.

    I have tried setting the route to be kAudioSessionOverrideAudioRoute_Speaker but it doesn’t seem to do anything. I was reading in your blog about Loopy that you had a problem with feedback because of audio coming out of the speaker, but how did you do it?

    Thanks

    • Chris
      Posted June 22, 2010 at 6:42 am | Permalink

      Hey Toby can you explain how you got audio passing from mic to headphones? I am trying to do the same thing, but I am not sure what to put in the recording callback to do this. I assume it involves using the function AudioUnitRender, then copying over the buffer? Any suggestions / sample code? Thanks

      • Posted June 22, 2010 at 10:46 am | Permalink

        Hi Chris – as far as I know, as long as the headphones are plugged in, that’s the route the audio should take, unless you’ve manually overridden the path (or perhaps if you’re using a session category that has that behaviour?).

        My problem was as Toby expressed above (and I totally, inexcusably missed his question at the time!), getting the audio to come out of the speaker rather than the receiver; What I’m doing is listening for changes in the audio route, which happens when headphones are plugged in, etc:

        // Listen for changes in audio route AudioSessionAddPropertyListener(kAudioSessionProperty_AudioRouteChange, routeChangeCallback, self);

        Then in the callback, checking the route, and overriding it if necessary:

        ... if ( [(NSString*)route isEqualToString:@"ReceiverAndMicrophone"] ) { ... // Re-route audio to the speaker (not the receiver) UInt32 newRoute = kAudioSessionOverrideAudioRoute_Speaker; status = AudioSessionSetProperty(kAudioSessionProperty_OverrideAudioRoute, sizeof(route), &newRoute); checkStatus(status);

      • Toby
        Posted June 22, 2010 at 11:00 am | Permalink

        Hi Chris,

        I can’t remember off the top of my head now I did it now, so long ago :)

        I can take a look when I get home, but I believe I used this example as a basis which is slightly different from Michael’s example.

        http://pastie.org/pastes/445149

  5. Massimo
    Posted December 11, 2009 at 9:03 am | Permalink

    It’s useful. But I still do not understand how to copy input samples to output. Is there a complete example of a playthru with, in the middle, a callback that perform some sort of modification (e.g. a volume)?

  6. Posted December 31, 2009 at 5:58 pm | Permalink

    Thank you for this simple walkthrough! I’m going to use this as reference while doing some experimentation with Pandora.

  7. Rudiger
    Posted March 24, 2010 at 8:23 pm | Permalink

    Your name is Mike Tyson… That’s awesome!

    Kinda like Michael Bolton from Office Space?

    • Posted March 25, 2010 at 11:42 am | Permalink

      …I told those fudge-packers I liked Michael Bolton’s music…

      Yeah, I can relate….

  8. Janaka
    Posted April 18, 2010 at 7:30 am | Permalink

    Thanks for posting this wonderful example.

    I am having trouble getting the example to work. Is there a place I can download your code?

  9. Ben
    Posted April 24, 2010 at 9:40 am | Permalink

    Hey, great code. In recordingCallback how would you process the ioData to get the amplitude of each sample. I have everything else apart form this sorted and im so stumped. Any help would be appreciated. Thanks.

  10. Ben "Another one!"
    Posted May 13, 2010 at 1:43 pm | Permalink

    Great looking bit of code…

    I have a question. Would something like this code allow me to record multiple samples to an audio file?

    I have a musical instrument app and I am desperately trying to record the output to an audio file. I have been trying everything and I think Audio Units is the answer but Apples Docs are far from helpful.

    Any advice would be greatly appreciated.

    Thanks

    Ben

    • Posted May 29, 2010 at 2:45 pm | Permalink

      Hi Ben – Sorry about the respond delay.

      This code doesn’t actually include any file operations, but you would probably put something in the record callback. The Audio File services are what you’re after.

  11. Andy Capon
    Posted July 16, 2010 at 9:54 pm | Permalink

    Hi guys,

    We have remoteio set up mostly from info in this article and working perfectly on the iPhone and iPad.

    The problem we have is that on the iPod (1st gen tested) the record playback is never called. We get no errors from any of the setup calls.

    Anyone any idea on this issue?

    Thanks for any help

    Andy

3 Trackbacks

  1. By [Time code]; on January 17, 2009 at 7:12 am

    [...] complexity and move on to Audio Toolbox (or perhaps even Core Audio… a DevForums thread and a blog by developer Michael Tyson report extremely low latency by using the RemoteIO audio unit [...]

  2. [...] I also wouldn’t have gotten anywhere on VocaForm without Michael Tyson’s post on using the remoteIO AU. [...]

  3. [...] where you can do a lot of useful stuff, and then at the lowest level there are two types of Audio Unit: Remote I/O (or remoteio) and the Voice Processing Audio Unit [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">

Subscribe without commenting