Blog

Using RemoteIO audio unit

I’ve had nasty old time trying to get some audio stuff going on the iPhone, no thanks to Apple’s lack of documentation. If you’re an iPhone developer interested in getting RemoteIO/IO Remote/whatever it’s called working on the iPhone… Do I have good news for you. Read on.

Wanna skip the Core Audio learning curve and start writing code straight away? Check out my new project:

The Amazing Audio Engine: Core Audio, Cordially

Update: Thanks to Joel Reymont, we now have an explanation for the “CrashIfClientProvidedBogusAudioBufferList” iPhone simulator bug: The simulator doesn’t like mono audio. Thanks, Joel!

Update: Happily, Apple have now created some excellent documentation on Remote IO, with some good sample projects. I recommend using that as a resource, now that it’s there, as that will continue to be updated.

Update: Tom Zicarelli has created a very extensive sample app that demonstrates the use of AUGraph, with all sorts of goodies.

So, we need to obtain an instance of the RemoteIO audio unit, configure it, and hook it up to a recording callback, which is used to notify you that there is data ready to be grabbed, and where you pull the data from the audio unit.


Overview

  1. Identify the audio component (kAudioUnitType_Output/ kAudioUnitSubType_RemoteIO/ kAudioUnitManufacturerApple)
  2. Use AudioComponentFindNext(NULL, &descriptionOfAudioComponent) to obtain the AudioComponent, which is like the factory with which you obtain the audio unit
  3. Use AudioComponentInstanceNew(ourComponent, &audioUnit) to make an instance of the audio unit
  4. Enable IO for recording and possibly playback with AudioUnitSetProperty
  5. Describe the audio format in an AudioStreamBasicDescription structure, and apply the format using AudioUnitSetProperty
  6. Provide a callback for recording, and possibly playback, again using AudioUnitSetProperty
  7. Allocate some buffers
  8. Initialise the audio unit
  9. Start the audio unit
  10. Rejoice

Here’s my code: I’m using both recording and playback. Use what applies to you!

Initialisation

Initialisation looks like this. We have a member variable of type AudioComponentInstance which will contain our audio unit.

The audio format described below uses SInt16 for samples (i.e. signed, 16 bits per sample)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
#define kOutputBus 0
#define kInputBus 1
 
// ...
 
 
OSStatus status;
AudioComponentInstance audioUnit;
 
// Describe audio component
AudioComponentDescription desc;
desc.componentType = kAudioUnitType_Output;
desc.componentSubType = kAudioUnitSubType_RemoteIO;
desc.componentFlags = 0;
desc.componentFlagsMask = 0;
desc.componentManufacturer = kAudioUnitManufacturer_Apple;
 
// Get component
AudioComponent inputComponent = AudioComponentFindNext(NULL, &desc);
 
// Get audio units
status = AudioComponentInstanceNew(inputComponent, &audioUnit);
checkStatus(status);
 
// Enable IO for recording
UInt32 flag = 1;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioOutputUnitProperty_EnableIO, 
                              kAudioUnitScope_Input, 
                              kInputBus,
                              &flag, 
                              sizeof(flag));
checkStatus(status);
 
// Enable IO for playback
status = AudioUnitSetProperty(audioUnit, 
                              kAudioOutputUnitProperty_EnableIO, 
                              kAudioUnitScope_Output, 
                              kOutputBus,
                              &flag, 
                              sizeof(flag));
checkStatus(status);
 
// Describe format
audioFormat.mSampleRate			= 44100.00;
audioFormat.mFormatID			= kAudioFormatLinearPCM;
audioFormat.mFormatFlags		= kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
audioFormat.mFramesPerPacket	= 1;
audioFormat.mChannelsPerFrame	= 1;
audioFormat.mBitsPerChannel		= 16;
audioFormat.mBytesPerPacket		= 2;
audioFormat.mBytesPerFrame		= 2;
 
// Apply format
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_StreamFormat, 
                              kAudioUnitScope_Output, 
                              kInputBus, 
                              &audioFormat, 
                              sizeof(audioFormat));
checkStatus(status);
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_StreamFormat, 
                              kAudioUnitScope_Input, 
                              kOutputBus, 
                              &audioFormat, 
                              sizeof(audioFormat));
checkStatus(status);
 
 
// Set input callback
AURenderCallbackStruct callbackStruct;
callbackStruct.inputProc = recordingCallback;
callbackStruct.inputProcRefCon = self;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioOutputUnitProperty_SetInputCallback, 
                              kAudioUnitScope_Global, 
                              kInputBus, 
                              &callbackStruct, 
                              sizeof(callbackStruct));
checkStatus(status);
 
// Set output callback
callbackStruct.inputProc = playbackCallback;
callbackStruct.inputProcRefCon = self;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_SetRenderCallback, 
                              kAudioUnitScope_Global, 
                              kOutputBus,
                              &callbackStruct, 
                              sizeof(callbackStruct));
checkStatus(status);
 
// Disable buffer allocation for the recorder (optional - do this if we want to pass in our own)
flag = 0;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_ShouldAllocateBuffer,
                              kAudioUnitScope_Output, 
                              kInputBus,
                              &flag, 
                              sizeof(flag));
 
// TODO: Allocate our own buffers if we want
 
// Initialise
status = AudioUnitInitialize(audioUnit);
checkStatus(status);

Then, when you’re ready to start:

1
2
OSStatus status = AudioOutputUnitStart(audioUnit);
checkStatus(status);

And to stop:

1
2
OSStatus status = AudioOutputUnitStop(audioUnit);
checkStatus(status);

Then, when we’re finished:

1
AudioComponentInstanceDispose(audioUnit);

And now for our callbacks.

Recording

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
static OSStatus recordingCallback(void *inRefCon, 
                                  AudioUnitRenderActionFlags *ioActionFlags, 
                                  const AudioTimeStamp *inTimeStamp, 
                                  UInt32 inBusNumber, 
                                  UInt32 inNumberFrames, 
                                  AudioBufferList *ioData) {
 
    // TODO: Use inRefCon to access our interface object to do stuff
    // Then, use inNumberFrames to figure out how much data is available, and make
    // that much space available in buffers in an AudioBufferList.
 
    AudioBufferList *bufferList; // <- Fill this up with buffers (you will want to malloc it, as it's a dynamic-length list)
 
    // Then:
    // Obtain recorded samples
 
    OSStatus status;
 
    status = AudioUnitRender([audioInterface audioUnit], 
                             ioActionFlags, 
                             inTimeStamp, 
                             inBusNumber, 
                             inNumberFrames, 
                             bufferList);
    checkStatus(status);
 
    // Now, we have the samples we just read sitting in buffers in bufferList
    DoStuffWithTheRecordedAudio(bufferList);
    return noErr;
}

Playback

1
2
3
4
5
6
7
8
9
10
11
static OSStatus playbackCallback(void *inRefCon, 
                                  AudioUnitRenderActionFlags *ioActionFlags, 
                                  const AudioTimeStamp *inTimeStamp, 
                                  UInt32 inBusNumber, 
                                  UInt32 inNumberFrames, 
                                  AudioBufferList *ioData) {    
    // Notes: ioData contains buffers (may be more than one!)
    // Fill them up as much as you can. Remember to set the size value in each buffer to match how
    // much data is in the buffer.
    return noErr;
}

Finally, rejoice with me in this discovery ;)

Resources that helped

No thanks at all to Apple for their lack of accessible documentation on this topic – They really have a long way to go here! Also boo to them with their lack of search engine, and refusal to open up their docs to Google. It’s a jungle out there!

Update: You can adjust the latency of RemoteIO (and, in fact, any other audio framework) by setting the kAudioSessionProperty_PreferredHardwareIOBufferDuration property:

float aBufferLength = 0.005; // In seconds
AudioSessionSetProperty(kAudioSessionProperty_PreferredHardwareIOBufferDuration, 
                        sizeof(aBufferLength), &aBufferLength);

This adjusts the length of buffers that’re passed to you – if buffer length was originally, say, 1024 samples, then halving the number of samples halves the amount of time taken to process them.

Another Update: In the comments below, Florian Bomers pointed out that I was incorrectly using the AudioUnitUninitialize to clean up the Audio Unit. This is incorrect, and should in fact be AudioComponentInstanceDispose. Further discussion here. Cheers Florian!

Tagged , , , . Bookmark the permalink. Both comments and trackbacks are currently closed.

272 Comments

  1. Titus Abraham
    Posted May 15, 2013 at 6:11 am | Permalink

    Hi,

    I noticed that When i record from MIC and give it to AU to play on speaker, the volume i hear is not that loud. Any means on how to set it high. I have noticed that the current hard ware volume is .5 (by getting property value for kAudioSessionProperty_CurrentHardwareOutputVolume).

    Any suggestions?

    Thanks, Titus

    • iBanzai
      Posted May 16, 2013 at 7:38 pm | Permalink

      I am also looking for suggestions about this.

  2. William Power
    Posted May 30, 2013 at 2:13 am | Permalink

    I have a RemoteIO AudioUnit set up for play-through. No graph. Only one callback. (Similar to the setup in the book “Learning Core Audio”, chapter 10)

    This is great when there are headphones attached, but there’s tremendous feedback when there are no headphones attached. I’ve learned that I cannot simply disable the playback using the AudioUnitSetProperty call while recording is taking place, and I can’t seem to find the proper invocation to get recording going WITHOUT play-through.

    Can you provide any feedback / guidance?

    Thanks

    • Brennon Bortz
      Posted May 30, 2013 at 3:50 am | Permalink

      Once you’ve done whatever you need to do with your samples (persist them to a file, etc.), just zero them out. Boom–no more playback.

      • William Power
        Posted May 31, 2013 at 4:43 am | Permalink

        Yes, absolutely. I’m using extended audio file services to write to file async. AFTER that call, zeroing out ioData totally works. Thanks!

    • Brennon Bortz
      Posted May 30, 2013 at 3:53 am | Permalink

      Or, just use the kAudioSessionCategory_RecordAudio session category.

      • William Power
        Posted May 31, 2013 at 4:42 am | Permalink

        Tried it. I’m using the render callback to save to file using extended audio file services’ write async. The category session stuff on a pass-through remote io setup (without using an augraph) causes nothing to be recorded.

        • Brennon Bortz
          Posted June 3, 2013 at 3:22 am | Permalink

          So, everything working now, or is something still funky?

          • William Power
            Posted June 3, 2013 at 11:59 pm | Permalink

            The Record-only audio session wasn’t working, but zeroing out the buffer after firing ExtAudioFileWriteAsync did.

          • Puneet Sharma
            Posted June 6, 2013 at 5:14 am | Permalink

            Hi

            I am facing a problem related to Audio recording. What i want to achieve is that suppose an Audio track is being played and at the same time recording is done, the playing track sound should not be recorded. But presently i am not able to achieve it. Always when i play and record simultaneous the played track is also merged in the recorded voice. Please help in this, it has been more than a week trying to sort this.

            Thanks, Puneet

  3. Posted June 5, 2013 at 12:39 pm | Permalink

    Hi

    I am facing a problem related to Audio recording. What i want to achieve is that suppose an Audio track is being played and at the same time recording is done, the playing track sound should not be recorded. But presently i am not able to achieve it. Always when i play and record simultaneous the played track is also merged in the recorded voice. Please help in this, it has been more than a week trying to sort this.

    Thanks, Puneet

  4. realloxer
    Posted July 3, 2013 at 4:27 pm | Permalink

    Hi, how can i set the number of Buffers? I have a VOIP app,I want to play continuously from socket bytes, can anyone help me?

  5. Lucas
    Posted August 5, 2013 at 9:44 pm | Permalink

    I copied and pasted ‘OSStatus status’ and ‘AudioComponentInstance audioUnit’ into my header file; put the rest of the initialization code in a method called ‘testMathod’ in my implementation file; and I’ve put ‘static OSStatus playbackCallback’ and ‘static OSStatus recordingCallback’ in my implementation file, but not inside a (objective -C) method. I want to be able to save the recording file to a URL I create and call it later to be played by an AVAudioPlayer in another method. I am currently getting the following failures: ‘audioFormat’ undeclared, ‘recordingCallback’ undeclared, ‘playbackCallback’ undeclared, ‘audioInterface’ undeclared, and nested functions are disabled, use -fnested-functions to reenable. <— those are the failures I am getting. The last thing I need to know is how to specify the URL to have the recording file saved to it.

  6. ksam
    Posted August 8, 2013 at 6:38 am | Permalink

    Hi all,

    I am trying to capture and play the audio simultaneously in iOS device using Remote IO audio unit, and it is working fine. What I am facing is, in iPhone 5 device when I am trying to capture the audio the captured audio is getting clipped and it is not smooth. How to over come this.

    When I am using the Voice processing audio unit then both the capture and play is happening but the captured data is not coming fine. It is looking like half duplex and VPIO is producing the zero sampled signals when I am using Voice chat to cancel the echo. So how to make use of inbuilt AEC effectively.

    Waiting for suggestions. Thank you in advance. ksam.

    • Posted August 8, 2013 at 7:44 am | Permalink

      Hi Kasam

      I also faced the same issue and after weeks of googling i came to know that, Voice processing is use full in case of VOIP as in that case the Voice latency is considerable. But the same approach is not useful if you are developing some Musical app or when the Output and Receiver both are same. So while recording you need to pause the playback of other sound. Hope this helps.

      Puneet

      • ksam
        Posted August 8, 2013 at 8:07 am | Permalink

        Punith,

        Thank you for your feedback. I am not concentrating on musical application, I want to use for VOIP in which low latency matters most. I am creating the separate Audio Units for both capturing and playback using kAudioVoiceProcessing unit. Here the captured audio is coming like half duplex one where as with RemoteIO it is coming fine. How to improve the captured audio quality when we are using VPIO.

  7. rahul parashar
    Posted October 11, 2013 at 1:07 pm | Permalink

    Hi Sir,

    i want to implement the functionality in my app. Like i have one bluetooth handsfree and one iPad device, now when i speak from bluetooth handsfree speaker my voice will transfer to iPad device via bluetooth connection establish between two ,and voice must come out through iPad speakers. Please suggest appropriate answer.

    Thanks

5 Trackbacks

  1. By [Time code]; on January 17, 2009 at 7:12 am

    […] complexity and move on to Audio Toolbox (or perhaps even Core Audio… a DevForums thread and a blog by developer Michael Tyson report extremely low latency by using the RemoteIO audio unit […]

  2. […] I also wouldn’t have gotten anywhere on VocaForm without Michael Tyson’s post on using the remoteIO AU. […]

  3. […] where you can do a lot of useful stuff, and then at the lowest level there are two types of Audio Unit: Remote I/O (or remoteio) and the Voice Processing Audio Unit […]

  4. By Again onwards | iOS adventures on October 4, 2012 at 4:12 pm

    […] form this guy until I realise we weren’t trying to do the same thing, looked a bit at Michael Tyson’s stuff and had a good read of the Apple docs on Audio Unit […]

  5. […] mayor parte del día la dedicamos al procesamiento de audio en tiempo real mediante remote IO Audio Units. Las Audio Units, son como plugins que procesan el sonido y pueden ser encadenados uno […]