Chuck Remes: cremes' CoreAudio and QuickTime Blog

Go to new blog

Sun, 27 Feb 2005 19:57:14 GMT

This blog is going bye-bye. Head on over to desperatepundit.com and check out the new digs. This page should auto-redirect you within 10 seconds or so. If not, click on the link in the previous sentence.

Don't read too much into the new name. I just like the word "pundit," that's all. Th new blog will continue to focus on OpenDarwin related things with an occasional look at other things.

Working FLAC QT Component!

Sat, 02 Oct 2004 21:42:28 GMT

For all of you waiting to see a FLAC QT component, I need to apologize for my absence. My day job has taken up all my time.

However, someone else has made some great progress. I urge you to visit this page and take a look at the work that programmer has done. Yep, it's a FLAC QT component, plus he provides the source.

I think my project will morph into translating this working plugin into CoreAudio.

This may happen yet!

Mon, 05 Jul 2004 17:25:31 GMT

Learned a few things at WWDC that have re-inspired me to work on this project. Ping me if I haven't posted an update by July 31, 2004.

See ya!

Sat, 26 Jun 2004 14:09:14 GMT

Off to WWDC today. Plane leaves at 5:30PM from O'Hare and gets me to SF around 8:30PM or so.

Watch this space during the week for some darwin-related blogging goodness.

FLAC QT Component project: Coma!

Tue, 25 May 2004 00:41:23 GMT

In case anyone has subscribed to the RSS feed for the FLAC QT Component, I'm posting this message for you.

This project isn't dead, but it's definitely in a coma. I don't intend to do any work on it until after WWDC this year (end of June 2004). I'm hoping to learn a few things about the future of QuickTime and creating components for it.

The old way of doing QT components is kind of ugly, and if it's going to change soon then I don't want to waste the time learning the old frameworks. Sorry if you were counting on this showing up sometime soon.

Lossless AAC?

Wed, 28 Apr 2004 12:54:53 GMT

Now that Apple has released iTunes 4.5, we see that it includes a lossless encoder. My desire to create and finish a FLAC QT Component is shrinking...

Here I am.

Fri, 02 Apr 2004 04:09:04 GMT

Still alive. Sorry for being quiet lately, but work has been particularly onerous as of late. Hope to do some updates soon.

Sorry!

Tue, 02 Mar 2004 04:29:53 GMT

I apologize if you've subscribed to this RSS feed and I haven't posted any new progress in a while. I've been off working on some important stuff for the next OpenDarwin release and didn't focus on this project.

I'll try and post some updates later this week!

Need a break from this project...

Mon, 16 Feb 2004 17:31:12 GMT

Oh, my head is throbbing. I've been banging it against the QT wall for the past two days. I'll be taking a break from this project today to work on my other project.

Here's an excerpt from a thread on the quicktime-api list. I'm having trouble getting the component to match by filename extension. This is necessary since FLAC files won't contain any regular QT data as they get shuffled around from platform to platform. The only way to tell it is FLAC would be if it is called "some_audio_file.flac". But QT isn't being very cooperative and it's pissing me off.

--- start message ---
From: cremes@mac.com
Subject: Re: QT Movie Import, filename matching
Date: February 16, 2004 11:02:47 AM CST
To: quicktime-api@lists.apple.com

On Feb 15, 2004, at 9:42 PM, chuck remes wrote:

>On Feb 15, 2004, at 5:50 PM, Steve Nicolai wrote:

>>QuickTime uses the thng and thga resources to find the importers. The component type must be 'eat '. The component subtype is
>> a finder file type if the component flag movieImportSubTypeIsFileExtension is not set, or an uppercase version of the file
>> extension if movieImportSubTypeIsFileExtension is set.
>>
>>If you have both a file type and an extension that you want to support, you need two components, generally a thng and a thga.
>>
>>Speculating on what QuickTime does internally, if it is given a file type, it will first search for a component:
>>
>>type 'eat '
>>subtype
>>manufacturer 0
>>flags 0
>>flagsMask movieImportSubTypeIsFileExtension
>>
>>If it doesn't find a component or a file type is not given, it then looks for an extension on the file. If it finds one, it then searches
>> for:
>>
>>type 'eat '
>>subtype
>>manufacturer 0
>>flags movieImportSubTypeIsFileExtension
>>flagsMask movieImportSubTypeIsFileExtension
>>
>>The mcfg resource isn't used by QuickTime to find components. I think it's used to register with Windows that QuickTime
>> supports these file types.
>>
> I've double-checked all of this and it appears just as you outlined. I'm now concerned that the reason it isn't working is due to
> some incorrect setting within the XCode project itself. I created the project from scratch and added files to it rather than cloning
> the ElectricImage project and replacing the files.
>
>I'd like to put a breakpoint in the code where QT interfaces to this component. Is there a recommended way of doing this? I've read
> all the archived messages on this list and coreaudio-api for using gdb with QT components, but none of them spelled out how to
> trap entry into the component. In other words, I'm not 100% certain this component is executing when the open file dialog comes
> up.

The experiment continues. I "cloned" the ElectricImage sample and added my files to it. After building, still no joy on matching by filename. At least I eliminated the possibility that it is related to my XCode project settings.

My next thought was to start changing things in the original ElectricImage sample until I broke the filename match. Imagine my surprise when it broke after I modified the EI_GraphicsImport.r resource file. This resource file contains a 'thng' and 'thga' resource of type 'grip' just like the EI_MovieImport.r resource file contains one of type 'eat '. However, the 'grip' resource seems to be the one controlling whether or not the OpenFile dialog box matches on the filename. If I remove this file from the project and compile the project with the MovieImporter providing the only 'thng' and 'thga' resources of type 'eat ', it will NOT match on filename. Adding it back in gets the match to work again.

I tried flipping the resource IDs so the MovieImport 'eat ' resource had a lower ID, but it didn't help.

So what I've learned from this is that I can't match by filename without a 'thng' and 'thga' resource that registers itself as type 'grip'. Having a properly configured 'thng' and 'thga' of type 'eat ' is insufficient for mapping by filename.

I am well and truly stuck. I'm trying to create an audio component, so the concept of a 'grip' component doesn't fit at all. Help?

cr
_______________________________________________
quicktime-api mailing list | quicktime-api@lists.apple.com
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/quicktime-api
Do not post admin requests to the list. They will be ignored.

--- end message ----

Debugging Pro In Just One Day?

Sun, 15 Feb 2004 20:51:47 GMT

Wow, I love learning from other people's experience. I did a search on the coreaudio-api list for some similar keywords ("debug gdb") and found this post on using the graphical gdb for debugging a QT component.

Here's the shorthand version:

Go to your PB/XCode project window and find your Target list on the left-hand side.
Ctrl-click your Target and choose Add->"New Custom Executable" from the pop-up menu.
Set the executable to be the QuickTime Player (find it using the standard file dialogs).

Now you can do option-cmd-Y or Debug->"Debug Executable" to run the code witin the graphical version of gdb. Very nice.

Debugger newbie

Sun, 15 Feb 2004 20:12:43 GMT

All of the programming I've done in the past 3 or 4 years has been inside the kernel. (See my other project.) Programming in the kernel is really tough because you can't take advantage of any kind of source-level debugger like gdb (unless you have a second machine and can debug over the network). Therefore, my knowledge and experience with debugging userland applications is nil.

I've made some reasonable progress in the last few hours, so I dropped the newly built QT component into /Library/QuickTime, fired up QuickTime Player and hit cmd-O to open up a new movie. I wanted to see if my component had registered correctly and I could open a FLAC audio file. CRASH! (Just QT Player, not the OS.)

So how do I debug this thing? Before asking for help from the wizards on the quicktime-api mailing list, I did a search of their archives looking for the words "debug breakpoint component" to see what others had done. I didn't get many helpful hints from the results with one exception... this message pointed me in the right direction for catching QT Player when it crashes. The 'bt' command for printing the backtrace is apparently now called "where" so I executed that command and found it crashing in doResolveComponentAlias(RegisteredComponent*, CMCombinedResRef*) with an EXC_BAD_ACCESS signal. Some memory allocation is probably fubar, so I'll take a peek and see what I can see.

Anyone have a good reference for gdb and how to use it?

Movie Importers again

Sun, 15 Feb 2004 18:33:39 GMT

I found an important new link for people to take a peek at. On the Apple site they discuss movie import and export data exchange components which are a necessary evil when using QT. For QT to recognize a file, that file must already be a QT file or there must be a component registered with QT that knows how to recognize and translate that file format into the native QT format. The data exchange components take care of that function within QT. The aforementioned link is so important, it now has a place of honor on the left side navigation bar for quicker reference.

Back in the old OS9 days (and before), the component would register a type and creator code with QT. With OSX there is far less reliance on type/creator codes. The emphasis has shifted to filename extensions (like .txt for a text file, for example). The Ogg Vorbis component contains a Movie Importer (data exchange component) that registers the .ogg filename extension. This tells QT who to pass the file off to when opening it up. The Ogg Vorbis component translates the .ogg file into the desired QT format for playing. This conversion is done "in place" without overwriting the original file.

ProjectBuilder and XCode configuration

Sat, 14 Feb 2004 23:23:27 GMT

When configuring the project and trying to get it to compile successfully, I ran into some problems with the *.r files (resource files). There is apparently a known bug (or feature) with ProjectBuilder and XCode that makes it necessary to do clean builds all the time when you have .r files in your project.

One of the annoying things about these files is you can't really specify the order in which they are "Rez-ed" by the Rez application. It looks like the Apple ElectricImage sample got around this by declaring a master .r file that #incluedes the other files in the order they like. To get this to work, after adding all the .r files to the project, you need to go in and uncheck all of the .r files except for this master one. If you don't do this, then PB/XCode just Rez them in whatever order they want which (in this case) results in a lot of errors about undefined variables.

Here's the example master .r file:

// Building PB Mac OS X Mach-O Component

#define TARGET_REZ_MAC_68K 0

#define TARGET_REZ_MAC_PPC 0

#define TARGET_REZ_CARBON_CFM 0

#define TARGET_REZ_CARBON_MACHO 1

#define TARGET_REZ_WIN32 0



#include "MovieImport.r"

//#include "MovieExport.r"

#include "Codec.r"

This example shows the definition of various TARGET variables that are used by the children .r files. This is the only file "checked" and available for compilation by the IDE.

If this isn't clear and someone wants a better explanation, mail me.

Can't stop me now

Fri, 13 Feb 2004 04:27:53 GMT

I started putting the XCode project together tonight, modeling it from the ElectricImage sample from Apple.

Setting up the project is kind of tricky since there isn't any clear documentation. There are also several methods allowed that will result in the correct component package. According to TN2012 you can do it as a Carbon Bundle or as a Dynamic Library. It looked odd to me, so I just created an Empty Project and then added a New Target and chose the Legacy Bundle type. I don't know if this is correct, but it looks good so far.

First thing I did was go in and click on the target to edit its settings. I changed the "wrapper extension" from bundle to component.

Next I created a Frameworks group within the project and added Carbon.framework, CoreServices.framework, CoreAudio.framework and QuickTime.framework. I know I'll eventually need all of them.

Last, I started copying the EI_MovieImport and EI_ImageDecompressor directories from the ElectricImage sample. I also copied the ImportExampleLinkedMachO.r file into my project directory. I'll rename these files and start editing them to use a new name like FLAC_Codec. Better yet, I'll call it codec_skeleton for the next step.

What I think I'll do now is create the bare bones necessary to do a successful MovieImport and call out to the Codec. When it compiles fine and shows up in QuickTime successfully then I'll know it worked correctly. I'll save it off and start again with FLAC_Codec as my project. That way if someone else wants to get started on building a new QT Component they won't have to go through all the extra effort and can start off with a skeleton component framework.

On the road again

Thu, 12 Feb 2004 13:16:05 GMT

Got an email from the great Steve Nicolai, author of the OggVorbis QT component. He answered the question(s) I posed yesterday and has set me on the right track.

In summary, the sequence of development should be: 1. Create a basic Movie Importer. This should be capable of reading in the FLAC file based on either filetype or filename extension. Upon successful import, QT should display the correct format name, time sequence, etc.

2. Write the decompressor that corresponds to the format specified by the MovieImporter. This allows QT to hand-off the imported file to the correct codec for the next step in processing.

There's more detail than that, but I'm pressed for time so I'll add that detail in a separate post later.

I'm a bit disappointed that I need to write an importer first, but at least this whole system is starting to make sense to me.

How does the system match up the right decompressor component?

Thu, 12 Feb 2004 04:20:02 GMT

Going through the AudioCodec sample code has confused me on a pretty basic point. How does the system know to match up a particular audio stream with a particular codec? The AudioCodec classes have no facility for reading/writing files, nor do I see any "probe" (like in I/OKit) where a codec can test a stream and decide if it matches or not.

For a hint, I looked at the OggVorbis QT component source. Even though it is based on the Carbon Sound Manager, there's a LOT of similarity between it and the CoreAudio stuff. Anyway, near as I can tell the only way the system knows that a file should be handed off to the OggVorbis component is via the filename extension (.ogg). If you remove ".ogg" from a filename, you can't play it from QT, import it into QT, preview it from the Finder, or drag-n-drop it into iTunes. That's because the .ogg extension is defined as part of the QT Importer/Exporter interface. Do I really need to write those too just to test?

Any hints on how to proceed would be appreciated.

Inch by inch...

Wed, 11 Feb 2004 05:00:06 GMT

Final post for the night.

I finished slogging through the ACAppleIMA4Codec and ACAppleIMA4Decoder classes. It's neat how C++ classes build upon themselves and use inherited functionality, but tracing execution through the subclasses and parent classes can be difficult.

Enough grousing... I think I get it. I'm looking at the decoder first because an obvious Day One target is being able to decode the FLAC stream and output linear PCM.

Here's what I don't get. How does QuickTime know how to play an audio format it hasn't seen before? If I drop in a fully functional FLAC decoder into /Library/QuickTime, QT would have the ability to decode the stream data but would QT be aware of this? How?

From the old Component Manager I can see how an application can seek to match itself up with a codec (by passing in a componentType, componentSubType, etc., etc.) but what about the MoviePlayer application. How can it open and recognize enough information from the file to match up with the right codec?

I don't even know where to look to find these answers. I'll probably ask on quicktime-api, but I'd appreciate any pointers in the comments section or via email to me.

Coding Style - variable naming

Wed, 11 Feb 2004 04:12:59 GMT

Minor but important style note regarding the Apple example source.

Any method variable that is named "in" (such as "inPropertyID") indicates that this variable is being passed in to the method for some operation.

Any method variable named "out" (such as "outWritable") indicates that this variables value will be returned to the caller and used in some fashion.

[edit] Any method variable named "io" (such as "ioPropertyDataSize") indicates that the caller is passing in a value for the method to use and the method may pass back a changed value to the caller.

Makes the code easier to read and understand.

First Ah-ha Moment

Wed, 11 Feb 2004 03:40:55 GMT

Looking through the ACCodec, ACBaseCodec and ACSimpleCodec classes I noticed that several things looked eerily similar to the old Sound Manager stuff. When I got to the file ACCodecDispatch.h I had an "ah ha" moment. The template class defined in that file now takes the place of all that screwy C preprocessor bullshit from the old Carbon Sound Manager. In the old Sound Manager, there would be a file in your project that had contents like:

	ComponentSelectorOffset (6)
	
	ComponentRangeCount (2)
	ComponentRangeShift (7)
	ComponentRangeMask	(7F)
	
	ComponentRangeBegin (0)
		ComponentError (Target)
		ComponentError (Register)
		StdComponentCall (Version)
		StdComponentCall (CanDo)
		StdComponentCall (Close)
		StdComponentCall (Open)
	ComponentRangeEnd (0)

This tied in to some *really* ugly stuff in the background. To call your FooCodec's Register function it would have to concatenate your codec name ("FooCodec") to a function basename (like "Register") and include the parameters along with any commas to separate those parameters. The whole mess would look like:

#define CALLCOMPONENT_BASENAME() FooCodec
#ifdef CALLCOMPONENT_BASENAME
	#ifndef CALLCOMPONENT_GLOBALS
		#define CALLCOMPONENT_GLOBALS() 
		#define ADD_CALLCOMPONENT_COMMA 
	#else
		#define ADD_CALLCOMPONENT_COMMA ,
	#endif
	#define CALLCOMPONENT_GLUE(a,b) a##b
	#define CALLCOMPONENT_STRCAT(a,b) CALLCOMPONENT_GLUE(a,b)
	#define ADD_CALLCOMPONENT_BASENAME(name) CALLCOMPONENT_STRCAT(CALLCOMPONENT_BASENAME(),name)

	EXTERN_API( ComponentResult  ) ADD_CALLCOMPONENT_BASENAME(Open) 
		(CALLCOMPONENT_GLOBALS() ADD_CALLCOMPONENT_COMMA ComponentInstance  self);

#endif	/* CALLCOMPONENT_BASENAME */

The system would use it to piece together your FooCodecOpen(FooCodecGlobalsPtr globals, ComponentInstance self) function name. All of that nonsense is now replaced by (edited for length):

template <class CodecClass>
ComponentResult ACCodecDispatch(ComponentParameters* inParameters, CodecClass* inThis)
{
	ComponentResult	theError = kAudioCodecNoError;
	
	try
	{
		switch (inParameters->what)
		{
			//	these selectors don't use the object pointer
			
			case kComponentOpenSelect:
				{
					CodecClass*	theCodec = new CodecClass();
					SetComponentInstanceStorage(((AudioCodecOpenGluePB*)inParameters)->inCodec, (Handle)theCodec);
				}
				break;
		};
	}
	catch(ComponentResult inErrorCode)
	{
		theError = inErrorCode;
	}
	catch(...)
	{
		theError = kAudioCodecUnspecifiedError;
	}
	
	return theError;
}

Much cleaner in my view. It uses the power of templates to sub in the correct names and dispatch the component function call to the appropriate class method. Ultimately both things perform the same function, but one way requires you to chase through multiple files to figure out what is going on while the other way just prints the code in a nice, clean manner. I never thought I'd say this, but the C++ way is the "right" way to do this. Nice job, CoreAudio guys!

When are you going to start coding?

Wed, 11 Feb 2004 02:18:35 GMT

To all of you wondering when I'll start coding, the answer is simple: when I understand what the hell I'm doing. I need to ask some questions, study some code, ask some more questions, study the code again, do some judicious copy & paste, test, crash my IDE, ask some more questions, etc. The code will come.

Glossary of terms

Wed, 11 Feb 2004 02:10:42 GMT

Back on the 9th I asked a question about definitions on audio concepts like sample, frame, etc. One of my reasons for doing so is that depending on the codec I look at, each one appears to use different words to describe the same concept. What I really needed was to figure out what Apple means by these words. That's the environment I'm in, so it's the one I need to grok. Googling around I found this PDF on the Apple developer site. It's a glossary of audio terms and what they mean in the Apple environment. Very useful! I can use that glossary to translate between Apple and (in this case) FLAC. For example, in the Apple world the term for a unit of uncompressed audio data (one or more samples) is "frame." In FLAC the same concept is "block." Similarly, Apple says calls one or more compressed samples a "packet." In FLAC they call this a "frame." See how they reuse the terms? The format documentation page at the FLAC home page uses these alternate definitions.

Looking at the examples...

Tue, 10 Feb 2004 06:01:25 GMT

I've also started taking a look at the CoreAudio examples that shipped with Panther and XCode. The example at /Developer/Examples/CoreAudio/AudioCodecs shows how to create a codec (compressor/decompressor). Even better, it shows how to wrap it all up in an interface that the Carbon Sound Manager can use. Apparently QuickTime hasn't been revved yet to directly interface to CoreAudio, so any codecs created in CA aren't visible to QT. To remedy this, the wizards at Apple generated SMAC (I think this stands for Sound Manager Audio Codec). SMAC makes the CA codec appear to be a Sound Manager codec so that QT can see it and use it. Nice.

Tue, 10 Feb 2004 05:56:49 GMT

All the documentation I'm reading on FLAC, OggVorbis, MP3 or any other codec uses lots of jargon that I'm not familiar with. What's a sample? What's a sample rate? What do they mean by frame and what does it contain? I found a pretty decent website that lists lots of definitions. Take a look at it here. It's mostly for telco stuff, but their business is audio so the definitions are probably right on.

RSS Feed

Mon, 09 Feb 2004 00:02:47 GMT

I highly recommend that readers do NOT subscribe to this RSS feed yet. There's a problem that I'm trying to work through. Also, give me a week to shake this thing down. There might be lots of sidebar link changes and test posts made. No sense in clogging your newsreader with test messages.

Sun, 08 Feb 2004 17:35:57 GMT

This blog is brand new. Follow along as I learn CoreAudio, QuickTime, some old Carbon Sound Manager stuff, and FLAC. The point is to create a FLAC QuickTime component so that there is system-wide support for this lossless audio compression format. Read this summary for some other details. Lately there appears to be several people working on this or about to work on this. More power to them.