Quicktime Atoms

Rant mode: Disabled
Here we are. Slighly cold day, working away on Squish in some attempt to extract various bits of information about the selected compression fromat, its frame rate, audio properties etc etc. I want to do this so I can show users an "executive summary" of what their compression preset is all about - without them having to enter the QT settings dialogs to find out. Of course all these things are stored in something called a QTAtomContainer.

OK. I can handle that. How hard can it be?

... a few hours pass ...

Rant mode: Enabled

I have now discovered the source of all woe in the world. QuickTime Atom Containers, Atoms, Atom IDs, Atom anythings. grrr. I can understand that this is a legacy carry over, from a time perhaps when space efficiency was important - but I can't think of anything more painful right now than using these things... aside from perhaps trying to cut my leg off using the back edge of a rusty razor blade.

Don't get me wrong here. I like QuickTime (the new parts). Rather, I like what QuickTime can DO for me. Lots. It's pretty cool. But these 1980's C API's are doing my HEAD IN. Every time I come to examine these APIs and use them, I cringe.... lets demonstrate the joy:

So - I want to get some details about a QTEffect.
Lets say I want to get the name of said effect - and that I've already gone through a bunch of pain to get the effect description QTAtomContainer. I can get the name by simply doing:

  
  QTAtom effectNameAtom = QTFindChildByIndex(container, kParentAtomIsContainer, kEffectNameAtom, effectIndex, nil); 
  long dataSize;
  Ptr atomData;
  OSErr err = QTGetAtomDataPtr(container, effectNameAtom, &dataSize, &atomData);
  if(err != noErr) {
   [NSException raise:@"Effects" format:@"Can't get name of the effect"];
  } else {
   [self setName:[NSString stringWithCString:atomData length:dataSize]];
  }

Easy huh? OK. Lets say I wanted to create an effect - and apply that effect to a video. I'll make it simple too - lets say that effect is to operate on just the existing video - it's not a transition. Righteo then. Lets begin.

Well, we need to setup a effect description ... simple enough:

 
 QTAtomContainer effectsDescription;
 QTNewAtomContainer(&effectsDescription);
 QTInsertChild(effectsDescription, 
      kParentAtomIsContainer, 
      kParameterWhatName, 
      kParameterWhatID, 
      0, 
      sizeof(effectCode), 
      &effectCode, 
      nil); 
 // EndianU32_NtoB(3);
 long value = 3;
 
 QTInsertChild(effectsDescription, 
      kParentAtomIsContainer, 
      'ksiz', 
      1, 
      0, 
      sizeof(value), 
      &value, 
      nil); 
 
 if(effectCode == kEdgeDetectImageFilterType) {
  value = 1;
  QTInsertChild(effectsDescription, 
       kParentAtomIsContainer, 
       'colz', 
       1, 
       0, 
       sizeof(value), 
       &value, 
       nil); 
 }
 OSType myType = [self sourceName];
 QTInsertChild(effectsDescription, kParentAtomIsContainer, kEffectSourceName, 1, 0, sizeof(myType), &myType, NULL);

Er, yay. But wait - that's not all! We need also to add this as a sample to a new track. This is how we tell QT that's it can apply this effect.

 
 Movie sourceMovie = GetTrackMovie(sourceTrack);
 
 // Now add an effect
 NSRect trackSize = [self trackRect:sourceTrack];
 theEffectsTrack = NewMovieTrack(sourceMovie, IntToFixed(trackSize.size.width), IntToFixed(trackSize.size.height), 0);
 if(theEffectsTrack == nil) {
  [self log:@" ** Unable to create the effects track"];
  return;
 }
 
 [self log:@"Created a new effects track with %@", NSStringFromSize(trackSize.size)];
 theEffectsMedia = NewTrackMedia(theEffectsTrack, VideoMediaType, GetMovieTimeScale(sourceMovie), nil, 0); 
 if(theEffectsMedia == nil) {
  [self log:@"Unable to add the new media to the video, for the effect track"];
 } else {
  [self log:@"Added effects media, with duration %lld", GetMovieDuration(sourceMovie)];
 }

 ImageDescriptionHandle sampleDescription = [self createEffectImageDescription:effectCode size:trackSize.size];
 BeginMediaEdits(theEffectsMedia);  
 // Add the sample to the media 
 TimeValue sampleTime = 0;
 OSStatus err = noErr;
 BAILSETERR(AddMediaSample(theEffectsMedia, 
         (Handle) effectsDescription, 
         0, 
         GetHandleSize((Handle) effectsDescription), 
         GetMovieDuration(sourceMovie), 
         (SampleDescriptionHandle) sampleDescription, 
         1, 
         0, 
         &sampleTime));  
 // End the media editing session 
 EndMediaEdits(theEffectsMedia); 
 BAILSETERR(InsertMediaIntoTrack(theEffectsTrack, 0, sampleTime, GetMediaDuration(theEffectsMedia), fixed1));

...

- (ImageDescriptionHandle) createEffectImageDescription:(OSType)effectCode size:(NSSize)effectSize {
 ImageDescriptionHandle sampleDescription = nil;
 [self log:@"Creating sample description for effect %@", [NSString osTypeToString:effectCode]];
 MakeImageDescriptionForEffect(effectCode, &sampleDescription);
 (**sampleDescription).vendor = kAppleManufacturer;
 (**sampleDescription).temporalQuality = codecNormalQuality;
 (**sampleDescription).spatialQuality = codecNormalQuality;
 (**sampleDescription).width = effectSize.width;
 (**sampleDescription).height = effectSize.height;
 return sampleDescription;
}

Whew. But guess what? YES! There's MORE! It's all well and good to describe an effect, but we also need to tell QT about how to link it all together. Should be simple enough:

 
 OSErr err = noErr;
 QTNewAtomContainer(&inputMap);
 
 long refIndex;
 QTAtom inputAtom;
 
 // Add a reference to the video track 
 BAILSETERR(AddTrackReference(theEffectsTrack, sourceTrack, kTrackModifierReference, &refIndex));
 
 // Add a reference into the input map
 QTInsertChild(inputMap, kParentAtomIsContainer, kTrackModifierInput, refIndex, 0, 0, nil, &inputAtom);
 
 OSType type = [self safeOStype:kTrackModifierTypeImage];
 QTInsertChild(inputMap, inputAtom, kTrackModifierType, 1, 0, sizeof(type), &type, nil);
 
 type = [self sourceName];
 QTInsertChild(inputMap, inputAtom, kEffectDataSourceType, 1, 0, sizeof(type), &type, nil);
 
 BAILSETERR(SetMediaInputMap(theEffectsMedia, inputMap));

Er, wow.
Now it's slightly unfair of me to complain here. A couple of things are going on that simple CoreImage Filtering can't match.

1) It's applying the filter to a video - not just a frame.
2) The API is allowing us to choose the source video track, so effects can be layered "easily".

However. It doesn't excuse the verbosity of this entire API/method. Every time I come to use this tech I'm absolutely Stunned (yes, stunned with a capital S) at how much work I have to do. Why couldn't we have an API like so? (OK, I'm intentionally ignoring everyone who doesn't use Objective-C here)...

 QTEffect *effect = [[QTEffect alloc] initWithName:kQTBlurEffect];
 NSArray *defaultParameters = [effect defaults];

 // ... go off and modify these - whatever.
 // In my example above they are hard coded).

 [effect setParameters:newParameters];
 [effect applyTo:myQtMovie];

I guess my rant is really that there appears to be no other way to do this "simple" task. Simple? OK, it's not simple. Application of video effects must be reasonably tricky. But it's now "simple" compared to other APIs available to us right now. CoreImage Filtering is a great example. It provides the paramter metadata and defaults as easily understood NSDictionary instances. Bam. Done. I spent a considerable time trying to extract the presets from a QTEffect but with no joy. I know the stuff is there, but it's soooo cryptic. Ghaaa.

I can only assume that QTEffects is dead. It's certainly not been kept up to date (even though underneath it I think has been udpated) from an API point of view. In fact for Squish I've taken two steps back now. Why would I use QTEffects (even though they do what I want) when CoreImage seems the future? Yes, I'm going to have to rewrite my recompression engine (because the change away from the method I'm using now so that I can use CoreImage is actually quite major. QT has a very cool function ConvertMovieToDataRef that can magically (given the right input) transcode one movie to another) - but I can't see why I would want to stay with API's like QTAtom and QTEffects. The pain is too great.

I guess I'd hoped that QTEffect would be a neat way for me to have effects applied to the video without having to unpack each source video frame, perform the effect myself and then recompress the video myself. It seems it is not to be. Ah well, I guess you don't get the best of both worlds (the cools stuff of QT, with the cool stuff of Core*) without doing some work.

Roll on Leopard.