QuickTime for Java: A Developer's Notebook/Editing Movies

From WikiContent

< QuickTime for Java: A Developer's Notebook(Difference between revisions)
Jump to: navigation, search
m (1 revision(s))
Current revision (13:22, 7 March 2008) (edit) (undo)
(Initial conversion from Docbook)
 
(One intermediate revision not shown.)

Current revision

QuickTime for Java: A Developer's Notebook

Playback is nice, but you have nothing to play if you lack tools to create media, and the most critical of these are editing tools. If you've ever used iMovie with your home movies, you know what I'm talking about: there's a huge difference between watching a cute collection of scenes of your kids playing, set to music, and watching the two hours of unedited raw footage you started with. Sometimes, less is more.

Contents

Copying and Pasting

The most familiar form of editing is copy-and-paste, which many users already are familiar with from the "pro" version of QuickTime Player. The metaphor is identical to how copy-and-paste works in nonmedia applications such as text editors and spreadsheets: select some source material of interest, do a "copy" to put it on the system clipboard, select an insertion point in this or another document, and do a "paste" to put the contents of the clipboard into that target.

In the simplest form of a QuickTime copy-and-paste, the controller bar (from MovieController) is used to indicate where copies and pastes should occur. By shift-clicking, a user can select a time-range from the current time (indicated by the play head) to wherever the user shift-clicks (or, if he is dragging, wherever the mouse is released).

Note

QuickTime Pro costs money ($29.99 as of this writing), but it allows you to exercise much of the QuickTime API from QuickTime Player, which can be a useful debugging tool.

How do I do that?

BasicQTEditor, shown in Example 3-1, will be the basis for the examples in this chapter. It offers a single empty movie window (with the ability to open movies from disk in new windows or to create new empty movie windows), and an Edit menu with cut, copy, and paste options.

Example 3-1. A copy-and-paste movie editor

package com.oreilly.qtjnotebook.ch03;
 
import quicktime.*;
import quicktime.qd.QDRect;
import quicktime.std.*;
import quicktime.std.movies.*;
import quicktime.app.view.*;
import quicktime.io.*;
 
import java.awt.*;
import java.awt.event.*;
 
import com.oreilly.qtjnotebook.ch01.QTSessionCheck;
 
public class BasicQTEditor extends Frame
  implements ActionListener {
 
  Component comp;
 
  Movie movie;
  MovieController controller;
  Menu fileMenu, editMenu;
  MenuItem openItem, closeItem, newItem, quitItem;
  MenuItem copyItem, cutItem, pasteItem;
  static int newFrameX = -1;
  static int newFrameY = -1;
  static int windowCount = 0;
  
  /** no-arg constructor for "new" movie
   */
  public BasicQTEditor ( ) throws QTException {
      super ("BasicQTEditor");
      setLayout (new BorderLayout( ));
      QTSessionCheck.check( );
      movie = new Movie(StdQTConstants.newMovieActive);
      controller = new MovieController (movie);
      controller.enableEditing(true);
      doMyLayout( );
  }
 
  /** file-based constructor for opening movies
   */
  public BasicQTEditor (QTFile file) throws QTException {
      super ("BasicQTEditor");
      setLayout (new BorderLayout( ));
      QTSessionCheck.check( );
      OpenMovieFile omf = OpenMovieFile.asRead (file);
      movie = Movie.fromFile (omf);
      controller = new MovieController (movie);
      controller.enableEditing(true);
      doMyLayout( );
  }
  /** gets component from controller, makes menus
   */
  private void doMyLayout( ) throws QTException {
      // add movie component
      QTComponent qtc =
          QTFactory.makeQTComponent (controller);
      comp = qtc.asComponent( );
      add (comp, BorderLayout.CENTER);
      // file menu
      fileMenu = new Menu ("File");
      newItem = new MenuItem ("New Movie");
      newItem.addActionListener (this);
      fileMenu.add (newItem);
      openItem = new MenuItem ("Open Movie...");
      openItem.addActionListener (this);
      fileMenu.add (openItem);
      closeItem = new MenuItem ("Close");
      closeItem.addActionListener (this);
      fileMenu.add (closeItem);
      fileMenu.addSeparator( );
      quitItem = new MenuItem ("Quit");
      quitItem.addActionListener (this);
      fileMenu.add(quitItem);
      // edit menu
      editMenu = new Menu ("Edit");
      copyItem = new MenuItem ("Copy");
      copyItem.addActionListener(this);
      editMenu.add(copyItem);
      cutItem = new MenuItem ("Cut");
      cutItem.addActionListener(this);
      editMenu.add(cutItem);
      pasteItem = new MenuItem ("Paste");
      pasteItem.addActionListener(this);
      editMenu.add(pasteItem);
      // make menu bar
      MenuBar bar = new MenuBar( );
      bar.add (fileMenu);
      bar.add (editMenu);
      setMenuBar (bar);
      // add close-button handling
      addWindowListener (new WindowAdapter( ) {
              public void windowClosing (WindowEvent e) {
                  doClose( );
              }
          });
  }
 
  /** handles menu actions
   */
  public void actionPerformed (ActionEvent e) {
      Object source = e.getSource( );
      try {
          if (source =  = quitItem) doQuit( );
          else if (source =  = openItem) doOpen( );
          else if (source =  = closeItem) doClose( );
          else if (source =  = newItem) doNew( );
          else if (source =  = copyItem) doCopy( );
          else if (source =  = cutItem) doCut( );
          else if (source =  = pasteItem) doPaste( );
      } catch (QTException qte) {
          qte.printStackTrace( );
      }
  }
 
  public void doQuit( ) {
      System.exit(0);
  }
 
  public void doNew( ) throws QTException {
      makeNewAndShow( );
  }
 
  public void doOpen( ) throws QTException {
      QTFile file =
          QTFile.standardGetFilePreview (QTFile.kStandardQTFileTypes);
      Frame f = new BasicQTEditor (file);
      f.pack( );
      if (newFrameX >= 0)
          f.setLocation (newFrameX+=16, newFrameY+=16);
      f.setVisible(true);
      windowCount++;
  }
 
  public void doClose( ) {
      setVisible(false);
      dispose( );
      // quit if no windows now showing
      if (--windowCount =  = 0)
          doQuit( );
  }
 
  public void doCopy( ) throws QTException {
      Movie copied = controller.copy( );
      copied.putOnScrap(0);
  }
 
  public void doCut( ) throws QTException {
      Movie cut = controller.cut( );
      cut.putOnScrap(0);
  }
 
  public void doPaste( ) throws QTException {
      controller.paste( );
      pack( );
  }
 
/** Force frame's size to respect movie size
   */
  public Dimension getPreferredSize( ) {
      System.out.println ("getPreferredSize");
      if (controller =  = null)
          return new Dimension (0,0);
      try {
          QDRect contRect = controller.getBounds( );
          Dimension compDim = comp.getPreferredSize( );
          if (contRect.getHeight( ) > compDim.height) {
              return new Dimension (contRect.getWidth( ) +
                                    getInsets( ).left +
                                    getInsets( ).right,
                                    contRect.getHeight( ) +
                                    getInsets( ).top +
                                    getInsets( ).bottom);
 
          } else {
              return new Dimension (compDim.width +
                                    getInsets( ).left +
                                    getInsets( ).right,
                                    compDim.height +
                                    getInsets( ).top +
                                    getInsets( ).bottom);
 
          }
      } catch (QTException qte) {
          return new Dimension (0,0);
      }
  }
 
  /** opens a single new movie window
   */
  public static void main (String[  ] args) {
      try {
          Frame f = makeNewAndShow( );
          // note its x, y for future calls
          newFrameX = f.getLocation( ).x;
          newFrameY = f.getLocation( ).y;
      } catch (Exception e) {
          e.printStackTrace( );
      }
  }
 
  /** creates "new" movie frame, packs and shows.
      used by main( ) and "new"
   */
  private static Frame makeNewAndShow( )
      throws QTException {
      Frame f = new BasicQTEditor( );
      f.pack( );
      if (newFrameX >= 0)
          f.setLocation (newFrameX+=16, newFrameY+=16);
      f.setVisible(true);
      windowCount++;
      return f;
  }
}

Note

With the downloaded book code, compile and run this with ant run-ch03-basicqteditor.

Figure 3-1 shows the BasicQTEditor class in action, with two windows open. The window on the left is the original empty movie window, with the user about to paste in some contents. The window on the right is a movie that was opened from a file. Note the small stretch of darker gray in the timeline, under the play head, which indicates the selected segment that was copied from the movie to the system clipboard.

Figure 3-1. BasicQTEditor with two movies open

BasicQTEditor with two movies open

Also note that when running in Windows, as pictured here, the menus are inside the windows. On Mac OS X, the usage of AWT means the File and Edit menus will be at the top of the screen in the Mac's "One True Menu Bar."

One usability note: for simplicity, I haven't tried to make this particularly smart about what the user "really wants," and that can be bad on the paste . The paste will replace whatever is selected in the target movie, and if there is no selection, it will paste to the beginning of the movie. It's probably more typical to add clips either to the end of the movie, or to the current time as indicated by the play head (i.e., to behave as if a lack of a selection should be interpreted as a zero-length selection beginning and ending at the movie's current time). It's simple enough to add this kind of intelligence to doPaste() and find a behavior that feels better.

What just happened?

This is a big example, so here's an overview.

The no-arg constructor, BasicQTEditor( ), initializes QuickTime with Chapter 1s QTSessionCheck, then creates a new empty Movie, gets a MovieController for it, and calls doMyLayout. A second constructor, BasicQTEditor (QTFile), is essentially identical, except that instead of creating an empty movie, it gets a movie from the provided QTFile. The movie and the controller instance variables are used by many methods throughout the application.

The doMyLayout( ) method sets up the menus and their ActionListeners and reminds us that building GUIs in code is a pain.

actionPerformed (ActionEvent) is used to farm out method calls from clicks on the various menu items.

doQuit( ) is a trivial call to System.exit(0). Remember that the QTSessionCheck call has set up a shutdown handler to close QuickTime when Java goes away.

doNew( ) trivially calls makeNewAndShow(), which is a convenience method to call the no-arg constructor (which creates an empty Movie), pack the frame, and move it down and to the right 16 pixels from the last place a new window was created.

Tip

Note that there's nothing here to keep new windows from going off-screen if the user creates enough of them. In a more polished application, you'd check the proposed x and y against the screen size reported by the AWT's Toolkit.getScreenSize() .

doOpen( ) brings up a file-open dialog and calls the file-aware constructor. It then pack( ) s the window and positions it in the same way makeNewAndShow() does.

doClose( ) closes the frame and, if it is the last open window, quits the application via doQuit( ) (yes, this is Windows-like behavior, as opposed to the typical Mac application which can hang around with no open windows).

doCopy( ) and doCut( ) are practically identical, and each needs only two lines to do its thing. They make a call to the MovieController to cut or copy the current selection and return the result as a new Movie. Then they put this movie on the system clipboard with the movie's putOnScrap( ) call.

doPaste( ) is even simpler: it just calls the controller's paste( ) method and then re-pack( )s the window.

The getPreferredSize() method overrides the default by indicating that the window needs to be large enough to contain the movie, its control bar, and any insets that might be set. This is why you should pack( ) after each paste: the original empty movie has no size other than its control bar, so when you paste into it, the size of the movie (and thus its controller) changes to accommodate the pasted contents, and you need the frame to adjust to that.

Warning

This really should be taken care of automatically in Java, because the use of a BorderLayout should allow the contents to achieve their preferred size on a pack( ). Unfortunately, on Mac OS X, the QTComponent exhibits a bizarre behavior where its preferred size is set once, when it's packed, and never again. So, a component built from an empty movie always thinks it's supposed to be zero pixels high by 160 pixels wide, even if you paste in contents much larger than that. Fixing this reveals the opposite problem on Windows: sometimes there's a good preferred size and a zero-height controller bound. The version here prefers whichever set of bounds has a greater height.

What about...

...that weird play head? That is odd, isn't it? The call to enableEditing(true) has changed the play head ball to an hourglass shape. Figure 3-2 shows it at an enlarged size.

Figure 3-2. MovieController scrubber bar with editing enabled

MovieController scrubber bar with editing enabled

My guess is that the shape is supposed to help you select the exact point for making a selection, instead of burying it under the center of the ball. That said, there's a reason you don't see this elsewhere: this default widget isn't terribly well-suited to editing. The QuickTime Player application that comes with QuickTime has a custom controller widget with two little triangles under the timeline to mark in and out points. But that control, like this one, shares the flaw that the accuracy of your edit is limited by the on-screen size of your movie. More serious editing applications, like Premiere and Final Cut Pro, have custom GUI components for editing, usually based on a timeline that can be "zoomed" to an arbitrary accuracy. Of course, one could do the same with AWT or Swing, tracking MouseEvents, paint()ing as necessary, and making programmatic calls to QTJ to perform actions.

Performing "Low-Level" Edits

Low-level edits are a separate set of editing calls that don't involve the clipboard or selection metaphors. They're called "low level" because instead of operating at the conceptual level of "paste the contents of the clipboard into the user's current selection," they work at the level of "insert a segment from movie M1, ranging from time A to time B, into movie M2 at time C."

Note

By way of comparison, although QuickTime has two sets of editing functions, Sun's Java Media Framework has no editing API at all.

How do I do that?

This version reimplements doCopy( ), doCut( ), and doPaste( ) to use low-level editing calls on the Movie instead of cut/copy/paste-type calls on the MovieController.

First, LowLevelQTEditor needs a static Movie, called copiedMovie, to keep track of what's on its virtual "clipboard" so that it can be shared across the new doCopy( ), doCut(), and doPaste( ) methods:

public void doCopy( ) throws QTException {
      copiedMovie = new Movie( );
      TimeInfo selection = movie.getSelection( );
      movie.insertSegment (copiedMovie,
                           selection.time,
                           selection.duration,
                           0);
  }
 
public void doCut( ) throws QTException {
      copiedMovie = new Movie( );
      TimeInfo selection = movie.getSelection( );
      movie.insertSegment (copiedMovie,
                           selection.time,
                           selection.duration,
                           0);
      movie.deleteSegment (selection.time,
                           selection.duration);
      controller.movieChanged( );
  }
 
public void doPaste( ) throws QTException {
      if (copiedMovie =  = null)
          return;
      copiedMovie.insertSegment (movie,
                                 0,
                                 copiedMovie.getDuration( ),
                                 movie.getSelection( ).time);
      controller.movieChanged( );
      pack( );
  }

Note

You can make ant compile and run this example with ant run-ch03-lowlevelqteditor.

The only thing the user might see as being different or odd in this example is that the cut or copied clip does not get put on the system clipboard because low-level edits don't touch the clipboard.

Tip

For what it's worth, this example was intended originally to be a drag-and-drop demo, for which these low-level, segment-oriented calls are particularly well-suited. Unfortunately, the QTComponent won't generate an AWT "drag gesture." I suppose it would be a little unnatural to drag the current image as a metaphor for copying a segment of a movie. Anyway, if you decide to do your own controller GUI, you can use this low-level stuff for your drag-and-drop.

What just happened?

The doCut( ), doCopy( ), and doPaste( ) methods all call Movie.insertSegment() ; either to put some part of a source movie into the clipboard-like copiedMovie or to put the copiedMovie into the target movie. This method takes four arguments:

  • The Movie to insert into
  • The start time of the segment, in the movie's time scale
  • The end time of the segment, in the movie's time scale
  • The time in the target movie when the segment should be inserted

In the case of a cut, the deleteSegment() call removes the segment that was just copied out. This method simply takes the beginning and end times of the segment to delete.

Note

Time scales are covered in Chapter 2, in the section Section 2.5."

In the doPaste( ) and doCut( ) methods, a call to MovieController.movieChanged( ) lets the controller know that the movie was changed in a way that didn't involve a method call on the controller, and that the controller now needs to update itself to adjust to the changed duration, current time, etc.

What about...

...any other low-level calls? There is an interesting method in the Movie class, called scaleSegment() , which changes the duration of a segment, meaning it either slows it down or speeds it up to suit the specified duration. This could be handy for creating a " slow-motion" or "fast-motion" effect from a normal-speed source, or stretching it out to fit a piece of audio.

Undoing an Edit

Critical to any kind of editing is the ability to back out of a change that had unintended or undesirable effects. Fortunately, controller-based cuts and pastes can be undone with some fairly simple calls.

How do I do that?

UndoableQTEditor builds on the original BasicQTEditor by adding an "undo" menu item. The doUndo( ) method it calls has an utterly trivial implementation:

public void doUndo( ) throws QTException {
  controller.undo( );
}

Note

Compile and run this example with ant run-ch03-undoableqteditor.

What just happened?

With a simple call to MovieController.undo() , the program gained the ability to undo a cut or paste, or any other destructive change made through the controller.

What about...

...multiple undoes? Or redoes? Ah, there's the rub. Hit undo again and the cut or paste is redone, in effect undoing the undo.

Sadly, this is your dad's "undo"...the undo from back in 1990, when a single level of undo was a pretty cool thing. Today, when users expect to perform multiple destructive actions with impunity, it's not too impressive.

Undoing and Redoing Multiple Edits

Fortunately, QTJ offers a unique opportunity to combine Swing's thoughtfully designed undo API, javax.swing.undo, with QuickTime's support for reverting a movie to a previous state. Combined, these features provide the ability to support a long trail of undoes and redoes.

How do I do that?

RedoableQTEditor again builds on BasicQTEditor, adding a Swing UndoManager that is used by both the doUndo( ) and doRedo( ) methods:

Note

Compile and run this example with ant run-ch03-redoableqteditor.

public void doUndo( ) throws QTException {
  if (! undoanager.canUndo( )) {
      System.out.println ("can't undo");
      return;
  }
  undoManager.undo( );
}
 
public void doRedo( ) throws QTException {
  if (! undoManager.canRedo( )) {
      System.out.println ("can't redo");
      return;
  }
  undoManager.redo( );
}

The information about a destructive edit is encapsulated by an inner class called QTEdit :

class QTEdit extends AbstractUndoableEdit {
  MovieEditState previousState;
  MovieEditState newState;
  String name;
  public QTEdit (MovieEditState pState,
                 MovieEditState nState,
                 String n) {
      previousState = pState;
      newState = nState;
      this.name = n;
  }
  public String getPresentationName( ) {
      return name;
  }
  public void redo( ) throws CannotRedoException {
      super.redo( );
      try {
          movie.useEditState (newState);
          controller.movieChanged( );
      } catch (QTException qte) {
          qte.printStackTrace( );
      }
  }
  public void undo ( ) throws CannotUndoException {
      super.undo( );
      try {
          movie.useEditState (previousState);
          controller.movieChanged( );
      } catch (QTException qte) {
          qte.printStackTrace( );
      }
  }
  public void die( ) {
      previousState = null;
      newState = null;
  }
}

Finally, doCut( ) and doPaste() are amended to create suitable QTEdits and hand them to the UndoManager:

public void doCut( ) throws QTException {
  MovieEditState oldState = movie.newEditState( );
  Movie cut = movie.cutSelection( );
  MovieEditState newState = movie.newEditState( );
  QTEdit edit = new QTEdit (oldState, newState, "Cut");
  undoManager.addEdit (edit);
  cut.putOnScrap(0);
  controller.movieChanged( );
}
 
public void doPaste( ) throws QTException {
  MovieEditState oldState = movie.newEditState( );
  Movie pasted = Movie.fromScrap(0);
  movie.pasteSelection (pasted);
  MovieEditState newState = movie.newEditState( );
  QTEdit edit = new QTEdit (oldState, newState, "Paste");
  undoManager.addEdit (edit);
  controller.movieChanged( );
  pack( );
}

When clicked, the Undo menu item now undoes a cut or paste. Redo redoes the edit, while a second "undo" will undo the previous edit, etc.

What just happened?

Obviously, the fun parts involve the destructive actions and how they save enough information to be undoable and redoable. In each case, they call Movie.newEditState() to create a MovieEditState , a QuickTime object that contains the information needed to revert the movie to the current state at some point in the future. Then they do the destructive action and create another MovieEditState to represent the post-edit state. These objects are passed to the QTEdit, which is then sent to the UndoManager to join its stack of edits.

Note

.

When the UndoManager.undo() method is called, it takes the first undoable edit, if there is one, and calls its undo( ) method. In this case, that means the manager is calling the QTEdit.undo( ) method, which takes the pre-edit MovieEditState and passes it to Movie.useEditState( ) to return the movie to that state. Similarly, a post-undo call to QTEdit.redo( ) also uses useEditState( ) to get to the post-edit state.

Saving a Movie to a File

Once a user has performed a number of edits and has a finished project, she presumably needs to save the movie to disk. In QuickTime, many different actions can be thought of as "saving" a movie. Perhaps the simplest and most flexible option is to let the user decide.

How do I do that?

The SaveableQTEditor uses a QTFile to keep track of where a movie was loaded from (null in the case of a new movie). This is used by the doSave( ) method to indicate where the saved file goes:

public void doSave( ) throws QTException {
  // if no existing file, then prompt for one
  if (file =  = null) {
      file = new QTFile (new File ("simplemovie.mov"));
  }
  int flags = StdQTConstants.createMovieFileDeleteCurFile |
      StdQTConstants.createMovieFileDontCreateResFile |
      StdQTConstants.showUserSettingsDialog;
  movie.convertToFile (file, // file
                       StdQTConstants.kQTFileTypeMovie, // filetype,
                       StdQTConstants.kMoviePlayer, // creator
                       IOConstants.smSystemScript, // scriptTag
                       flags);
}

Note

Compile and run this example with ant run-ch03-saveableqteditor.

When the user hits the Save menu item, she'll see the QuickTime Save As dialog as shown in Figure 3-3.

Figure 3-3. QuickTime Save As dialog

QuickTime Save As dialog

This dialog's Export selector gives the user four choices:

Movie
Saves a QuickTime reference movie , a tiny (typically 4 or 8 KB) file that contains just references (pointers) to the media in their original locations
Movie, self-contained
Copies all the media, in their original encodings, into a new QuickTime movie file
Movie to Hinted Movie
Creates a self-contained movie but lets the user adjust the hinting settings for use in a streaming server
Movie to QuickTime Movie
Creates a self-contained movie, but lets the user choose different compressors and settings to re-encode the audio and video

Some of these options give the user additional choices. Saving a "self-contained" movie presents an Options... button that lets the user specify the audio and video codecs to be used in the saved movie, their quality and bitrate settings, etc. A "Use" pop up contains canned settings with appropriate choices for distributing the movie on CD-ROM, over dial-up, etc.

Once the user clicks Save, the program saves the movie to disk. This is a very fast operation for the reference movie option and a potentially slow operation for the other options because the media might be re-encoded into a new format as part of the save.

What just happened?

The key is the Movie.convertToFile() method. The version shown here takes five parameters:

  • The QTFile to save to.
  • An int to represent the old Mac OS file "type." Use the constant kQTFileTypeMovie , which gives it the QuickTime movie type moov.
  • An int to represent the old Mac OS file "creator." The boilerplate option is kMoviePlayer , which associates it with the default QuickTime Player application.
  • An int to represent the old Mac OS "scriptTag," which indicates what kind of "script system" (character encoding, writing direction, etc.) is to be used. Common practice is to use the constant smSystemScript to use whatever the operating system's current script is.
  • Behavior flags to affect the save operation, logically ORed together. The most important flag for this example is the showUserSettingsDialog ; without it, the program would silently save the file with Apple's ancient "Video" codec and uncompressed sound. This example also uses the flag createMovieFileDeleteCurFile to delete any file already at the target location and createMovieFileDontCreateResFile to force the file to exist in a single data "fork," instead of using the old Mac OS' "resource" fork. This is required for making QuickTime movies that run on multiple platforms.

Note

Most of the time, it's appropriate to use boilerplate code for things like type, creator, and system script, and not to have to read some Inside Macintosh book from 10 years ago.

What about...

...other interesting behavior flags? The docs for the native ConvertMovieToFile function offer two that aren't shown here because they seem to indicate behavior that is already the default:

  • movieFileSpecValid indicates that the file passed in actually exists and should be shown as the default save location.
  • movieToFileOnlyExport restricts the dialog to showing only the data export components that are actually present.

Can anything be done about the interminable wait when saving "Movie to QuickTime Movie"? One thing that helps is to provide a "progress function," which provides a visual representation of the progress being made on the long save operation. You can set up the default progress function with a one-line call right before convertToFile():

movie.setProgressProc( )

This will bring up a progress dialog like the one shown in Figure 3-4.

Figure 3-4. Default QuickTime progress dialog

Default QuickTime progress dialog

The Movie class also has a setProgressProc( ) method that takes a MovieProgress object as a parameter. The idea here is that of a typical callback arrangement—during a long save, MovieProgress.execute() is called repeatedly with four parameters: the movie being monitored, a "message" int, a "what operation" int, and a float that represents the percentage done on a scale from 0.0 to 1.0. Unfortunately, this interface has a couple of problems. First, the constants for the "message" aren't defined in QTJ (a few printlns here and there show that the values are 0 for start, 1 for update, and 2 for done). More importantly, using this callback seems extremely unstable in QTJ 6.1—I find I often get an exception with an "Unknown Error Code," and the movie doesn't save. So, maybe the default behavior is the safe choice for now.

Flattening a Movie

Saving a movie can mean different things in QuickTime: saving a reference movie, saving a self-contained movie, or exporting to a different format. Typically, though, the idea of creating a self-contained movie is what users think of as "saving"—they want a single file that doesn't depend on any others, so they can put it on a server, email it to mom, etc. This process is called "flattening."

Note

"Flattening" is also an old Mac OS term for turning a file with both a resource fork and a data fork into a single-fork file, suitable for use on non-Mac disk formats. In this book, we use "flatten" only in its QuickTime sense.

How do I do that?

The FlattenableQTEditor is similar to the SaveableQTEditor, adding the menu item and its typical GUI and action-handling support. The flattening is done in a doFlatten( ) method:

public void doFlatten( ) throws QTException {
  // always attempts to save to a new location,
  // so prompt for filename
  FileDialog fd = new FileDialog (this,
                                  "Flatten...",
                                  FileDialog.SAVE);
  fd.setVisible(true); // blocks
  if ((fd.getDirectory( ) =  = null) ||
      (fd.getFile( ) =  = null))
      return;
  QTFile flatFile =
      new QTFile (new File (fd.getDirectory( ),
                                   fd.getFile( )));
  if (flatFile.exists( )) {
      // JOptionPane is a bit of cheat-for-clarity here,
      // building a working AWT dialog would be punitive
      int choice = 
          JOptionPane.showConfirmDialog (this,
                                           "Overwrite " + 
                                        flatFile.getName( ) + "?",
                                        "Flatten",
                                        JOptionPane.OK_CANCEL_OPTION);
      if (choice != JOptionPane.OK_OPTION)
          return;
  }
  movie.flatten(StdQTConstants.flattenAddMovieToDataFork |
                StdQTConstants.flattenForceMovieResourceBeforeMovieData, 
                flatFile, // fileOut
                StdQTConstants.kMoviePlayer, // creator
                IOConstants.smSystemScript, // scriptTag
                StdQTConstants.createMovieFileDeleteCurFile, 
                StdQTConstants.movieInDataForkResID, // resID
                null); // resName
}

Note

Compile and run this example with ant run-ch03-flattenableqt-editor.

When run, this creates a self-contained QuickTime movie file at the specified location, using whatever video and audio encoding was used in the original sources. This can result in some playback jitters if the user has mixed in different kinds of codecs—for example, pasting in some MPEG-4 video with some Sorenson 3 video. Flattening doesn't change encoding; it just resolves references and puts all the media into one file.

What just happened?

The Movie.flatten( ) call creates the self-contained movie file, taking seven parameters to control its behavior:

Note

Many of these are the same parameters used by Movie.convertToFile( ), covered in the previous lab.

  • Behavior flags for the flatten operation, logically ORed together. This example uses flattenAddMovieToDataFork to create a single-fork movie that is more suitable for non-Mac operating systems. Using flattenForceMovieResourceBeforeMovieData creates a "quick start" movie, so named because all its metadata comes before its media samples, which allows QuickTime to start playing the movie from a stream, even an http://-style URL, before all the data is loaded, because all the information QuickTime needs (what tracks are present, what size the video is, how loud the audio is, etc.) is loaded first.
  • The file to flatten to.
  • The Mac OS "creator," typically kMoviePlayer.
  • The Mac OS script tag, typically smSystemScript.
  • The behavior flags that are used for the create file operation. createMovieFileDeleteCurFile is used here to delete any file already at the target file location.
  • Resource ID. For cross-platform reasons, it's usually best to use movieInDataForkResID instead of old Mac OS-style resources.
  • Resource name. Irrelevant here, so null will do.

What about...

...behavior flags for the flatten operation? The native docs for FlattenMovie define a bunch, but the ones not used here are largely esoteric.

flattenDontInterleaveFlatten
Turns off "interleaving," an optimization that mixes audio and video samples together so that they're easier to read at playback time (if a movie had a couple of megabytes' worth of video samples, followed by a couple of megabytes' worth of audio samples, the hard drive would have a difficult time zipping back and forth between the two; interleaving puts the samples for the same time period in the same place so that they can be read together). The default behavior is a good thing, so this constant isn't used often.
flattenActiveTracksOnly
Doesn't include disabled tracks from the movie in the flattened file.
flattenCompressMovieResource
Compresses the movie's resource, and its organizational and metadata structure, if stored in the data fork. Like you care.
flattenFSSpecPtrIsDataRefRecordPtr
This is meaningless in QTJ.

Saving a Movie with Dependencies

The opposite of flattening is saving a movie with dependencies. In this type of a save, the resulting file just contains pointers to the sources of the media in each track. The file typically is tiny, usually just 8 KB or less.

How do I do that?

The RefSaveableQTEditor example extends the FlattenableQTEditor with a "Save w/Refs" menu item that calls doRefSave():

public void doRefSave( ) throws QTException {
  // if no home file, then prompt for one
  if (file =  = null) {
      FileDialog fd = new FileDialog (this,
                                      "Save...",
                                      FileDialog.SAVE);
      fd.setVisible(true); // blocks
      if ((fd.getDirectory( ) =  = null) ||
          (fd.getFile( ) =  = null))
          return;
      file = new QTFile (new File (fd.getDirectory( ),
                                   fd.getFile( )));
  }
  // save ref movie to file
  if (! file.exists( )) {
      file.createMovieFile(StdQTConstants.kMoviePlayer,
                           StdQTConstants.createMovieFileDontCreateResFile);
  }
  OpenMovieFile outFile =
      OpenMovieFile.asWrite(file);
  movie.updateResource (outFile,
                        StdQTConstants.movieInDataForkResID,
                        null);
}

Note

Compile and run this example with ant run-ch03-refsaveableqt-editor.

When run, this creates a movie file that, despite its tiny size, behaves exactly like any other movie file. Double-click it and it will open in QuickTime Player, just like a self-contained movie. QuickTime completely isolates the user from the fact that the file contains nothing more than metadata and pointers to the source media files.

Of course, there are limits to what QuickTime can do if those pointers cease to be valid. A user can move the source files and the movie still will play, but if the source movies are deleted, or if the reference movie is transferred to another system, QuickTime won't be able to resolve the references. This typically will result in a "searching..." dialog, followed by a dialog asking the user to locate the missing media, as shown in Figure 3-5.

Figure 3-5. Unresolvable media reference dialog

Unresolvable media reference dialog

What just happened?

First, a call to QTFile.createMovieFile() creates the file on disk, if it doesn't exist already. This method takes two parameters:

  • A Mac OS "creator," for which StdQTConstants.kMoviePlayer is the typical boilerplate value.
  • Behavior flags. The constant createMovieFileDontCreateResFile commonly is used to create cross-platform, single-fork files.

With the file created, the reference movie data can be put into the file with the updateResource() method. This method takes three parameters:

Note

The name updateResource( ) seems to be another Classic Mac OS legacy that doesn't make much sense today.

  • An OpenMovieFile, opened for writing.
  • A resource ID, for which the appropriately cross-platform, no-resource-fork value is movieInDataForkResId.
  • An updated name for the resource; null is appropriate here.

What about...

...the fragility of reference movies? Because a reference movie is fragile, why would anyone ever create one? This technique is very handy for the saving state in editing applications because it allows the user to quickly save his edited movie without the I/O grinding of flattening. Editing, after all, can be seen as a process of arranging pointers to source materials; in the professional realm, a document called an Edit Decision List (EDL) is a simple list of "in" and "out" points from source media that you can use to produce the edited media. The reference movie is equivalent to the EDL: it's just a collection of pointers, with the nice advantage that it continues to behave as a normal QuickTime movie. So, the reference movie can be used to save the progress of the user's editing work, and when finished, a final self-contained movie can be generated via flattening or exporting (see Chapter 4).

Editing Tracks

Often, it makes sense to perform edits on all tracks of a movie. But for serious editing applications, sometimes you need to work at the track level, to add and remove tracks, or to work on just one track in isolation from the others. This task will provide a taste of that by adding a second audio track to a movie.

How do I do that?

The AddAudioTrackQTEditor builds on FlattenableQTEditor by adding another Add Audio Track... menu item, calling the doAddAudioTrack( ) method:

public void doAddAudioTrack( ) throws QTException {
  // ask for an audio file
  QTFile audioFile =
      QTFile.standardGetFilePreview (QTFile.kStandardQTFileTypes);
  OpenMovieFile omf = OpenMovieFile.asRead (audioFile);
  Movie audioMovie = Movie.fromFile (omf);
  // find the audio track, if any
  Track audioTrack =
          audioMovie.getIndTrackType (1,
                                 StdQTConstants.audioMediaCharacteristic,
                                 StdQTConstants.movieTrackCharacteristic);
  if (audioTrack =  = null) {
      JOptionPane.showMessageDialog (this,
                                     "Didn't find audio track",
                                     "Error",
                                     JOptionPane.ERROR_MESSAGE);
      return;
  }
  // now make new audio track and insert segment
  // from the loaded track
  Track newTrack =
      movie.newTrack (0.0f, // width
                      0.0f, // height
                      audioTrack.getVolume( ));
  // ick, need a dataref for our "new" media
  // http://developer.apple.com/qa/qtmtb/qtmtb58.html
  SoundMedia newMedia =
      new SoundMedia (newTrack,
                      audioTrack.getMedia( ).getTimeScale( ),
                      new DataRef (new QTHandle( )));
  newTrack.getMedia( ).beginEdits( );
  audioTrack.insertSegment (newTrack,
                            0,
                            audioTrack.getDuration( ),
                            0);
  controller.movieChanged( );
}

Note

Compile and run this example with ant run-ch03-addaudiotrackqteditor.

This method is admittedly contrived—it prompts the user to open another file, and if an audio track can be found in the file, the program adds that track to the movie, starting at time 0. If the user has done only a few short pastes and then adds an audio track from a typical iTunes MP3 or AAC, the result probably will be a movie in which the new soundtrack is much longer than the pasted contents.

Also, QuickTime will eat more CPU cycles playing this movie, because it has to decode two compressed soundtracks at once. Like I said, it's a contrived example, but it covers some interesting ground.

What just happened?

The program tries to find an audio track with Movie.getIndTrackType( ) , passing audioMediaCharacteristic as the search criterion. Assuming an audio track is found in this movie, the program needs to create a new track in the movie being edited. Movie.newTrack( ) creates the new track, taking as parameters the width, height, and volume of the new track.

This new track is useless without a Media object to hold the actual sound data, so the next step is to construct a new SoundMedia object. The constructor takes the track that the media is to be associated with, a time scale, and a DataRef to indicate where media samples can be stored.

Interestingly, although the edit methods this program uses are in the Track class, first I have to call Media.beginEdits( ) to inform the track's underlying media that it's about to get edited. Having done this, the program then can call Track.insertSegment() , which is identical to its low-level-editing Movie equivalent, taking a target track, source in and out times, and a destination-in time. Following this, the program calls movieChanged( ) on the movie controller to let it know that a change was made to the movie behind the controller's back.

The result is an additional audio track in the movie. If the user then flattens the movie and opens it up with QuickTime Player, a "Get Info" shows the extra audio track, as seen in Figure 3-6. In this case, I imported clips from an MPEG-4 file and added an MP3 soundtrack.

Note

No, I'm not swearing in this filename. I combined a video of my son in an inflatable boat with an MP3 of a song called "Dam Dariram" from the video game "Dance Dance Revolution"; thus, "dam-boat.mov".

Figure 3-6. QuickTime Player "Get Info" for movie with multiple audio tracks

QuickTime Player "Get Info" for movie with multiple audio tracks

What about...

...that crazy-looking new DataRef (new QTHandle( )) parameter in the SoundMedia constructor? OK, scary edge case—here's the story. Zoom out for a second: movies have tracks, tracks have media, media have samples. Those samples need to live somewhere. It's not a problem when you open a movie from disk, but when you create new media in a new movie, QuickTime has no idea where it's supposed to put any samples that you add, whether by way of inserting segments from other tracks or by adding individual samples one by one (which will be covered in Chapters Chapter 7, Chapter 8, and Chapter 9). So, this example uses the SoundMedia constructor that takes a DataRef, which represents a location to store the samples. This DataRef can be practically anything, even a zero-length buffer in memory, which is pretty much what this example passes in by constructing a new DataRef out of a new, empty QTHandle.

Tip

For more on this icky little gotcha, and if you don't mind a C-oriented technote, see "BeginMediaEdits -2050 badDataRefIndex error after calling NewMovie" at http://developer.apple.com/qa/qtmtb/qtmtb58.html.

Also, what about the control bar? It tells the user nothing about the tracks in the movie. You're absolutely right. Being playback-oriented, the provided GUI is weak for editing movies, and utterly useless for editing tracks. It gives the user no idea how many tracks a movie has, where there's video without sound or vice versa, etc. Moreover, there's no default widget in QTJ to replace it. If you want to provide track-oriented editing, you'll need to develop your own GUI components to display tracks and their contents. I haven't provided one here, because the appearance and behavior of such a component would vary wildly with the kind of application it was needed for (a home movie editor, an MP3 playlist builder, etc.) and because it easily could contain more than 1,000 lines of AWT code with maybe a dozen lines of QuickTime...not exactly ideal for the format of this book.

What about other track-editing methods? Fortunately, many of the concepts from the low-level Movie editing lab from earlier in the chapter apply to tracks. Along with Track.insertSegment() are a deleteSegment() and a scaleSegment() that work like their Movie equivalents. The insertEmptySegment() does what its name implies, and could be useful for building a track in nonconsecutive segments. There's also a Track.insertMedia() that will be used in later chapters to build up a Media object from raw samples.

As for how the tracks relate to their parent movies, this example uses Movie.newTrack( ) , though it also is possible to use addEmptyTrack() , which takes a prototype track and a DataRef. Tracks can be removed with Movie.removeTrack( ) and temporarily turned on and off with Track.setEnabled() .

Personal tools