Quartz Composer

Quartz for Revolution Document Part 1 of 2 By: Obleo Beck

February 2009

Quartz for Revolution Quartz is an umbrella technology and includes many other technologies part of it. While we can access anything we want from any giving system, here we are going to focus on the use of Quartz Compositions as a processing method. This method will include Core Video, QuickTime, Core Image, Core Audio and a few others by the use of Quartz Compositions, QuickTime and shell for the most part. Overview of the Quartz Process to use with Runtime Revolution and mac OS X. Quartz Compositions is just one way we can access quartz related processes. The tutorials I present in the most part will start here. The reason I will focus on this method the most is because this method stays in line to Runtime Revolution rapid development model. Using, and building Quartz Compositions are Rapid to develop and use, just like Runtime Revolution. You will find there is no faster and more simpler way to add high end graphic processing to a revolution built app targeting deployment to OSX 10.5 and later. There are ways to included this process with OSX 10.4, that starts removing the word rapid development. The OS X 10.4 -10.4.11 method will be presented in an advance tutorial at a later time. What differs between using Quartz Compositions from OSX v Tiger and Leopard. The quartz related technologies that included Quartz Compositions and system wide support began in OSX Tiger. It introduced a simple way to process stunning graphics to any mac OS X built app with out knowing advance mathematics and doing very low level coding. This method in Tiger was more geared to object-c built apps, but also included methods of using python as a bridge to 3rd party development IDE’s that had any kind of shell support. Runtime Revolution does have shell support with the syntax; (set the shellcommand to “/bin/sh/py”) and then linking a py script to the shell to run, using the shell function with in Revolution. That is how we can add quartz processing to OS X 10.4 -10.4.11 . I personally feel this method (for OSX 10.4 Tiger) is an more advance method because we have to use a second language “Python” to get this process to work. In Mac OS X 10.5 and later (Leopard) .... Continued on next page...

Quartz for Revolution What differs between using Quartz Compositions from OSX v Tiger and Leopard. Continued from last page... In Mac OS X 10.5 and later (Leopard) we do Not have to program in any other language, other than write a few shell/command line terms. These terms are simple to do and there is really no learning curve if you already have scripted with Runtime Revolution. What was added to this process in OS X Leopard gives us a great advantage. First off quartz is at the center of all things graphics in OSX 10.5, this includes how all things are drawn to screen. Apple really added some key things in Leopard that make it simple for us to use quartz processes with in our Revolution built apps. The addition of Quartz Extreme and GPU processing off screen are major keys to how we will access these processes. The protocols set up in Leopard with quartz compositions, bridge support for every framework and command line tools built native into the default system are a major help to us. These additions into the operating system with Leopard give us the key to open a door and walk threw. What you will need to add Quartz processing with in Runtime Revolution built apps. First off Quartz is a mac only process. Started in OS X 10.4 but that method is not being presented at this time. mac OS X 10.5 will be needed to develop and to deploy to. (meaning the end user needs OSX 10.5 or later) This does eliminate many developers. You may be able to developed with a windows and linux system with this process for an OSX 10.5 deployment. When I say may, what I mean is I find it very hard to test the processes script with out having access to the OS I am writing for. Many times there can be errors in a script and without access to an OS version it’s hard to deliver a error free solution to your user base. This means you can script this method with any version revolution supports to, as long as you test it on the deployed version (OS X 10.5) first. (If you plan on writing these scripts from windows and linux and have access to a mac running Leopard. The best thing to do is to run the commands from the Terminal first to make sure it works with out error. Actually It is wise to do that anyhow it will cut down time debugging any problems down the road.) Continued on next page...

Quartz for Revolution What you will need to add Quartz processing with in Runtime Revolution built apps. Continued from last page... While you do not need to use Quartz Composer to do this process, If you want to do a process not included in the built in Quartz Composition Repository you will need to have access to Quartz Composer Developer tool part of xcode. The Quartz Composer Developer tool is a very simple graphical tool that is simple to use. The xcode tools are part of the OS Install disc and can be added to your system for free. If you skip using Quartz Composer you will be limited to using only the built in Quartz Compositions that Apple built for 3rd party developers. These also limit the use of many parameters that give control of certain functionality to the end user. With Quartz Composer we can build a composition to almost anything we want. We can set up the protocols or parameters of the composition to allow control to alter the composition at run time. It is highly recommended that you use Quartz Composer to build custom compositions because it will allow your app to stand out and do something unique. Also We should note QuickTime 7.2 or later should be installed on the system. Because it is installed default in Leopard does not mean it’s user did not remove it. Setting up your revolution app to check the QuickTime version and OS version on pre open is wise. The syntax to do so is built into Revolution. (example scripts are enclosed stacks) And last you need Runtime Revolution. I have tested this method on Revolution versions 2.9 and 3.0 and they work. I have not ran them on any other version for use with 10.5. They should work on previous versions of Runtime Revolution. How Quartz compositions differs from a xcode Cocoa project to a Revolution project. You will not need to know this to use this information. This will give insight how Quartz Processes work. Understanding how a process works help’s grasp how to control that process it self. Quartz Compositions are files produced with Quartz Composer that end with the extension “.QTZ” This files is also called an application by Apple with in there own documentation. (more info on this is in all documentation related to quartz, quartz compositions and related technologies by Apple, This documentation is available to all mac developers at Apple Developer Connection and are included with xcode for further reading.) Continued on next page...

Quartz for Revolution How Quartz compositions differs from a xcode Cocoa project to a Revolution project. Continued from last page... Quartz Compositions built in Cocoa rely heavily on Open GL for rendering. Part of Cocoa’s Interface builder is pre made widgets that make viewing compositions, there parameters and picking between different compositions super easy to develop with. The use of these widget’s built by Apple software engineers allow developers not to write one line of code to get them to work. In Runtime Revolution we do not have such widgets. While we can access some of Open GL to render these Quartz Compositions. The main way we will have to do it currently is by using Open GL to process the Compositions off screen, on the GPU or Graphics and Video cards. This changes one aspect. A Cocoa built app does live processing by using openGL. In Revolution we will not get live processing, but will still get a fast process. (about 1 to 4 seconds.) Trading off live for only a few seconds is not bad at all. In Runtime Revolution we also will have to make our own composition pickers. We could do this using a list field, list magic or in 3.5 the data grid object. We could also use buttons or some custom made solution of your choosing. Last the parameter view is not automatic. Your need to build one. One solution is to build all these views and controls once and stick them on a library stack as a group. Then build them live setting that templateGroup syntax in revolution to the active stack. This way you can poll the composition parameters and it will activate the correct set of controls on screen. There is many different ways to build the UI elements that can control these processes. Last we do not have an openGL view widget to put in a window. If we look at the technologies that work and are part of quartz them self. Reviewing the header files of the frameworks gives us this information. We can use QuickTime to view both a QTZ file and a processed file. While the technologies are a little different, the end product is exactly the same. Some other interesting facts on quartz processes include. Image export from a cocoa built app is a screen capture of the composition Rect. (Well we can do that.) Another fact is quartz composition processes in cocoa use quicktime for file output.(We can do that) These two facts help us out greatly, We can use QuickTime and we can take screen captures from a rect out of the box in Runtime Revolution. With the shell process we can directly output any QuickTime supported file . To go a step further thanks to Blue Mango we also have accesses to use a QuickTime External to encode and edit these output files from the shell command we will use. Continued on next page...

Quartz for Revolution How Quartz compositions differs from a xcode Cocoa project to a Revolution project. Continued from last page... Now that we have understanding how Cocoa does this, we know how to control this process from Runtime Revolution. Based of what we just learned from learning how Apple uses quartz Compositions from a cocoa built app. And finding out about some key aspects to the quartz technologies we now have are keys to use this from Revolution. To review we will use: 1. openGL will process the Compositions off screen as a GPU process. We will do this by a simple shell script. Note we will need to garbage collect. 2. We will use QuickTime to view and output threw. The viewing process in QT is nothing new. QuickTime is the most versatile media tool on the planet. Yes we can load a .qtz composition into a player. By the shell command we can Import any file QuickTime supports. Also we can export to any file QuickTime format. The fact this is built in to quartz compositions means it is automatic. The only aspect that not automatic is full encoding of codecs. We still could use the Blue Mango awesome Enhanced QT External or the podcast producer command line tool for codec encoding and file flatten. 3. We can take a screen capture of a Rect. The sheer idea that this is how this works from cocoa is amazing to me. One would thing they are using a complete different method, but that not the case. Well we have access to screen capture with in revolution and a player has a rect. Bingo. We use the exact same process as a cocoa built app. It does not get easier than this.

Continued on next page...

Quartz for Revolution Review of using parameters. Protocols and Key Value Coding. Presented in one of the video (part_3 and part_4) are these methods. I wanted to document some of this because some parts of this (in the video) did not overview the full aspect of what this is and how this works. Any of the published inputs on a quartz composition can be a parameter. These published ports are set up by using a protocol. Some protocols are required to run these processes and some are optional. The optional one’s do not have to be Apple defined methods, the optional one’s can be set just like you could set a custom property in Runtime Revolution. The Template view in Quartz Composer uses these standard protocols and it is the simplest way to finding out details on what is standard (what must be there) If you do not have Quartz Composer installed it almost near impossible to set these and is also hard to read them. But if the file is opened in a text editor you can read the protocols. The published ports use what called KVC. (key value coding) KVC has a key: and a key value. The text inputed into the published port is the KEY. The value that past into the composition at run time is the key value. KVC uses ascii only so you can not pass Unicode. Also it needs to be noted that the published port key or protocol text can not included any white space at all or it will not run. By using the under score “_” you can break words apart as in this example:( _protocolInput_Image) That is the standard required protocol to input an image. (_protocolOutput_Image) is the standard protocol to output an image. An image is a bitmap image, a vector image or a movie file. If you pain on doing image processing and the composition you plain to use does not have these standard protocols it will not work. It will return error. In QC the template’s overview including standard protocols are:

Continued on next page...

Quartz for Revolution In QC the template’s overview including standard protocols are:

Graphic Animation A template for creating generic motion graphic compositions. These compositions have no required input or output values but are required to render to the screen. An example use for a Graphic Animation would be a background animation within Keynote. Optional Inputs: Primary Color - Color value that represents the primary color used within the composition. Secondary Color - Color value that represents a secondary color used within the composition. X - Number value representing the X position of a primary point of interest within the composition. This value is expressed within the Quartz Composer coordinate system. Y - Number value representing the Y position of a primary point of interest within the composition. This value is expressed within the Quartz Composer coordinate system. Pace - Number value representing how quickly or dramatically the composition perfoms its animation. Preview Mode - Boolean value representing if the composition is running in Preview mode.

Quartz for Revolution In QC the template’s overview including standard protocols are:

Graphic Transition A template for creating compositions that transition from a source image to a destination image. This transition occurs for time in the [0,1] range. Concretely, this means that when the Patch Time equals zero the Source Image should be shown and when the Patch Time equals one the Destination Image should be shown. Required Inputs: Source Image - The first image to be shown. Destination Image - The resulting image. Optional Inputs: Preview Mode - Boolean value that indicates if the composition is running in 'Preview' mode

Quartz for Revolution In QC the template’s overview including standard protocols are:

Image Filter A template for creating compositions that apply visual effects to images. Required Inputs: Input Image - The image to process. Optional Inputs: X - Number value representing the X position of the center point of the effect. This value is in the [0,1] range, where 0 represents the left edge of the image. Y - Number value representing the Y position of the center point of the effect. This value is in the [0,1] range, where 0 represents the bottom edge of the image. Preview Mode - Boolean value that indicates if the composition is running in Preview mode. Required Outputs: Output Image - The processed image..

Quartz for Revolution In QC the template’s overview including standard protocols are:

Music Visualizer A template for creating compositions that visualize music. Required Inputs: Audio Peak - A number representing the instantaneous audio peak. Audio Spectrum - A structure of 16 number values, each representing the signal strength of a given frequency. Optional Inputs: Track Signal - A boolean signal indicating when a track has changed by changing from false to true for a single frame and then back to false. Track Info - A structure containing the current track information such as the track artist, album name, track name, and track duration among others Track Position - A number representing the current position in the track expressed in seconds.

Quartz for Revolution In QC the template’s overview including standard protocols are:

RSS Visualizer A template for creating compositions that parse and display RSS Feeds. Required Inputs: RSSFeed URL - A string representing the RSS Feed URL. Optional Inputs: RSSArticle Duration - A number representing the duration to display each feed article.

Quartz for Revolution In QC the template’s overview including standard protocols are:

Screen Saver A template for creating compositions that can be used as Screen Savers. Optional Inputs: Screen Image - An image representing the contents of the screen at the start of the Screen Saver. If this image is used within a composition, the fade normally performed at the start of a Screen Saver does not occur. This allows seamless transitions to and from the Screen Saver composition. Preview Mode - A Boolean value that indicates if the composition is running in Preview mode. Optional Outputs: Web Page URL - A string representing the webpage URL to open in the default web browser when the Screen Saver exits.

Playing around. If you look at these protocols and match them up with other processes you can get neat processes. We can take from the screensaver a method of building a video screen capture composition. If we take a protocol from RSS we can do some neat networking compositions. The key thing is to play around and test things out. We can do just about anything with Quartz Composer to make powerful quartz compositions. Giving Revolution processes not found into the built in functionality.

Quartz for Revolution Whats Next.... Will be presented as downloaded stacks. With working examples of a number of different examples. The video’s and stacks will be available soon for download and on Rev Mentor. This will included detailed script notes. This document was made for the fill in the blanks, from the quartz with revolution videos part 1 threw 4. This whole process was filled on 15 notebooks (paper) and while Jerry Daniels ask the question “can you show us how this works, Obleo” about a month ago. It has taken a lot of time to present this info from my notebooks (paper) to digital form, so other revolution developers may use it this quartz method. I been trying to present this in the least confusing way. I hope someone finds this helpful and appreciates the massive amount of work this takes to present to all of you.

I thank you for your patients as a viewer as I make a new screen cast each weak. The Scripted stacks are taken a bit longer as I want to make sure the notes in the scripts are well documented. (so you can follow along how this is working.) I can see this getting some what confusing if you did not spend the past 4 years working with quartz compositions and these type of processes. They are coming soon. Thank you, Obleo Beck The terms Quartz, Quartz Compositions, Core Image, Core Video, Core Audio, Core Animation, QuickTime, Quartz Composer, OS X Tiger, OS X Leopard, Xcode, ADC, Cocoa, Podcast and Object-C are trademarks of Apple, Inc. Runtime Revolution is a Trademark of Runtime Revolution, LTD. Enhanced QT is a Blue Mango Learning Product.

Thanks to Jerry Daniels and Mary Jane Mara for presenting Quartz for Revolution on Rev Mentor. my name Obleo is pronounced: oh-blee-leo Any questions fell free to send them my way.