Author: Tom Hancocks

An update on the QuickDraw and Pictures

It turns out PICT is one of those formats that is just crap.  It’s needlessly complex and for every way of encoding something, there is potentially 2 or 3 others ways of doing so as well. On top of that, some areas of the spec seem to be vague, repetitive or contradictory and just there to make life hell.

I’m not a fan of this format at all.

Why the rant?

So it turns out EV Nova’s data files use the PICT format for many things. UI graphics, targeting graphics, ship icons, landing screens, etc. Lots of them. However there currently appear to be 3 types of PICTs used. I’ve accounted for two of them, but the final one is infuriating, as finding documentation or reasons is impossible. I doubt this is the fault of the original designers. It was the software they used that did the encoding.

Single pixel width images are the problem. These are used by the middle graphics of buttons in the UI. These fail to parse properly.

On the bright side…

So as I’ve been searching for more information about this format, I have been able to find something interesting thankfully.

QuickDraw.png
QuickDraw

This is buried in the depths of the system, nested inside other frameworks. It’s the QuickDraw framework. In which all the functionality for QuickDraw and it’s format exists. This framework is deprecated and use of it is not really recommended these days. It will limit the app to 32-bit only and will not handle retina displays correctly.

But it does mean I have a reference point. How?

QD-Hopper-Retina.png
QuickDraw – _DrawPicture in Hopper Disassembler

Hopper Disassembler! A great tool for peeking inside and inspecting compiled binaries. That QuickDraw framework includes a binary that I am able to pull apart and study. Hopefully some of the secrets of the Picture resource will begin to reveal themselves.

 

The ResourceFork – A file of files.

The ResourceFork. A relic from a bygone age of the Mac. It was an elegant solution to the problem of software resources. All of the assets and resources for a file or program would be contained within the ResourceFork. To the user this would then appear as a single file on the file system.

However it had issues. If you tried to move a file that had a resource fork to a machine that used a different file system, or just plain didn’t understand them then you would run into trouble. The ResourceFork of the file would be truncated and you were left with an empty file. Oops. This was particularly common when sending files over the internet or transferring them on USB Thumb Drives to a Windows machine.

Apple has since supplanted the ResourceFork with the concept of the “Bundle”, a folder that can appear to the user as a single file. Mac OS X Applications are bundles, and contain a number of files inside related to the Application.

In fact, Apple supplanted the ResourceFork so long ago, that it has never been a first class citizen on Mac OS X. That’s nearly 17 years since it was effectively “deprecated”. Given that it was a predominantly a feature from the classic era of Mac OS, the API’s have lived on in the Carbon framework (which is on the way out currently). The actual functionality of the ResourceFork is now just a consequence of HFS+, which is itself being replaced by the newer file system, AFS.

Luckily EV Nova’s resource forks were flattened into data forks and live on as ndat files or rez files. However huge scores of plugins out there are probably still original ResourceForks.

But the ResourceFork situation is the case with everything in EV Nova. The ResourceFork’s house a number of formats which are all also deprecated and becoming relics of a bygone era. Functionality is fast disappearing. At some point of Apple will release the ultimate death blow to EV Nova, and old classic era software. They will remove Carbon and HIToolbox from the system. When that day arrives EV Nova will lose the ability to read anything from its data files.

That makes this project more than just a quest to recreate the engine for modern machines. It makes it an endeavour to preserve these old formats from being lost to the sands of time.

QuickDraw and the Picture Resource

Anyone that has worked with me will know that I love low level and/or old technology. So when you mix the two together, I am in my element! QuickDraw is an old, low level library for handling media. It first appeared on the earliest versions of the Macintosh System Software. It should have been right up my street.

Archaic Formats

It turns out old technology is not always documented well. In some cases it’s not really documented at all. Once upon time QuickDraw was very well documented. Well it was well documented from an API standpoint. The internals were… well there was some information about how it worked, but precious little. A document does still exist online for it. It’s in the archived section of Apple’s old websites. It’s basically a PDF of a scanned document. Not fantastic, but better than nothing.

QuickDraw it seems went through several iterations. Thankfully, by the time the Escape Velocity series came on to the scene. Version 2 of QuickDraw the default, and thus all picture resources appear to be encoded as such. This is both a blessing and curse.

Version 2 is the only one that I’ve had any luck in finding documents for. Even then there isn’t much. That document mentions a tiny amount about Version 1. The only thing I could learn was, that Version 1 was basically easier to parse. Damn. That means more work in trying to get the picture decoder up and running.

The Challenges

There are 2 major difficulties with the Picture Resources in QuickDraw. These are of course only the challenges that I’ve encountered. It turns out that EV Nova data files only seem to use a thin slice of the Picture Resource functionality.

  1. Op-codes
    Yes. The Picture Resource uses a series of op-codes to encode data. This is done primarily so that pictures can be encoded as a type of vector graphic. This is pretty cool considering when the format was designed. However there is only really one op-code that is needed for EV Nova. That is the Draw Direct Packed Bits operation. This opcode basically reads a massive chunk of data from the resource. This data is an encoded bitmap.
  2. PackBits
    This is an algorithm that is used to compress/pack bitmap information into a smaller space. The exact algorithm is notoriously difficult to find online and there appears to be multiple variants of the algorithm.

On top of those issues, the colour space for the original pictures was RGB 555, which is no longer a ready defined colour space on the Mac. So the bitmap information once unpacked needs to be translated into RGBA 8888.

When it goes wrong…

CorruptPicture.png
When decoding a QuickDraw picture goes wrong.

There is so much minutia in the format that it is extremely common to end up with results that appear something like you can see above, or worse. A lot of images will still fail to parse correctly due to some unexpected opcodes that I’ve yet to handle or decipher.

And this is just the picture resource. There is still the cicn, rlë, ppat, and other QuickDraw formats to handle.

OpenNova. Rebuilding a classic Mac OS game.

A few months ago I decided to undertake a slightly ludicrous side project. To re-implement a modern version of the classic Mac OS game, Escape Velocity Nova. The game is at this point essentially abandoned. Ambrosia Software is but a ghost of shell of their former selves and the game receives very little in the way of support. Furthermore it also relies on technologies from the Classic Macintosh days such as ResourceForks, QuickDraw and various others. Apple are advancing macOS each year and its becoming apparent that a lot of the old classic games are at the end of their lives.

EV Nova already requires an unofficial patch (or tweaking of the application binary) to run on systems later than macOS 10.11. Sooner or later, Apple is going to deliver the final nail to the coffin.

This is where my idea was spawned. I had once been a member of the community for EV Nova (particularly on ev-nova.net). I loved the game and would play it constantly. I was much younger at the time (true for everyone, unless you’re Benjamin Button) and had just started to really learn how to develop software. I decided to try my hand at creating a plugin development tool for EV Nova. The tool I made was crap, but it gave me insight into how the game worked. So, drawing upon nostalgia and old knowledge of the game I decided that rebuilding the game engine, with modern technologies, was the only solution.

This presented a slight problem…

ResourceForks & HFS+

With the release of Mac OS X almost 17 years ago, Apple declared that the ResourceFork was “legacy” technology and should no longer be used for new projects. However the API’s were still present for old programs. Great!

With the migration to 64-bit, Apple declared that the Carbon framework was now deprecated and end-of-life. The Carbon framework is where the ResourceFork functionality lives/lived. Uh-oh.

As Apple continued marching onwards the ResourceFork became less and less reliable. EV Nova received an update to deal with this. They migrated all the data files over to the ndat format. ndat is nothing special. It’s simply the ResourceFork data in a simple DataFork file. However the contents are a proprietary Apple format. No problem, the old Carbon functions could parse it. They maybe deprecated but they weren’t going anywhere anytime soon!

But it poses a problem. The old Carbon frameworks will be thrown out at some point. And when they are EV Nova will stop working. Those frameworks are the only part of Apple’s systems that can read the old ResourceFork format.

Now it’s true that the windows version doesn’t use the ndat format. It uses a custom rez format. So in theory that functionality could be moved over to the Mac version. But that would be a huge undertaking, and given the state of Ambrosia Software these days, probably not very likely.

So there is only one future proof solution… make a custom implementation of the parser for the ndat format (and one for rez at the same time).

QuickDraw

Awesome! We’re going to be up and running in no time! We’ll have all the old resources extracted out and we’ll be able to start building a game engine.

Uh-oh! Turns out resources such as PICT, cicn, RLË, ppät, etc, were all encoded/decoded via QuickDraw functionality, and that has long since gone the way of the dodo on macOS. Though curiously Preview can still render a PICT? Either way there is no publicly available way to handle those resources.

So that means a need to reverse engineer those as well… this may take a while.

A slight hint of a silver lining

There is some good that will come out of this project.

  1. It will hopefully result in a modern version of the game engine that will continue to work for years to come.
  2. Old total conversions such as Polycon EV, and even the Escape Velocity, and Escape Velocity Override conversions will work in this new engine with modification to the files.
  3. It’s modern and will have custom parsers included so it wouldn’t be too much a stretch to get it running on iOS (though controls maybe an issue.)

I plan on using the blog to keep people who are interested up to date with the progress of work and to document some of the technical aspects as well.

Hopefully this will be of interest to some people…