Plugin plans in Kestrel

In my last post I talked about how the EV: Override revival project is now a thing and how the engine is being called Kestrel. In that post I also mentioned about the plans for plugin development and how maintaining backwards compatibility is an important factor. At the same time it will be important to expand the capabilities and features afforded to plugin developers.

This post will explain how this will be done and my current plans on accomplishing it.


I have talked in the past about ResourceForks and how they are a central aspect of how the old Escape Velocity games worked. Everything was stored in ResourceForks, and that allowed the engine to quickly located the required resources, and for plugin developers to quickly identify, modify and replace existing resources in the game.

This functionality will be included into kestrel and will allow it to load all of the old legacy data files and plugins. Plugins using this format will be constrained to the same limitations as they were back on the original engines, as some of the constraints actually came from the format itself.

Newer content should be able to make use of a custom extension on ResourceForks.

Extended ResourceForks

So this is a purely custom thing, based upon the ResourceFork format. The format will allow for larger offset numbers and Resource IDs, thus removing limitations (for all practical purposes) of the format. These values will likely become 64-bit values, rather than 16-bit values.

This will be the extent of the change to the ResourceFork format.

Distinguishing between the two formats

So how will the engine distinguish between the formats, and what will it mean for plugin developers?

Well it won’t really mean anything for plugin developers, or even much for the engine itself. Most of the heavy work will be handled by libResourceFork which is part of the Diamond Project. That will handle loading from both formats and then providing back a common set of data structures and interfaces.

Internally though there will be a small addition to the header of the ResourceFork to denote that it is in the extended format.

Let’s take a look at the header (preamble) of the original ResourceFork.

Standard ResourceFork Preamble

This provides information about the layout of the ResourceFork. We can visualise this slightly with the following diagram.

The basic layout / structure of a ResourceFork

The two offset values represent the position within the file for that particular structure. The “data_offset” being where the actual data for all of the resources is located, and the “map_offset” being where all of the resource meta data is located (including types, ids, names, flags, etc). The corresponding sizes simply state how much data of each there is.

Note: Technically the resource map could be located before the resource data, but everything I’ve seen has this ordering.

So let’s take a look at the proposed preamble for the extended format.

Extended ResourceFork Preamble

There are two big changes. Firstly is an extra field at the start of the structure, encoding information about the format.

Due to the extremely unlikely scenario of both the “data_offset” and “map_offset” having a combined value of 1, this is the value we expect “format_version” to return. Anything else causes the file to be treated as a standard ResourceFork.

The second change is the use of 64-bit values, effectively making the capacity of the ResourceFork infinite (unless you have access to 18.5EB of storage and can fill it.)

So why/how does this “format_version” field trick work?

Let’s take a look at the initial 20 bytes of a standard ResourceFork.

A standard ResourceFork preamble

Each grouping of 8 digits represents a single 32-bit value (each pair of digits is a single byte). Using the above structure for the standard preamble we can see that the “data_offset” field will have a value of “00000100”, which translates to 256. We can also see that the “map_offset” field will have a value of “00007520” which translates to 29,984.

Now if we apply the extended preamble format we need to take 16 digits in order to have a 64-bit value. This will give our “format_version” field a value of “000001000007520” which translates to 1,099,511,657,760, which is quite a bit bigger than 1!

Because the offsets are from the beginning of the file, we can be certain that neither offset is going to be 0, and any value greater than 0 in “data_offset” will cause “format_version” to have a value in excess of 4,294,967,296.

So now that we have a method of distinguishing between standard and extended ResourceForks, let’s take a look at the resource map, and limitations that it imposes.

A diagram illustrating the layout of the resource map

This whole structure represents all of the meta data in the ResourceFork. Resource types, resource IDs, sizes, flags, resource names, etc. It is this structure that causes us most of our headaches and limitations.

The data structure representing the “resource map layout”

This data structure is only using 16-bit fields! Given that both offsets included are from the beginning of the resource map, this means we have a limitation on the number of resources that can be included in the ResourceFork. Obviously we can split the resources into multiple files to get around this issue but that is not always a nice or elegant solution (plus I’m not totally certain on what the internal limitations of the Resource Manager were, so there may be further limitations there!)

So let’s work out, best case scenario, how many resources we could house in a single ResourceFork. We’ll assume that we have just a single resource type for this. Let’s take a look at the structure of the type list.

The resource type list contains a count of how types are included in the file, followed by a list of type definitions, which in turn point to a list of resource definitions.

This structure may seem complex, but in reality it is not that bad. The type count field at the start is a 16 bit value, which means a total of 65,536 types can be included in a single ResourceFork. For all practical purposes this would never be reached.

Each type entry is 8 bytes long. 4 bytes denoting a type code such as ‘PICT’, 2 bytes for the offset to the first resource of that type and then a further 2 bytes representing how resources of that type are included.

The resource entry is 12 bytes long. 2 bytes for the resource ID, 2 bytes for the offset to the resource name, 1 byte for the resource flags, 3 bytes for the offset to the resource data and then 4 ignored bytes.

Note: These bytes are not actually ignored and used by the system for an unknown purpose, likely storing a reference value.

So we started with a total of 65,536 bytes of space. Immediately we can take away 6 bytes for the resource map structure, followed by a further 2 bytes for the resource type count, and then finally another 8 bytes for our type definition.

This gives us a total 65,520 bytes of space to keep track of our resources, which when divided between resource definitions allows us to store a maximum of 5,460 resources in a single ResourceFork! This means that a game such as EV Nova requires its data to be split across multiple files. Assuming those resources do not need more than 16MB of data (due to the 3-byte data offsets the format uses.)

In reality memory constraints on earlier systems will have been a much more limiting factor.

How the Extended ResourceFork will change things.

I already mentioned the changes to the preamble structure at the start of the ResourceFork. This is purely to help differentiate the two formats. The next change will be to make all of the 16-bit (2 byte) values in to 64-bit values.

In addition to that all resource names will be UTF-8 encoded rather than Mac Roman Encoded, allowing for modern representation of text in the resource names.

However this new format will need a proper specific defining at some point, but keen plugin developers will have noticed an issue with this proposal. It will break existing plugin editors!

The Kestrel Plugin Development Kit (KPDK)

This is a tangental project to Kestrel and will be extremely barebones, likely requiring the community to make it nice (kind of like how Nova Tools and Mission Computer made EV Nova plugin development nice).

So what is the plan with this? What is it?

Many people have expressed concern over the use of resource forks, and rightly so. They are an old format, and if backwards compatibility was not a concern they would not be getting used. But this does not mean plugin developers should be subjected to them. For this reason the KPDK will be compiler of sorts that compiles simple text based definitions into plugins compatible with Kestrel or EV Nova.

Here is a simple draft definition for a plugin.

; Define the planet Earth
StellarObject {
    New(id = #128, name = "Earth") {
        PlanetType = 0
        Government = #128
        ; Other fields for the stellar object.

; Define a planet description for Earth
Description {
    New(id = StellarObject("Earth"), name = "Earth Landing Description") {
        Body = "Welcome to Earth. The home of Humanity."
        LandingPicture = PNG(/path/to/image/file.png)

The compiler would know how to parse this and how to build the appropriate resources. It would be able synthesise the appropriate resources and reference resources with in definitions, allowing a plugin developer to not worry about remembering specific id ranges or how to link things.

There will be more information on this KPDK later on once it has been fleshed out more, and once Kestrel is actually loading some data files.

But I do plan to have compiler be able to target both Kestrel and EV Nova, as well as provide warnings on issues in the definitions being compiled and to be able to decompile a Kestrel plugin, or EV Nova plugin in to this definition script.

It should also mean that the tooling and features to plugin developers can evolve and expand over time without being tightly bound and dependant on the engine.

Wrap up

There was a lot of information in this post. I hope you enjoyed reading it, and it gives some insight into the direction I hope to go with regards to plugins.


Here we go… time to start properly building this thing, and the first step of the journey is going to be Graphite.

What is Graphite?

Graphite is going to be a Swift package containing all the modern implementations of old legacy code found on the Macintosh platform in the Carbon era. It is fully written in Swift, and the name is a play on the name Carbon, of which Graphite is a type.

The early versions of Graphite will basically provide enough functionality for me to get my EV Nova clone up and running. Read resource files, handle some QuickDraw media and play some sound clips. Certainly not enough to provide a fully compliant Carbon environment. Maybe it’ll get to that point one day, but I certainly don’t see a need for that.

As a result this means that it is a combination of ResourceKit and ClassicKit, two earlier frameworks that I developed for accomplishing these goals. However they are written in Objective-C and strongly coupled to Apple’s Cocoa environment.

Graphite, on the other hand is written in Swift and, in theory, is not coupled to Apple’s ecosystem.

I’m not quite ready to setup the Open Source project page for Graphite just yet, as I want to get some tests in place and ensure the code and project are nicely documented. However I hope to get this up soon.

Run Length Encoding – RLË Resources

EV Nova encodes all of its sprites in the RLË format. Specifically split between rlë8 and rlëD in the Nova engine. The 8 and D correspond to the colour/bit depth of the sprite images. For modern computers we’re only ever going to be concerned about rlëD.

RLË as it turns out is very similar, but not as complex, as the PICT​ format. It uses a series of opcodes to instruct the decoder on how to product an image. Luckily, it turns out I made a crude prototype of one quite a while ago. All I had to do was update the code and it worked.

The Starbridge. The icon of EV Nova, literally.

Well as it turns out, we’re now loading in all graphical assets except for the CICN and PPAT assets. Everything is just game data.

The ResourceFork – A file of files.

The ResourceFork. A relic from a bygone age of the Mac. It was an elegant solution to the problem of software resources. All of the assets and resources for a file or program would be contained within the ResourceFork. To the user this would then appear as a single file on the file system.

However it had issues. If you tried to move a file that had a resource fork to a machine that used a different file system, or just plain didn’t understand them then you would run into trouble. The ResourceFork of the file would be truncated and you were left with an empty file. Oops. This was particularly common when sending files over the internet or transferring them on USB Thumb Drives to a Windows machine.

Apple has since supplanted the ResourceFork with the concept of the “Bundle”, a folder that can appear to the user as a single file. Mac OS X Applications are bundles, and contain a number of files inside related to the Application.

In fact, Apple supplanted the ResourceFork so long ago, that it has never been a first class citizen on Mac OS X. That’s nearly 17 years since it was effectively “deprecated”. Given that it was a predominantly a feature from the classic era of Mac OS, the API’s have lived on in the Carbon framework (which is on the way out currently). The actual functionality of the ResourceFork is now just a consequence of HFS+, which is itself being replaced by the newer file system, AFS.

Luckily EV Nova’s resource forks were flattened into data forks and live on as ndat files or rez files. However huge scores of plugins out there are probably still original ResourceForks.

But the ResourceFork situation is the case with everything in EV Nova. The ResourceFork’s house a number of formats which are all also deprecated and becoming relics of a bygone era. Functionality is fast disappearing. At some point of Apple will release the ultimate death blow to EV Nova, and old classic era software. They will remove Carbon and HIToolbox from the system. When that day arrives EV Nova will lose the ability to read anything from its data files.

That makes this project more than just a quest to recreate the engine for modern machines. It makes it an endeavour to preserve these old formats from being lost to the sands of time.

OpenNova. Rebuilding a classic Mac OS game.

A few months ago I decided to undertake a slightly ludicrous side project. To re-implement a modern version of the classic Mac OS game, Escape Velocity Nova. The game is at this point essentially abandoned. Ambrosia Software is but a ghost of shell of their former selves and the game receives very little in the way of support. Furthermore it also relies on technologies from the Classic Macintosh days such as ResourceForks, QuickDraw and various others. Apple are advancing macOS each year and its becoming apparent that a lot of the old classic games are at the end of their lives.

EV Nova already requires an unofficial patch (or tweaking of the application binary) to run on systems later than macOS 10.11. Sooner or later, Apple is going to deliver the final nail to the coffin.

This is where my idea was spawned. I had once been a member of the community for EV Nova (particularly on I loved the game and would play it constantly. I was much younger at the time (true for everyone, unless you’re Benjamin Button) and had just started to really learn how to develop software. I decided to try my hand at creating a plugin development tool for EV Nova. The tool I made was crap, but it gave me insight into how the game worked. So, drawing upon nostalgia and old knowledge of the game I decided that rebuilding the game engine, with modern technologies, was the only solution.

This presented a slight problem…

ResourceForks & HFS+

With the release of Mac OS X almost 17 years ago, Apple declared that the ResourceFork was “legacy” technology and should no longer be used for new projects. However the API’s were still present for old programs. Great!

With the migration to 64-bit, Apple declared that the Carbon framework was now deprecated and end-of-life. The Carbon framework is where the ResourceFork functionality lives/lived. Uh-oh.

As Apple continued marching onwards the ResourceFork became less and less reliable. EV Nova received an update to deal with this. They migrated all the data files over to the ndat format. ndat is nothing special. It’s simply the ResourceFork data in a simple DataFork file. However the contents are a proprietary Apple format. No problem, the old Carbon functions could parse it. They maybe deprecated but they weren’t going anywhere anytime soon!

But it poses a problem. The old Carbon frameworks will be thrown out at some point. And when they are EV Nova will stop working. Those frameworks are the only part of Apple’s systems that can read the old ResourceFork format.

Now it’s true that the windows version doesn’t use the ndat format. It uses a custom rez format. So in theory that functionality could be moved over to the Mac version. But that would be a huge undertaking, and given the state of Ambrosia Software these days, probably not very likely.

So there is only one future proof solution… make a custom implementation of the parser for the ndat format (and one for rez at the same time).


Awesome! We’re going to be up and running in no time! We’ll have all the old resources extracted out and we’ll be able to start building a game engine.

Uh-oh! Turns out resources such as PICT, cicn, RLË, ppät, etc, were all encoded/decoded via QuickDraw functionality, and that has long since gone the way of the dodo on macOS. Though curiously Preview can still render a PICT? Either way there is no publicly available way to handle those resources.

So that means a need to reverse engineer those as well… this may take a while.

A slight hint of a silver lining

There is some good that will come out of this project.

  1. It will hopefully result in a modern version of the game engine that will continue to work for years to come.
  2. Old total conversions such as Polycon EV, and even the Escape Velocity, and Escape Velocity Override conversions will work in this new engine with modification to the files.
  3. It’s modern and will have custom parsers included so it wouldn’t be too much a stretch to get it running on iOS (though controls maybe an issue.)

I plan on using the blog to keep people who are interested up to date with the progress of work and to document some of the technical aspects as well.

Hopefully this will be of interest to some people…