Plugin plans in Kestrel

In my last post I talked about how the EV: Override revival project is now a thing and how the engine is being called Kestrel. In that post I also mentioned about the plans for plugin development and how maintaining backwards compatibility is an important factor. At the same time it will be important to expand the capabilities and features afforded to plugin developers.

This post will explain how this will be done and my current plans on accomplishing it.


I have talked in the past about ResourceForks and how they are a central aspect of how the old Escape Velocity games worked. Everything was stored in ResourceForks, and that allowed the engine to quickly located the required resources, and for plugin developers to quickly identify, modify and replace existing resources in the game.

This functionality will be included into kestrel and will allow it to load all of the old legacy data files and plugins. Plugins using this format will be constrained to the same limitations as they were back on the original engines, as some of the constraints actually came from the format itself.

Newer content should be able to make use of a custom extension on ResourceForks.

Extended ResourceForks

So this is a purely custom thing, based upon the ResourceFork format. The format will allow for larger offset numbers and Resource IDs, thus removing limitations (for all practical purposes) of the format. These values will likely become 64-bit values, rather than 16-bit values.

This will be the extent of the change to the ResourceFork format.

Distinguishing between the two formats

So how will the engine distinguish between the formats, and what will it mean for plugin developers?

Well it won’t really mean anything for plugin developers, or even much for the engine itself. Most of the heavy work will be handled by libResourceFork which is part of the Diamond Project. That will handle loading from both formats and then providing back a common set of data structures and interfaces.

Internally though there will be a small addition to the header of the ResourceFork to denote that it is in the extended format.

Let’s take a look at the header (preamble) of the original ResourceFork.

Standard ResourceFork Preamble

This provides information about the layout of the ResourceFork. We can visualise this slightly with the following diagram.

The basic layout / structure of a ResourceFork

The two offset values represent the position within the file for that particular structure. The “data_offset” being where the actual data for all of the resources is located, and the “map_offset” being where all of the resource meta data is located (including types, ids, names, flags, etc). The corresponding sizes simply state how much data of each there is.

Note: Technically the resource map could be located before the resource data, but everything I’ve seen has this ordering.

So let’s take a look at the proposed preamble for the extended format.

Extended ResourceFork Preamble

There are two big changes. Firstly is an extra field at the start of the structure, encoding information about the format.

Due to the extremely unlikely scenario of both the “data_offset” and “map_offset” having a combined value of 1, this is the value we expect “format_version” to return. Anything else causes the file to be treated as a standard ResourceFork.

The second change is the use of 64-bit values, effectively making the capacity of the ResourceFork infinite (unless you have access to 18.5EB of storage and can fill it.)

So why/how does this “format_version” field trick work?

Let’s take a look at the initial 20 bytes of a standard ResourceFork.

A standard ResourceFork preamble

Each grouping of 8 digits represents a single 32-bit value (each pair of digits is a single byte). Using the above structure for the standard preamble we can see that the “data_offset” field will have a value of “00000100”, which translates to 256. We can also see that the “map_offset” field will have a value of “00007520” which translates to 29,984.

Now if we apply the extended preamble format we need to take 16 digits in order to have a 64-bit value. This will give our “format_version” field a value of “000001000007520” which translates to 1,099,511,657,760, which is quite a bit bigger than 1!

Because the offsets are from the beginning of the file, we can be certain that neither offset is going to be 0, and any value greater than 0 in “data_offset” will cause “format_version” to have a value in excess of 4,294,967,296.

So now that we have a method of distinguishing between standard and extended ResourceForks, let’s take a look at the resource map, and limitations that it imposes.

A diagram illustrating the layout of the resource map

This whole structure represents all of the meta data in the ResourceFork. Resource types, resource IDs, sizes, flags, resource names, etc. It is this structure that causes us most of our headaches and limitations.

The data structure representing the “resource map layout”

This data structure is only using 16-bit fields! Given that both offsets included are from the beginning of the resource map, this means we have a limitation on the number of resources that can be included in the ResourceFork. Obviously we can split the resources into multiple files to get around this issue but that is not always a nice or elegant solution (plus I’m not totally certain on what the internal limitations of the Resource Manager were, so there may be further limitations there!)

So let’s work out, best case scenario, how many resources we could house in a single ResourceFork. We’ll assume that we have just a single resource type for this. Let’s take a look at the structure of the type list.

The resource type list contains a count of how types are included in the file, followed by a list of type definitions, which in turn point to a list of resource definitions.

This structure may seem complex, but in reality it is not that bad. The type count field at the start is a 16 bit value, which means a total of 65,536 types can be included in a single ResourceFork. For all practical purposes this would never be reached.

Each type entry is 8 bytes long. 4 bytes denoting a type code such as ‘PICT’, 2 bytes for the offset to the first resource of that type and then a further 2 bytes representing how resources of that type are included.

The resource entry is 12 bytes long. 2 bytes for the resource ID, 2 bytes for the offset to the resource name, 1 byte for the resource flags, 3 bytes for the offset to the resource data and then 4 ignored bytes.

Note: These bytes are not actually ignored and used by the system for an unknown purpose, likely storing a reference value.

So we started with a total of 65,536 bytes of space. Immediately we can take away 6 bytes for the resource map structure, followed by a further 2 bytes for the resource type count, and then finally another 8 bytes for our type definition.

This gives us a total 65,520 bytes of space to keep track of our resources, which when divided between resource definitions allows us to store a maximum of 5,460 resources in a single ResourceFork! This means that a game such as EV Nova requires its data to be split across multiple files. Assuming those resources do not need more than 16MB of data (due to the 3-byte data offsets the format uses.)

In reality memory constraints on earlier systems will have been a much more limiting factor.

How the Extended ResourceFork will change things.

I already mentioned the changes to the preamble structure at the start of the ResourceFork. This is purely to help differentiate the two formats. The next change will be to make all of the 16-bit (2 byte) values in to 64-bit values.

In addition to that all resource names will be UTF-8 encoded rather than Mac Roman Encoded, allowing for modern representation of text in the resource names.

However this new format will need a proper specific defining at some point, but keen plugin developers will have noticed an issue with this proposal. It will break existing plugin editors!

The Kestrel Plugin Development Kit (KPDK)

This is a tangental project to Kestrel and will be extremely barebones, likely requiring the community to make it nice (kind of like how Nova Tools and Mission Computer made EV Nova plugin development nice).

So what is the plan with this? What is it?

Many people have expressed concern over the use of resource forks, and rightly so. They are an old format, and if backwards compatibility was not a concern they would not be getting used. But this does not mean plugin developers should be subjected to them. For this reason the KPDK will be compiler of sorts that compiles simple text based definitions into plugins compatible with Kestrel or EV Nova.

Here is a simple draft definition for a plugin.

; Define the planet Earth
StellarObject {
    New(id = #128, name = "Earth") {
        PlanetType = 0
        Government = #128
        ; Other fields for the stellar object.

; Define a planet description for Earth
Description {
    New(id = StellarObject("Earth"), name = "Earth Landing Description") {
        Body = "Welcome to Earth. The home of Humanity."
        LandingPicture = PNG(/path/to/image/file.png)

The compiler would know how to parse this and how to build the appropriate resources. It would be able synthesise the appropriate resources and reference resources with in definitions, allowing a plugin developer to not worry about remembering specific id ranges or how to link things.

There will be more information on this KPDK later on once it has been fleshed out more, and once Kestrel is actually loading some data files.

But I do plan to have compiler be able to target both Kestrel and EV Nova, as well as provide warnings on issues in the definitions being compiled and to be able to decompile a Kestrel plugin, or EV Nova plugin in to this definition script.

It should also mean that the tooling and features to plugin developers can evolve and expand over time without being tightly bound and dependant on the engine.

Wrap up

There was a lot of information in this post. I hope you enjoyed reading it, and it gives some insight into the direction I hope to go with regards to plugins.


Kestrel – EV: Override Remaster

In case you are not a frequent visitor of the r/evnova subreddit (or just missed it), Peter Cartwright recently announced a project to revive and remaster Escape Velocity Override. So what does this mean for OpenNova? Well OpenNova will becoming the foundation for the new version of EV: Override. Additionally the engine itself will be adopting the name Kestrel.

Open Source

Good news to everyone who has been following OpenNova for a while. Kestrel will be open source. Whilst the exact license has not yet been decided, it will be a generally permissive license as to allow people to use the main engine as a basis for their own games.

The biggest question in deciding a license is going to be around forks pushing changes back to the main Kestrel project, commercial limitations and attribution requirements. For the time being these are still being considered and weighed up.

Kestrel will not be source available during the initial run of development. This is mainly because I may decide to make sweeping changes to the foundation of the game and I don’t want to worry about managing the project and merge conflicts whilst I’m in this initial phase.

What will become of this blog?

This blog will become the primary development blog for Kestrel. I’ll discuss design decisions, structure and implementation details of Kestrel. Some of the posts will be more technical than others, but I’ll generally try and keep them understandable.

I’m also going to try and aim for weekly/bi-weekly updates. In the past OpenNova has been an infrequent side project for me and as a result has not allowed for many posts to be made. Kestrel on the other hand will be much more active, with me dedicating a couple of hours a day (where possible) to it.

The structure and format of this blog may change over time and evolve as the project develops and the community grows.

Some information about Kestrel

Disclaimer: The project is still in its very early stages, so do not take all of this information as being set in stone. Things may change and/or evolve.

Kestrel is being developed from the ground up in an engine dedicated to accurately representing the Escape Velocity experience. Further to this it will maintain backwards compatibility with all of the old plugins and data files, whilst bringing in some new features. There will be a follow up post on the future of plugin compatibility in Kestrel in the coming days.

As it stands at the moment, I am developing Kestrel in C. This may evolve in to C++ in the future, but for the moment I’m wanting to keep it to C. This is mostly a personal preference and that C can compile to anything.

The result of this is that there will be several components of the project.

  1. The core library of Kestrel
    This will include all of the graphics layer abstraction, basic engine functionality etc. This library should generally be platform agnostic.
  2. The main Kestrel game library
    This library is the main functionality of Kestrel. The actual game itself. This library should be completely platform agnostic.
  3. The platform specific binaries
    These are the foundational code bases that are used to make an actual executable or application bundle. The core library and main game will be linked into these to produce an actual functional game.

    For example; one of the macOS versions of the game will use Metal to render any graphics. The platform specific game will setup the environment and Metal for use, and provide any hooks that the core library will use to render sprites to the screen, and receive events.

There will be some overlap between these components, but in general it should be minimal. Once again there will be a blog post about the architecture and structure of the engine in the future.

The functionality for old Macintosh features such as ResourceForks, QuickDraw, etc will becoming from the Diamond Project and not directly replicated into Kestrel.

Current State

So what is the current state of Kestrel?

Well I’m currently getting the Metal rendering pipeline and functions setup for macOS, and dispatching rendering commands from the main Kestrel game library.

Humble beginnings for Kestrel – A test of the render pipeline using Apple’s Metal Graphics API.

This may not look like much, and to an extent it is not, but this actually has much of the graphical requirements fulfilled for Kestrel. The ability to render quads, textures (sorry that the texture is just a yellow square), move objects around, etc. The only missing aspect is the rendering of text, key presses and mouse interaction.

Once these aspects are implemented, it will be on to developing the actual game.