Took me a while

In order to keep up the good tradition of this blog, famous for its quality content, I had to meditate for months.

Work, as usual, has not been forgiving with my spare time. Many things also happen outside the event horizon of a computer screen, and it eventually becomes increasingly hard to clear up thoughts and write down stuff.

Blabbering aside, I've managed to finally write a working prototype of the famous Entity System. Not only that, though. The Rainweaver Framework (guess I'll stick to this name for the time being) has been given lots of love as well, and despite my constant refactoring, I can say I'm 80% satisfied with the result. I made a post at Gamedev.net but stirred no interest. Ah well. For fun and not for fame, right?

This Framework of mine is a hobby project. I'm giving myself the chance to learn new technologies such as WPF as well. It has a steep learning curve. It's true. If you want to learn WPF, you gotta have the time to. Time to sit down, open two Visual Studio instances, download Family.Show and get dissecting. Time to wade through thousands of tutorials looking for best practices with a dim light.

With this spirit, I decided to take up a WPF project related to the Entity System. It's called Prototyper. You can still see in the changelogs that the project was named ArchProto, then ProtoArch, it turned WPF from WinForms (just like growing adult), and got its final name.

Project conception aside, I'll explain briefly what it's supposed to do.

Every Entity System has a Schema and a Runtime. The first says how things work, the second makes them happen. Between them, there are Runtime Globals and Scripts. Runtime Globals are implementation-specific methods or properties exposed by the runtime to the schema. Scripts can be modified at runtime (atomically, i.e. no undefined state across modifications) and are generally kept in some kind of storage (files, db).

Prototyper is being written to allow designers to define components, messages and prototypes in a visual fashion; associate a Runtime Globals type for scripts; associate scripts from a typed script storage object to message handlers; compile a schema to CLR Objects.

The Entity System was a challenge and still is, as it's not 100% done. But a bigger challenge lies ahead with this Prototyper thing.

If you feel brave, head over at http://rainweaver.codeplex.com/ and download the latest code drop, play with it, examine the code.

Thanks for reading.

Bye-bye!

Oooh.

My best virtual croatian friend Joe Basic and I have opened a Google Code page.

http://code.google.com/p/rainweaver/

The Rainweaver Framework (it's a codename, whatever) is a set of libraries for game developers. You can read about our evil plans on the first page. There's something you can download, as well. Let us know what you think, of course!

If you know C# and you're the reliable type, consider joining us. We're busy with work, university, and all the rest, so it's a long term project.

Thanks for reading.

Of Cyclical Rewriting

Sometimes I wonder why I even bother. It all started because there was a bug in the NTL compiler apparently not recognizing handler tokens. I could have sat down and probably fixed it in a bunch of hours. But no. I had to put the whole NTL language idea aside (as I said to Zak, it was mainly a "programming masturbation", but a cool one ;( ) and fall back to XML.

Well, I think it's eventually a good choice since XML is widespread, can be read by almost anything, espresso machines included, and is an accepted standard.

Luckily I didn't have to rewrite all the code that actually creates CLR assemblies out of the entity system definitions you spam in, or it would have been serious pain. Much pain.

Anyway. While I was at it, I also removed a feature which I didn't like. When the entities were processed and turned to IL, they were added a field named Prototype, which contained an EntityPrototype object basically describing the objects referenced by the entity; this means components, handlers and handled messages. All of this just to speed up entity creation at runtime.

However, I had forgotten once again about Attributes, and why they're in the CLR. So, armed with loads of patience but handicapped by lack of proper sleep, I set to remove this prototype stuff and go on with attributes. Now I'm in the middle of realizing what's the next step.

The most annoying part is that I am storing Types via their TypeHandles (actually their RuntimeTypeHandle.Values ToInt32()ed); this basically means creating a type storage via indices; this so reminds me of the database stuff I have going on at work - though, when I started to ramble about entity systems, I kind of expected it.

I had a nice demo going on but it's no more. Raaargh. Here's hoping it won't be long until I have something actually doing its job again.

I'm out.

07 Sept. Edit: Well, it wasn't as bad as I initially thought. I managed to get things up and running again, now including some things like proper component installation and uninstallation. Since my programming life is full of happy gummybears again, I can commit some time to the actual entity system editor. I'll keep you posted.

Entity System and Lua

Lately the Entity System code has been getting more and more complex, to the point that I had to read what I was writing very carefully (and sometimes reading aloud too... go figure).
I have some news about it... about time!

The good news is that it is almost complete! It's a joyful day for me. Object pooling, message dispatching... stuff that works! I still have to finish adding and removing components dynamically (right now we just get the components we defined per entity in an NTL file).

The bad news is that Lua hates me, and it's giving me all kind of exceptions. I think I know what's going on and the relative fix, but if said fix did not work, I'd be in serious troubles. Basically, Lua needs to know what types you're working with, otherwise it would just sit and cry pitifully, making you feel guilty for such carelessness. And it makes sense, of course. LuaInterface is a mysterious world, I still have to figure out thoroughly how the .NET bridging works - luckily the source code is freely available.

At some point of yesterday evening, while bashing my head on the desk for the umpteenth exception, I even felt like writing a Lua compiler / interpreter from scratch in order to get rid of problems when interfacing with the CLR. Sadly, it's a task beyond my available time and most likely experience as well. It would be a great challenge, though.

I still haven't performed any performance tests, but I know in advance I will have to make sure a) the object pool isn't raping the heap b) entity creation is quick and painless, and so is its disposal.

I know it's all a bit vague, but I'll be sure to release the NTL environment as soon as I have something worth attention.

Have a good and productive .NET day! (...rolololflol.)

Getting back on track...

Well, these have been very busy days. I managed to rewrite a good part of the Entity System, which now supports object pooling.

I have to brush up my Lua knowledge before implementing message handlers properly.

Here's hoping it won't be too long before I post again with more substantial news.

Of Disappointment

I came to the conclusion that I really wanted to implement object pooling for the Entity System, but I think I've hit two or three (or many more) snags along the way.

The more I try to add this feature to the ES, the more I realize I have to rewrite a good part of the compiler too (make it 80%). I have based the design of the ES upon different assumptions, and now stitching new stuff is proving to be very difficult.

Moreover, I'm not sure all of this will be eventually worth the effort.

So, as much as I'd love to release an ES with an object pool, I don't seem able to pull it off with the current design. It'll have to wait for a new ES version, when perhaps I'll have the time to rewrite it from scratch. This will also mean I'll break the compatibility with assemblies produced with the old compiler.

Bah.

Edit: hell, since I love to rewrite code, I think I'm going to redo the compiler from scratch, this time taking in account everything. I haven't been analysing the problem at hand enough, and that's the result! Don't follow my example! ç_ç/

Entity System (Part 4)

As I said in part 2, the core of this entity system is how messages are dispatched to entities (technically, to their message handlers). Since handlers have a state and a priority flag, it'd make sense to create an ordered list of handlers, and shoot the message down this list. Of course, upon component installation, the order must be kept and it depends on the above flags.

At this point, I don't care if the "message pipe" must be a list, a tree, an array, whatever; I'm worried about the performance of the system, to be precise about allocations.

In a game, especially such as ours, it is common to have many similar enemies which share the same type of components (to keep the es terminology) - we know that all of them, at least, have a Health component. Since I mentioned allocations, the attentive reader might have already understood that I'm about to talk about object pooling.

Why pooling? I find it logical to reuse some objects over and over, since I'm not sure how long they'll live. They might be short-termed (entity insta-kill), or long-lived, or anything in between. We know that GCs kick in post allocations, and that short-lived objects are fast to collect. We also know that caches need a policy, or they're just disguised memory leaks (and if Rico Mariani says so...).
However, in our case scenario, I cannot predict the life of the objects composing an entity (plus the entity itself). Should I just allocate whatever I want and focus my attention on, say, a funny youtube video? Or should I consider creating a pool of reusable objects, instead of allocating without a specific order? Still, allocations are very fast in the CLR.

And the story isn't over, yet. A message pipe contains many references to handlers. And there will be x message pipes equal to the number of entities times the number of system messages, each of which will contain y references to y different handlers. It's a lot of references to be tracked by the garbage collection, and I'm scared to just think what'd happen if these references kept some objects alive instead of letting the GC collect them.

My solution (and it's far from being the right one, I suppose) would be to pool stuff, even if I have just advised against it in a recent gamedev.net post. With pooling, I have collections of objects ready to be used or already used, and I can access them via an integer "handle" (namely an index). All my message pipes would contain is a list of these integers - thus no actual pointers but indices. Would my garbage collector be happy? Am I wasting my time? I tend to think the latter, since I haven't managed to make up my mind about this matter yet.

I gotta say I find many .net perf advices a bit obscure.

There's also something else which has kept my mind a bit busy between a general ledger entry table and a C/AL codeunit this morning (brrrrr!). Should I suggest the GC to perform a collection when, say, the user reloads the level or changes map? I'd say so, but it isn't necessarily right. I must remember to test this as well at some point...

Premature optimization, anyone?

Entity System (Part 3)

Well, the compiler looks almost done; a ntl file in, a .net assembly out. I'm sure there will be a few bugs to iron out, but it conforts me to just think I'm not too far from its release.

The next step will be releasing an editor for the language; for now, I think I'm going to use Roger Alsing's Syntaxbox and some other stuff found in the net.

Another aspect worth discussing is that the Entity System will work on its own thread, and it will communicate with the main thread via the so-called "action gateway". Every action sent to the ES is queued and executed serially; the inverse is alike, however the main thread has to take care of serializing the actions the ES has sent and execute them after whatever comes first (usually rendering and physics update).

Now, in my mind this looks like a good plan, but I know it's bound to change somehow. I can't wait to have some working results to show.

Good weekend, Raine out (nerdish goodbye).

Part 4 is here.

Entity System (Part 2)

It's Friday, I guess I'm going to keep rambling a bit.

I came up with a simple declarative language, which happens to be very similar to IL (I like the leading full stops) to ease the editing of entities, components and system messages (simply messages from now on):

// build options
.assembly entities.dll;
.namespace EntitySystem;
.debug true;

// declarations
.component HealthComponent
{
// .flags
.field System.Int32 Health;

.handler DamageReceived
{
.priority normal;
.state impl;

// .extern scripts/ondamagereceived.lua;
.body
{
--lua code here
}
}

}

.message DamageReceived
{
.field System.Int32 Damage;
}

.entity Player
{
.components
{
HealthComponent
}
}

The good news is that the parser works; the bad news is that the first version of the compiler refused to work properly (for some unknown reasons CreateType complained some methods didn't have any implementation).

How does it work? Basically, you can define some predefined entities by listing their components. Components have fields and handlers (message listeners) and messages have fields in order to contain the data they carry (what else? :) ).
Handlers have two special members: the .priority and .state declarators. These things specify in which order the handler is installed into the Entity. You cannot specify an index, rather the Entity System inserts these handlers in a stack fashion in the right place.

.priority allows values such as "SystemHighPriority", "SuperHighPriority", "HighPriority", "NormalPriority" etc, whereas .state allows only "pre", "impl", "post". I have to thank the good Mr John Brewer for this idea, which is basically the core of how the message dispatching works.
You can send messages from one entity to another, or from one entity to a family of entities (which is defined by the flags field, which is made to containthe numerical value behind OR'ed enum values).

I'll be releasing the compiler (and probably the source code) as soon as it's completed. This will be plugged into newborn's StoryED in order to allow further customization. You compile the NTL (eNTity definition Language), you reference the produced assembly and the ES assembly into your application and you're done. End.

I like this stuff. What do you think?

Part three is here.

Entity System (Part 1)

Long blog post incoming. And it's, like, my third one.

Entity
1596, from M.L. entitatem (nom. entitas), from L. ens (gen. entis), proposed by Caesar as prp. of esse "be" (see is), to render Gk. philosophical term to on "that which is."

System
1619, "the whole creation, the universe," from L.L. systema "an arrangement, system," from Gk. systema "organized whole, body," from syn- "together" + root of histanai "cause to stand" from PIE base *sta- "to stand" (see stet). Meaning "set of correlated principles, facts, ideas, etc." first recorded 1638. Meaning "animal body as an organized whole, sum of the vital processes in an organism" is recorded from 1683; hence fig. phrase to get (something) out of one's system (1900). Computer sense of "group of related programs" is recorded from 1963. All systems go (1962) is from U.S. space program.

http://www.etymonline.com/

There are posts around the net about entity systems. Not sure there's a particular implementation to follow. I've been reading some related threads over at gamedev.net, but none of them particularly stirred my interest either.

Entity systems are made to favour aggregation over inheritance. There are reasons to use the former, and reasons to use the latter. I'm not going to say one is better than the other. The right tool for the right blah blah blah.

I'll go the Entity System route with Darkmana, mostly because - provided my implementation will work as intended - it'll allow me to define custom behaviours with little effort.

Imagine for a moment your main character is standing near a bonfire, ready with his composite bow. The wannabe game designer in me thinks: wouldn't it be cool to put the arrow on fire by standing close to the bonfire? This means the next shot will also do some fire damage. Cool stuff, yes.

Now, what am I going to do to implement is? What's the way? Many a way. Logically speaking, my arrow is a simple arrow at first (it has some default properties), and then I add some other attributes; that is, it is on fire. From a code standpoint, I have my Arrow object, probably derived from a Missile : GameObject class. Now, it's a FieryArrow object, derived from the Arrow class. So basically, create the new one (probably with a constructor accepting the old Arrow object?) and use it.

I could go on forever (in fact I was about to, I just deleted a good bunch of lines worth of examples), but the problem still remains. Whatever you do, it's hardcoded AND stuck into a specific object hierarchy, which you can't free yourself from.

Scripting comes in rather handy. Well, you still have the fixed hierarchy issue. Why is it an issue? From Cowboy Programming:

"The result of implementing functionality near the root of the hierarchy is an overburdening of the leaf objects with unneeded functionality. However, the opposite method of implementing the functionality in the leaf nodes can also have unfortunate consequence. Functionality now becomes compartmentalized, so that only the objects specifically programmed for that particular functionality can use it. Programmers often duplicate code to mirror functionality already implemented in a different object. Eventually messy re-factoring is required by re-structuring the class hierarchy to move and combine functionality." (http://cowboyprogramming.com/2007/01/05/evolve-your-heirachy/)

Let's get rid of it, shall we? How am I going to define my game objects now? Here's my approach.

The rationale behind removing hierarchies is that entities are made of components, pretty much like a body is made of organs performing different stuff. An entity in itself isn't much; it's the way it reacts to the surrounding system rules that matters; and the way it reacts depend on what composes it.

So, now I'm going to have an object which allows me to add other objects, and these objects will take care of defining the way the owner reacts to some system messages.

I mentioned system rules. They are indipendent from the entities. Entities act into a system with a predefined set of rules, rules which they cannot change (we're talking about virtual environments and not politics) - basically the boundaries set by the game designers.

I'm going to post some more later. I need a cigarette and the lunch break's almost over. Let me know what you think.

Part two is here.