Sunday, 11 July 2010

DXStudio progress

Well, it is easy to get distracted.

I have been working for a couple of weeks with DXStudio and it seems pretty good.  No Mac support as yet, but that is coming.  I've written a client for my GameServer code in C++ and linked it as a DLL (dxeffect) into DXStudio.  Recalling my previous C++ experience and getting multithreading (for the Network side) and winsock2 to work has been instructive, and hard work for several days.

The Javascript wrapped dxeffect files in DXStudio are pretty powerful, you can hook into Update and Rendering passes and implement anything you can think of - and do in Windows (will be OPENGL restrictions I think when v4 comes out).


In terms of achieving MMO'ness this is the way to go for me.  Running in a browser is much lower friction trying to get something out there and a growing userbase (what we all yearn for).

The server side is all going to be C#, so as soon as development can be based there, the better.  If I can sort out safe C# scripting via a web browser for the server gameboard management, that should be a productive development environment.

Saturday, 26 June 2010

Persistence and interaction

Two topics to think about, while bearing in mind that this is a minimal MMO.  How much persistence is required and how much interaction ?

Persistent information is retained.  Non-persistent information is lost.  For example, many players expect their character(s) in MMO games to develop and grow over multiple play sessions.  Players like to 'level up' and acquire special items for their 'toons' (perhaps armour or weapons in an RPG context).  Some multiplayer games do not support this, for example the Age Of Empires (AoE) series start everyone off from the same newbie starting point every game.  AoE is seen as a RTS (Real Time Strategy) rather than a MMO to most people, but because of the large number of potential opponents it played not unlike a purely instanced MMORPG.  EXCEPT for the issue of persistence.  I think it is reasonable to make a case that any MMO must support persistence of player abilities development (other than a scoreboard) so that more developed players are able to perform more actions or better actions in the world.

Some MMORPG games, Lotro and Vanguard for example, allow the creation or customisation of player and guild housing.  This allows the player to make change the simulated world, to leave their mark.  Second Life is almost entirely about players creating things and places in-world.  Yes, SL has socialising, apparently,... but I'm a programmer, so...

World of Warcraft, currently the most populated MMO (? is Farmville getting there on Facebook ?) does not have any housing or any real way for players to persist changes in the world - except by the immediate short-term actions of their characters.  This omission does not seem to have done the game any lasting harm.


Persistence is subtly tied to interaction between players.  If the only interaction is very indirect, through an auction house for trading items perhaps, then each player is effectively seeing a private version of the world.  Such a compartmentalised game could (more) easily provide a great deal of customisation to players, no-one else would ever see it so potentially the whole customisation could be client side with no information about it going to the server.

If the players are able to interact directly or indirectly then there must be persistence of the required information at the server.  If I can drop a cube in a shared area somewhere and you walk past later on, perhaps when I am logged off, then you should see the cube sitting where I dropped it.  It gets more complicated if there is a cube sitting somewhere and either of us can move it.  Who gets to move it ?  Is it picked up and dropped (by one person at a time) or do we both tug at it and the server resolves the conflict ?

With a small 'p' this is where the physics of an MMO gets complicated.  Some things are entirely client-side and no conflicts arise - the exact position of the particles making up a fire effect probably does not 'matter' in the context of the game.  In the case of a fire then all the clients need to know is that there is a fire object and where it is.  In terms of interaction with the fire, either player might pour water on it to extinguish it.  This works once (and your 'water pouring' skill goes up by one point).  If twenty players are sitting watching the fire and someone puts it out then all twenty clients need to be informed of what has happened to the state of the fire.  Additionally some clients whose players have also tried to put out the fire need to be able to deduce that they failed.  While it is relatively easy to block a simple action which can be 'performed' or 'fail' at the server depending on an object's state it is tougher to sequence events correctly when the action being taken is manipulating a system such as a pile of bricks.

The aim probably needs to be to allow the largest possible number of clients to succeed in any legitimate actions taken, but without ever having to 'wind reality back' at the server.  Most MMO clients will allow character movement, for example, without waiting for a response from the server - but if your movement fails for some reason (you were stunned by another player before you could start to move) then your view of the world will sometimes 'snap back' or 'rubber band' to a previous position.  This effect is often noticed by players of popular MMO games, particularly in congested areas or at busy times of day.

Thursday, 24 June 2010

Slightly sidetracked

While thinking about client software I revisted possible web browser hosted 3D client environments and this time came up with a new possible product.... DX Studio has been getting positive comments from people having a hard time with Unity - and who don't need Unity's iPhone support.

So, over to http://www.dxstudio.com/ and download the free non-commercial version for an evaluation.

It suffers from the bits and pieces everywhere problem common to integrated engines (and the lovely Smalltalk) but on the other hand their support seems good... even a newbie post from me got a quick and useful series of replies and seems to have identified a minor bug in one of their many demos (when published via a web page).

The publish to HTML feature is very seductive.  It generates a .zip, .html and .js file, which when placed onto a webserver can display nice looking (and fast) 3D on all 3 machines on which I tried it out.  There is a minor problem that the Chrome auto-install of their viewer doesn't work properly at present, but this seems to be something which is being worked on.

So, although I am not moving to this for the client now - because it isn't necessary to run in a web browser for my 'minimal' condition - it is still an interesting development that something as good as DX Studio is available at such a low price.  They do have a server model, but without investing time I don't think it would serve my needs.

I may try to set aside a few days to build some kind of game with the DX Studio tools though... perhaps yet another implementation of my old pygame GiggleBubble... just to see what can be done.

There can/could be only one

I was thinking about game design, an activity which occupies a lot of my time.  I like the idea of incorporating physics into an MMO somewhere other than on the client side and got sidetracked into thinking about the client software, again.

It would be easier in many ways to have ONE program, which contains all server and client code.  This simplifies some aspects of testing and building the application.  The program runs in different modes depending on configuration files, no graphics on the server for example.  Some effort is required to draw the line which separates server and client aspects, but this would be required whether one executable is both (my current preference) or there are separate client and server programs (the norm, I suspect, for almost any large Client-Server system).

Well.  It may be the norm, but that doesn't mean considering something else might not yield interesting progress.

Firstly I am talking about a MINIMAL MMO.  Putting everything in one place might help keep the size down.
Developer convenience is an important factor.  Switching devpt. environments and designing intermediate protocols could - to some extent - be eliminated.

I obviously still have to send messages from the clients to the server.  Even a lobby+client group system needs a server to hold the 'actual' state of the world (well, okay peer-to-peer network is one possibility).  But if I'm careful designing the separation between different parts of the code then the messaging might be able to be pretty transparent, or even non-existant, for a while.

In a similar vein, coding for 2 or 4 separate client windows within a single executing copy of the program - and having them communicate with the server separately - might prove useful in testing.  User input handling would be slightly unnatural (active console select), so maybe running multiple copies of the code would be better... hmmm.

Monday, 21 June 2010

What do I know about tense ?

Perhaps the title of this Blog should be "How I want to make an MMOG" or even "Exercises in futile wishful thinking".  But I am a fan of the positive outlook, even though not actually capable of maintaining one constantly, and so getting the tense wrong on purpose seemed like a good idea.

I should though, I think, try to be reasonably honest in describing what is going on in my MMOG development.  Therefore I will say that at present there is no MMOG, nor anything which could be described as such.  The culmination of 18 months of trying out different languages, platforms, libraries and tools has led me back to an old friend, C#.  Now I realise  that many people roll their eyes when C# comes up in conversation and I have chosen it for one main reason only.  It is a very friendly development environment, and for the time being, free to use the express versions of the tools.

This is great.  I have experience with Linux, my last company used Linux, Perl, C and Java exclusively for our online web service and this worked really well.  But for writing a MINIMAL (see scope control) MMO I want the easiest, simplest, most supportive development platform and C#, XNA and Windows OSs is it.

I've considered C++ or Python with OGRE3D as a library, because those are also great languages in which I enjoy coding things.  But C# with XNA is more robust and does what I expect more often.

This is version 1, or will be, I reserve the expectation that version 2 will be completely rewritten from the ground up.


That said, I adopt a technical control before top-down design approach to coding.  In other words I want a collection of device drivers for the muscles and eyes  before I write my robots OS.  This might seem like a daft approach to some developers, but in my experience solving the technically difficult problems from the bottom up creates useful insights which can guide the much more difficult top-down design of the complex parts of a project.  Bringing it all together and structuring the game is complicated, but not difficult.  Working with a physics engine is not complicated, but it is difficult (at least it goes in the difficult box until it works).

I recently signed a licence agreement for the excellent BEPU physics engine.  So one small, but significant, part of the world is in place and no longer difficult - at least not for me.  Why use a physics engine for a simple MMO I hear you cry ?  Because a friend with more experience in these matters recommended it to me, repeatedly (not the BEPU engine, but using a physics engine more generally) and eventually I crumbled and tried it. He was right, it was not too difficult, it integrated into my test build framework easily and it worked after solving a few stupid problems of my own creation.

Why use a physics engine at all ?  Well two reasons.  First, and most importantly, it does the collision detection for your models.  Secondly it is available if you end up using physics anywhere in your game, which I think I would like to include.  Adding a physics engine later on would be unpleasant, so it was one of the first things built into the framework.

So far I have created a mini-game framework in XNA (I've even built an Xbox version and tested it, but I'm not planning to support that platform in version 1).  My framework NOW has working: 3D sound with listener located at the camera and sounds placed in the world; a collection of cameras from First Person and Fly Anywhere to some of my own devising (more on that later); a working webcam capture to 3D object surface texture (not sure I need this, but I wanted to be able to do it in the same framework (!); BEPU physics integrated and compound physics models working (e.g. 3 cylinders joined to 'wrap' a sword with cross-hilt); A collection of simple models from Blender (courtesy of #2 son); Code to import Blender .fbx export models at runtime (or compile time); Models can be scaled to a fixed coordinate system (including their wrapping physics models) and skybox code.

A lot of the understanding of XNA has come from reading books.  I thoroughly (and completely) recommend that anyone thinking of playing with XNA should obtain a copy of Riemer Grootjans excellent recipes book and work their way through it.




So... once you've done that... welcome back!
(if you don't want to do it now, that's okay, but it is essential reading for a developer who wants to 'get' XNA quickly).

Next up we have some posts on design and game building with emphasis on trimming the scope to keep things achievable in my limited time-frame.  I really want to get to the stage where I can upload a trial game and see what people think but this currently is more than 2 months away.

Second Life

I enjoyed my time creating things in Second Life.  I knew I wouldn't be designing clothes or creating beautiful buildings, any talents which I have are not in the visual arts.  Early on in SL I decided to focus on programming tools for other people to use.  I created some fun software which could create giant pyramids, boxes, spheres and the like out of hundreds of objects by creating child objects and then flying them to their final resting place.  I got quite good at this and at one point was selling maybe 5 building machines a day.  However trade started to fall off, and even at 5 a day I was only earning £5 a day... This was not going to become a way to earn a living.

I ended up giving away my code for free, but it was all a useful learning experience so I'm not bitter ... *cracks knuckles*... 

[Aside: I haven't connected to SL for at least a year, so things may have improved.  From a casual reading around they probably have not]

I think that Second Life is a good place to start for anyone thinking of programming a 3D environment.  Why ?  Because it gives you exposure to textured surfaces, rotations, coordinate systems, quaternions and lag.  Lots of lag.  Second Life allows people to upload their own textures and other people to see them wrapped around objects, this creates lag even when you stand still and someone with lovely clothing walks past you.  MMO environments such WoW and Lotro do not suffer from this sort of lag.  MMO games have all the visual assets in the game installation on the client hard disk, the only information* travelling to and from the server is distributed physics stuff telling things where to go - more on the whole distributed physics thing in another post (or five) later.

*OK there is some other stuff on the network: Chat, Database flags (you have mail), Auction House data, Trade object info. for inter-player trading and so on.  But the distributed physics is the main thing.  I include in physics the whole world simulation and sharing of behaviours of objects (thus avoiding the need to separate combat, donning different clothes and spellcasting for example).

So... if you haven't already... download Second Life, program some objects to do something half-way interesting and, breathe...

Back ?

Having done that you have probably encountered some SL specific amusements such as code spread around all over the place - almost literally as misbehaving objects head for (0,0,0) outside your control.  If you fancy a laugh then visit a collection of SL sims and look at 0,0,0 ... It is not surprising to programmers how often things end up here, in virtual limbo.

The main problems facing anyone trying to build an MMO in SL - and I did consider it as a platform (No, I wished repeatedly that it was good enough) - is the nightmare itty-bitty-bits distributed code problem.  For personal reasons I call this the Smalltalk problem, but it applies to many highly productive environments such as Lisp, Forth, Prolog and so on.  It is truly wonderful when programmer A can leverage abstractions to build structures to the clouds... until the resulting houses of cards have to be maintained by programmer B.  

The thing is, SL script is not beautiful in the abstract like Lisp.  It is not fantastic in its declarative power like Prolog.  It is a highly specialised, pretty weird and clever way of allowing developers to implement objects with behaviours.  It is NOT a productive software development environment for building anything complex.  No doubt with vast hacks and lots of intricate wiring something MMOish could be created.  But it would make Little Big Planet seem entirely straightforward by comparison.

But you should go and play with it, so if you didn't last time then off you go !


OK... That's enough Second Life.  It is instructive and helps me to reduce the scope of what I want to achieve in Version 1.  A Good Thing!

Stabbing at the scope

I've been building bits in and pieces for online games for about 18 months now, and the result is a considerable gain in experience and understanding of just how difficult bringing everything together coherently is going to be.

I have played with Torque, Unity, C# (and XNA), DirectX (v8), Python, LISP, Actionscript (and as little Flash as possible), Blender, Paintshop Pro and many, many (sob, many many many) more tools and libraries.

I have also played with WoW, Lotro, Vanguard, EQ, Runescape, Second Life and been hunted mercilessly in COD4 and COD6 (by my two teenage sons, on the family Xbox).

One conclusion I draw, after luxuriating in all this fun and learning for far too long, is that 'paralysed by choice' is a great aphorism for the wall of any organisation building software, let alone open-ended creative fun like games.

At least when you are writing a device driver to interface a gigantic flat-bed plotter to AutoCad you have the advantage that you know what will qualify as a solution... something which makes the plotter move as desired when AutoCad tells it to move.  Problems like device driver writing may well be difficult, and require arcane incantations in assembler or C and fighting with obscure APIs (don't try to tell me AutoCad in the mid '80s had an easy to understand API developer manual).  BUT at least these problems have an end-point.  YES it works, YES you finished.  Thanks and here's your cheque.

I _wish_ that creating a multiplayer online game was so straightforward.  Here's the spec... now off you go.

It is not that simple.

Firstly, and most significantly, I'm doing it by myself.  Well I should credit my sons, again.  When they are not chasing me around in COD4 or COD6 (Favela is their favourite killing ground) then they are playing football or chatting on MSN.  But when even MSN gets boring, which it apparently does after 5 or 6 hours in one session, then they sometimes make sound effects or music (#1 son) or Blender models (#2 son) for me.  This support activity has dropped off somewhat recently, but I think that is because of my lack of direction, rather than their lacking enthusiasm when I am clear about what I want.

But, as I digress... Firstly, for the second time of asking, I am doing the coding and engine building myself.
I considered using a world building engine, but these are really geared towards asset builders first and software developers second.  No doubt plenty of programming challenges would exist if I were to use some fantastic (and hugely expensive) MMO engine, but the world would just be a clone... and the result a poor imitation of AION, Vanguard, WoW, Lotro and the rest...

Perhaps I should expand a little on what I mean by 'scope'.  The scope of a project of any complexity is the full range of resources and activities required in order to complete the project.  I have learned the hard way that what the client thinks the sales team has promised them is very often different to what the senior project designer has 'documented' in their blame-transferring spec. email, which has been designed mainly to ensure that everyone except the programmers avoid responsibility when delivery fails completely or falls below expectations.

I am reminded of a famous example from my estimating training more than two decades ago.  Question:  How long will it take you to make tea ?  The ONLY correct answer from a professional (except a professional gambler perhaps) is: What do you mean by 'make tea'  ?

[if you're not sure why, then think about it... Do they want a cup of tea, Devonshire Cream Tea, tea at precisely 3.30pm, mint tea ?? does it ever stop]

In other words, the only answer is a question.  This sort of thing has tied philosophers and theologians in knots for millenia and you only have to glance at their output of successful MMOGs to realise it is the wrong direction.  So we steel our resolve and must draw our sharp incisive sword of professional software development and cut vigorously through the Gordian Knot.

Cutting the scope knot professionally (as a one-man MMOG builder):
Whenever a question is raised we will think about the wide ranging scope of possibilities attempting to spread before our eyes, and enlarge the project into it's third decade, and then choose the most obvious answer which shrinks the scope but without making version 1 unplayable.

It's a bit like having a buddy developer in my head who is both lazy and direct.  Whenever he notices me considering including this feature or that feature, he will scream NO NO NO A THOUSAND TIMES NO at me.  This is pretty much the approach of any successful developer of projects which actually get used in the real world to their immediate clients.  Concomitant strategies are YES BUT I CAN NO LONGER GUARANTEE ANY PARTICULAR DEADLINE and YES BUT IT WILL COST A POTENTIALLY INFINITE AMOUNT OF MONEY.

If you want something to work, then build something which works. THEN refine it, carefully, continuously checking to make sure it still works.

Oh, you say, I thought this was an MMOG ?  I want to see dragons, oh yes and flying bats in swarms and teleporting and PvP and self-assembling goblin robots made of sludge and rocks and, and, and...

YES we all want those things, really we do, but NOT first.  FIRST we want something which actually works and includes all the required layers which, in my imagination, an AAA MMO would have.

[Aside: at this point I notice that CAPITALS are becoming more frequent and am therefore going to stop this post pretty soon and get a drink of water.  Emotion and blogging seem to be caught in an intimate and unavoidable embrace]


So, what do we have so far ?

1. I know some stuff
2. Scope control is tricky and even worse when I am my own client.
3. Blogging begets a passionate writing style if you're not careful.

I was hoping to actually start to pin down something of the scope of the project.  That will be for another post.  I also want to describe my Second Life programming experience because it is perhaps the environment which is closest to what I want to achieve - while simultaneously being completely unlike what I want.

More later.