08.10.06 -- Rendering Optimisations and loadings issues
~ The (unmastered) art of thinking ahead of the road
As I wrote the other day: one of the fun and sometimes difficult aspects of working with tools that are still being continusouly developped is that you have to think ahead of those. In the case of the plugin we have a clear road: the objective is to have all the features of the Uru engine, and end with something as close as possible to the tools Cyan use. Ambitious right? Yah, probably a bit too much even; but at least that gives us some clear objectives.
And how does that applies to Ahra Pahts? Ahra is a very ambitious Age, so when I jumped in that boat I made a risky but firm decision: to help design and create the Age as if we already had all the tools available. That is: with all the possible Uru features in mind... and hope that our beloved plugin developers can deliver.
Now you can call me crazy, but so far that bet has worked well. In fact if we had limited ourselves to the features of the plugin, Ahra Pahts simply wouldn't have been possible and the project would have been dropped, or would have been realized but with lower objectives. For instance: water. We knew right from the start that we lacked many tools to properly create water (animated textures, transparency, reflections) yet we designed water everywhere. A couple months ago it was decided to'turn it to ice' to make it look good while we waited for the missing plugin features. And last week we got part of those and next version of the Age will have water. Bet won so far.
I could say that of other things (alpha textures, cameras, light flares, nexus scripting etc) but there's one thing that is really far away in terms of plugin features and yet that is both critical to the Age success and a big part of the Age design: rendering optimisations and loading issues. Ahra Pahts is a huge Age that contains lots of objects and textures and will contain even more as shells start to open.. Because of this we have two big problems: initial loading time of the Age (so that not all the Age & Shells load at linking) and performance issues in the Age (so that not all of the Age is calculated when there's only part of it in sight). Both problems will benefit from the same solution: cleanly cutting the Age in pieces.
Speculations & planning
The loading process of Uru Ages is a bit of a mystery for me, so for practical reasons I assumed it is somehow tied to the whole 'rendering optimisations' gang. What are those? Simple stuff that is done 'on the fly' by Uru, like not calculating objects that are behind the camera; and more complex stuff, like what to load/not to load in memory. Some of this more complex stuff has to be done by hand, and has to be handled through the plugin. Basically it all falls down to cutting the Age in pieces and communicating to the plugin those different pieces and tell it when to load or render which ones. A good example of this is the community room in the Hoods. If you import the Neighborhoord in Blender you'll notice that part of the ceiling of the room can be seen from the outside. Although it is not visible in Uru when you are near the foutain, because the room is only loaded when you actually enter it. I don't know how this could works in terms of plugin interface, but at some point we will need that to be able to handle larger Ages.
Maybe this might be done with regions. For instance when the Avatar links in Ahra he'll be in a region that will tell Uru (via Python scripting) to load and render the surrounding Shells and streets, the sky of the Age, and a couple other things. And when the avatar will move to the next street it'll enter a new region that'll tell Uru to unload what was previously loaded, and to load new stuff. Basically we need to load what is in sight. Maybe it will be done automatically, the tools will calculate automatically I think I read somewhere that Uru support some form of BSP, which is a way to do this (and commonly used in other games). At this point this is pure speculation on my part.
But what is important is that it had to be applied to Ahra. In most Ages you have natural obstacles that cut them in pieces: lanscapes, buildings etc. Given Ahra is a city how do we do that? Quite simply in fact: when we divided the city in districts I decided to make walls to separate those districts, high walls. It would look nice, it would help explorers find their way around, and it would totally block the view of other districts. It was a great way to cut up the Age in equal parts. Also this had to limit the height of the shells. If shells structures were higher than the district walls we would have to load them anyway so that they do not 'pop up' when walking closer. Which would defeat the whole purpose of the districts walls. At first the idea was to have the shells be smaller than the city Hub; but I had to lower that number some more to align it with the height of the district walls. Currently they are 60 units high (which is 10 times the height of the avatar); if the walls were any higher we couldn't see the Hub from the outer parts of the city.
Buffer zones
So, this loading/unloading regions businness is cool, but where to do it? To simplify things I made dedicated zones for that, called 'buffer' zones. Those are the zones between the 'key' areas: between the districts, around the larger courtyards, around the hub. They are narrow zones designed to totally block the view at key points to make the transition smoother; think of it as 'airlocks'. This is a common trick in games. Of course right now, this isn't available yet for our tools. But in the future when that will be available, I'll be really glad to have these zones ready, because they are a fundamental part of the city layout; and if I hadn't included those we would be in trouble.. In fact I was at some point. A couple versions ago just before releasing the shells I realized I lacked space for those zones. I had to spend several evenings to move the districts 40 feet away from each other, and to redo the district walls.
Now the 27 buffer zones are positionned like this:
There are 3 differents styles of zones (district, courtyards & hub) currently there are only placeholder structures for those around the Hub and around the larger courtyards. You may notice there are zones that are on the outer edges of the districts; those are a 'just in case' thing. If (and hopefully 'when') all the shells will be filled new districts will be opened and we'll need to have buffer zones ready for those as well. For now they will be dead-ends. We might put there a plaque saying "Congratulations you have reached the end of the Age come back next semester for more." or something along that line. Ok maybe not.
Designing and modelling all those zones is the first thing on my list once the current bugs will be fixed.
Finally there are other kinds of rendering optimisations I haven't thouched on; but this journal entry is getting much longer than I had planned, so I'll be do a quick list:
- LOD meshes: switching regular object meshes to lower LOD (level of detail) ones when beyond a certain distance from the camera. Again this is a common technique, but thinking about it I can't find a good example of this in Uru beside the avatars. This will be necessary for the Hub (given we can see it from everywhere but only at a distance) and most 'smaller' objects like lampposts and book pedestals.
- Billboards: this is used extensively in the City (the 'real' City: Ae'Gura), to switch an object or a full building for a plain flat billboard (like the Guild Hall or the Library). If done well that can be a very efficient trick.
- Removing objects altogether: when beyond a certain distance certain objects simply disapear even if they are still in sight, like cones or other objects of small size (small lamps in the hood light gardens). This can be tricky to use but a good resources saver.
- The City.CSV file: This thing is a total mystery for me. It seems to indicate which region can be seen from where and vice-versa, but it is the only Age to use such a thing; odd. Still I think it''d be interesting to understand what it is, because the City is the biggest Age of Uru (in terms of objects & datas) and there's surely plenty to learn from it.
Just to end on a more positive note: the Age has evolved quite a lot over the last couple versions, and will evolve even more in the upcoming ones, and this is due to the tremendous progress of the plugin. This Age largely started on a bet and so far it has been won. I'm confident that things will keep going this way. At least I'll keep doing everything I can so they do and we can complete this impossible Age.
