Trylon's guide to understanding PyPRP

Announcements and discussion regarding any projects related to Cyan Worlds' Plasma Engine including (but not limited to) CyanWorlds.com Engine, Drizzle, OfflineKI, PyPRP, and libHSPlasma.

Trylon's guide to understanding PyPRP

Postby Trylon » Sat Oct 13, 2007 4:40 am

Currently PyPRP (the blender plugin) is the only available way to write ages. And I guess some of the more curious programmers have tried once or twice to take a look at the code and understand it - and quite frankly, I wouldn't be surprised if you couldn't get a grip on it, as it's not exactly a prime example of coding standards (and even it it was, it still is quite complex)

To take my mind off of some tedious programming work (on PyPRP, I admit) I'm planning a number of short pieces in this thread, that 're aimed to help understanding the nature of PyPRP.

So, let's start...

History
(I'm typing this up from memory, so not everything is in logical order, and some things might be a little incorrect)

PyPRP came out around September 2005, back then, Until Uru had just started about a year before, and Uru was effectively dead. Hacking of prp files and vault files had been going on around the COBBS.ca forums for quite some time, and in August 2005 a hacker from spain - Almlys - had been working on this plugin, and released the first versions of it.

As Almlys main project had been Alcugs, a server for uru, that had been started on since before Until Uru. The plugin then naturally had some resemblance of that, most notably in the name of properties, who almost universally started with "alc". You can still find remnants of this in the filenames of the current versions, most of them still start with "alc".

Now, the first plugin didn't allow for much yet, no textures, no lights, only objects (meshes) and colors.
Pretty soon simple UV texturing was added, and usable lights.

Now all is well, but how to load an age into URU (besides some fericous hacking, or using programmer only tricks)? A programmer named Dustin came with the solution, he wrote UAM, the Uru Age manager. A simple (though non-intuitive) tool that added links into a single linking book in Relto, and could do downloads of ages that were submitted. Nowadays time, UAM has (due to lack of support when Dustin left) been replaced by ULM the UruLibraryManager, which is more extensible and intuitive (though not on all fronts, I admit).

Back then, in the beginning, .sum files were not generated by the Plugin, and had to be generated by UAM.
Also, many additions like clickables needed to be hacked together yourself, like this:
The plugin had a feature where an age could be imported. Then all objects inside the prp file were stored in files in a subfolder, and used on subsequent exports. So if you exported your age, imported it again, changed the raw files, and then exported again, you had something working.
That actually was as complex as it sounds, and luckily is no longer neccessary. (At the time of writing, the functionality is still in PyPRP, but expect it to be gone soon). Actually, even for things like adding regions, exporting and re-importing was the recommended procedure (glad we fixed that :))

Somewhere around mid-2006 the name of the plugin was officially changed to PyPRP, and most logic properties were renamed from "alc..." to "prp..."

Slowly more features were added, including:
- Transparency
- Swimming regions
- Generic region support
- Clickables
- Camera support
- Advanced texturing - lighted textures
- Sprites (objects are always facing you)
- Climbing regions/Ladder regions
- Soft volumes

Hacked together
Looking through the plugin's code - expecially in the older or more obscure objects, you can still find how many things were hacked together. Analyzing the prp objects was a tedious work, and sometimes required guesswork. The meaning of variious fields had to be figured out by simply changing the values and see how uru reacts. Use of the logic properties in blender, where named values were just put in was common, and it still is for some things, though lately more and more properties have been tied to blender properties.
Evidence of this you can still find when importing your age, that often adds a whole lot of properties to your object - all of those have direct counterparts in the prp objects.

Different styles of reading and writing
In the plugin you can also see two mainstream ways of writing to the output buffer:
1) the old way:
Code: Select all
self.int32value = struct.unpack("I",buf.read(4))
self.floatvalue = struct.unpack("F",buf.read(4))

As this was a very tedious way of writing, the new way came up
2) the new way:
Code: Select all
self.int32value = buf.Read32()
self.floatvalue = buf.ReadFloat()


While the second form is currently popular (due to it's inherent simplicity), the first still exists in a lot of code, even the Camera code which was written a short while after the second form was introduced. (Yeah, I plead guilty to implemeting it in an ugly deprecated way ;)).
EDIT: Meanwhile, I've redone the camera classes, so that's fixed.

Among other things, like just working with a couple of different programmers, and limited communication at some points, this accounts for a lost of werdness you may find.

Next subject will be the file structures.

File Structure

Ok, as promised a part about the file structure..
Now, before we start, I want to confess that of the whole PyPRP code I probably only understand half at best.
A short example why: Python doesn't allow for typed variables, which is a definite pain when you have a lot of objects that transfer a "parent" of some sort all to each object they make. Good luck finding out what type of object that "parent" is.
Ah well, maybe typing up all this will help me understand a little more of it myself.

Now, on to the file structure...
the files are roughly divided into two parts (exceptions available):
1) The files starting with alc_. These are the Core files of the plugin, the "engine" as it were.
2) The file starting with uru. These are the bridge between blender and PyPRP, and are both responsible for starting the import/export process, and adding special kinds of objects to your blender file.

The core files are the most complex. There aren't any set-in-stone standards for what goes into which file. A rough guide follows:

- Keyed PRP Object Classes
alc_AbsClasses.py - Abstract Object Bases. These objects (even though typed) aren't found in any prp files, but form the base of derived objects. Like for example, the plBitmap object, which is only used in it's derivatives like the plMipMap.
alc_Classes.py - PRP Objects that aren't categorized into another file. This file is obviously pretty large, as it houses the bulk of the classes for the supported PRP objects.
alcdraw.py - The plDrawableSpans class, and supporting classes. A pretty complex object storing the visual geometry.
alc_MatClasses.py - The set of classes for the PRP objects that are related to Materials, Layers, MipMaps and Cubic environment maps.
alcrespmod.py - A rough implementation of the plResponderModifier

- Other Uru PRP Classes
alcurutypes.py - The base classes (abstract) for all classes describing PRP Objects
alc_VolumeIsect.py - Support for Softvolumes
alcspecialobjs.py
alcurugeom.py - Classes for describing geometry Uru-Style
alcprpasm.py - Assemble and Dissemble PRP files
alcprpfile.py - PRP file Structure classes - also contains a list of Keyed Objects
alcprpparse.py - Parser for PRP files
alcpypack.py - Interesting, didn't notice this one before, appearently encodes/decodes python .pak files
alc_hsStream.py - Support for the basic I/O stream of PyPRP (and Plasma incidentally)
alc_Messages.py - Quite a number of possible messages, used in e.g. in the plResponderModifier. FootstepRegions are using one of these messages.
alc_Creatable.py- Defines various interesting classes (Yeah, I have no clue :) )

- Encryption/Decryption
alcm5crypt.py - Myst 5 Decryption
alcmfsgen.py - Functions to create the summary XML file and the ages .sum (checksum) file

- Resource Manager
alcresmanager.py - This is the main parser of blender data, here it is determined which object is parsed by which subclass

- PRP Page

- Helper classes
alcdxtconv.py - Contains classes to handle the DXT and JPEG images in the mipmaps
alcConvexHull.py - Classes/Functions to help Build a convex hull
alcGObjects.py - Various functions used to transform objects or make regions
alchexdump.py - A function to dump a hexadecimal whatsit for a whosa. (Ok, I admit, I have no idea why this is here :) )
enum.py - Enumeration support


- XML Support Classes
alcXMLConfig.py
alcxmllogic.py
alcxmlparser.py
alcxmlparser2.py


- Blender Interface Classes
uruprp_addbook.py - Functions under Scripts->Add->PyPRP
Used to add book structures and linkinpoints as well as regions of various types
uruprp_addsv.py - Functions under Scripts->Add->PyPRP
uruprp_export.py - The export functions under Scripts->Export->PyPRP
uruprp_import.py - The import functions under Scripts->Import->PyPRP
uruprp_objectopts.py - Functions under Scripts->Object->PyPRP
alcimportblend.py - Adds support for importing another blend file. (appearently blender doesn't or didn't natively support that)

- GUI Files
An attempt was made at a gui property editor quite some time ago, this is probably that, I don't know th latest status
prpgui_common.py
prpgui_export.py
prpgui_intro.py
prpgui_properties.py
pyprp.xml
pyprp_gui.py


- Configuration
alcconfig.py - the old config file
pyprp.xml - the new config (no idea which one is used :) )

Well, that's quite a list, and admitedly, it's quite a chaotic bunch of files. But then again, I refer you to the previous post about PyPRP's history.


Next part - PRP format


PRP Format 101

No, I'm not gonna describe the prp format, I'm just gonna introduce the basics

Basicaly, an age's data is stored in several "PRP" files. PRP stands for Plasma Resource Page. Each page describes a certain region of an age.
All PRP files combined make up the age. And, the regions of the age described by the PRPs blend into each other.

Take the Cleft for example, there is a separate PRP file describing the Desert, there is a PRP file describing the Interior of the Cleft, one describing the BookRoom and many more. Even the notes (this is generally a rule in Cyan's ages) have their own prp's for the GUI (image that pops up when you click the note)

The basic structure of an age consists of three PRP's minimum:

[agename]_District_BuiltIn.prp - Contains only two objects, used for linking an age itself to a python class.
[agename]_District_Textures.prp - Contains all the textures for the age. This is just a convention of Cyan, Plasma also accepts textures in separate prps (Which actually needed in Ahra Pahts shells)
[agename]_District_[Region].prp - Contains all the other info, per region

In Cyans ages there are also PRPs describing GUI's (pop-up screens) and Age-specific Avatar movements.

Now you should know that the PRP is just a container format. To be precise, it contains various objects, usually called "PRP Objects".
These objects have a specific number per object type, a plMipMap object has type nr 0x0004 for example, and each have a unique format.
Thus, these PRP Objects are also called "Keyed Objects"
These formats need to be separately worked out and are quite unique per object. Most of the essential objects are known and implemented, but there are many that aren't yet.

Essentially these objects are serialized (written down to disk) versions of objects (think Object Oriented Programming) in the Plasma engine itself.

Of course, as you can imagine, these Classes are implemented in PyPRP just the same, as classes.

To gain a better understanding of these classes, take a copy of PrpExplorer or PlasmaShop, and open an age with them (preferably, a PyPRP exported age, as they're the simplest). Browse through them and see the links between them.

Last edited by Trylon on Wed Oct 31, 2007 3:35 pm, edited 9 times in total.
One day I ran through the cleft for the fiftieth time, and found that uru held no peace for me anymore.
User avatar
Trylon
 
Posts: 1446
Joined: Fri Sep 28, 2007 11:08 pm
Location: Gone from Uru

Part n

Postby Trylon » Sat Oct 13, 2007 4:41 am

Reserved for future additions.
One day I ran through the cleft for the fiftieth time, and found that uru held no peace for me anymore.
User avatar
Trylon
 
Posts: 1446
Joined: Fri Sep 28, 2007 11:08 pm
Location: Gone from Uru

Re: Trylon's guide to understanding PyPRP

Postby Trylon » Sat Oct 13, 2007 4:41 am

Reserved for future additions.
One day I ran through the cleft for the fiftieth time, and found that uru held no peace for me anymore.
User avatar
Trylon
 
Posts: 1446
Joined: Fri Sep 28, 2007 11:08 pm
Location: Gone from Uru

Re: Trylon's guide to understanding PyPRP

Postby Trylon » Sat Oct 13, 2007 4:41 am

Reserved for future additions.
One day I ran through the cleft for the fiftieth time, and found that uru held no peace for me anymore.
User avatar
Trylon
 
Posts: 1446
Joined: Fri Sep 28, 2007 11:08 pm
Location: Gone from Uru

Re: Trylon's guide to understanding PyPRP

Postby Trylon » Sat Oct 13, 2007 4:42 am

Reserved for future additions.
One day I ran through the cleft for the fiftieth time, and found that uru held no peace for me anymore.
User avatar
Trylon
 
Posts: 1446
Joined: Fri Sep 28, 2007 11:08 pm
Location: Gone from Uru

Re: Trylon's guide to understanding PyPRP

Postby BAD » Sat Oct 13, 2007 8:24 pm

WOW thanks Trylon!

I had loosely followed the story of this stuff, but it is great to read about it from someone who was there in the trenches. :)
BAD is as good as he gets
User avatar
BAD
 
Posts: 832
Joined: Sat Sep 29, 2007 9:44 am

Re: Trylon's guide to understanding PyPRP

Postby Jennifer_P » Sat Oct 13, 2007 10:52 pm

Yeah, I've been wondering about pyPRP--thanks for the story. :)
Jennifer_P
 
Posts: 729
Joined: Fri Sep 28, 2007 10:54 pm

Re: Trylon's guide to understanding PyPRP

Postby Jojon » Sun Oct 14, 2007 9:39 am

"Me too!" / "Yeah that!" :)
Jojon
 
Posts: 1116
Joined: Sun Sep 30, 2007 5:49 am

Re: Trylon's guide to understanding PyPRP

Postby Trylon » Wed Oct 31, 2007 1:47 pm

Added an update.
One day I ran through the cleft for the fiftieth time, and found that uru held no peace for me anymore.
User avatar
Trylon
 
Posts: 1446
Joined: Fri Sep 28, 2007 11:08 pm
Location: Gone from Uru

Re: Trylon's guide to understanding PyPRP

Postby Jojon » Wed Oct 31, 2007 2:30 pm

Trylon wrote:Added an update.
Thanks again, much appreciated!
Even though the only "programming" I've ever done myself is some 6502 and plain 68000 assembler (and the sight of any C-like language turns my stomach), I can absorb a tiny hint of the stuff and better understand what's going on under the hood.
Hmm, so PRP Objects are designated using a char, rather than a roomier value? Well, 256 should last a while, I suppose... :7
Jojon
 
Posts: 1116
Joined: Sun Sep 30, 2007 5:49 am

Next

Return to Plasma Development

Who is online

Users browsing this forum: No registered users and 4 guests

cron