Presenting UruDistro

Announcements and discussion regarding Chacal's UruDistro

Moderator: Chacal

Presenting UruDistro

Postby Chacal » Tue Jul 07, 2009 1:22 am

Here's a short description of the UruDistro project. More details will follow shortly.

UruDistro is a set of scripts for the easy packaging and installation of changes to Uru files. This achieves fine granularity because changes are made at the object level inside Uru files. It is then possible to distribute changes to existing files, such as Cyan original Ages, without distributing whole files.

Hackers use tools such as PlasmaShop, PageEditor, UruExplorer for discovering the inner workings of the Uru files, testing and making changes to objects inside those files. Those changes often involve dozens of manual operations. It becomes difficult for hackers to distribute such changes to the user community, and it is difficult and risky for users to try and apply those changes manually. Take for example my own changes to the Relto, City and RestorationGuild Ages, as described here, here and here. As you can see, instructions for even such simple changes can be quite convoluted.

When changes consist only of adding objects, such objects are grouped in prp files that can be distributed and added into Uru by Drizzle and the off-line KI. However it is not always possible to do so. Such is the case when changes are made to existing files. Distributing whole files as a replacement for existing files is not always an option, considering such issues as size of distribution, copyright infringement, or conflict with other changes.

Take for example the addition of a new feature that requires adding a variable to an existing SDL file. The traditional way of doing this is to distribute the whole SDL file. Installing this will wipe out any previous change the user had made to his existing SDL file. In contrast, UruDistro will simply add the required variable to the user's SDL file. Changes applied this way are reversible. Every change performed by the install script has a corresponding reverse change performed by an uninstall script.

For the user, installation of a distribution is simple. A complex distribution (for example, making the Gahreesen wall work again) can be applied by the most clueless user in a few clicks, and then can be uninstalled without affecting other changes. The install script reads changes described in an xml file and applies them using libPlasma command-line tools developped by Zrax. The user needs to have installed UruDistro and libPlasma, which is quite easy.

For the author, packaging a distribution is simple. It consists merely of putting together some object files (.prc files), any other needed files (prp, pak, ogg, etc.), editing an xml file describing the changes, and zipping all this together.

UruDistro is almost ready for release. It is working well already. What started as a few batch files has evolved into hundreds of lines of scripts, mainly because of the required ease of use and paranoid error checking. I am now looking for good real-life test cases, apart from my own referenced above.

More details later.
Last edited by Chacal on Tue Jul 07, 2009 11:02 am, edited 1 time in total.
Chacal


"The weak can never forgive. Forgiveness is an attribute of the strong."
-- Mahatma Gandhi
User avatar
Chacal
 
Posts: 2443
Joined: Tue Nov 06, 2007 2:45 pm
Location: Quebec, Canada

Re: Presenting UruDistro

Postby Whilyam » Tue Jul 07, 2009 5:14 am

Nice. I've always wanted a Disco in Uru, and now you say you're making an Uru Disco? :lol:
User avatar
Whilyam
 
Posts: 1023
Joined: Sat Sep 29, 2007 5:55 pm

Re: Presenting UruDistro

Postby Robert The Rebuilder » Tue Jul 07, 2009 6:07 am

Nice work, Chacal! Can't wait to test it out.

One real life case could be the Offline KI, which comes with modified versions of Cyan's URU files. If they use UruDistro to package the necessary changes, then Offline KI could be distributed without any dispute over copyright infringement.
Can we rebuild it? Yes, we can - here's how.

MOULagain KI# 1299

Myst Movie coming soon - spread the word!
User avatar
Robert The Rebuilder
 
Posts: 1383
Joined: Sat Sep 29, 2007 7:24 am
Location: Virginia, US

Re: Presenting UruDistro

Postby diafero » Tue Jul 07, 2009 9:11 am

Hopefully we can soon test it on a smaller package of modifications before converting the whole Offline KI to it - but generally, that sounds like a good idea.
However I doubt this will really silence the people complaining about distributing Cyan files since the Python files still would have to be distributed. It could reduce the size of the download though.

Will there also be something like a diff system to create the UruDistro packages? For example the GlobalClothing files were changed by GPNMilano and I have no clue what exactly she did there. The GUI and Nexus files are taken from UU and can not easily be replaced by a patch based on the TPOTS files.
I prefer e-mails to "diafero arcor de" (after adding the at and the dot) over PMs.

"Many people's horizon is a circle with a radius of zero. They call it their point of view."

Deep Island Shard | Offline KI
diafero
Deep Island Admin
 
Posts: 2828
Joined: Mon May 05, 2008 5:50 am
Location: Germany

Re: Presenting UruDistro

Postby Chacal » Tue Jul 07, 2009 10:27 am

@Whilyam: There's already one in Ahra Pahts! BTW, I'm not too found of the name UruDistro, but it was the first one that sprang to mind and it stuck. I'm looking for a better name.
@Robert: Thanks. See diafero's answer about the Ki.
@diafero:
You're quite right. There are plenty of other packages and also I'm all for "if it isn't broken, don't fix it". Also, sometimes it isn't possible to avoid including copyrighted material.
About a diff system: that's easily done. Manually, I would extract all objects from both prps into .prc files using PrpExtract, then use free GUI tools like WInMerge and SyncBack for comparing the objects.
In a script, I would:
Code: Select all
 extract all objects from both prps into .prc files using PrpExtract
 list objects in both prps
 for each object in the GPNMilano prp
          if it is not in the normal prp, add it to "objects to be added"
          if it is different from the one in the normal prp, add it to "objects to be added"
 for each object in the normal prp
          if it is not in the GPNMilano prp, add it to "objects to be deleted"
Chacal


"The weak can never forgive. Forgiveness is an attribute of the strong."
-- Mahatma Gandhi
User avatar
Chacal
 
Posts: 2443
Joined: Tue Nov 06, 2007 2:45 pm
Location: Quebec, Canada

Re: Presenting UruDistro

Postby diafero » Tue Jul 07, 2009 11:10 am

Then we just have to manually sort out the recompression-changes and it should work... sounds good!
I prefer e-mails to "diafero arcor de" (after adding the at and the dot) over PMs.

"Many people's horizon is a circle with a radius of zero. They call it their point of view."

Deep Island Shard | Offline KI
diafero
Deep Island Admin
 
Posts: 2828
Joined: Mon May 05, 2008 5:50 am
Location: Germany

Re: Presenting UruDistro

Postby Robert The Rebuilder » Tue Jul 07, 2009 6:19 pm

diafero wrote:However I doubt this will really silence the people complaining about distributing Cyan files since the Python files still would have to be distributed. It could reduce the size of the download though.


The Uru:CC Python files that are included in Offline KI can be replaced with diffs that are patched in, using Chacal's tool. [You gotta agree that Python patches should be supported as well, right?] And if you provided an "offline-KI-lite" that simply didn't include moul.pak and mystv.pak, then there would be no Cyan content inside - just diffs.

[Note: it could detect whether you have MystV installed and apply patches. But nothing can be done about MOUL, since our GameTap images didn't have the Python files.]
Can we rebuild it? Yes, we can - here's how.

MOULagain KI# 1299

Myst Movie coming soon - spread the word!
User avatar
Robert The Rebuilder
 
Posts: 1383
Joined: Sat Sep 29, 2007 7:24 am
Location: Virginia, US

Re: Presenting UruDistro

Postby Chacal » Tue Jul 07, 2009 6:57 pm

Robert The Rebuilder wrote:But nothing can be done about MOUL, since our GameTap images didn't have the Python files.


MOUL python files Show Spoiler


By editing payiUrwinBrain.py, you can make Urwin do what you want...
:D

About diffs: UruDistro is not using diffs. It is using objects, which are:
- objects in prp files;
- variables in SDL files;
- settings in fni files;
- pages in age files.

Otherwise it just adds files, as UAM and ULM do.
For python files I was planning to do just that, since python will always use the latest version anyway. So python source will be distributed.

Diafero is right: for some changes, it will not be possible (or practical) to completely avoid distributing copyrighted material. It is only a question of how big the material is.

Edit: after consideration, I don't see why not eventually provide support for diffs for text files, providing I can find a good command-line diff tool for Windows that doesn't come with all kinds of dependencies. The tool must be able to generate and apply diff files.
This will make processing python files more complex, since I will now have to unpack, decompyle, modify, recompyle, repack. Also more error-checking.
This will go on my to-do list, but copyright protection is not very high in my personal priorities.
Chacal


"The weak can never forgive. Forgiveness is an attribute of the strong."
-- Mahatma Gandhi
User avatar
Chacal
 
Posts: 2443
Joined: Tue Nov 06, 2007 2:45 pm
Location: Quebec, Canada

Re: Presenting UruDistro

Postby GregW11 » Tue Jul 07, 2009 11:02 pm

As long as no one's making money off of it, techincally companies wouldn't waste the time to prosecute such a thing, especially since you'd need a purchased copy of Uru anyway to do this =/

Sounds interesting, I'm getting tired of forgetting that the Wall in Gahreesen causes Uru to exception every time without fail (most likely random scraps of code from Prologue that were forgotten). While it wouldn't amount to much, since you'd need two teams to make the Wall function at all, it'd be nice to see it up close. Looking in the folders makes it seem like all the code to make the wall work was functional, there were some particular avatar animations for falling off ladders if I remember right.
GregW11
 
Posts: 96
Joined: Sun Aug 03, 2008 10:15 pm

Re: Presenting UruDistro

Postby Chacal » Wed Jul 08, 2009 12:00 am

GregW11 wrote:I'm getting tired of forgetting that the Wall in Gahreesen causes Uru to exception every time without fail (most likely random scraps of code from Prologue that were forgotten). While it wouldn't amount to much, since you'd need two teams to make the Wall function at all, it'd be nice to see it up close.


Oh, just delete the plSceneObject "SfxRegSen-MaintOnStone01" from Garrison_District_WallRoom.prp, using prpExplorer 0.6 and import the object in the zip file attached to this post. Resum Garrison.sum using Plasmashop.

Yes, that's the kind of thing UruDistro is made for.
Attachments
SfxRegSen-MaintOnStone01.zip
(458 Bytes) Downloaded 200 times
Chacal


"The weak can never forgive. Forgiveness is an attribute of the strong."
-- Mahatma Gandhi
User avatar
Chacal
 
Posts: 2443
Joined: Tue Nov 06, 2007 2:45 pm
Location: Quebec, Canada

Next

Return to UruDistro

Who is online

Users browsing this forum: No registered users and 1 guest