Need some libPlasma tools

[diafero]

I'm not yet sure whether it can be actually useful. I never really liked manually mounting stuff, and for unmounting the FUSE filesystem I even need to be root (or is there some other way to unmount it without "umount"?). But it is definitely cool, and if it can actually replace the process of unpacking and re-packing prp files, it should also save some time when hacking in prp files. So I indeed vote for adding some more read/write support

[Zrax]

I'm not yet sure whether it can be actually useful. I never really liked manually mounting stuff, and for unmounting the FUSE filesystem I even need to be root (or is there some other way to unmount it without "umount"?). But it is definitely cool, and if it can actually replace the process of unpacking and re-packing prp files, it should also save some time when hacking in prp files. So I indeed vote for adding some more read/write support

"fusermount -u" should allow you to unmount any FUSE filesystem without needing root privileges. Like I say, it's mostly a toy project, but if it ends up being useful to someone, that's even better

[Chacal]

Thanks a lot Zrax!
I've begun testing this new tool. Here's a few bugs that I found. I hope this is useful.

Bug01 (may not be a bug)
As previously noticed, modifying a prp will change the size of drawablespans.
Here's my test results, I don't know if this is normal or not.
Steps:
1- Execute "prppack -x Abysos_District_Mainroom.prp" to extract all objects from the prp into .po files in a subfolder.
2- Execute "prpmod abysos.prp del plGenericPhysical:Cave1" to delete Cave1 from the prp.
3- Execute again "prppack -x Abysos_District_Mainroom.prp" to extract all objects from the prp into .po files in a subfolder.
4- Compare the contents of the two subfolders.

Result:
- object Cave1 is deleted as expected
- the file size of the prp file has dropped from 1530K to 623K
- plDrawableSpan objects have shrunk to less than half their size
- other unrelated objects have lost some bytes: all plCoordinateInterface objects and the one plDirectionalLightInfo object in the prp.
- examination of lamp.po using a hex editor shows this: before running PrpMod there were seven 00 bytes after the Urustring for "lamp", after running PrpMod there are only four such bytes.

test.jpg (140.31 KiB) Viewed 5055 times

Abysos is a fan Age, it is possible that the texture compression was incorrect in the first place.
Repeating the test with the Gira prp from Cyan results in a smaller shrinkage: from 7193K to 7189K.

Bug02
Starting with a clean copy of Abysos:
1- Execute "prpmod abysos.prp extract plGenericPhysical:Cave1" to extract the object into file "[003F]Cave1.po."
2- execute "prpmod abysos.prp add [003F]Cave1.po" to add the object even though it already exists.
This results in an error trapped by PrpMod and displayed in the console:

* Loading: mainRoom (Abysos)
Location: <16858|0>
Version: PotS/CC
pdUnifiedTypeMap::PlasmaToMapped ERR: Got type 2100
pdUnifiedTypeMap::PlasmaToMapped ERR: Got type 2100
pdUnifiedTypeMap::ClassName ERR: Got type FFFF
pdUnifiedTypeMap::PlasmaToMapped ERR: Got type 2100
Warning: NOT reading type [FFFF](null)
Failure parsing [003F]Cave1.po

Repeating the test and first deleting the object before adding it results in the same error.
Repeating the test with the Gira prp from Cyan results in the same error.



Bug03
Starting with a clean copy of Gira:
1- Execute "prpmod gira.prp extract plGenericPhysical:Bone05 --prc" to extract the object into file "[003F]Bone05.prc."
2- execute "prpmod gira.prp add [003F]Bone05.prc" to add the object even though it already exists.
This results in a crash; console shows no error, only the normal output:
prpmod Gira.prp add [003F]Bone05.prc
* Loading: giraCanyon (Gira)
Location: <24|1>
Version: PotS/CC

The size of the prp file has INCREASED from 7193K to 7195K.
Repeating the test and first deleting the object before adding it give the same results.

All tests ran on Windows Vista 64bits Business edition with MS C++ Runtime library 9.
Same results with XP 32 bits SP3.

[Zrax]

Thanks for the report, Chacal

Regarding Bug #1: The recompiling of DrawableSpans is a known issue, and I do actually have a fix in the works for it. However, the issue you're seeing here is more likely due to the fact that, AFAIK, PyPRP still doesn't compress verts by default. libPlasma, on the other hand, always does (including if it read a file with decompressed verts). The fix I mentioned above will still compress verts if they're read uncompressed, but I may include a config option to some of the tools to specify writing uncompressed vert buffers... The other file size differences, however, are not actually bugs, but optimizations that libPlasma does automatically -- specifically, Cyan frequently writes empty hsBitVectors with a size of 1 and content of { 0 }... libPlasma compacts the BitVector before writing it, resulting in a size of 0 and content of { }. Generally, if libPlasma reads the original with no errors, and reads the modified object also with no errors, the object is probably fine, even if it's not in precisely the same format as the original. Also note that, while Geometry Buffer recompression *is* done by libPlasma, Texture recompression is *not* (which is the correct behavior), so your textures should be unaffected, unless you change them.

Regarding Bug #2: That is definitely a problem . From what you describe, I think I might be trying to parse the object as a PRP, which is, of course, not the correct behavior... I'll look into it, and make sure there aren't any other issues with replacing an existing object.

Regarding Bug #3: Again, I'll look into what's going on here... This could either be a bug in the GenericPhysical itself, or an issue with not actually deleting an object that was marked for deletion. Either way, I'll look into it and let you know what I find.

Thanks again!

EDIT: Maybe I should make a libPlasma bug tracker, heh...

[Zrax]

Ok, the Vertex Buffer recompression issue should be fixed now... Unless you modify (or re-PRC-compile) the DSpan object, the compressed buffer should remain untouched. If the source buffer is uncompressed, libPlasma will still compress it for you, but it won't touch compressed buffers anymore unless it needs to

[Chacal]

Confirmed. The shrinkage for Abysos is now 4K, like with Gira.
Thanks!

[Zrax]

Ok, bugs 2 and 3 should be fixed now, as well as a couple of other things I found... Let me know how it works for you.

[Chacal]

Actually no, but the files in the archive are dated 2009-07-01 13:13, which is before the version that fixed bug #1.

[Zrax]

You probably have the download cached... I checked the copy on the server, and it is the correct one (from 7/2 4:00 PM).

[Chacal]

Got it. So far so good, all tests perform flawlessly.
Great job, Zrax! It is now much easier to hack prp files.
My distribution system is almost ready, this is what I was waiting for.

Not a bug probably, but I have this weird behavior:
1- Extract an object into a prc file using prpmod extract --prp
2- Import it again using prpmod add. Works fine.
3- Modify the prc file using UltraEdit32
4- Import it again using prpmod add. Works fine but adds message: WARN: Ignoring extraneous token ´╗┐

This is repeatable.

Looking at the two prc files with WinMerge and with a hex editor, there is no difference between the files. Not a single byte.
Weird, isn't it?

Suggestion: by default, prcdc and prcc write an output file called "out.prc" or "out.po".
It would be better if the default name was changed to be the same as prpmod and PageEditor: [003F]Bone05.prc or [003F]Bone05.po.
Makes it easier in scripts because we don't have to calculate the output file name.