Light baking in Korman

Announcements and discussion regarding any projects related to Cyan Worlds' Plasma Engine including (but not limited to) CyanWorlds.com Engine, Drizzle, OfflineKI, PyPRP, and libHSPlasma.

Light baking in Korman

Postby Sirius » Fri Feb 05, 2016 5:51 pm

Greetings !

I noticed something that looks like a "Big Bad Problem" with how Korman handles light baking. Maybe one of the guys who know Korman best might be able to confirm this...

I'm currently working on my Canyon Age in Blender with Korman. As you might remember, I also use my own plugin for baking light in a scene, which I made shortly before Korman was first released. Up until recently, I was still using this script alongside with Korman, because it felt a bit more convenient for various reasons (better handling of objects with modifiers being one).
Then a few days ago I decided I'd stop using my script and let Korman do most of the baking stuff on its own (mostly due to my script not being compatible with how Korman stores textures for consecutive exports, which means longer export times).

Now, my Age has a lot of objects in it (some of them pretty big), which must all be baked, using mainly vertex shading and a few lightmaps. It's a perfect test file to test how fast a script can process a big scene.
However, my Age never exports when I tell Korman to bake lighting - most of the time, Blender is stuck at exporting, which means I cancel the export after 15 minutes.
On the other hand, my lightbaking script can rebake the whole vertex shading in under 60 seconds. Then Korman can export it in less than 30 seconds. Huh, quite a difference.

Actually, I found out why tonight by tweaking my own script. The problem lies in how Korman processes light baking. When, to be more precise.

What my script does:
  • select all objects which must be baked
  • call bpy.ops.object.bake_image()
You can see bake_image is called only once, and it processes all selected objects in a row.

What Korman does (from what I've seen):
  • pick an object in the scene
  • call bpy.ops.object.bake_image()
  • export the object
  • pick an object in the scene
  • call bpy.ops.object.bake_image()
  • export the object
  • ...

Exporting the object being processed takes little time. The problem is we call bake_image several times.
Now, why is it bad ? Let's take a look at what bake_image() does internally:
  • 1. Prepare scene data (single threaded): this step puts the whole rendered geometry in memory, which sometime involve recomputing modifiers, etc.
  • 2. Build Raytree (single threaded): computes the path of shadows for the whole scene.
  • 3. Actual rendering (multi-threaded): for each selected object, computes color for each vertex (or when dealing with lightmaps, "compute each pixel in each texture applied to each object").
  • 4. Trigger scene update (I'm not sure about that one, but most operators do)

As you might be guessing now, the difference is that my script triggers step 1, 2 and 4 only once - Korman triggers it once per object in my scene. Step 1 and 2 alone take roughly 10 seconds. Now, imagine those 10 seconds multiplied by the number of objects in the scene (more than 700). Uh oh, this means theoretically it would take roughly 2 hours to bake.

BTW, this is actually simple to verify: have Korman export a big scene. Because step 1 and 2 are single threaded, your CPU usage will rarely rise above 100/(num cores) percent. Now, if you instead select all objects in your scene and hit "Bake", you'll notice your CPU usage being at 100% most of the time, indicating most of the computing is done in step 3.



Anyhow, on to the big question.
Korman is designed to process meshes and light groups in a specific order, because translating Blender data to Plasma is a complex process.
Would it be feasible to call bake_image only once or twice per page ? :?
User avatar
Sirius
 
Posts: 1506
Joined: Mon Jul 26, 2010 4:46 am
Location: France

Re: Light baking in Korman

Postby Tsar Hoikas » Fri Feb 05, 2016 7:06 pm

Yikes! I had never tested baking tons and tons of lighting, but that is the expected use case. In Korman, I tend to prefer a clean design over tricks for the sake of speed, but I've been doing a bit of profiling to speed things up in certain tight areas. I had noticed that updating the scene was slow and tweaked the lighting code a bit already, I just hadn't realized how slow bake_image is.

I can spend some time refactoring the lightmap modifier for v0.04 to work more like your script. It already has some weird special cases and fragile code, so it needs some TLC. I was also planning on allowing the artist to select which layers to bake, so this is a win-win. There's still some room for speed improvements in other parts of the code that I plan on addressing as well :)
Image
Tsar Hoikas
Councilor of Technical Direction
 
Posts: 2180
Joined: Fri Nov 16, 2007 9:45 pm
Location: South Georgia

Re: Light baking in Korman

Postby Sirius » Sat Feb 06, 2016 11:40 am

Tsar Hoikas wrote:It already has some weird special cases and fragile code, so it needs some TLC. I was also planning on allowing the artist to select which layers to bake, so this is a win-win.
At least you won't have to refactor this code twice ! ;)

For now I'm using my script for vertex light, and I let Korman bake lightmaps, since there aren't many in my Age... Cheers !
User avatar
Sirius
 
Posts: 1506
Joined: Mon Jul 26, 2010 4:46 am
Location: France

Re: Light baking in Korman

Postby Tsar Hoikas » Sun Feb 14, 2016 7:10 pm

OK, I believe I have addressed this in #30. https://github.com/H-uru/korman/pull/30
Image
Tsar Hoikas
Councilor of Technical Direction
 
Posts: 2180
Joined: Fri Nov 16, 2007 9:45 pm
Location: South Georgia

Re: Light baking in Korman

Postby Sirius » Mon Feb 15, 2016 2:01 pm

It exported in about 1 minute 30 seconds, so good job ! :D even though my log file says it took 400 seconds to complete. What the heck.

My Age was quite dark as the sun was left out of the scene lighting, but that sounds like a wrong lightgroup on my end :)
User avatar
Sirius
 
Posts: 1506
Joined: Mon Jul 26, 2010 4:46 am
Location: France


Return to Plasma Development

Who is online

Users browsing this forum: No registered users and 4 guests