10,000 faces max export?

[tachzusamm]

The worst-case scenario is completely non-contiguous UV maps, meaning each triangle is its own UVW "island". In such a case one unique vertex has to be generated for each corner of each triangle, resulting in a vertex count of 3xtotal_triangles.

I just remembered - the original PyPRP exporter had buggy code where every triangle would have its own vertices.

I just made a test.

Added a flat plane, subdivided, and converted Quads to Triangles, and got 16384 Triangles.
On export, the number of vertices was 49152, so exactly 3*16384.

Another plane, with 16383 Quads, got exported to 65532 vertices, which is 4*16383.

A third, mixed plane, with 10240 Tris + 5120 Quads, exported to 51200 vertices (3*10240 + 4*5120).

Okay, seems I was wrong assuming there's some "projection" made. It's just 4 vertices per Quad, and 3 per Triangle. Always, it seems.

But, wait, you want to say that this is sub-optimal? Can graphics cards really "share" vertices? I was under the assumption the hardware always needs faces or vertices separated.

[Aloys]

For what my opinion is worth on this subject:
Having a mesh with 35000++ vertices is a bad idea in the first place. Not that it would bring any computer to its knees; but it certainly entices modelers to be more lazy; and that is a not a good thing. Optimising run time rendering in Plasma with vis regions, occluders and whatnot is a bit of a pain, but this is a set of skills that has to be learned to create large/detailed Ages. And in this regard upping the limit of vertices per object is exactly what shouldn't be done.
In any case even if someone somehow needs to have a very large mesh in an Age (landscape etc) it takes litterally 10 seconds to separate it into several meshes..

So, if anything needs to be done IMO it's simply to change the error message to something more explicit.

It's just 4 vertices per Quad, and 3 per Triangle. Always, it seems.

Does it mean Plasma (or PyPRP?) does not make any use of triangle strips? That would be a bad thing.

[Branan]

Plasma uses indexed rendering, which is how the GPU likes to look at data. Hardware hasn't natively supported triangle strips for over a decade. Internally the driver will convert them to indexed batches.

In indexed rendering, you have two lists of data: One is a list of unique vertices, and one is a list of triangles. So if I have a Quad, like so:

Code: Select all

._.
|\|
.-.

It'll have 4 vertices: <0,0> <0,1> <1,0> and <1,1>

When that's converted to triangles during export, the exporter will create 6 indices, in two groups of three: <0,1,2> <2,1,3>. On all but the most recent hardware these are (as I said earlier) unsigned shorts. Bytes are usually supported, but that limits you to 256 unique vertices.

In short: we only push the four unique vertices to the GPU, and tell it how to put those vertices together into triangles.

When you're dealing with normals and UV maps, it's not just the position that goes into making a vertex unique, of course. If there was a UV seam between the two triangles, it wouldn't be able to share those vertices since they'd have a different UVW component. (which is what tachzusamm was saying earlier).

EDIT: The point of this being that the exporter should be looking at triangles and creating *unique* vertices. That is, only duplicating them per-triangle if there's a discontinuity in the UVW or Normal data. Duplicating them for every triangle is a "shortcut", but it's inefficient.

EDIT2: Regarding really big meshes... GPUs these days can push millions of triangles, and they prefer batches of several thousand if you can manage it. This is why most engines sort by *material* rather than by object - they can merge multiple objects that share one material into a large batch for the GPU.

[Aloys]

Yeah, I'm not up to date on the exact low level hardware optimization methods. Thanks for the lesson that's informative.

The point of this being that the exporter should be looking at triangles and creating *unique* vertices.

Yes, it should; but reading Tachzusamm post it doesn't seem to do it. Or am I reading that wrong?

Regarding really big meshes... GPUs these days can push millions of triangles, and they prefer batches of several thousand if you can manage it.

I know, and in this regard objects of 30,000 tris is not really a problem; but I see that limit as a good way to educate Age creators in watching their polycounts. Maybe I'm just being over-zealous though.

[Branan]

The point of this being that the exporter should be looking at triangles and creating *unique* vertices.

Yes, it should; but reading Tachzusamm post it doesn't seem to do it. Or am I reading that wrong?

You're reading it correctly. It cheats and assumes every face requires each vertex to be unique. In most cases this is probably wrong, and explains part of why PyPRP generates such large mesh data files.

Regarding really big meshes... GPUs these days can push millions of triangles, and they prefer batches of several thousand if you can manage it.

I know, and in this regard objects of 30,000 tris is not really a problem; but I see that limit as a good way to educate Age creators in watching their polycounts. Maybe I'm just being over-zealous though.

Well, it depends on what you're trying to say. If they just split one big mesh into several smaller ones, performance will go down because of the smaller batches. If they actually lower their polycount, performance might improve.

[Trylon]

The point of this being that the exporter should be looking at triangles and creating *unique* vertices.That is, only duplicating them per-triangle if there's a discontinuity in the UVW or Normal data. Duplicating them for every triangle is a "shortcut", but it's inefficient.

Yes, it should; but reading Tachzusamm post it doesn't seem to do it. Or am I reading that wrong?

You're reading it correctly. It cheats and assumes every face requires each vertex to be unique. In most cases this is probably wrong, and explains part of why PyPRP generates such large mesh data files.

On the other hand, I took a quick peek at the Drawable Export code yesterday, and as far as I can tell, it does just what it's supposed to do: Push out a list of vertices, and then a list of indices. I couldn't find any hint of code that pours out vertices for each face. Which fit's which what I remember of rewriting that code - as far as I can remember, I took care of using indices and unique vertices except for UV discrepancies.

EDIT:
P.S.
By default, PRP also uses vertex compression, to minimize impact.

EDIT2:
Just took a good look through the code again, and whaddyaknow, I was partly mistaken.
In exporting the vertices ndeed are processed on a per face basis - potentially allowing for vertices to be duplicated, BUT:
After a vertex has been processed, the code will look through the list of previously processed vertices to see if a vertex fully equal to this one has already been processed. If so, it isn't added again, but the original one is used in the indexing.

Now, there could be a bug in the code that checks if two vertices are equal, but other than that, the code is there....

EDIT3:
HEre is he relevant code....

The Vertex Class (eqality check)

Code: Select all

class Vertex:
    def __init__(self,x=0,y=0,z=0):
        self.x=x
        self.y=y
        self.z=z
        #normals
        self.nx=0
        self.ny=0
        self.nz=0
        #Base vertex (x=px + offset/1024.0), so ((x-px)*1024 = offset)
        ##self.base=[0,0,0]
        ##self.off=[0,0,0]
        #weights - bone influence (0-1)
        self.blend=[]
        ###self.blendbase=[]
        ###self.blendoff=[]
        #bones - link to bone
        self.bones=[]
        #color
        self.color=RGBA()
        #Texture coordinates
        self.tex=[]
        ###self.texbase=[]
        ###self.texoff=[]


    def isequal(self,v):
        return(self.x==v.x and self.y==v.y and self.z==v.z)

    def isfullyequal(self,v):
        # this check goes in order of importance, to ensure quick loading

        # check vertex location
        if not (self.x==v.x and self.y==v.y and self.z==v.z):
            return False

        # check all uv coordinates
        if len(self.tex) != len(v.tex):
            return False
        else:
            for i in range(len(self.tex)):
                if not (self.tex[i][0] == v.tex[i][0] and self.tex[i][1] == v.tex[i][1]):
                    return False

        # check vertex color

        if not self.color.equals(v.color):
            return False

        # check blend weights
        if len(self.blend) != len(v.blend):
            return False
        else:
            for i in range(len(self.blend)):
                if not (self.blend[i] == v.blend[i]):
                    return False

        # check skin indices
        if len(self.bones) != len(v.bones):
            return False
        else:
            for i in range(len(self.bones)):
                if not (self.bones[i] == v.bones[i]):
                    return False

        # check vertex normal
        if not (self.nx==v.nx and self.ny==v.ny and self.nz==v.nz):
            return False

        # if we arrive here, the vertex is the same
        return True


Excerpt from he DrawableInterface Class"

Code: Select all

        for mface in mesh.faces:
            if (len(mface.verts) < 3) or (len(mface.verts) > 4):
                continue # ignore this face (too many verts)

            vertIdxs = []
            baseVertexIdx = len(materialGroups[mface.mat]["vertices"])

            index = 0
            for vector in mface.verts:
                # convert to Plasma vertex
                v = Vertex()

                # coordinates
                v.x = vector.co[0]
                v.y = vector.co[1]
                v.z = vector.co[2]

                # normal
                v.nx = vector.no[0]
                v.ny = vector.no[1]
                v.nz = vector.no[2]

                # vertex colors
                if mesh.vertexColors:
                    if len(ColorLayers) > 0:
                        try:
                            col_a=255
                            for vc in range(len(ColorLayers)):
                                if(ColorLayers[vc].lower() == "col"): #Mesh vertex colours
                                    mesh.activeColorLayer = ColorLayers[vc]
                                    col_r=mface.col[index].r
                                    col_g=mface.col[index].g
                                    col_b=mface.col[index].b
                                    #if col_a >= 1.0: #See if there is alpha on the colour map
                                    #    col_a = mface.col[index].a
                                elif(ColorLayers[vc].lower() == "alpha"): #Mesh vertex alpha
                                    mesh.activeColorLayer = ColorLayers[vc]
                                    col_a = (mface.col[index].r + mface.col[index].g + mface.col[index].b) / 3.0
                                    materialGroups[mface.mat]["vtxalphacol"] = True
                                    blendSpan = True

                            v.color = RGBA(col_r,col_g,col_b,col_a)
                        except IndexError:
                            pass

                # Blend weights.
                if weightCount > 0:
                    bone,weight = mesh.getVertexInfluences(vector.index)
                    v.blends = [weight,]

                # UV Maps
                for uvlayer in UVLayers:
                    mesh.activeUVLayer = uvlayer

                    tex_u = mface.uv[index][0]
                    tex_v = 1-mface.uv[index][1]
                    v.tex.append([tex_u,tex_v,0])

                # Sticky Coordinates Next
                if materialGroups[mface.mat]["Sticky"]:
                    sticky = [ vector.uv[0], 1 - vector.uv[1], 0 ]
                    v.tex.append(sticky)

                v_idx = -1 # to avoid unneccesary copying of vertices

                # This adds a really long waiting time...
                #Anyone know of a better way of doing this? >.>
                if True:
                    # see if we already have saved this vertex
                    try:
                        VertexDict = materialGroups[mface.mat]["vtxdict"]
                        for j in VertexDict[v.x][v.y][v.z]:
                            vertex = materialGroups[mface.mat]["vertices"][j]

                            if vertex.isfullyequal(v):
                                # if vertex is the same, set index to that one
                                v_idx = j
                                break # and end the search
                    except:
                        pass

                # if vertex is unique, add it
                if v_idx == -1:
                    materialGroups[mface.mat]["vertices"].append(v)
                    v_idx = len(materialGroups[mface.mat]["vertices"]) -1

                    # Store this one in the dict
                    VertexDict = materialGroups[mface.mat]["vtxdict"]
                    if not VertexDict.has_key(v.x):
                        VertexDict[v.x] = {}
                    if not VertexDict[v.x].has_key(v.y):
                        VertexDict[v.x][v.y] = {}
                    if not VertexDict[v.x][v.y].has_key(v.z):
                        VertexDict[v.x][v.y][v.z] = []
                    VertexDict[v.x][v.y][v.z].append(v_idx)

                # and store the vertex index in our face list
                vertIdxs.append(v_idx)
                index += 1

            if len(mface.verts) == 3:
                materialGroups[mface.mat]["faces"].append(vertIdxs)
            elif len(mface.verts) == 4: # a quad must be separated into two triangles
                materialGroups[mface.mat]["faces"].append([vertIdxs[0],vertIdxs[1],vertIdxs[2]]) # first triangle
                materialGroups[mface.mat]["faces"].append([vertIdxs[0],vertIdxs[2],vertIdxs[3]]) # second triangle



[tachzusamm]

I did some quick debugging attempts with debug outputs to a text file.
The method "isfullyequal" is called often when exporting an age, but *never* returned True.
My guess is that some of the various checks simply cannot evaluate to a valid condition.
Hopefully I find time to track the bug later - but I have to stop for now (shopping for BBQ today )


Okay, update:
isfullyequal always exits with False on this check:

Code: Select all

        # check vertex color

        if not self.color.equals(v.color):
            return False


I currently don't see what's wrong with this.
Maybe the "equals" method in class RGBA is wrong? Shouldn't it get "self" as first parameter as well?


Update 2:
At least this seem to solve it.
I replaced:

Code: Select all

    def equals(col):
        if self.r == col.r and self.b == col.b and self.g == col.g and self.a == col.a:
            return True
        else:
            return False


with this:

Code: Select all

    def equals(self,col):
        if self.r == col.r and self.b == col.b and self.g == col.g and self.a == col.a:
            return True
        else:
            return False


in prp_GeomClasses.py, class RGBA (line 247)

Now vertices seem to be compared correctly - but I would feel better if you could confirm that this is a correct bugfix, Trylon - or any other person of the PyPRP script developers.

Cheers, tach

[Trylon]

Yeah, that's definitely a valid bug. Good catch!
The fix is correct too AFAICT

That should make the exports a bit smaller - though perhaps it might cause a change in the appearance of the meshes.

[D'Lanor]

A bit smaller? I tried this fix and depending on the age I am getting a 25% to 75% decrease in prp size! Great find! And so far I have not noticed any visual differences.

[Trylon]

Cool.