Package Editor: edit/save/load Custom Triangle Sorting-array

[TranscendedOne]

I'm trying to convert a number of Sims 4-hairstyles to Mass Effect Legendary Edition.

There are a bunch of hoops needing to jump through in order to convert them properly, and one of them is the fact that there's apparently a difference with how Sims and UE3 handle Translucency-sorting.

Usually when we look at a (semi-) transparent mesh in-game, or in a tool like Blender or 3DS Max, everything is rendered in a proper order. However, when converting from Sims to UE3 (ME(LE)), the proper order is somehow lost in translation, and the following issue arises: Image Link

Notice how the hair ends have hard pixelated edges. This is because the game renders the hair strips behind them as if they were in front of them, but ONLY where the front hair-strips are translucent (hair diff's alpha channel).

The issue can be resolved with Translucent Hair Sorting from within UDK 3: https://docs.unrealengine.com/udk/Three/TranslucentHairSorting.html

With the TRISORT_Custom option you can manually move hair strips in front or behind others, and with TRISORT_CustomLeftRight you can have it dependant on from which direction you're looking at the mesh (though I'm not sure CustomLeftRight works properly in MELE).

The annoying thing about this, is that when there's a need to adjust the mesh in something like Blender, the sorting thing needs to be redone too, and it can become quite time-consuming that way. UDK doesn't offer a viable option to just copy the Custom sorting array, so my request thus becomes:

A way to copy/paste/save/load/edit the Custom Triangle Sorting array from within Package Editor.

Thanks!

[Mgamerz]

This will be part of the features we are looking into implementing after we push the next stable.

[TranscendedOne]

Looking forward to it, thanks!

[ElysiumFic]

Possibly relevant to this discussion...

Disclaimer: I don't actually know what I'm doing or what a lot of the tools are doing; I'm just sort of fumbling through trying to find what works.

In the process of poking at Blender and UDK to try to work out how to create my own hair mod conversions from Sims mods, I stumbled upon some interesting stuff.

It seems like we're sort of dealing with the mesh, material, and texture elements of these mods all separately in the external tools, like Blender and UDK, and then putting them together in Legendary Explorer. But there are settings and properties important for the material component of these mods that don't appear to be available in Legendary Explorer.

So, let's say I have a mesh that I created in Blender, and I want to import it to UDK in order to package it in such a way that I can import it in Legendary Explorer, as described in this tutorial.

(First of all, I will note that instead of exporting a .psk from Blender and importing it into UDK, it appears possible to export an .fbx from Blender and that the .fbx can have material and texture included already, and that seems like a simpler way to go, but that's outside the scope of this discussion, for the moment, since we're dealing with what the tutorial I linked above is recommending, that being the approach most people are going to be taking, I imagine.)

Importing the .psk itself is pretty straightforward, but if we want to see the mesh with its texture (in order to spot those transparency sorting issues more easily) we would then import the texture, as either a .png or .tga, and in the process, we get a lot of options thrown at us:

Going simply on guesswork since there's no tutorial for this portion of things, I check the following options you see in the screenshots above:

Alpha to Opacity Create Material? RGB To Diffuse

I also select "BLEND_Translucent" in the blending dropdown (though I'm not sure if that is right, maybe "BLEND_AlphaComposite" is more appropriate?)

This creates a texture and material in the project in UDK, and double-clicking on the material gives a whole menu of options as well:

Again, going on guesswork alone due to lack of guidance, I'm assuming most of these are safely ignored, but that "Translucency" section is relevant to this discussion in particular:

"Two-Sided Separate Pass" indicates that it's used for hair rendering, so this seems like we need to check it.

"Allow Fog" is checked by default, and I think that may be important, because one of the biggest issues I personally have had with converted hair mods is that they look weird anywhere where there is smoke or fog. They just don't render properly so the hair looks too solid for being in a smoky area.

The tooltip for "Translucency receive dominant shadows from static" has a tooltip that, again, indicates its necessary for hair rendering. Probably needs to be checked.

The tooltip for "Allow translucency DoF" indicates that its necessary for items with "BLEND_translucent" selected. Which we did, so we probably want to check that too.

The tooltip for "Use Lit Translucency Depth Pass" specifically refers to hair sorting, so this I'm almost absolutely certain needs to be checked.

"Use Lit Translucency Post Render Depth Pass" indicates its necessary for fog and DoF of hair, which we've already called out as being an issue so yes, let's check that, too.

So, we save that, then we go into AnimSet Editor and sort our triangles.

Then, if we save a .upk from UDK, it contains a material file and texture, and if you look at the .upk in Package Edtor in Legendary Explorer, you see there's a whole bunch of properties in the material file that aren't available in the material files we see with vanilla hairs in game. There doesn't appear to be any way to import this material with all these properties to the material being used with our mesh we're importing from the .upk, either. (I tried exporting the binary from the .upk and then importing that into the .pcc, but that doesn't work.)

So it seems like all those settings are getting lost, because they're not stored in the mesh, but in the material.

Since things like the translucency sorting and rendering hair in fog aren't a problem for vanilla hairs in the game, I assume these options are not stored in the material for those hairs, but wherever they are stored, they don't get applied to any non-vanilla hairs that are created. I don't think? (Again, my apologies, working completely on guesswork here.)

The same applies for Blender. There are some relevant properties for transparency sorting in the material properties that we can set for a mesh, but no way to import them along with the mesh.

I assume if we set these properties in Blender, then exported the project as a .fbx and imported that .fbx into UDK, those properties would come along for the ride if we checked the relevant boxes upon import:

So it seems like maybe a lot of what we need isn't necessary an option for sorting (though that is necessary too) but an option to import or set these material properties? Maybe?

[Mgamerz]

A lot of stuff here to go over, but a few starting points:

We cannot import materials from UDK. Materials are tied to shaders, and we can't compile shaders, nor are the ones compiled by UDK compatible with our games.

Some properties in UDK aren't available in LE games as the engine versions are different.

Hair has always had weird translucent issues, even in vanilla.

The mesh data is mostly what our import from udk is for.

There's more to post but it's pretty late here so I'll look it over tomorrow.

On Fri, Jan 28, 2022, 8:43 PM ElysiumFic @.***> wrote:

Possibly relevant to this discussion...

Disclaimer: I don't actually know what I'm doing or what a lot of the tools are doing; I'm just sort of fumbling through trying to find what works.

In the process of poking at Blender and UDK to try to work out how to created my own hair mod conversions from Sims mods, I stumbled upon some interesting stuff.

It seems like we're sort of dealing with the mesh, material, and texture elements of these mods all separately in the external tools, like Blender and UDK, and then putting them together in Legendary Explorer. But there are settings and properties important for the material component of these mods that don't appear to be available in Legendary Explorer.

So, let's say I have a mesh that I created in Blender, and I want to import it to UDK in order to package it in such a way that I can import it in Legendary Explorer, as described in this tutorial https://github.com/ME3Tweaks/LegendaryExplorer/wiki/How-to-port-a-hairstyle-from-the-Sims-4#porting-in-blender .

(First of all, I will note that instead of exporting a .psk from Blender and importing it into UDK, it appears possible to export an .fbx from Blender and that the .fbx can have material and texture included already, and that seems like a simpler way to go, but that's outside the scope of this discussion, for the moment, since we're dealing with that the tutorial I linked above is recommending, as that is the approach most people are going to be taking, I imagine.)

Importing the .psk itself is pretty straightforward, but if we want to see the mesh with its texture (in order to spot those transparency sorting issues more easily) we would then import the texture, as either a .png or .tga, and in the process, we get a lot of options thrown at us:

[image: udk1] https://user-images.githubusercontent.com/18093925/151645713-1d0b8cb5-90d2-472b-8af4-23e7396daed0.jpg

Going simply on guesswork since there's no tutorial for this portion of things, I check the following options you see in the screenshots above:

Alpha to Opacity Create Material? RGB To Diffuse

I also select "BLEND_Translucent" in the blending dropdown (though I'm not sure if that is right, maybe "BLEND_AlphaComposite" is more appropriate?)

This creates a texture and material in the project in UDK, and double-clicking on the material gives a whole menu of options as well:

[image: udk2] https://user-images.githubusercontent.com/18093925/151645720-8e365df3-65ac-4ecd-8ae8-8016a7f8991d.jpg

Again, going on guesswork along due to lack of guidance, I'm assuming most of these are safely ignored, but that "Transparency" section is relevant to this discussion in particular:

[image: udk3] https://user-images.githubusercontent.com/18093925/151645727-41b83b36-173a-4979-9aaf-994a9d8fc538.jpg

"Two-Sided Separate Pass" indicates that it's used for hair rendering, so this seems like we need to check it.

[image: udk4] https://user-images.githubusercontent.com/18093925/151645735-9c39b8c0-9300-40d7-80fb-96f12aa111af.jpg

"Allow Fog" is checked by default, and I think that may be important, because one of the biggest issues I personally have had with converted hair mods is that they look weird anywhere where there is smoke or fog. They just don't render properly so the hair looks too solid for being in a smoky area.

[image: udk5] https://user-images.githubusercontent.com/18093925/151645738-5e0387f1-e68b-4787-913e-8a4742f190f4.jpg

The tooltip for "Translucency receive dominant shadows from static" has a tooltip that, again, indicates its necessary for hair rendering. Probably needs to be checked.

[image: udk6] https://user-images.githubusercontent.com/18093925/151645740-bf86fb0f-5b8b-47df-b809-6fe571078e8d.jpg

The tooltip for "Allow translucency DoF" indicates that its necessary for items with "BLEND_translucent" selected. Which we did, so we probably want to check that too.

[image: udk7] https://user-images.githubusercontent.com/18093925/151645749-0ffb80ef-a852-4c89-a87f-8499120b5c5e.jpg

The tooltip for "Use Lit Translucency Depth Pass" specifically refers to hair sorting, so this I'm almost absolutely certain needs to be checked.

[image: udk8] https://user-images.githubusercontent.com/18093925/151645753-68bd81a3-a3e4-4063-8a08-2598a3454acd.jpg

"Use Lit Translucency Post Render Depth Pass" indicates its necessary for fog and DoF of hair, which we've already called out as being an issue so yes, let's check that, too.

So, we save that, then we go into AnimSet Editor and sort our triangles.

Then, if we save a .upk from UDK, it contains a material file and texture, and if you look at the .upk in Package Edtor in Legendary Explorer, you see there's a whole bunch of properties in the material file that aren't available in the material files we see with vanilla hairs in game. There doesn't appear to be any way to import this material with all these properties to the material being used with our mesh we're importing from the .upk, either. (I tried exporting the binary from the .upk and then importing that into the .pcc, but that doesn't work.)

So it seems like all those settings are getting lost, because they're not stored in the mesh, but in the material.

Since things like the translucency sorting and rendering hair in fog aren't a problem for vanilla hairs in the game, I assume these options are not stored in the material for those hairs, but wherever they are stored, they don't get applied to any non-vanilla hairs that are created. I don't think? (Again, my apologies, working completely on guesswork here.)

The same applies for Blender. There are some relevant properties for transparency sorting in the material properties that we can set for a mesh, but no way to import them along with the mesh.

[image: blender3] https://user-images.githubusercontent.com/18093925/151645843-fbd5553f-0c3a-489d-848d-d951823b7ab9.jpg [image: blender4] https://user-images.githubusercontent.com/18093925/151645844-fc3c0994-4e9a-4139-b355-07091cb1b6d9.jpg

I assume if we set these properties in Blender, then exported the project as a .fbx and imported that .fbx into UDK, those properties would come along for the ride if we checked the relevant boxes upon import:

[image: udk9] https://user-images.githubusercontent.com/18093925/151645970-c8cfb491-2ccb-463e-91cd-720bae7c7908.jpg

So it seems like maybe a lot of what we need isn't necessary an option for sorting (though that is necessary too) but an option to import or set these material properties? Maybe?

— Reply to this email directly, view it on GitHub https://github.com/ME3Tweaks/LegendaryExplorer/issues/282#issuecomment-1024824355, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU4VFDMTG6NUBD4C6Y4ENTUYNO67ANCNFSM5FIAMDRA . You are receiving this because you modified the open/close state.Message ID: @.***>

[ElysiumFic]

A few months on, I'm more familiar with Blender and UDK now, and since Blender is open source, I think I may have found a clue as to how to (somewhat reliably) accomplish this a bit easier than in UDK. Especially since we already have a visual rendering of the mesh in Mesh Explorer.

In Edit Mode of Blender, there is a Sort Elements option. I've been experimenting with this, and have found that I can generally do a "pre-clean" of a mesh so that it imports into UDK a lot less messed up. There may still be some fine-tuning/tweaking required, but not nearly as much.

Basically, UDK is rendering vertices in order from highest index to lowest index. So the element that has a vertex 0 will be rendered "in front" of the element with vertex 15000. Somehow, these vertices end up scrambled and that is why we get "rear" (i.e. should be less visible) strands being rendered in front of the "front" strands that should be more visible.

So the strand of hair that is right smack over the middle of the forehead should definitely have the lowest indices. And while it may seem logical that the one on the nape of the neck should have the highest indices, that may not be the case. Because if the strand at the nape has another strand hidden under it, then THAT strand should actually have the highest indices.

There are two approaches to get to the same place using "Sort Elements" in Blender. The first is to get up "under" the mesh and select the least visible strand of the entire mesh, then choose "Mesh -> Sort Elements -> Selected." What this will do is put all the vertices on that strand in first places (i.e. starting at 0). Then I hide that strand, and select the second least visible strand and do the same thing (which will then change the indices of the first strand so that they come after the indices of the second strand.) Rinse and repeat until the final strand remaining is what should be the most visible strand (i.e. the one right in the middle of the forehead). So the strands will then be sorted in ascending order from most visible to least visible.

However, since it's harder to visualize what should be most/least visible working from the inside out, the other option is to start with the most visible strand and do the same steps as above, hiding the strands like removing layers of an onion from the outside inward. Then when everything is done (with the least visible strand having the lowest indices), unhide all the strands, select the entire mesh, and choose "Mesh -> Sort Elements -> Reverse."

There may still be a few "jig saw" spots when viewing the mesh in UDK, where a few vertices are out of order, but it won't be nearly as bad.

Doing it this way, I was able to import a VERY complex hair into UDK almost entirely correctly sorted:

The few pieces that you see aren't correct were me incorrectly prioritizing which strand should come after which, but compared to what it looked like when imported into UDK before I tried sorting it in Blender first:

So maybe what's in Mesh Explorer is some way to make the visual representation of the mesh 1) more accurately reflect what the result will be when it's rendered, and 2) more interactive, so we can select a strand (i.e. all the linked vertices, like CTRL-L does in Blender when you click on a face/edge/vertex) and designate them as being high priority/high visibility, and then recalculate the indices accordingly.