Optimizer Improvements

[StandingPadAnimations]

Opening this to keep track of the improvements I'm thinking of adding to the optimizer (though these are mostly ideas). I'll open a pull request once the next version of MCprep (which I assume is 3.4.0) is released

So far my ideas are:

• [ ] Overloading the F12 key to do a quick optimization pass before rendering
• [x] Automatic scrambling distance (as an unsafe but reversible feature)
• [ ] Removing mesh lights (as an unsafe and irreversible feature)
• [ ] Expanding on the previous idea, adding lights to replace mesh lights (as an unsafe and irreversible feature. This seems like the best way to handle this for now as EEVEE has a light limit)
• [ ] Removing unused nodes from any MCprep materials (as an unsafe and irreversible feature which would be enabled by default and would also work in EEVEE)
• [ ] Use existing materials when prepping (mostly useful when a user deletes and imports a newer version of the same OBJ)
• [x] Use of homogeneous volumes when possible
• [x] Setting volumetric stepping rate based on how volumes are being used
• [x] Having "True Glossy" and "True Transmissive" counters for PBR materials to further improve the light bounce optimizations

I think I'll work from simple to difficult and see how it goes

[TheDuckCow]

Sounds good @StandingPadAnimations - I would recommend a couple practical points for development progress:

• Create branches directly in this MCprep repo vs others, it makes it easier for me to checkout your code and test deploy (yes I can do the same with adding additional remotes, it just gets finicky). Only you specifically have this permission, being added as a direct github collaborator, so use your power wisely!
• Create branches for individual, complete features; this is something I'm going to start doing more myself. So from your check list above, I could see that being 8 different branches and thus 8 different pull requests (maybe some are related enough that they fit together and it's only 5 branches). This means you can have more of your work merged in pieces while other parts are still in progress, otherwise you get stuck with an all-or-nothing merge which is tricky and just delays the goodness to users.
• No reason to delay work until the 3.4.0 release (unless you want to ofc). I see you know how to rebase back to dev so you'd just do that after a release or otherwise just periodically.
• Agreed with moving from simple to difficult! Like I said, pick at most 1 or 2 items that are related and make that its own branch to be reviewed on its own. If it's fast enough, might even make it in for 3.4.0 (but I won't delay waiting for it).

As for some specific points or ideas on the optimizations above:

• Removing lamps: Irreversible yes, but I think what you'd actually want to do is make the object emissive just to the camera (and possible reflection?), but then otherwise treat it as a solid shader for other light passes. This would work for Cycles, and for Eevee either way there's not much cost to having it be emitting since someone might still want to use probes, so adding lights further around the blocks is a bonus
• Overloading the F12 : I'm weary auto running code just before render, especially since someone might not always be working on Minecraft scenes :) It could lead to unexpected results. If it's off by default and needs to be enabled, how different is that from just having the user press optimize scene the one time anyways? Unless you are thinking some functions need continually re-running?
• Removing unused nodes: Yeah I agree a post-processor could be good, particular for image nodes. This could be added as a part of the prep materials operator without much consequence.
• For light bounce and volumetric numbers, do we think there's really computational answer to these? The "right" number would change even depending on camera angle. I personally think we are better off just coming up with a good enough average case number and setting that static number.

[StandingPadAnimations]

For light bounces, I was thinking of using variables like "True glossy" and "JSON glossy" and then averaging them out if they're both different and if we're dealing with PBR materials. JSON glossy would be using the JSON file and True glossy would be based on if a material has a specular map and the JSON file.

For volumetrics, it's easier as we're simply adjusting the stepping rate. It would start at something like 5, and then for every volume node without a texture controlling the density we'll increase by 2. For any volume node with a texture, we'll decrease by 2 (basically tug-of-war but with volumetrics). I was also thinking of something similar for automatic scrambling distance but with things it doesn't like dealing with (such as volumetrics)

[StandingPadAnimations]

"True Glossy" and "True Transmissive" counters have been replaced with a system of "if the value is the default set by MCprep, then simply don't count it. If it uses a map, then count it"

[StandingPadAnimations]

One more thing I want to add to the list: reusing already existing materials

I've noticed that when MCprep preps materials, it creates duplicates. Currently the optimizer deals with this by purging orphan data, but it would be simpler to just reuse existing materials during the prepping process

Of course, this is less of an optimizer thing and more of a prepping thing

[TheDuckCow]

Under what circumstances is prep materials making duplicates? Hm that doesn't sound right, should modify existing materials. May be worth looking into how to resolve this directly. Also be aware of the combine materials feature that already exists, could be that optimizer calls this, but it can also have unintended consequences (eg if you swap skins but don't rename the material. Hm maybe swap skins SHOULD attempt to rename the material idk)On Jul 2, 2022 3:50 PM, StandingPad @.***> wrote: One more thing I want to add to the list: reusing already existing materials I've noticed that when MCprep preps materials, it creates duplicates. Currently the optimizer deals with this by purging orphan data, but it would be simpler to just reuse existing materials during the prepping process Of course, this is less of an optimizer thing and more of a prepping thing

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

[StandingPadAnimations]

Prep materials seems to make duplicates whenever I import a new OBJ or even the same OBJ. Here's what I mean:

1. I import the OBJ once

2. I import the same OBJ and prep. Despite combine materials being enabled (btw, I'm curious as to why it's disabled by default), all the materials are indexed and I see both duplicates. The optimizer gets around this by purging orphan data twice (which is the amount of times needed to remove duplicate materials) but it's 2 extra calls that have to be done

In particular, it's annoying when I do a quick change to something, reimport the same OBJ and have to set up stuff like water shaders (which is made more annoying when the optimizer purges orphan data)

[StandingPadAnimations]

I'll be looking into enabling Fast GI for the next update. The main reason it's not used right now is because I'm not sure if it impacts scenes too much. I'll have to do some tests to see how it compares and if it's a good tradeoff

What I'll likely do is enable Fast GI for CPU/OpenCL users by default and make it optional

[StandingPadAnimations]

One other change I might do is actually not mess with caustics for Blender 3.0+. In Blender 3.0 it seems like caustics have been improved performance wise, and they have a benefit with reflective and refractive materials.

As such, it just makes sense to not mess with caustics settings, but again this will require some testing on my end.

I've been watching a video on Cycles optimization and it seems like Cycles X changed a few things, so expect some differences for 3.0+

[TheDuckCow]

Ah yes, importing new obj's is just using blenders OBJ importer which indeed itself will generate new materials. It's actually an interesting idea that we could specifically look at only the newly generated materials during import, and seek to replace those with prior matching generalized names pre import. I don't have combine mats on by default because there can be artifacts, namely for the reason I mentioned where if you select everything and happen to have rigs there with different skins applied, you can accidentally merge two intentionally different materials together. That too probably could be made smarter to dub dub checks between two materials to be merged to see if they have the same source image name (even if the path is different.. since it could be the copied MCprep resource pack vs the original export next to the obj).Indeed data would be cleaned up after opening and reloading blender. I don't purge right away as the user may not be expecting that to happen and they have have other materials or unused data they aren't expecting to lose. If we do include that, there should be a warning at least or a way to skip the purge part.On Jul 2, 2022 4:41 PM, StandingPad @.***> wrote: Prep materials seems to make duplicates whenever I import a new OBJ or even the same OBJ. Here's what I mean:

I import the OBJ once

I import the same OBJ and prep. Despite combine materials being enabled (btw, I'm curious as to why it's disabled by default), all the materials are indexed and I see both duplicates. The optimizer gets around this by purging orphan data twice (which is the amount of times needed to remove duplicate materials) but it's 2 extra calls that have to be done

In particular, it's annoying when I do a quick change to something, reimport the same OBJ and have to set up stuff like water shaders (which is made more annoying when the optimizer purges orphan data)

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

[StandingPadAnimations]

Closing as not planned