SkyrimSE-Linux-Modding icon indicating copy to clipboard operation
SkyrimSE-Linux-Modding copied to clipboard

SSELodgen failing to complete

Open ecbfry opened this issue 5 years ago • 17 comments

I notice you list SSELodgen as working in the wiki - for me this always seems to get stuck on large worldspaces (Tamriel, Solstheim, Sovngarde), but does complete without errors on smaller areas e.g. mod added.

Do you have any tips on versions/settings used?

ecbfry avatar May 05 '19 11:05 ecbfry

I just tried to generate LOD for Tamriel and it worked fine for me. Took around 5 minutes. I'm not sure what the issue could be for you. It does take some time to generate on some systems. If you check htop (or something similar) is the process doing anything? i.e CPU usage.

Does it get stuck on trees or objects? Does it freeze at a certain point? LODGen is very verbose and is outputting a lot of messages, maybe there's an error?

You can also try to run it this way: STEAM_COMPAT_DATA_PATH=~/.steam/steam/steamapps/compatdata/489830/ ~/.steam/root/compatibilitytools.d/skyrim-proton/proton waitforexitandrun [PATH TO EXE]/SSELODGen.exe

spooknik avatar May 05 '19 13:05 spooknik

Thanks, running with proton with the command you suggested seems to be working! Previously I was running using wine and had tried various builds, but never tried proton. I'm not sure why this would be - but I do notice that with proton, all my CPU cores are being used, whereas with stock wine I only ever had 1 core running at 100%, and it seemed to get stuck and never complete.

I also ran into the problem with Texconv failing with an error "Invalid value specified with -m (0)" - as you have described with DynDOLOD. This was with xLODGEN beta 42. Using xLODGEN beta 38 ran OK. I'm using BC1 texture formats.

ecbfry avatar May 06 '19 10:05 ecbfry

Glad to hear that worked. That was the point of getting xterm (or terminal) to launch from Steam when you click 'play' on Skyrim SE, you can just run whatever .exe you need for modding Skyrim via proton and its own wineprefix for Skyrim.

Yea, Texconv.exe doesn't play nice with Wine and the way it handles certain Windows C and C++ applications (See here) . Sheson updated DynDOLOD to try to be more friendly with wine, but even with the most current version (2.61) it still fails when trying to generate mipmaps. Time will tell if we'll get a fix for this. It's at the top of my 'to solve' list because I really want to use DynDOLOD it's a huge upgrade even over SSELODGen.

spooknik avatar May 06 '19 11:05 spooknik

It seems I can run DynDOLOD 2.54 - tree LOD generates OK, but object LOD fails with error messages along the lines of: "Error reading meshes\lod\nordicexterior\nortempleexteriorribfree03_lod.nif from BSA/BA2 Frame checksum is invalidlz4". This happens for every worldspace - the files referenced do appear to be ones that are packed in BSAs, not loose files. There don't seem to be any errors earlier when the BSAs are loaded.

DynDOLOD does nevertheless finish running and generates the esp and esm, which work in game, though I assume I'm only getting the benefit of tree LOD.

DynDOLOD 2.61 fails with the error about mipmaps.

ecbfry avatar May 06 '19 16:05 ecbfry

Oh that's interesting. Would you mind sharing the files for 2.54? I'd like to test this out myself. i can report this to Sheson and he might be able to patch it in the next version.

In regards to this error message, you might have to install Visual C++ 2015 or 2017. You can do this in a few ways. Easiest is to just install is via winetricks. So the command would be something like:

WINEPREFIX=~/".steam/steam/steamapps/compatdata/489830/pfx" winetricks vcrun2015

spooknik avatar May 06 '19 17:05 spooknik

DynDOLOD-Standalone.2.54 DynDOLOD-Resources-SE.2.45 Both from December 2018

It's possible I made an error with the version numbers when I was running it earlier - they're quite confusing (standalone and resources are not necessarily the same). I'll try again later with the 2.45 resources to see if that solves the error, if not I'll try installing vcrun2015 / vcrun2017.

ecbfry avatar May 06 '19 18:05 ecbfry

Thanks for the links, I couldn't find them.

Okay, so 2.54 runs but still generates the error with Texconv.exe. It just carries on and doesn't halt the whole program right away... It still eventually fails though because it's trying to generate mipmaps... so the work around it seems is to set

StaticLODDiffuseFormat=87
StaticLODNormalFormat=88
TreeLODDiffuseFormat=87
TreeLODNormalFormat=88

in DynDOLOD_SSE.ini. Found in DynDOLOD/Edit Scripts/DynDOLOD

This means that the textures DynDOLOD will not be compressed and will be very large. So the next step is figuring out how to compress them.

I did get a similar error in the terminal while DynDOLOD was running:

lz4
Error reading meshes\terrain\Sovngarde\Sovngarde.4.4.20.btr from BSA/BA2 Invalid IL code in <Module>:LZ4_decompress_safe_continue (LZ4_streamDecode_u*,sbyte modopt(System.Runtime.CompilerServices.IsConst) modopt(System.Runtime.CompilerServices.IsSignUnspecifiedByte)*,sbyte modopt(System.Runtime.CompilerServices.IsSignUnspecifiedByte)*,int,int): IL_00e7: shr.un 

But DynDOLOD did complete okay. So... not sure if that's a problem or not. There is no meshes in Output/Meshes/Terrian/[Worldspace] so perhaps no terrain objects are being generated. Needs more testing in game, but it's late and i'll check it out tomorrow.

spooknik avatar May 06 '19 20:05 spooknik

Where are you seeing the mipmap related errors? I run TexGenx64.exe with the following settings:

Diffuse Alpha format: BC1 (DXT1)
Diffuse format: BC1 (DXT1)
Normal Specular format: BC3 (DXT5)
Normal format: BC1 (DXT1)

and it seems to run fine.

The errors I get running DynDOLODx64 are when it's trying to generate object LOD, and fails to read *.nif files (or fails to validate them with a checksum). Like you, I have no files in meshes/terrain/worldspace/objects, but I do have files in meshes/terrain/worldspace/trees, and in textures/terrain/worldspace/trees. Changing DynDOLOD resources and installing vcrun2017 don't seem to have made any difference.

ecbfry avatar May 06 '19 21:05 ecbfry

I get the error: Invalid value specified with -m (0) somewhere in the log when Texconv.exe runs. The -m is the argument for mipmaps with Texconv. I talked with sheson and because of the error with mipmaps he suggested using uncompressed textures because this forces Texconv not to make mipmaps. I think i'll have to experiment a little bit.

However.. the bigger issue at hand is with this type of error:

Error reading meshes\terrain\Sovngarde\Sovngarde.4.4.20.btr from BSA/BA2

I believe for whatever reason it's not able to open and read BSA/BA2. I tried googling this error and nothing really comes up. I'll have to ask sheson if he knows anything about it.

However a work around is to extract everything to the data folder and just have it loose so it doesn't have to search inside an archive for it. BAE runs well under wine. You need the meshes/lod and meshes/terrain folders to be loose. Then it shouldn't need to read anything from an archive.

Note: When DynDOLOD runs on large worldspaces proton will error with: eventfd: Too many open files. Fix is here.

spooknik avatar May 07 '19 05:05 spooknik

I’m not seeing Texconv run when I use DynDOLOD 2.54, and I can’t see the mipmap error in the log. I’m running TexGen with the settings I mentioned above and DynDOLOD on ‘medium’.

My error when trying to read meshes is also slightly different (see above), and I’ve now seen it with a loose file, so I’m not sure it is the BSA extraction step - it still gives the error ‘Checksum is invalid’. Both your error message and mine mention lz4 decompression, so maybe it’s a bug with this - but this is all beyond my knowledge! Perhaps It will make sense to Sheson.

ecbfry avatar May 07 '19 07:05 ecbfry

@spooknik Did you ever resolve the issue with Texconv.exe failing under wine? It now supports WSL2 under windows, maybe that somehow helps our linux cause?

JanThomas118 avatar Mar 20 '21 14:03 JanThomas118

I wanted to leave a note that I'm running xLODGen for terrain LOD now under wine/proton via Mod Organizer 2. Prior to the current run, I'd get a lot of the Invalid values specified with -m errors from Texconvx64.exe starting shortly after the terrain LOD generation would start. So far it appears that upgrading texconv to the latest DirectXTex binary from Microsoft resolved the issue (at least for the 64-bit progs); I replaced xLODGen/Edit Scripts/Texconvx64.exe with the texconv.exe from https://github.com/microsoft/DirectXTex/releases/tag/apr2021 and xLODGen has run for 12 minutes so far without error (lots of "Finished LOD level..." messages, but not a single error or printing out of texconv's help message).

For a bit of further detail, I'm running the terrain LOD generation according to https://stepmodifications.org/wiki/SkyrimSE:1.0.0 as I have the full set of mods from there plus some expansion mods like Ordinator added in.

kira78 avatar Apr 26 '21 13:04 kira78

@kira78 which version of xLODGen did you use? When I start xLODGen64.exe beta 81 (and 77), it instantly crashes on startup with an EAccessViolation.

cverstege avatar May 19 '21 09:05 cverstege

I got it working guys! xLODGen and DynDOLOD! U need:

  • xLODGen beta 84 from https://stepmodifications.org/forum/topic/13451-xlodgen-terrain-lod-beta-84-for-fnv-fo3-fo4-fo4vr-tes5-sse-tes5vr-enderal-enderalse/
  • DynDOLOD Standalone 2.96 beta from https://stepmodifications.org/forum/topic/13894-dyndolod-beta-for-skyrim-special-edition-skyrim-vr-and-enderal-se-296/

U have to patch both tools' "Edit Scripts" folder. Download the 3.00 Alpha of DynDOLOD https://stepmodifications.org/forum/topic/15606-dyndolod-300-alpha-53/ and copy and replace following files in the Edit Scripts folder into both the xLODGen beta 84 and DynDOLOD Standalone 2.96 Edit Scripts folder:

  • 7z.dll
  • 7z64.dll
  • LODGen.exe
  • LODGenx64.exe
  • Texconv.exe
  • Texconvx64.exe

Then xLODGen beta 84 and DynDOLOD Standalone 2.96 beta should work!

coajaxial avatar Oct 21 '21 23:10 coajaxial

Can yopu make a video guide on how to do this and does it work with the newest version of dyndolod aka 3.00?

KnightJeffrey avatar Jan 13 '22 06:01 KnightJeffrey

@KnightJeffrey Not sure if you figured it out yet, but yes, the same method works with 3.00 Resources when using the 2.96 beta. Reddit Post right here has a little bit more info on how to install it, but still based off of this Github post.

FormBurden avatar Jul 13 '22 00:07 FormBurden

Is there an update for this. I am using DynDOLOD 3.00 Alpha 155 for Skyrim SE. After a while, when TexGenx64 generates the textures, it just freezes and stops generating TexConv processes. The whole thing can be observed within the first 5 minutes.

ErikSteiner avatar Oct 13 '23 19:10 ErikSteiner