Depthmap failure - Error: CUDAError: driver shutting down

[Mother-Woolf]

Describe the problem Red bar when trying to compute depthmap. All seems fine in all the nodes before this but I am consistently getting failures on computing the depthmap on any attempt. This is even happening on previously successful models (ie. I was able to fully complete a render last week but when opening the same save file and rerendering it fails on the depthmap). I have updated my CUDA drivers too. I have multiple warnings regarding tile buffer size: Each time I change a setting and try to compute again the warning for file buffer size gives a lower and lower measurement for maximum image size. I had previously experienced failed depthmap computions but this was fixable by increasing downscale value to 8 rather than 4.

I'm totally new to figuring out such issues so please explain it like I'm five wherever possible. Thank you!

Log [2024-02-25 17:25:31.660214] [0x00002180] [trace] Embedded OCIO configuration file: 'C:\Users\Woolfrey\Downloads\Meshroom-2023.3.0\aliceVision/share/aliceVision/config.ocio' found. Program called with the following parameters:

• autoAdjustSmallImage = 1
• chooseTCamsPerTile = 1
• colorOptimizationEnabled = 1
• colorOptimizationNbIterations = 100
• customPatchPatternGroupSubpartsPerLevel = 0 (default)
• customPatchPatternSubparts = Unknown Type "class std::vector<struct aliceVision::depthMap::CustomPatchPatternParams::SubpartParams,class std::allocator<struct aliceVision::depthMap::CustomPatchPatternParams::SubpartParams> >" (default)
• downscale = 4
• exportIntermediateCrossVolumes = 0
• exportIntermediateDepthSimMaps = 0
• exportIntermediateNormalMaps = 0
• exportIntermediateTopographicCutVolumes = 0
• exportIntermediateVolume9pCsv = 0
• exportIntermediateVolumes = 0
• exportTilePattern = 0
• imagesFolder = "C:/Users/Woolfrey/Desktop/Artwork/Photogrammetry/WalkHome/meshroom/MeshroomCache/PrepareDenseScene/20e773532cd435772417714b4d4a5855a4b95380"
• input = "C:/Users/Woolfrey/Desktop/Artwork/Photogrammetry/WalkHome/meshroom/MeshroomCache/StructureFromMotion/bb915e686cca6d7d78240df7894cfc4c3d1d645a/sfm.abc"
• maxCoresAvailable = Unknown Type "unsigned int" (default)
• maxMemoryAvailable = 18446744073709551615 (default)
• maxTCams = 10
• maxViewAngle = 70
• minViewAngle = 2
• nbGPUs = 0
• output = "C:/Users/Woolfrey/Desktop/Artwork/Photogrammetry/WalkHome/meshroom/MeshroomCache/DepthMap/66ad77765264a229999dc738674acb73eae0c556"
• rangeSize = 12
• rangeStart = 0
• refineEnabled = 1
• refineGammaC = 15.5
• refineGammaP = 8
• refineHalfNbDepths = 15
• refineInterpolateMiddleDepth = 0
• refineMaxTCamsPerTile = 4
• refineScale = 1
• refineSigma = 15
• refineStepXY = 1
• refineSubsampling = 10
• refineUseConsistentScale = 0
• refineUseCustomPatchPattern = 0
• refineWSH = 3
• sgmDepthListPerTile = 1
• sgmDepthThicknessInflate = 0
• sgmFilteringAxes = "YX"
• sgmGammaC = 5.5
• sgmGammaP = 8
• sgmMaxDepths = 1500
• sgmMaxSimilarity = 1
• sgmMaxTCamsPerTile = 4
• sgmP1 = 10
• sgmP2Weighting = 100
• sgmScale = 2
• sgmSeedsRangeInflate = 0.2
• sgmStepXY = 2
• sgmStepZ = -1
• sgmUseConsistentScale = 0
• sgmUseCustomPatchPattern = 0
• sgmUseSfmSeeds = 1
• sgmWSH = 4
• tileBufferHeight = 1024
• tileBufferWidth = 1024
• tilePadding = 64
• verboseLevel = "info"

Hardware : Detected core count : 8 OpenMP will use 8 cores Detected available memory : 11445 Mo

[17:25:31.666210][info] CUDA-Enabled GPU. Device information: - id: 0 - name: NVIDIA GeForce GTX 1080 Ti - compute capability: 6.1 - clock frequency (kHz): 1657500 - total device memory: 11263 MB - device memory available: 10200 MB - per-block shared memory: 49152 - warp size: 32 - max threads per block: 1024 - max threads per SM(X): 2048 - max block sizes: {1024,1024,64} - max grid sizes: {2147483647,65535,65535} - max 2D array texture: {131072,65536} - max 3D array texture: {16384,16384,16384} - max 2D linear texture: {131072,65000,2097120} - max 2D layered texture: {32768,32768,2048} - number of SM(x)s: 28 - registers per SM(x): 65536 - registers per block: 65536 - concurrent kernels: yes - mapping host memory: yes - unified addressing: yes - texture alignment: 512 byte - pitch alignment: 32 byte CUDA build version: 11.6 [17:25:31.864183][info] Supported CUDA-Enabled GPU detected. [17:25:34.731106][info] Found 1 image dimension(s): [17:25:34.731106][info] - [478x850] [17:25:35.038919][info] Overall maximum dimension: [478x850] [17:25:35.038919][warning] Tile buffer size (width: 1024, height: 1024) is larger than the maximum image size (width: 119, height: 212). [17:25:35.038919][warning] Single tile computation, override SGM maximum number of T cameras per tile (before: 4, now: 10). [17:25:35.038919][warning] Single tile computation, override Refine maximum number of T cameras per tile (before: 4, now: 10). [17:25:35.038919][warning] Single tile computation, override SGM step XY (before: 2, now: 1). [17:25:35.038919][info] Tiling information: - parameters: - buffer width: 1024 px - buffer height: 1024 px - padding: 64 px - maximum downscale: 2 - maximum image width: 119 px - maximum image height: 212 px [17:25:35.038919][info] Maximum image size is smaller than one tile, use only one tile. [17:25:35.038919][info] SGM parameters: - scale: 2 - stepXY: 1 [17:25:35.038919][info] Refine parameters: - scale: 1 - stepXY: 1 [17:25:35.038919][info] Number of GPU devices: 1, number of CPU threads: 8 [17:25:35.248862][info] DepthMapEstimator::getNbSimultaneousTiles(): limit the number of simultaneous RC due to the max constant memory for camera params from 9 to 4 [17:25:35.248862][info] Device memory: - available: 8160 MB - requirement for the first tile: 855.043 MB - # computation buffers per tile: 851.867 MB (Sgm: 757.867 MB, Refine: 94 MB) - # input images (R + 10 Ts): 3.17583 MB (single mipmap image size: 0.288712 MB) [17:25:35.248862][info] Parallelization: - # tiles per image: 1 - # simultaneous depth maps computation: 4 - # simultaneous tiles computation: 4 [17:25:35.325813][info] Device memory (device id: 0): - used: 4489.75 MB - available: 6774 MB - total: 11263.8 MB

CUDAError: no error file: C:\Release\sources\AliceVision\src\aliceVision\depthMap\cuda\imageProcessing\deviceMipmappedArray.cu function: aliceVision::depthMap::cuda_createMipmappedArrayFromImage line: 327

================================================================================ ====================== Command line failed with an error =======================

CUDA Error: no error file: C:\Release\sources\AliceVision\src\aliceVision\depthMap\cuda\imageProcessing\deviceMipmappedArray.cu function: aliceVision::depthMap::cuda_createMipmappedArrayFromImage line: 327

================================================================================

CUDAError: driver shutting down file: C:\Release\sources\AliceVision\src\aliceVision\depthMap\cuda\host\DeviceMipmapImage.cpp function: aliceVision::depthMap::DeviceMipmapImage::~DeviceMipmapImage line: 25

Desktop (please complete the following and other pertinent information):

• OS: [e.g. win 10, osx, ]
• Python version [e.g. 2.6]
• Qt/PySide version [e.g. 5.15.2]
• Meshroom version: please specify if you are using a release version or your own build
  • Binary version (if applicable) [e.g. 2019.1]
  • Commit reference (if applicable) [e.g. 08ddbe2]

Additional context I'm not sure why it refers to CUDA version 11.6 no matter what my CUDA version is (currently 12.3) The last time this process worked for me was before these windows updates were installed: February 13, 2024—KB5034763 (OS Builds 19044.4046 and 19045.4046)