youtu-graphrag
youtu-graphrag copied to clipboard
大规模数据集的构建知识图谱的"超级节点"时候,如何很好的解决OOM的问题
[2025-10-26 14:35:51] ERROR main:159 - Failed to build knowledge graph for 数据集: CUDA out of memory. Tried to allocate 43.43 GiB. GPU 0 has a total capacity of 79.15 GiB of which 26.50 GiB is free. Process 1537359 has 52.64 GiB memory in use. Of the allocated memory 52.16 GiB is allocated by PyTorch, and 8.62 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 尝试过更频繁的内存清理,checkpoint,混合精度,(尝试多张卡并行似乎不太可行?),求助有效的解决方案谢谢