godot
godot copied to clipboard
Adding a translation CSV results in errors on initial import for many types of resources
Tested versions
Reproducible in 4.3.beta [811ce36c6]
System information
Windows 11 (10.0.22631)
Issue description
(I believe this is a duplicate of #92262, but there wasn't much of an MRP provided in that issue.)
When a project includes a "CSV Translation" file, that file will emit .translation
files (one for every language/locale) next to the .csv
file. The writing of these .translation
files will (in the same callstack) seemingly cause every scene in the project to be loaded, which in turn will try to load every resource referenced by those scenes, most of which will (on initial import) fail, since many of these resources won't have been imported at that point.
The type of resource matters to some degree, due to the ordering of the reimports that happens here:
https://github.com/godotengine/godot/blob/811ce36c6090013f1a48676c0388892bf1621288/editor/editor_file_system.cpp#L2660
... as this will sort the imports by the importer name (ResourceImporter::get_importer_name()
), where "csv_importer"
will often get sorted first, before things like "texture"
, like in the provided MRP.
Here is the callstack when one of these texture resources fail to load due to not having been imported:
ResourceLoader::_load(const String &p_path, const String &p_original_path, const String &p_type_hint, ResourceFormatLoader::CacheMode p_cache_mode, Error *r_error, bool p_use_sub_threads, float *r_progress) (core/io/resource_loader.cpp:283)
ResourceFormatImporter::load(const String &p_path, const String &p_original_path, Error *r_error, bool p_use_sub_threads, float *r_progress, ResourceFormatLoader::CacheMode p_cache_mode) (core/io/resource_importer.cpp:154)
ResourceLoader::_load(const String &p_path, const String &p_original_path, const String &p_type_hint, ResourceFormatLoader::CacheMode p_cache_mode, Error *r_error, bool p_use_sub_threads, float *r_progress) (core/io/resource_loader.cpp:269)
ResourceLoader::_thread_load_function(void *p_userdata) (core/io/resource_loader.cpp:333)
ResourceLoader::_load_start(const String &p_path, const String &p_type_hint, ResourceLoader::LoadThreadMode p_thread_mode, ResourceFormatLoader::CacheMode p_cache_mode) (core/io/resource_loader.cpp:535)
ResourceLoaderText::load() (scene/resources/resource_format_text.cpp:472)
ResourceFormatLoaderText::load(const String &p_path, const String &p_original_path, Error *r_error, bool p_use_sub_threads, float *r_progress, ResourceFormatLoader::CacheMode p_cache_mode) (scene/resources/resource_format_text.cpp:1392)
ResourceLoader::_load(const String &p_path, const String &p_original_path, const String &p_type_hint, ResourceFormatLoader::CacheMode p_cache_mode, Error *r_error, bool p_use_sub_threads, float *r_progress) (core/io/resource_loader.cpp:269)
ResourceLoader::_thread_load_function(void *p_userdata) (core/io/resource_loader.cpp:333)
ResourceLoader::_load_start(const String &p_path, const String &p_type_hint, ResourceLoader::LoadThreadMode p_thread_mode, ResourceFormatLoader::CacheMode p_cache_mode) (core/io/resource_loader.cpp:535)
ResourceLoader::load(const String &p_path, const String &p_type_hint, ResourceFormatLoader::CacheMode p_cache_mode, Error *r_error) (core/io/resource_loader.cpp:454)
EditorFileSystem::_get_scene_groups(const String &p_path) (editor/editor_file_system.cpp:1939)
EditorFileSystem::_update_scene_groups() (editor/editor_file_system.cpp:1895)
EditorFileSystem::_update_pending_scene_groups() (editor/editor_file_system.cpp:1914)
EditorFileSystem::_process_update_pending() (editor/editor_file_system.cpp:1854)
EditorFileSystem::update_files(const Vector<String> &p_script_paths) (editor/editor_file_system.cpp:2076)
EditorFileSystem::update_file(const String &p_file) (editor/editor_file_system.cpp:1946)
EditorNode::_resource_saved(Ref<Resource> p_resource, const String &p_path) (editor/editor_node.cpp:6261)
ResourceSaver::save(const Ref<Resource> &p_resource, const String &p_path, unsigned int p_flags) (core/io/resource_saver.cpp:149)
ResourceImporterCSVTranslation::import(const String &p_source_file, const String &p_save_path, const HashMap<StringName,Variant,HashMapHasherDefault,HashMapComparatorDefault<StringName>,DefaultTypedAllocator<HashMapElement<StringName,Variant>>> &p_options, List<String,DefaultAllocator> *r_platform_variants, List<String,DefaultAllocator> *r_gen_files, Variant *r_metadata) (editor/import/resource_importer_csv_translation.cpp:146)
EditorFileSystem::_reimport_file(const String &p_file, const HashMap<StringName,Variant,HashMapHasherDefault,HashMapComparatorDefault<StringName>,DefaultTypedAllocator<HashMapElement<StringName,Variant>>> &p_custom_options, const String &p_custom_importer, Variant *p_generator_parameters) (editor/editor_file_system.cpp:2434)
EditorFileSystem::reimport_files(const Vector<String> &p_files) (editor/editor_file_system.cpp:2716)
EditorFileSystem::_update_scan_actions() (editor/editor_file_system.cpp:779)
EditorFileSystem::_notification(int p_what) (editor/editor_file_system.cpp:1465)
EditorFileSystem::_notificationv(int p_notification, bool p_reversed) (editor/editor_file_system.h:140)
Object::notification(int p_notification, bool p_reversed) (core/object/object.cpp:873)
SceneTree::_process_group(SceneTree::ProcessGroup *p_group, bool p_physics) (scene/main/scene_tree.cpp:965)
SceneTree::_process(bool p_physics) (scene/main/scene_tree.cpp:1042)
SceneTree::process(double p_time) (scene/main/scene_tree.cpp:528)
Main::iteration() (main/main.cpp:4099)
(Note that I disabled import/use_multiple_threads
in the MRP to make debugging more straight-forward, hence the callstack, but that setting is inconsequential to this issue.)
As far as I can tell (after bisecting) this is a regression in 4.3, caused by #60965, as EditorFileSystem::_update_pending_scene_groups
loading all the scenes in EditorFileSystem::_process_update_pending
is largely what's causing this problem.
However, it does seem to me like the CSV importer itself is also part of the problem here, as every other import artifact that triggers EditorNode::_resource_saved
will simply early out in EditorFileSystem::_find_file
here due to those import artifacts being in the .godot
folder, as opposed to being generated next to the source file like with .csv
files.
Steps to reproduce
- Ensure there is no existing
.godot
folder in the MRP - Open the MRP in the editor
- Note errors about failing to open/load
logo1.png
,logo2.png
and their respective import artifacts - Close the project
- Open the project again
- Note the lack of errors