draft solution for corrupted file while renaming when using Docker volumes on Windows
When running the library on a Docker volume with a Windows host, there seems to be some problem when the helper thread creates the Segment and the mmap that belongs to it. The mmap holds a file open, which leads to a file appearing to be corrupted when trying to rename it from another thread (this happens when creating closed-* files). This issue appears only in this specific setup. Even running a bare example with Rust on Windows doesn't allow to reproduce the issue.
Here is the code that allows to reproduce the situation:
use std::fs::{self, OpenOptions};
use std::thread;
use std::time::Duration;
use memmap2::MmapOptions;
use std::io::Write;
fn main() -> std::io::Result<()> {
// Spawn a new thread that creates and maps a file
let handle = thread::spawn(|| {
// Create and write to a file
fs::write("file.txt", b"Hello, World!").unwrap();
// Open the file with read and write options
let file = OpenOptions::new()
.read(true)
.write(true)
.create(false)
.open("file.txt").unwrap();
// Memory-map the file
let mmap = unsafe {
MmapOptions::new()
.map_mut(&file)
.expect("Failed to map the file")
};
println!("File is mapped.");
thread::sleep(Duration::from_secs(2));
// Drop (close) the file handle
drop(file);
println!("File handle is dropped.");
// leave a thread running
thread::sleep(Duration::from_secs(120));
});
// Give the other thread a head start to ensure the file is created and mapped
thread::sleep(Duration::from_secs(10));
// Rename the file
fs::rename("file.txt", "renamed_file.txt").unwrap();
println!("File is renamed.");
// Keep the main thread alive to observe the behavior
thread::sleep(Duration::from_secs(120));
Ok(())
}
When trying to list the file from some other terminal, it will appear as it is "corrupted".
(in the example, the file renamed_file.txt would be in this state)
This will happen when running on a Docker volume mounted on Windows.
I implemented a simple solution that involves dropping the mmap before renaming the file and the reloading it again on the main thread. I'm not sure whether it's the most clean way, but I'm not proficient with Rust yet so any suggestions on how to improve it are welcome.
Would adding some tests using Docker for it be required? I can see there is one workflow on Windows machine, but I'm not sure if the overhead for that isn't too big.
@generall
After a few more tests it looks like calling .flush before unloading the mmap allows dropping the file handle immediately after unload, so the retries aren't necessary.