Bug - When Telegram files have the same name the program stops with an error

[art435]

I tried to download some files that have the same name (duplicates) in Telegram channel. But the program stops with this error callback: rename downloads\January Session.rtf.tmp downloads\January Session.rtf.rtf: The process cannot access the file because it is being used by another process.

[iyear]

Sorry I'm replying now because I've been busy recently.

Yes, this problem exists. What do you think is the best way to solve it?

1. Add random strings to file name: the file name may be too long
2. Skip if files have the same name: some files with the same name may be missed

[art435]

The program should offer 2 options to users.

1. Don't download new files that have identical names. Fast but with small risks.
2. Download the new files in a separate folder, calculate the hashes, and delete if they are the same.

This should be done real time, and the download should continue for other files. See here.

[art435]

This issue is fixed from version 0.6.0

[art435]

I reopen this issues, because it happens again. It occurs when more file are downloaded in parallel (-l bigger than 1) and files have the same name. This has forcing me to download files with (-l=1), just one file at a time.

[iyear]

Is it possible to modify the file name template to achieve no duplication?

[art435]

To prevent this issue? How? My template for file name is {{ .FileName }}, just the filename.

[iyear]

Is it acceptable to add some random string after the file name?

[art435]

This solution doesn't interfere with the deduplication? If the file name in source chat is abc.flac and is renamed abc1.flac, and later tries to download an identical file abc.flac, it won't find the duplicate in chat because the file name has changed. abc.flac is now abc1.flac

[rsyqvthv]

same issue here with Version: 0.7.2

not sure if this is actually means file name duplicate, as the error shows "file is being used by another process." would ending the process will solve the issue?

as I use template {{.MessageDate}}-{{.FileName}} when downloading, I don't think the destination file name will be duplicated, as by the file system limitation, uploader won't able to upload duplicated file in a very same time.

added:

this issue won't occur when I remove the --template argument, that means the default template works, I think that must something of the template custom process conflict with the rename process.

Hope this will help @iyear address the issue.

[locvez]

@iyear - I have this issue when downloading files using only {{.FileName}} template. If you could add (1) to the filename, or (2) if filename(1) exists, etc, etc, I think this will help massively. IF the file contains the same number of bytes, do not download, if the file size differs, rename the new file.

If the filename is too large, would the app throw an error? If so you could handle the error and present a "number of files with too large filenames = x" once it finished scraping.

[locvez]

This solution doesn't interfere with the deduplication? If the file name in source chat is abc.flac and is renamed abc1.flac, and later tries to download an identical file abc.flac, it won't find the duplicate in chat because the file name has changed. abc.flac is now abc1.flac

No, it doesn't mess with de-dupe, because abc.flac already exists, abc.flac is downloaded again (with diff. file size) it is renamed abc1.flac, both files now exist. If another abc.flac is downloaded, with a different file size, it would be renamed abc2.flac. This means abc.flac, abc1.flac and abc2.flac all exist with different file sizes.

IF abc.flac had the same file size as the downloaded abc.flac then it would skip