elodie
elodie copied to clipboard
jmathai
Original checksum returned None for multiple files.
Hi,
Great tool and very useful!
I'm getting this error for a lot of my files from Google Take Out:
Original checksum returned None for /source/gto/takeout/fer/Takeout/Google Photos/Familj och vänner/IMG_5683.JPG. Skipping...
Just to test I extracted the parts that does the checksum and ran it manually:
# python3 filetest.py import --file "/source/gto/takeout/fer/Takeout/Google Photos/Familj och vänner/IMG_5683.JPG"
pass
/source/gto/takeout/fer/Takeout/Google Photos/Familj och vänner/IMG_5683.JPG
/source/gto/takeout/fer/Takeout/Google Photos/Familj och vänner/IMG_5683.JPG
9ca92b26f96e9aace58420b4dca2d0a13f3aba55b5c4f8331aead18bcbbfd18b
9ca92b26f96e9aace58420b4dca2d0a13f3aba55b5c4f8331aead18bcbbfd18b
/elodie # cat filetest.py
import sys
import hashlib
import click
def _decode(string, encoding=sys.getfilesystemencoding()):
"""Return a utf8 encoded unicode string.
Python2 and Python3 differ in how they handle strings.
So we do a few checks to see if the string is ascii or unicode.
Then we decode it if needed.
"""
if hasattr(string, 'decode'):
# If the string is already unicode we return it.
try:
if isinstance(string, unicode):
return string
except NameError:
pass
return string.decode(encoding)
return string
def checksum(file_path, blocksize=65536):
"""Create a hash value for the given file.
See http://stackoverflow.com/a/3431835/1318758.
:param str file_path: Path to the file to create a hash for.
:param int blocksize: Read blocks of this size from the file when
creating the hash.
:returns: str or None
"""
hasher = hashlib.sha256()
with open(file_path, 'rb') as file:
buf = file.read(blocksize)
while len(buf) > 0:
hasher.update(buf)
buf = file.read(blocksize)
return hasher.hexdigest()
return None
@click.command('import')
@click.option('--file', type=click.Path(dir_okay=False),
help='Import this file, if specified.')
def _import(file):
print(file)
_file = _decode(file)
print(_file)
print(checksum(file))
print(checksum(_file))
@click.group()
def main():
print("pass")
main.add_command(_import)
if __name__ == '__main__':
main()
Perhaps this is just a debug issue, while reviewing the log (had to re-run elodie on my ~250'000 files) I saw this:
/source/photos/Bilder/Peru/P1010097.JPG already at /destination/2009-03-Mar/Unknown Location,/2009-03-16_10-27-50 dmc-lx3 p1010097.jpg.
Original checksum returned None for /source/photos/Bilder/Peru/P1010097.JPG. Skipping...
/source/photos/Pictures/Rebuilt Library/Masters/2013/02/13/20130213-053230/P1000291.JPG already at /destination/2012-02-Feb/Unknown Location,/2012-02-10_11-32-50 dmc-lx7 p1000291.jpg.
Original checksum returned None for /source/photos/Pictures/Rebuilt Library/Masters/2013/02/13/20130213-053230/P1000291.JPG. Skipping...
So it seems that the message perhaps should be something different.
