szurubooru-toolkit icon indicating copy to clipboard operation
szurubooru-toolkit copied to clipboard

Could Upload Media include Tags from Sidecar txt file

Open gljames24 opened this issue 2 years ago • 4 comments

Is the currently a way to import images from a folder with sidecars that programs like gallery-dl make with --write-tags instead of having to set the tag to "tagme" and relying on the auto tagger? Additionally, would it be possible to have a boolean in the config so that if the similarity is too high, the tags get copied over anyway like if the same image was downloaded twice, but from two different boorus with two different sets of tags?

gljames24 avatar Oct 16 '23 21:10 gljames24

Do you mean that you want to be able to set tags individually for each image with the upload-media script?

Regarding the second request: You mean that the duplicate post should use the tags from the existing post while also adding the new ones?

reluce avatar Oct 24 '23 19:10 reluce

When I use the upload-media script, I would like to be able to have an image, let's call it 'abc.jpg', and for it to be imported with the tags given from a text file 'abc.txt' if it exists. If the image is already in the booru, there could be an option to either do nothing, overwrite, or append the tags from that text file to the image it matched with.

gljames24 avatar Oct 27 '23 04:10 gljames24

@reluce I ended up using the import-from-url script to import from a folder by adding in txt and no sidecar options for images found in a folder and just downloaded an image already in my szurubooru as import-from-url would already import images in the tmp directory. It would probably be better to move this importing functionality to upload-media and call it in import-from-url after the images and json files had been imported. I need to take a look at the 1.0 changes, but I think adding in my merge tags option and this addition would make importing media a lot easier with a backlog of data already downloaded with gallery-dl.

gljames24 avatar Feb 25 '24 04:02 gljames24

files = [file for file in glob.glob(f'{download_dir}/*') if  Path(file).suffix not in ['.psd', '.json','.txt']]

    logger.info(f'Downloaded {len(files)} post(s). Start importing...')

    saucenao_limit_reached = False

    
    for file in tqdm(
        files,
        ncols=80,
        position=0,
        leave=False,
        disable=config.import_from_url['hide_progress'],
    ):
        if os.path.exists(str(file) + '.json'):
            with open(file + '.json') as json_file:
                metadata = json.load(json_file)
                metadata['site'] = site
                metadata['source'] = generate_src(metadata)

                if 'rating' in metadata:
                    metadata['safety'] = convert_rating(metadata['rating'])
                else:
                    metadata['safety'] = config.upload_media['default_safety']

                if 'tags' in metadata or 'tag_string' in metadata or 'hashtags' in metadata:
                    metadata['tags'] = set_tags(metadata)
                else:
                    metadata['tags'] = []

                # As Twitter doesn't provide any tags compared to other sources, we try to auto tag it.
                if site == 'twitter':
                    metadata['tags'] += extract_twitter_artist(metadata)
                    config.auto_tagger['md5_search_enabled'] = True
                    config.auto_tagger['saucenao_enabled'] = True
                else:
                    config.auto_tagger['md5_search_enabled'] = False
                    config.auto_tagger['saucenao_enabled'] = False
        elif os.path.exists(str(file) + '.txt'):
            with open(str(file) + '.txt') as txt_file:
                tags = txt_file.read().splitlines()
            metadata = {
                'safety': config.upload_media['default_safety'],
                'tags': tags,
                'site': '',
                'source': ''
            }
        else:
            metadata = {
                'safety': config.upload_media['default_safety'],
                'tags': 'tagme',
                'site': '',
                'source': ''
            }
        with open(file, 'rb') as file_b:
            saucenao_limit_reached = upload_media.main(file_b.read(), Path(file).suffix[1:], metadata, saucenao_limit_reached)

gljames24 avatar Feb 25 '24 05:02 gljames24