Inference.py script is not returning the custom labels on test images

[nitin-dominic]

Hello:

I recently tried to run "inference.py" script on my custom dataset with custom labels (only 3). I trained the model EfficientNet and pushed it to the HuggingFace platform. Now, when i try to reload the checkpoints to run inference tests on my own dataset, I get this error:

"ERROR: Cannot deduce ImageNet subset from model, no labelling will be performed."

Any fixes or solutions will be appreciated.

[rwightman]

@nitin-dominic if the model is pushed with label_names and optionally label_descriptions and those are in the config.json it will work with the Hub API inference but yeah, inference.py is not handling this properly...

The API does this

        self.dataset_info = None
        label_names = self.model.pretrained_cfg.get("label_names", None)
        label_descriptions = self.model.pretrained_cfg.get("label_descriptions", None)

        if label_names is None:
            # if no labels added to config, use imagenet labeller in timm
            imagenet_subset = infer_imagenet_subset(self.model)
            if imagenet_subset:
                self.dataset_info = ImageNetInfo(imagenet_subset)
            else:
                # fallback label names
                label_names = [f"LABEL_{i}" for i in range(self.model.num_classes)]

        if self.dataset_info is None:
            self.dataset_info = CustomDatasetInfo(
                label_names=label_names,
                label_descriptions=label_descriptions,
            )

the inference.py code is just doing

    to_label = None
    if args.label_type in ('name', 'description', 'detail'):
        imagenet_subset = infer_imagenet_subset(model)
        if imagenet_subset is not None:
            dataset_info = ImageNetInfo(imagenet_subset)
            if args.label_type == 'name':
                to_label = lambda x: dataset_info.index_to_label_name(x)
            elif args.label_type == 'detail':
                to_label = lambda x: dataset_info.index_to_description(x, detailed=True)
            else:
                to_label = lambda x: dataset_info.index_to_description(x)
            to_label = np.vectorize(to_label)
        else:
            _logger.error("Cannot deduce ImageNet subset from model, no labelling will be performed.")

Also need a mechanism to allow loading those fields from a stand alone json for models that aren't on the hub or pushed with the needed metadata (e.g. just load a yaml or json with those fields)

An example model config with those labels & descriptions https://huggingface.co/timm/eva02_large_patch14_clip_336.merged2b_ft_inat21/raw/main/config.json