CovPoolFER icon indicating copy to clipboard operation
CovPoolFER copied to clipboard

Nans when inferring and error when loading classif model

Open DaddyWesker opened this issue 3 years ago • 1 comments

Hello.

THanks for your code. I've been trying to write a script to infer your model on the webcam. Here is my code

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import cv2
import tensorflow.compat.v1 as tf

tf.disable_v2_behavior()
import numpy as np
import argparse
import framework
import os
import sys
import math
import pickle
from sklearn.svm import SVC
import dlib


def extract_face(image, rect):
    startX = rect.left()
    startY = rect.top()
    endX = rect.right()
    endY = rect.bottom()
    startX = max(0, startX)
    startY = max(0, startY)
    endX = min(endX, image.shape[1])
    endY = min(endY, image.shape[0])
    w = endX - startX
    h = endY - startY
    return image[startX:endX, startY:endY]

face_detector = dlib.get_frontal_face_detector()

video_capture = cv2.VideoCapture(0)

with tf.Graph().as_default():
    with tf.Session() as sess:
        framework.load_model("../../models/model2/20170815-144407/")
        stub = 0
        images_placeholder = tf.get_default_graph().get_tensor_by_name("input:0")
        embeddings = tf.get_default_graph().get_tensor_by_name("embeddings:0")
        embeddings = tf.nn.l2_normalize(embeddings, 1)
        embedding_size = embeddings.get_shape()[1]
        phase_train_placeholder = tf.get_default_graph().get_tensor_by_name("phase_train:0")
        faces = []
        while True:
            _, frame_bgr = video_capture.read()
            frame = cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB)
            rects = face_detector(frame)
            face = extract_face(frame, rects[0])
            face = cv2.resize(face, (100, 100))
            faces.append(face)
            if (len(faces) < 128):
                continue
            feed_dict = {images_placeholder: np.asarray(faces), phase_train_placeholder: False}
            arr = sess.run(embeddings, feed_dict=feed_dict)
            with open("/home/daddywesker/EmotionRecognition/CovPoolFER/models/model2/svc_144407.pkl", 'rb') as infile:
                (model, class_names) = pickle.load(infile)

Here are two problems when running this code:

  1. arr = sess.run(embeddings, feed_dict=feed_dict) give me NaNs as output. DOn't think that is normal.

Here is a problem with TF version i guess. I have tf 2.5 and i throws me NaNs. I've tried tf 1.15 (which will be deprecated after some time) and it gives me some numbers, not NaNs. But what should we do, when there will be no tf 1.15 anymore?..

  1. (model, class_names) = pickle.load(infile) gives me No module named 'sklearn.svm.classes'

2nd problem could be related to scikit-learn version mismatch with yours, but i can't install 0.18.1. My version is 0.24. Min version i can install is 0.20.1 and it throws me another error UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 4: ordinal not in range(128)

DaddyWesker avatar Aug 05 '21 05:08 DaddyWesker

Well, problem is solved by installing py2.7. But it is kinda not exactly what i wanted. Is there are way to make it work on python3?

Also, is this right for predictions of this model? EMOTIONS = ["angry", "disgust", "scared", "happy", "sad", "surprised","neutral"] I mean, model gives label and i need label to emotion conversion.

DaddyWesker avatar Aug 05 '21 06:08 DaddyWesker