Help in output parser to get bounding box when use Tensorflow scoring Yolov3

[huanbd]

trafficstars

Hi, I'm use this tensorflow yolov3 source to train my custom object, and result is a frozen model (pb) file. I've followed this sample to scoring this model with some modify in code to fit with model

public string ModelInput { get; } = "x";
public string ModelOutput { get; } = "Identity_5";
public string[] Labels { get; } =
{
    "vietnam_map", "vietnam_img"
};
private readonly (float x, float y)[] anchors = new (float x, float y)[]
{
    (1.25F,1.625F),(2.0F,3.75F),(4.125F,2.875F),(1.875F,3.8125F),(3.875F,2.8125F),(3.6875F,7.4375F),(3.625F,2.8125F),(4.875F,6.1875F),(11.65625F,10.1875F)
};

Scoring use ML.NET, this model return a array float size [3549‬] (in ML.NET it flattens dimensional into one dimensional array 1x13x13x3x7 ) (in TensorFlow it is a tensor float32 with shape (1, 13, 13, 3, 7))

I'm struggling to parser array float to get bounding box.

Can anyone help, modify in OnnxOutputParser.cs to run with that score?

Width ModelInput = "x" and ModelOutput = "Identity_5" I've tested in TensorFlow code and it run perfect, you can try this file

[luisquintanilla]

Hi @huanbd

This is a great suggestion. Unfortunately at the moment there are no plans to update our samples to use YoloV3. However, our samples are open source, so if anyone from the community would like to contribute, we'd welcome those contributions.

[LordTrololo]

I am working on the same problem. In the OnnxModelScorer.cs I am able to get 3 probabilites outputs (as the yolov3 I use has 3) like so:

            IEnumerable<float[]> probabilitiesIdentity0 = scoredData.GetColumn<float[]>(YoloModelSettings.Identity0);
            IEnumerable<float[]> probabilitiesIdentity1 = scoredData.GetColumn<float[]>(YoloModelSettings.Identity1);
            IEnumerable<float[]> probabilitiesIdentity2 = scoredData.GetColumn<float[]>(YoloModelSettings.Identity2);

However I am not sure how would one merge this into 1 result or how would one extract the bounding boxes information out of it.

Second problem is - the model I am using needs images in the floating format, like so:

                mat.ConvertTo(mat, DepthType.Cv32F);
                mat = mat / 255.0;

Sadly, I do not know how could one apply such transform to IDataView imageDataView format. I tried playing with the ml.Contex.Tranforms in the var pipeline but I couldnt get it to work.

.Append(mlContext.Transforms.NormalizeBinning(outputColumnName: TinyYoloModelSettings.ModelInput, inputColumnName: nameof(ImageNetData.ImagePath), maximumBinCount: 255);

[BobLd]

Hi @huanbd, @LordTrololo, maybe have a look at my repos YOLOv3MLNet. Two different YOLO v3 models are used.

Also, you can check how to apply the 1 / 255.0 transform here.