Doesn't work for segmentation

[tim-fan]

Hello, I'm trying to get this running on my segmentation model. I'm making progress, but have now run out of free credits so can't debug any further!

On first attempt to run the script, I got the error No JPEG data found in image (similar to https://github.com/roboflow/video-inference/issues/2).

Checking the docs, I see segmentation models run on a different url: https://detect.roboflow.com

Correcting that, I also noticed that the url returns json data containing b64 encoded image data, so I used jq and base64 to get it into a file.

Finally I noticed the received image data is in png format, so I changed the file suffix to match, to make ffmpeg happy.

At that point the script completes without error and produces a video, however the video is entirely black. I'm not sure if that is due to a remaining bug, or if it is because I am out of credits.

Here's my full diff right now, could probably be cleaned up a bit:

diff --git a/infer.sh b/infer.sh
index 3dbe286..9d54493 100755
--- a/infer.sh
+++ b/infer.sh
@@ -56,7 +56,7 @@ _arg_model=
 _arg_video_in=
 _arg_video_out=
 # THE DEFAULTS INITIALIZATION - OPTIONALS
-_arg_host="https://detect.roboflow.com"
+_arg_host="https://segment.roboflow.com"
 _arg_confidence="50"
 _arg_overlap="50"
 _arg_stroke="5"
@@ -343,7 +343,7 @@ done
     exit 1;
 }
 
-inference_url="$host/$model?api_key=$ROBOFLOW_KEY&format=image&confidence=$confidence&overlap=$overlap&stroke=$stroke"
+inference_url="$host/$model?api_key=$ROBOFLOW_KEY"
 if [ $labels = "on" ]; then
     inference_url="$inference_url&labels=on"
 fi
@@ -393,14 +393,14 @@ do
             if [ ! -z "$verbose" ]; then
                 echo "Running inference on frame $f..."
             fi
-            cat $tmp/roboflow_in/$f | base64 | curl -s -d @- $inference_url > "$tmp/roboflow_out/$f" &
+            cat $tmp/roboflow_in/$f | base64 | curl -s -d @- $inference_url | jq -r .segmentation_mask | base64 --decode > "$tmp/roboflow_out/$f.png"
         fi
     done
 done
 
 rm -f $out
 echo "Rendering final video ($out)."
-ffmpeg -i $tmp/roboflow_out/frame%05d.jpg -vf fps=$fps_out $out
+ffmpeg -i $tmp/roboflow_out/frame%05d.jpg.png -vf fps=$fps_out $out
 
 rm -rf $tmp/roboflow_in
 rm -rf $tmp/roboflow_out

Any chance you could reimburse me some credits in exchange for this bug report? I wanted to work on my project, but ended up spending all my time and credits debugging this script! My roboflow account is associated with this github account.

If the intention is not to support segmentation models with this script, at least it would be nice if that was indicated clearly in the README, such that others do not lose time and credits as I did.

[josephofiowa]

@tim-fan Thanks so much for the report!

Any chance you could reimburse me some credits in exchange for this bug report?

As a quick(er) win while digging in, happy to get you credits. I doubled your allocation on the free plan.

[tim-fan]

Thanks, much appreciated! Actually my account is still showing I'm over the limit on inference allowance, so I still can't check if the diff fixes the problem. I'll wait for the new month and try again.