Yolov8-Segmentation-on-Browser icon indicating copy to clipboard operation
Yolov8-Segmentation-on-Browser copied to clipboard

use tf in image processing instead of opencv.js

Open shimaamorsy opened this issue 1 year ago • 0 comments

I want to perform image processing without using opencv.js

`const preprocessing = (source, modelWidth, modelHeight, stride = 32) => {
  const mat = cv.imread(source); // read from img tag
  const matC3 = new cv.Mat(mat.rows, mat.cols, cv.CV_8UC3); // new image matrix
  cv.cvtColor(mat, matC3, cv.COLOR_RGBA2BGR); // RGBA to BGR

  const [w, h] = divStride(stride, matC3.cols, matC3.rows);
  cv.resize(matC3, matC3, new cv.Size(w, h));

  // padding image to [n x n] dim
  const maxSize = Math.max(matC3.rows, matC3.cols); // get max size from width and height
  const xPad = maxSize - matC3.cols, // set xPadding
    xRatio = maxSize / matC3.cols; // set xRatio
  const yPad = maxSize - matC3.rows, // set yPadding
    yRatio = maxSize / matC3.rows; // set yRatio
  const matPad = new cv.Mat(); // new mat for padded image
  cv.copyMakeBorder(matC3, matPad, 0, yPad, 0, xPad, cv.BORDER_CONSTANT); // padding black

  const input = cv.blobFromImage(
    matPad,
    1 / 255.0, // normalize
    new cv.Size(modelWidth, modelHeight), // resize to model input size
    new cv.Scalar(0, 0, 0),
    true, // swapRB
    false // crop
  ); // preprocessing image matrix

  // release mat opencv
  mat.delete();
  matC3.delete();
  matPad.delete();

  return [input, xRatio, yRatio];
};`

However when I used tf in this way

`const preprocessing = async (
      source,
      modelWidth,
      modelHeight,
      stride = 32
    ) => {
      // Load the image using TensorFlow.js
      const tensor = tf.browser.fromPixels(source);

      // Resize the image
      const resizedTensor = tf.image.resizeBilinear(tensor, [
        modelHeight,
        modelWidth,
      ]);

      // Normalize the pixel values
      const normalizedTensor = resizedTensor.div(255.0);

      // Convert the tensor to a 2D array representing the image data
      const inputData = await normalizedTensor.array();

      // Calculate the padding ratios (since TensorFlow.js doesn't require explicit padding)
      const [xRatio, yRatio] = [
        modelWidth / source.width,
        modelHeight / source.height,
      ];

      // Dispose intermediate tensors
      tensor.dispose();
      resizedTensor.dispose();
      normalizedTensor.dispose();
      console.log("[inputData, xRatio, yRatio]",[inputData, xRatio, yRatio])
      return [inputData, xRatio, yRatio];
    };`

I got this error

Uncaught (in promise) TypeError: preprocessing is not a function or its return value is not iterable
    at detectImage (index_cpu.js:294:39)
    at img.onload (index_cpu.js:452:9)

Can you help me to replace opencv.js with tf

shimaamorsy avatar May 16 '24 08:05 shimaamorsy