face-api.js
face-api.js copied to clipboard
justadudewhohacks
solved: face-api is not compatible with tfjs 2.x or tfjs 3.x
Now that TFJS 2.0 has been released, are there any plans to update face-api models to be compatible with it?
Currently it fails due to batchNormalization being obsoleted in tfjs 2.0.
I'm using tfjs with multiple models on the same image to classify/detect all different types of objects (not just faces) and since concurrently loading multiple different versions of tfjs is not possible thus I cannot upgrade entire project because of a face-api dependency on tfjs 1.x.
True that!
Unhandled Rejection at: TypeError: backend.batchNormalization is not a function
at engine_1.ENGINE.runKernelFunc.x (/var/task/node_modules/face-api.js/node_modules/@tensorflow/tfjs-core/dist/ops/batchnorm.js:280:27)
at /var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3229:55
at /var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3075:22
at Engine.scopedRun (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3085:23)
at Engine.tidy (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3074:21)
at kernelFunc (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3229:29)
at /var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3240:27
at Engine.scopedRun (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3085:23)
at Engine.runKernelFunc (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3238:14)
at batchNorm_ (/var/task/node_modules/face-api.js/node_modules/@tensorflow/tfjs-core/dist/ops/batchnorm.js:279:31)
at Object.batchNorm (/var/task/node_modules/face-api.js/node_modules/@tensorflow/tfjs-core/dist/ops/operation.js:46:29)
at /var/task/node_modules/face-api.js/build/commonjs/ssdMobilenetv1/mobileNetV1.js:9:18
at /var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3075:22
at Engine.scopedRun (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3085:23)
at Engine.tidy (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3074:21)
at Object.tidy (/var/task/node_modules/face-api.js/node_modules/@tensorflow/tfjs-core/dist/globals.js:176:28)
at depthwiseConvLayer (/var/task/node_modules/face-api.js/build/commonjs/ssdMobilenetv1/mobileNetV1.js:7:15)
at /var/task/node_modules/face-api.js/build/commonjs/ssdMobilenetv1/mobileNetV1.js:38:19
at Array.forEach (<anonymous>)
at /var/task/node_modules/face-api.js/build/commonjs/ssdMobilenetv1/mobileNetV1.js:35:24
at /var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3075:22
at Engine.scopedRun (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3085:23)
at Engine.tidy (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3074:21)
at Object.tidy (/var/task/node_modules/face-api.js/node_modules/@tensorflow/tfjs-core/dist/globals.js:176:28)
at Object.mobileNetV1 (/var/task/node_modules/face-api.js/build/commonjs/ssdMobilenetv1/mobileNetV1.js:17:15)
at /var/task/node_modules/face-api.js/build/commonjs/ssdMobilenetv1/SsdMobilenetv1.js:29:42
at /var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3075:22
at Engine.scopedRun (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3085:23)
at Engine.tidy (/var/task/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:3074:21)
at Object.tidy (/var/task/node_modules/face-api.js/node_modules/@tensorflow/tfjs-core/dist/globals.js:176:28)
at SsdMobilenetv1.forwardInput (/var/task/node_modules/face-api.js/build/commonjs/ssdMobilenetv1/SsdMobilenetv1.js:26:19)
at SsdMobilenetv1.<anonymous> (/var/task/node_modules/face-api.js/build/commonjs/ssdMobilenetv1/SsdMobilenetv1.js:58:35)
at step (/var/task/node_modules/face-api.js/node_modules/tslib/tslib.js:139:27)
at Object.next (/var/task/node_modules/face-api.js/node_modules/tslib/tslib.js:120:57)
at fulfilled (/var/task/node_modules/face-api.js/node_modules/tslib/tslib.js:110:62) {}
Check to see if you have a node_modules/ inside the node_modules/face-api.js folder -- this can have the older version of the tensor library and then they will conflict, because the face-api will initially load this version which then conflicts with the version in the main node_modules folder... Once I nuked this extra node_modules folder; everything worked fine w/ v2.x
Check to see if you have a
node_modules/inside the node_modules/face-api.js folder -- this can have the older version of the tensor library and then they will conflict, because the face-api will initially load this version which then conflicts with the version in the main node_modules folder... Once I nuked this extra node_modules folder; everything worked fine w/ v2.x
Can you elaborate on that? IMO, loading any specific version of TFJS is not a problem, but if TFJS 2.0 is loaded Face-API will not work since TFJS 2.0 obsoleted some functions - primarily batchNormalization(). And that is a dependency inside models, so models need to be recompiled, it's not just a library code issue.
Sometimes NPM is stupid; so what happens is you have this:
[YourProject] -> [node_modules] ---> [...Your other package.json modules...] ---> [@tensorflow/ ] (All v2.x) ---> [faceapis.js] ---------> [node_modules] -----------------[@tensorflow/] (v1.x)
So when faceapi loads tensorflow it loads the v1.0 library from its node_modules; which then when it attempts to load tsflow-node; it gets the v2.0 from the root @tensorflow 2.0 version. Of course v1 doesn't work with v2 and you get that nice cryptic error message.
If you NUKE/DELETE the [node_modules] under the faceapi.js folder; then when faceapi.js goes to load tensorflow it will load the 2.0 version from the root node_modules, and that 2.0 will load the tsflow-node which is also 2.0 and everything will work.
As I said, the issue is not forcing load of TFJS 2.0 - even if you do that, the models used by face-api.js rely on batchNormalization() function that was deprecated and removed in TFJS 2.0 - see: https://github.com/tensorflow/tfjs/pull/3238 and https://github.com/tensorflow/tfjs/issues/3232
Anyhow, I've tried your method (and few others)...
- Nuke private tfjs within face-api.js so it tries to use tfjs 2.0 from node_modules/@tensorflow
rm -rf node_modules/face-api.js/node_modules/@tensorflow
As expected, it doesn't load TFJS at all, it just fails on first usage of any method since instance is undefined.
"Cannot read property 'fetch' of undefined"
- Update face-api.js dependencies in-place
cd node_modules/face-api.js
../.bin/ncu -u
@tensorflow/tfjs-core 1.7.0 → 2.3.0
tslib ^1.11.1 → ^2.0.1
@tensorflow/tfjs-node 1.7.0 → 2.3.0
@types/jasmine ^3.5.9 → ^3.5.12
@types/node ^13.9.2 → ^14.0.27
jasmine ^3.5.0 → ^3.6.1
jasmine-core ^3.5.0 → ^3.6.0
karma ^4.4.1 → ^5.1.1
karma-jasmine ^3.1.1 → ^4.0.1
karma-typescript ^5.0.1 → ^5.1.0
rollup ^2.1.0 → ^2.26.2
rollup-plugin-typescript2 ^0.26.0 → ^0.27.2
ts-node ^8.7.0 → ^8.10.2
typescript ^3.8.3 → ^3.9.7
npm i
Result is the same as in first case.
- Download and rebuild face-api.js from sources, not npm package
git clone https://github.com/justadudewhohacks/face-api.js
cd face-api.js/
npm i
This will do a recompile which may fail due to missing system dependencies, in which case install them manually. E.g.:
sudo apt install libjpeg-dev libgif-dev
Then force package update and do a full rebuild
npm i npm-check-updates
node_modules/.bin/ncu -u
Upgrading face-api.js/package.json
@tensorflow/tfjs-core 1.7.0 → 2.3.0
tslib ^1.11.1 → ^2.0.1
@tensorflow/tfjs-node 1.7.0 → 2.3.0
@types/jasmine ^3.5.9 → ^3.5.12
@types/node ^13.9.2 → ^14.0.27
jasmine ^3.5.0 → ^3.6.1
jasmine-core ^3.5.0 → ^3.6.0
karma ^4.4.1 → ^5.1.1
karma-jasmine ^3.1.1 → ^4.0.1
karma-typescript ^5.0.1 → ^5.1.0
rollup ^2.1.0 → ^2.26.2
rollup-plugin-typescript2 ^0.26.0 → ^0.27.2
ts-node ^8.7.0 → ^8.10.2
typescript ^3.8.3 → ^3.9.7
npm i
npm run build
Fails, so it's not just models that need work
Error: /home/vlado/dev/face-api.js/src/dom/NetInput.ts(131,72): semantic error TS2344: Type 'Rank.R4' does not satisfy the constraint 'Tensor<Rank>'.
Hi,
I had initially followed the sites instructions and did this for my project:
npm i face-api.js canvas @tensorflow/tfjs-node
Here is what I'm running: node: v12.16.3 tsc: V3.8.3
All I can report is what worked for me. I got the exact same message as Unhandled Rejection at: TypeError: backend.batchNormalization is not a function the very first time I ran the examples, and so that is how I found this issue. :grinning:
The npm i Unfortunately, creates the nested node_modules because the face-api.js package.json file asks for @tensorflow/tfjs-core": "1.7.0" however the npm i @tensorflow/tfjs-node part get whatever is the latest TF (which in my case was 2.1).
My projects node_modules contains these versions of libraries
- @tensorflow/[email protected]
- @tensorflow/[email protected]
- @tensorflow/[email protected]
- @tensorflow/[email protected]
- @tensorflow/[email protected]
- @tensorflow/[email protected]
- @tensorflow/[email protected]
- @tensorflow/[email protected]
- [email protected]
- [email protected]
Originally under the node_modules/face-api.js their was another node_modules with the v1.7 tensorflow module; this is what I nuked.
If I take any of the node-js examples that are on this repo; build them with TS, and then run them with node they ALL work.
(This is my out directory)
If I add code to any of the demos to output the TensorFlow version; this is what I get:
The code I'm using to get the version is I just add this to the very top of any of the examples to output TensorFlow version...
const tf = require('@tensorflow/tfjs');
console.log(`Initializing TensorFlow/JS version ${tf.version_core}`);
As you can see from the output; IT is using TensorFlow 2.1, Node is loading TF 2.1 with face-api; and face-api is using it.
Now to triple check; If I manually edit the node_modules/face-api.js/build/commonjs/index.js and add the exact same console.log right under where the face-api loads tensorflow like so:
I also get the exact same output:
Once from the added code in the example; and once from when face-api loads tensorflow.
Ok, I dug a bit deeper...
face-api.js includes multiple models:
- tinyFaceDetector: This one WORKS with TFJS 2.0+. This is the model that is used in most of examples anyhow.
- ssdMobilenetv1: This one depends on obsolete function batchNorm() so no chance to make it work with TFJS 2.0. I used this model since in my testing it seemed to have better precision than tiny (although a touch slower).
- mtcnn: Mostly obsolete, but still present
- tinyYolov2: Code is there, but weights are missing since forever, so I guess this is more of an abandonware.
So if you're ok using tinyFaceDetector, TFJS 2.0+ works just fine.
You could nuke local node_modules like NathanaelA stated or you can upgrade face-api.js packages.json and make it use newer tfjs or anything else.
FYI, I've forked face-api.js, updated it for tfjs 2.0, cleaned up obsolete code and modernized build process.
And it's about 2.5x smaller than original.
See https://www.npmjs.com/package/@vladmandic/face-api if you want to use it instead of original face-api.js.
There are no changes except what's noted, so original docs still apply.
@NathanaelA I'd be happy to write a PR for minor code changes and update dependencies, but it's really up to the author to clean up models - as it is, there is code for 4 different models and only 1 is working - removing that much code is not something that should be done in a PR. Also, build process itself is quite old and targeting old standards which did cause some issues for me when loading face-api concurrently with tfjs itself in a different model. That's why I did a fork, not a PR.
I find the accuracy of the tinyFaceDetector model not good for my use-case compared to the ssdMobilenetv1 model. For those who want still to use the ssdMobilenetv1 model, the workaround is downgrading your tfjs to version 1.7.0.
I hope this gets fixed soon.
cc. @justadudewhohacks
@focux I agree that ssdMobilenetv1 is generally a bit better than tinyFaceDetector and I wish author retrains model to be compatible with [email protected]+ although not sure how much can be done since mobilenet v1 is a really old model. Ideally, it should be trained with either ssd & mobilenet v2 (that would be simplest as it's very close to existing model) or a newer & better alternative.
i've noticed some work has been done in a separate branch https://github.com/justadudewhohacks/face-api.js/tree/face_detection_save, but there are no weights and it hasn't been updated in 3 months.
btw, if you really need ssdMobilenetv1, use latest [email protected], not 1.7.0 as there are more than few bugfixes and v1.7.4 is final version in 1.x branch.
i'll reopen this issue so it can be monitored...
One more note, ssd_mobilenetv1 model actually comes from a diferent project, https://github.com/yeephycho/tensorflow-face-detection.
And that project has actually been updated for [email protected]+ compatibility.
However, latests weighs released there https://drive.google.com/open?id=0B5ttP5kO_loUdWZWZVVrN2VmWFk are in TF Frozen format.
They do convert nicely to TF Graph model (and optionally quantize for reduced size),
> summarize_graph \
--in_graph="./frozen_inference_graph_face.pb" \
--print_structure=false
Found 4 possible outputs: (name=detection_boxes, op=Identity) (name=detection_scores, op=Identity) (name=detection_classes, op=Identity) (name=num_detections, op=Identity)
> tensorflowjs_converter \
--input_format tf_frozen_model \
--output_format tfjs_graph_model \
--skip_op_check \
--strip_debug_ops=True \
--quantize_uint16 \
--weight_shard_size_bytes 4194304 \
--output_node_names detection_boxes,detection_scores,num_detections \
./frozen_inference_graph_face.pb \
./converted/
but...face-api.js model-weights list to iterate and load weights, not full model.json. If someone wants to dig deeper, I'm sure this can be used, I just don't have any more time for this.
As it is, tinyFaceDetector works as-is and ssdMobilenetv1 can probably be made to work. Other models are obsolete.
Done!
I've just pushed updated package to https://www.npmjs.com/package/@vladmandic/face-api Both tinyFaceDetector and ssdMobileNetv1 work:
Model: {name: "FaceAPI SSD/MobileNet v1", modelPath: "models/faceapi/", exec: "ssd", score: 0.3, topK: 1, size: 416 },
Result: {
age: 21.586017608642578
expressions: FaceExpressions {...}
gender: "female"
genderProbability: 0.960713230073452
alignedRect: FaceDetection {...}
descriptor: Float32Array(128) [...]
detection: FaceDetection {...}
landmarks: FaceLandmarks68 {...}
}
{ name: 'FaceAPI TinyYoloDetector', modelPath: 'models/faceapi/', exec: 'yolo', score: 0.3, topK: 1, size: 416 },
Result: {
age: 24.492658615112305
expressions: FaceExpressions {...}
gender: "female"
genderProbability: 0.964848417788744
alignedRect: FaceDetection {...}
descriptor: Float32Array(128) [...]
detection: FaceDetection {...}
landmarks: FaceLandmarks68 {...}
}
Wow! That's awesome, great work @vladmandic, I will definitely use your package. Thank you.
Awesome work @vladmandic thank you for sharing! Can I buy you a beer/do you have a tip jar anywhere? :D
(and @justadudewhohacks too if you're still around 😃 )
