onepanel
onepanel copied to clipboard
Published
20 hours ago •
onepanelio
onepanelio
Explore CVAT serverless implementation, and design our knative implementation
Explore CVAT implementation https://github.com/openvinotoolkit/cvat/pull/1767 and https://github.com/openvinotoolkit/cvat/tree/e7585b8ce900cec2c9dd6ef2dc1092e6567780b1/cvat/apps/lambda_manager so we can design our Knative implementation in a way that it is easily pluggable into CVAT.
CVAT kicks off the serverless piece with an HTTP request (trigger)
- Right now, each serverless function returns an HTTP response
# GET /api/v1/lambda/functions - get list of functions
# GET /api/v1/lambda/functions/<int:fid> - get information about the function
# POST /api/v1/labmda/requests - call a function
# { "function": "<id>", "mode": "online|offline", "job": "<jid>", "frame": "<n>",
# "points": [...], }
# GET /api/v1/lambda/requests - get list of requests
# GET /api/v1/lambda/requests/<int:rid> - get status of the request
# DEL /api/v1/lambda/requests/<int:rid> - cancel a request (don't delete)
CVAT deploys the functions for Nuclio with deploy.sh.
- There are functions and configuration files inside
serverless
Nuclio, for k8s, converts the code into an image, CRD in k8s, and relevant resources in k8s (ingress, deployment, services)
- init "context", a shared object
- main function, takes "context, event"
CVAT has lambda_manager as a django app.
- There are a couple of functions there, matched to the Routes
- Some of the routes are used by the UI to list the models that can be executed
- The offline / online piece is if the serverless should report to the UI or execute just on the server
@rushtehrani This is the response for the "models" page in CVAT.
I deployed all their models via a deploy.sh file they have, running nuclio locally.
Some comments on the fields:
- id: Not displayed on the models page, used for requests
- kind: This is the "interactor" or "detector" you see when annotating an image, and selecting a function to annotate that image
- labels: They pass in all the labels that are supported by the model, used by "Labels" column
- state: Not rendered in models page
- description: Models page description
- framework: Models page column "framework"
- name: Not rendered in models page
The request is simple
GET http://localhost:7000/api/v1/lambda/functions
Response:
[
{
"id": "openvino.dextr",
"kind": "interactor",
"labels": [],
"state": "ready",
"description": "Deep Extreme Cut",
"framework": "openvino",
"name": "DEXTR"
},
{
"id": "openvino.omz.intel.person-reidentification-retail-0300",
"kind": "reid",
"labels": [],
"state": "ready",
"description": "Person reidentification model for a general scenario",
"framework": "openvino",
"name": "Person reidentification"
},
{
"id": "openvino.omz.intel.text-detection-0004",
"kind": "detector",
"labels": [
"text"
],
"state": "ready",
"description": "Text detector based on PixelLink architecture with MobileNetV2-like as a backbone for indoor/outdoor scenes.",
"framework": "openvino",
"name": "Text detection v4"
},
{
"id": "openvino.omz.public.faster_rcnn_inception_v2_coco",
"kind": "detector",
"labels": [
"person",
"bicycle",
"car",
"motorcycle",
"airplane",
"bus",
"train",
"truck",
"boat",
"traffic_light",
"fire_hydrant",
"stop_sign",
"parking_meter",
"bench",
"bird",
"cat",
"dog",
"horse",
"sheep",
"cow",
"elephant",
"bear",
"zebra",
"giraffe",
"backpack",
"umbrella",
"handbag",
"tie",
"suitcase",
"frisbee",
"skis",
"snowboard",
"sports_ball",
"kite",
"baseball_bat",
"baseball_glove",
"skateboard",
"surfboard",
"tennis_racket",
"bottle",
"wine_glass",
"cup",
"fork",
"knife",
"spoon",
"bowl",
"banana",
"apple",
"sandwich",
"orange",
"broccoli",
"carrot",
"hot_dog",
"pizza",
"donut",
"cake",
"chair",
"couch",
"potted_plant",
"bed",
"dining_table",
"toilet",
"tv",
"laptop",
"mouse",
"remote",
"keyboard",
"cell_phone",
"microwave",
"oven",
"toaster",
"sink",
"refrigerator",
"book",
"clock",
"vase",
"scissors",
"teddy_bear",
"hair_drier",
"toothbrush"
],
"state": "ready",
"description": "Faster RCNN inception v2 COCO via Intel OpenVINO toolkit",
"framework": "openvino",
"name": "Faster RCNN"
},
{
"id": "openvino.omz.public.mask_rcnn_inception_resnet_v2_atrous_coco",
"kind": "detector",
"labels": [
"person",
"bicycle",
"car",
"motorcycle",
"airplane",
"bus",
"train",
"truck",
"boat",
"traffic_light",
"fire_hydrant",
"stop_sign",
"parking_meter",
"bench",
"bird",
"cat",
"dog",
"horse",
"sheep",
"cow",
"elephant",
"bear",
"zebra",
"giraffe",
"backpack",
"umbrella",
"handbag",
"tie",
"suitcase",
"frisbee",
"skis",
"snowboard",
"sports_ball",
"kite",
"baseball_bat",
"baseball_glove",
"skateboard",
"surfboard",
"tennis_racket",
"bottle",
"wine_glass",
"cup",
"fork",
"knife",
"spoon",
"bowl",
"banana",
"apple",
"sandwich",
"orange",
"broccoli",
"carrot",
"hot_dog",
"pizza",
"donut",
"cake",
"chair",
"couch",
"potted_plant",
"bed",
"dining_table",
"toilet",
"tv",
"laptop",
"mouse",
"remote",
"keyboard",
"cell_phone",
"microwave",
"oven",
"toaster",
"sink",
"refrigerator",
"book",
"clock",
"vase",
"scissors",
"teddy_bear",
"hair_drier",
"toothbrush"
],
"state": "ready",
"description": "Mask RCNN inception resnet v2 COCO via Intel OpenVINO",
"framework": "openvino",
"name": "Mask RCNN"
},
{
"id": "openvino.omz.public.yolo-v3-tf",
"kind": "detector",
"labels": [
"person",
"bicycle",
"car",
"motorbike",
"aeroplane",
"bus",
"train",
"truck",
"boat",
"traffic light",
"fire hydrant",
"stop sign",
"parking meter",
"bench",
"bird",
"cat",
"dog",
"horse",
"sheep",
"cow",
"elephant",
"bear",
"zebra",
"giraffe",
"backpack",
"umbrella",
"handbag",
"tie",
"suitcase",
"frisbee",
"skis",
"snowboard",
"sports ball",
"kite",
"baseball bat",
"baseball glove",
"skateboard",
"surfboard",
"tennis racket",
"bottle",
"wine glass",
"cup",
"fork",
"knife",
"spoon",
"bowl",
"banana",
"apple",
"sandwich",
"orange",
"broccoli",
"carrot",
"hot dog",
"pizza",
"donut",
"cake",
"chair",
"sofa",
"pottedplant",
"bed",
"diningtable",
"toilet",
"tvmonitor",
"laptop",
"mouse",
"remote",
"keyboard",
"cell phone",
"microwave",
"oven",
"toaster",
"sink",
"refrigerator",
"book",
"clock",
"vase",
"scissors",
"teddy bear",
"hair drier",
"toothbrush"
],
"state": "ready",
"description": "YOLO v3 via Intel OpenVINO",
"framework": "openvino",
"name": "YOLO v3"
},
{
"id": "openvino.omz.semantic-segmentation-adas-0001",
"kind": "detector",
"labels": [
"road",
"sidewalk",
"building",
"wall",
"fence",
"pole",
"traffic light",
"traffic sign",
"vegetation",
"terrain",
"sky",
"person",
"rider",
"car",
"truck",
"bus",
"train",
"motorcycle",
"bicycle",
"ego-vehicle",
"background"
],
"state": "ready",
"description": "Segmentation network to classify each pixel into typical 20 classes for ADAS",
"framework": "openvino",
"name": "Semantic segmentation for ADAS"
},
{
"id": "pth.foolwood.siammask",
"kind": "tracker",
"labels": [],
"state": "ready",
"description": "Fast Online Object Tracking and Segmentation",
"framework": "pytorch",
"name": "SiamMask"
},
{
"id": "tf.faster_rcnn_inception_v2_coco",
"kind": "detector",
"labels": [
"person",
"bicycle",
"car",
"motorcycle",
"airplane",
"bus",
"train",
"truck",
"boat",
"traffic_light",
"fire_hydrant",
"stop_sign",
"parking_meter",
"bench",
"bird",
"cat",
"dog",
"horse",
"sheep",
"cow",
"elephant",
"bear",
"zebra",
"giraffe",
"backpack",
"umbrella",
"handbag",
"tie",
"suitcase",
"frisbee",
"skis",
"snowboard",
"sports_ball",
"kite",
"baseball_bat",
"baseball_glove",
"skateboard",
"surfboard",
"tennis_racket",
"bottle",
"wine_glass",
"cup",
"fork",
"knife",
"spoon",
"bowl",
"banana",
"apple",
"sandwich",
"orange",
"broccoli",
"carrot",
"hot_dog",
"pizza",
"donut",
"cake",
"chair",
"couch",
"potted_plant",
"bed",
"dining_table",
"toilet",
"tv",
"laptop",
"mouse",
"remote",
"keyboard",
"cell_phone",
"microwave",
"oven",
"toaster",
"sink",
"refrigerator",
"book",
"clock",
"vase",
"scissors",
"teddy_bear",
"hair_drier",
"toothbrush"
],
"state": "ready",
"description": "Faster RCNN from Tensorflow Object Detection API",
"framework": "tensorflow",
"name": "Faster RCNN via Tensorflow"
},
{
"id": "tf.matterport.mask_rcnn",
"kind": "detector",
"labels": [
"BG",
"person",
"bicycle",
"car",
"motorcycle",
"airplane",
"bus",
"train",
"truck",
"boat",
"traffic_light",
"fire_hydrant",
"stop_sign",
"parking_meter",
"bench",
"bird",
"cat",
"dog",
"horse",
"sheep",
"cow",
"elephant",
"bear",
"zebra",
"giraffe",
"backpack",
"umbrella",
"handbag",
"tie",
"suitcase",
"frisbee",
"skis",
"snowboard",
"sports_ball",
"kite",
"baseball_bat",
"baseball_glove",
"skateboard",
"surfboard",
"tennis_racket",
"bottle",
"wine_glass",
"cup",
"fork",
"knife",
"spoon",
"bowl",
"banana",
"apple",
"sandwich",
"orange",
"broccoli",
"carrot",
"hot_dog",
"pizza",
"donut",
"cake",
"chair",
"couch",
"potted_plant",
"bed",
"dining_table",
"toilet",
"tv",
"laptop",
"mouse",
"remote",
"keyboard",
"cell_phone",
"microwave",
"oven",
"toaster",
"sink",
"refrigerator",
"book",
"clock",
"vase",
"scissors",
"teddy_bear",
"hair_drier",
"toothbrush"
],
"state": "ready",
"description": "An implementation of Mask RCNN on Python 3, Keras, and TensorFlow.\n",
"framework": "tensorflow",
"name": "Mask RCNN via Tensorflow"
}
]
@aleksandrmelnikov how are things like labels, kind, framework, etc stored? Are they actually coming from the wrapper code for each model?
They are actually stored via function.yaml file for each model.
For example:
serverless/tensorflow/matterport/mask_rcnn/nuclio/function.yaml
They store the information via metadata in the spec, and they load it inside main.py, inside the init_context function.
Example yaml:
metadata:
name: tf.matterport.mask_rcnn
namespace: cvat
annotations:
name: Mask RCNN via Tensorflow
type: detector
framework: tensorflow
spec: |
[
{ "id": 0, "name": "BG" },
{ "id": 1, "name": "person" },
{ "id": 2, "name": "bicycle" },
{ "id": 3, "name": "car" },
{ "id": 4, "name": "motorcycle" },
{ "id": 5, "name": "airplane" },
{ "id": 6, "name": "bus" },
{ "id": 7, "name": "train" },
{ "id": 8, "name": "truck" },
{ "id": 9, "name": "boat" },
{ "id": 10, "name": "traffic_light" },
{ "id": 11, "name": "fire_hydrant" },
{ "id": 12, "name": "stop_sign" },
{ "id": 13, "name": "parking_meter" },
{ "id": 14, "name": "bench" },
{ "id": 15, "name": "bird" },
{ "id": 16, "name": "cat" },
{ "id": 17, "name": "dog" },
{ "id": 18, "name": "horse" },
{ "id": 19, "name": "sheep" },
{ "id": 20, "name": "cow" },
{ "id": 21, "name": "elephant" },
{ "id": 22, "name": "bear" },
{ "id": 23, "name": "zebra" },
{ "id": 24, "name": "giraffe" },
{ "id": 25, "name": "backpack" },
{ "id": 26, "name": "umbrella" },
{ "id": 27, "name": "handbag" },
{ "id": 28, "name": "tie" },
{ "id": 29, "name": "suitcase" },
{ "id": 30, "name": "frisbee" },
{ "id": 31, "name": "skis" },
{ "id": 32, "name": "snowboard" },
{ "id": 33, "name": "sports_ball" },
{ "id": 34, "name": "kite" },
{ "id": 35, "name": "baseball_bat" },
{ "id": 36, "name": "baseball_glove" },
{ "id": 37, "name": "skateboard" },
{ "id": 38, "name": "surfboard" },
{ "id": 39, "name": "tennis_racket" },
{ "id": 40, "name": "bottle" },
{ "id": 41, "name": "wine_glass" },
{ "id": 42, "name": "cup" },
{ "id": 43, "name": "fork" },
{ "id": 44, "name": "knife" },
{ "id": 45, "name": "spoon" },
{ "id": 46, "name": "bowl" },
{ "id": 47, "name": "banana" },
{ "id": 48, "name": "apple" },
{ "id": 49, "name": "sandwich" },
{ "id": 50, "name": "orange" },
{ "id": 51, "name": "broccoli" },
{ "id": 52, "name": "carrot" },
{ "id": 53, "name": "hot_dog" },
{ "id": 54, "name": "pizza" },
{ "id": 55, "name": "donut" },
{ "id": 56, "name": "cake" },
{ "id": 57, "name": "chair" },
{ "id": 58, "name": "couch" },
{ "id": 59, "name": "potted_plant" },
{ "id": 60, "name": "bed" },
{ "id": 61, "name": "dining_table" },
{ "id": 62, "name": "toilet" },
{ "id": 63, "name": "tv" },
{ "id": 64, "name": "laptop" },
{ "id": 65, "name": "mouse" },
{ "id": 66, "name": "remote" },
{ "id": 67, "name": "keyboard" },
{ "id": 68, "name": "cell_phone" },
{ "id": 69, "name": "microwave" },
{ "id": 70, "name": "oven" },
{ "id": 71, "name": "toaster" },
{ "id": 72, "name": "sink" },
{ "id": 73, "name": "refrigerator" },
{ "id": 74, "name": "book" },
{ "id": 75, "name": "clock" },
{ "id": 76, "name": "vase" },
{ "id": 77, "name": "scissors" },
{ "id": 78, "name": "teddy_bear" },
{ "id": 79, "name": "hair_drier" },
{ "id": 80, "name": "toothbrush" }
]
spec:
description: |
An implementation of Mask RCNN on Python 3, Keras, and TensorFlow.
runtime: "python:3.6"
handler: main:handler
eventTimeout: 30s
env:
- name: MASK_RCNN_DIR
value: /opt/nuclio/Mask_RCNN
build:
image: cvat/tf.matterport.mask_rcnn
baseImage: tensorflow/tensorflow:2.1.0-py3
directives:
postCopy:
- kind: WORKDIR
value: /opt/nuclio
- kind: RUN
value: apt update && apt install --no-install-recommends -y git curl libsm6 libxext6 libxrender-dev
- kind: RUN
value: git clone https://github.com/matterport/Mask_RCNN.git
- kind: RUN
value: curl -L https://github.com/matterport/Mask_RCNN/releases/download/v2.0/mask_rcnn_coco.h5 -o Mask_RCNN/mask_rcnn_coco.h5
- kind: RUN
value: pip3 install -r Mask_RCNN/requirements.txt
- kind: RUN
value: pip3 install pycocotools tensorflow==1.13.1 keras==2.1.0 pillow pyyaml
triggers:
myHttpTrigger:
maxWorkers: 2
kind: "http"
workerAvailabilityTimeoutMilliseconds: 10000
attributes:
maxRequestBodySize: 33554432 # 32MB
platform:
attributes:
restartPolicy:
name: always
maximumRetryCount: 3
These were investigated from create new task -> actions -> automatic annotation
Kick off serverless function. http://localhost:7000/api/v1/lambda/requests POST /api/v1/lambda/requests JSON Payload
/* Request */
{
"threshold": 0.5,
"max_distance": 50,
"task": 1,
"function": "openvino.omz.intel.person-reidentification-retail-0300"
}
/* Response */
{
"id": "6dd018f2-64fa-4c8f-948b-121c92be4bd1",
"function": {
"id": "openvino.omz.intel.person-reidentification-retail-0300",
"threshold": 0.5,
"task": 1
},
"status": "queued",
"progress": 0,
"enqueued": "2020-11-18T19:05:49.161191",
"started": null,
"ended": null,
"exc_info": null
}
Status and progress check on the serverless execution.
GET http://localhost:7000/api/v1/lambda/requests/6dd018f2-64fa-4c8f-948b-121c92be4bd1
{
"id": "6dd018f2-64fa-4c8f-948b-121c92be4bd1",
"function": {
"id": "openvino.omz.intel.person-reidentification-retail-0300",
"threshold": 0.5,
"task": 1
},
"status": "finished", //queued, started, finished
"progress": 100, //starts with 0, goes to 100
"enqueued": "2020-11-18T19:05:49.161191",
"started": "2020-11-18T19:05:49.174984",
"ended": "2020-11-18T19:05:49.313751",
"exc_info": null
}
Cancelling a serverless execution
DELETE /api/v1/lambda/requests/f73af3f3-bb09-4afb-9ad7-fc0084af9781
Returns 204 No Content
Recording request and responses for annotation kicked off from annotation page (of a given task).
Request
POST http://localhost:8080/api/v1/lambda/functions/openvino-mask-rcnn-inception-resnet-v2-atrous-coco
Body
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false, //deletes task data from database
"frame": 0,
"task": 1
}
POST http://localhost:8080/api/v1/lambda/functions/openvino-omz-intel-text-detection-0004
Body
{
"mapping": {
"text": "person"
},
"cleanup": false,
"frame": 0,
"task": 1
}
Response
[
{
"confidence": null,
"label": "person",
"points": [
496,
132,
531,
132,
531,
142,
496,
142
],
"type": "polygon"
},
{
"confidence": null,
"label": "person",
"points": [
528,
210,
555,
210,
555,
224,
528,
224
],
"type": "polygon"
},
//...continues...
]
Request
POST http://localhost:8080/api/v1/lambda/functions/openvino-omz-public-faster_rcnn_inception_v2_coco
Body
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
Response
[
{
"confidence": "0.99303883",
"label": "person",
"points": [
1176.3516235351562,
374.1847229003906,
1280.0,
699.4732475280762
],
"type": "rectangle"
},
{
"confidence": "0.98096424",
"label": "person",
"points": [
83.2307243347168,
359.85846519470215,
198.68478775024414,
662.8982162475586
],
"type": "rectangle"
}
//...
]
Request
POST http://localhost:8080/api/v1/lambda/functions/openvino-omz-public-yolo-v3-tf
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
Response
[
{
"confidence": "0.99985194",
"label": "person",
"points": [
61,
344,
217,
670
],
"type": "rectangle"
},
{
"confidence": "0.9997302",
"label": "person",
"points": [
849,
354,
1061,
720
],
"type": "rectangle"
}
//...
]
Request
POST http://localhost:8080/api/v1/lambda/functions/openvino-omz-semantic-segmentation-adas-0001
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
Response
[
{
"confidence": null,
"label": "person",
"points": [
0.0,
323.37236285124135,
6.0,
324.0474398505345,
15.0,
312.00384387108494,
20.0,
311.0982009090489,
30.0,
322.080764686252,
40.0,
314.0945640011426,
50.0,
315.0491316160951,
60.0,
328.34740633562683,
73.04631569391837,
321.0,
80.0,
311.1112883342544,
90.0,
320.3486430694041,
94.0,
307.0201630502398,
105.0,
310.0227706337829,
110.0,
317.0251536983245,
120.0,
311.09508015085714,
132.494829009255,
312.0,
150.0,
307.0128049933534,
156.98935181762772,
315.0,
159.0,
331.0842018812,
172.0,
318.55084849423577,
182.0,
316.06506673666183,
190.67774707788055,
332.0,
199.0,
333.1516860066772,
209.0,
325.35884974491347,
231.0,
324.0506705816824,
239.02297941787285,
304.0,
250.0,
297.10454904961733,
260.6684635517673,
313.0,
277.68845747767904,
323.0,
296.0,
317.0183710297214,
301.0,
306.59491616867854,
319.0,
307.08144052685134,
331.94103217703014,
316.0,
343.0,
331.0726127529786,
350.0757518263273,
331.0,
359.0,
315.25797276182834,
368.0,
317.0117430921824,
375.0,
329.0462554779512,
404.0,
310.02247539566923,
415.0,
312.4455892753989,
428.0,
324.04767394694977,
445.0,
314.0945640011426,
461.0,
314.0849363832643,
471.0,
308.4651703698857,
484.0,
316.12599086164903,
505.0,
319.0545221374546,
528.0,
313.52408225677027,
550.0,
319.0900326085039,
576.0,
312.0088626333916,
628.0,
316.10284709862407,
642.0,
307.0320390882156,
658.0,
313.0383187096331,
694.0,
301.2618609651209,
714.0,
302.1594923954951,
729.0,
287.3053112685377,
754.0,
292.14905146947785,
772.0,
287.079127770434,
788.0,
287.5540832648252,
799.0,
295.1567198469649,
804.0,
288.06639523736555,
822.0,
288.1418354820172,
829.0,
281.03676724827875,
831.0,
284.01719323093164,
838.0,
281.0283968255487,
863.0,
285.1049240938825,
868.0,
275.06743184072565,
879.0,
282.1023008393633,
898.0,
281.0743777913505,
911.0,
274.0102790191164,
934.0,
284.04028249934066,
938.0,
276.0921489779281,
943.0,
276.0908430935164,
944.9886432930049,
279.0,
938.0,
288.07953746229254,
958.0,
288.1256184498406,
964.0,
276.0186700565305,
976.0,
283.0408561949083,
989.0,
278.3350249308497,
1027.0,
288.1418354820172,
1050.0,
283.16421718096063,
1049.0211426102305,
278.0,
1054.0,
274.00917291257224,
1057.7679326741975,
284.0,
1076.0,
281.01218381599904,
1085.0,
284.037867207903,
1089.0,
281.01088333135954,
1095.8480959069284,
288.0,
1106.0,
290.6108268456344,
1107.0,
305.40559189265895,
1110.0,
307.1118507747409,
1120.5099102053518,
285.0,
1131.0,
278.12004829065535,
1135.3977600890803,
283.0,
1128.8712204679641,
284.0,
1126.9446128703871,
291.0,
1137.0,
292.102884668471,
1145.9832256890882,
304.0,
1132.8949558637546,
310.0,
1126.9510378657465,
333.0,
1140.9442632599732,
366.0,
1139.0,
369.89451412645786,
1129.0,
367.48427734725595,
1127.9797654671972,
381.0,
1132.9943118357103,
397.0,
1124.8850799165436,
407.0,
1123.3905951373736,
420.0,
1113.7858082079847,
422.0,
1109.8670001252606,
432.0,
1106.9129402826734,
490.0,
1111.0,
495.0173565216377,
1147.0,
497.02181807038977,
1164.0,
506.01106013026066,
1179.0,
522.0475830583266,
1188.003492593912,
445.0,
1183.014346038191,
390.0,
1189.092902282544,
384.0,
1177.0644757566283,
377.0,
1173.0146811187192,
352.0,
1187.10297640843,
327.0,
1184.0261863397004,
311.0,
1190.0,
308.05909303028216,
1199.1221619262933,
312.0,
1203.0121486702901,
292.0,
1201.001135134162,
282.0,
1209.0,
280.0856100641939,
1208.7395762108913,
294.0,
1219.0,
295.01689906365664,
1239.0,
323.18907329811844,
1244.0,
320.0713265237664,
1246.9324869216975,
323.0,
1251.8820231451618,
332.0,
1254.8356035999204,
356.0,
1258.8434430712487,
360.0,
1265.0,
357.09284142137903,
1268.0,
348.02790104589263,
1269.0,
354.2320348768142,
1274.0,
351.08694218918197,
1279.0,
357.096503035196
],
"type": "polygon"
},
{
"confidence": null,
"label": "person",
"points": [
1034.7221559550428,
330.0,
1044.3403315205298,
315.0,
1026.952746453593,
319.0,
1027.8689252091986,
325.0,
1034.7221559550428,
330.0
],
"type": "polygon"
},
//...
]
Request
POST http://localhost:8080/api/v1/lambda/functions/openvino.omz.intel.text-detection-0004
{
"mapping": {
"text": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
POST http://localhost:8080/api/v1/lambda/functions/openvino.omz.public.faster_rcnn_inception_v2_coco
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
POST http://localhost:8080/api/v1/lambda/functions/openvino.omz.public.mask_rcnn_inception_resnet_v2_atrous_coco
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
POST http://localhost:8080/api/v1/lambda/functions/openvino.omz.public.yolo-v3-tf
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
POST http://localhost:8080/api/v1/lambda/functions/openvino.omz.semantic-segmentation-adas-0001
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
POST http://localhost:8080/api/v1/lambda/functions/tf-faster-rcnn-inception-v2-coco
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
Response
[
{
"confidence": "0.9955258",
"label": "person",
"points": [
852.5807189941406,
373.9480447769165,
1062.0193481445312,
713.2855939865112
],
"type": "rectangle"
},
{
"confidence": "0.9941553",
"label": "person",
"points": [
1175.0753784179688,
370.0285005569458,
1280.0,
695.2166891098022
],
"type": "rectangle"
}//...
]
POST http://localhost:8080/api/v1/lambda/functions/tf-matterport-mask-rcnn
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
POST http://localhost:8080/api/v1/lambda/functions/tf.faster_rcnn_inception_v2_coco
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
POST http://localhost:8080/api/v1/lambda/functions/tf.matterport.mask_rcnn
{
"mapping": {
"person": "person",
"car": "car"
},
"cleanup": false,
"frame": 0,
"task": 1
}
Tracker across frames
POST http://localhost:8080/api/v1/lambda/functions/pth-foolwood-siammask
{
"task": 1,
"frame": 0,
"shape": [
1045.1845703125,
487.4638671875,
1230.0082702636719,
714.9391784667969
]
}
Response
{
"shape": [
1045.1845703125,
487.4638671875,
1230.0082702636719,
714.9391784667969
],
"state": {
"im_h": "720",
"im_w": "1280",
"p": "{\"py/object\": \"utils.tracker_config.TrackerConfig\", \"instance_size\": 255, \"base_size\": 8, \"out_size\": 127, \"seg_thr\": 0.35, \"penalty_k\": 0.04, \"window_influence\": 0.4, \"lr\": 1.0, \"total_stride\": 8, \"ratios\": [0.33, 0.5, 1, 2, 3], \"scales\": [8], \"round_dight\": 0, \"score_size\": 25, \"anchor_num\": 5, \"anchor\": {\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [3125, 4]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f4\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"<giant_base_64_string_of_image>\"}]}]}}",
"avg_chans": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [3]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"vZp4VqhsWUDiWdFINI1YQN+8mnhi1FlA\"}]}]}",
"window": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [3125]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"<base_64_image\"}]}]}",
"target_pos": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [2]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"AAAAvGLGkUAAAAC4nMmCQA==\"}]}]}",
"target_sz": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [2]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"AAAAwFsaZ0AAAADANW9sQA==\"}]}]}",
"net.zf": "{\"py/reduce\": [{\"py/function\": \"torch._utils._rebuild_tensor_v2\"}, {\"py/tuple\": [{\"py/reduce\": [{\"py/function\": \"torch.storage._load_from_bytes\"}, {\"py/tuple\": [{\"py/b64\": \"<base_64_image>\"}]}]}, 64, {\"py/tuple\": [1, 256, 7, 7]}, {\"py/tuple\": [57600, 225, 15, 1]}, true, null]}]}"
}
}
POST http://localhost:8080/api/v1/lambda/functions/pth-foolwood-siammask
{
"task": 1,
"frame": 2,
"state": {
"im_h": "720",
"im_w": "1280",
"p": "{\"py/object\": \"utils.tracker_config.TrackerConfig\", \"instance_size\": 255, \"base_size\": 8, \"out_size\": 127, \"seg_thr\": 0.35, \"penalty_k\": 0.04, \"window_influence\": 0.4, \"lr\": 1.0, \"total_stride\": 8, \"ratios\": [0.33, 0.5, 1, 2, 3], \"scales\": [8], \"round_dight\": 0, \"score_size\": 25, \"anchor_num\": 5, \"anchor\": {\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [3125, 4]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f4\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"=\"}]}]}}",
"avg_chans": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [3]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"vZp4VqhsWUDiWdFINI1YQN+8mnhi1FlA\"}]}]}",
"window": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [3125]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"==\"}]}]}",
"target_pos": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [2]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"AACAJhRzkUAAAKC2uuWCQA==\"}]}]}",
"target_sz": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [2]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"4C/1Ka6AZkDg3/zEX2RtQA==\"}]}]}",
"score": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray.scalar\"}, {\"py/tuple\": [{\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f4\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, {\"py/b64\": \"Nv5/Pw==\"}]}]}",
"ploygon": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [4, 2]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f4\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"0X6XRJpNM0SFHYFEVvwyRJdRgUQkP/ND47KXRKzh80M=\"}]}]}",
"net.zf": "{\"py/reduce\": [{\"py/function\": \"torch._utils._rebuild_tensor_v2\"}, {\"py/tuple\": [{\"py/reduce\": [{\"py/function\": \"torch.storage._load_from_bytes\"}, {\"py/tuple\": [{\"py/b64\": \"===\"}]}]}, 64, {\"py/tuple\": [1, 256, 7, 7]}, {\"py/tuple\": [57600, 225, 15, 1]}, true, null]}]}"
}
}
Response
{
"shape": [
1190.830078125,
718.5213012695312,
1016.8630981445312,
716.9252319335938,
1018.958984375,
488.47662353515625,
1192.926025390625,
490.07269287109375
],
"state": {
"im_h": "720",
"im_w": "1280",
"p": "{\"py/object\": \"utils.tracker_config.TrackerConfig\", \"instance_size\": 255, \"base_size\": 8, \"out_size\": 127, \"seg_thr\": 0.35, \"penalty_k\": 0.04, \"window_influence\": 0.4, \"lr\": 1.0, \"total_stride\": 8, \"ratios\": [0.33, 0.5, 1, 2, 3], \"scales\": [8], \"round_dight\": 0, \"score_size\": 25, \"anchor_num\": 5, \"anchor\": {\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [3125, 4]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f4\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"=\"}]}]}}",
"avg_chans": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [3]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"vZp4VqhsWUDiWdFINI1YQN+8mnhi1FlA\"}]}]}",
"window": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [3125]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"==\"}]}]}",
"target_pos": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [2]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"AAAAv+kvkUAAAJCe+OeCQA==\"}]}]}",
"target_sz": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [2]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f8\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"dp3rDXpSZkBVSUusLz1tQA==\"}]}]}",
"score": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray.scalar\"}, {\"py/tuple\": [{\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f4\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, {\"py/b64\": \"bPh/Pw==\"}]}]}",
"ploygon": "{\"py/reduce\": [{\"py/function\": \"numpy.core.multiarray._reconstruct\"}, {\"py/tuple\": [{\"py/type\": \"numpy.ndarray\"}, {\"py/tuple\": [0]}, {\"py/b64\": \"Yg==\"}]}, {\"py/tuple\": [1, {\"py/tuple\": [4, 2]}, {\"py/reduce\": [{\"py/type\": \"numpy.dtype\"}, {\"py/tuple\": [\"f4\", 0, 1]}, {\"py/tuple\": [3, \"<\", null, null, null, -1, -1, 0]}]}, false, {\"py/b64\": \"kNqURF2hM0Q9N35ENzszRGC9fkQCPfRDoh2VRE4J9UM=\"}]}]}",
"net.zf": "{\"py/reduce\": [{\"py/function\": \"torch._utils._rebuild_tensor_v2\"}, {\"py/tuple\": [{\"py/reduce\": [{\"py/function\": \"torch.storage._load_from_bytes\"}, {\"py/tuple\": [{\"py/b64\": \"==\"}]}]}, 64, {\"py/tuple\": [1, 256, 7, 7]}, {\"py/tuple\": [57600, 225, 15, 1]}, true, null]}]}"
}
}
Request
POST http://localhost:8080/api/v1/lambda/functions/openvino-dextr
{
"frame": 1,
"points": [
[
741.5517578125,
713.9072265625
],
[
697.884765625,
585.9521484375
],
[
732.412109375,
508.7724609375
],
[
665.3876953125,
459.0126953125
]
],
"task": 1
}
Response
[
[
708,
530
],
[
707,
530
],
[
705,
534
],
//...
]
Request
POST http://localhost:8080/api/v1/lambda/functions/pth-saic-vul-fbrs
{
"frame": 1,
"points": [
[
781.15625,
580.8740234375
],
[
771.0009765625,
382.8486328125
]
],
"task": 1
}