cpp backend logging and load model

[maaquib]

Description

• Folly log integration: I tried to implement a factory pattern to keep logging decoupled to some extent but Generics and Macros were giving a lot of trouble. So I have kept it tightly coupled to folly logging for the time being. Will do type erasures to fix this once the other functionality has been implemented
• Added encoding for successful load model response
• TODO: Add more tests for OTF protocol and logging macros

Create sample request for testing

import struct

buf = bytearray()
buf += b'L'
model_name = b'mnist'
buf+=struct.pack("!i", len(model_name))
buf+=model_name
model_path=b'/tmp/models/b946065d4921408b88bc89f0b1bc5cd6'
buf+=struct.pack("!i", len(model_path))
buf+=model_path
#Batch Size
buf+=struct.pack("!i",1)
handler=b'BaseHandler'
buf+=struct.pack("!i", len(handler))
buf+=handler
#GPU ID
buf+=struct.pack("!i",-1)
#Envelope
envelope=b''
buf+=struct.pack("!i", len(envelope))
buf+=envelope
buf+=struct.pack("!?", True)

with open('req.bin', 'rb') as f:
    f.write(buf)

Test happy case

$TS1> cat logging.config
INFO:default; default=file:path=/tmp/ts.log,async=false
$TS1> ./build.sh && ./_build/src/backends/model_worker_socket --model_dir /tmp/models/b946065d4921408b88bc89f0b1bc5cd6 --logger_config_path logging.config
$TS2> nc 127.0.0.1 9000 < req.bin > response.txt

Test response

import struct

with open('response.txt', 'rb') as fb:
    status_code = struct.unpack("!i", fb.read(4))
    length = struct.unpack("!i", fb.read(4))
    message = fb.read(length[0])
    no_pred = struct.unpack("!i", fb.read(4))

status_code, length, message, no_pred

Model dir

> tree /tmp/models/b946065d4921408b88bc89f0b1bc5cd6/
/tmp/models/b946065d4921408b88bc89f0b1bc5cd6/
├── MAR-INF
│   └── MANIFEST.json
└── mnist_cnn.pt
> cat /tmp/models/b946065d4921408b88bc89f0b1bc5cd6/MAR-INF/MANIFEST.json     
{
  "createdOn": "23/08/2022 11:51:46",
  "runtime": "python",
  "model": {
    "modelName": "mnist",
    "serializedFile": "mnist_cnn.pt",
    "handler": "BaseHandler",
    "modelFile": "mnist.py",
    "modelVersion": "1.0"
  },
  "archiverVersion": "0.5.3"
}