How to log model?

[mashu]

Hi,

It would be great if there was an example showing how to log model architecture Is that possible currently ?

[john-waczak]

^ I was just looking for the same. Is the only way to log the models via saving them as artifacts? If so, what would it take to set up model logging? I'd be happy to take a crack at it!

[mcamp-ata]

I dont know how much ehlp this is but this is what I have crafted to be able to do this kind of...

using AWSS3
using AWS: global_aws_config, AWSConfig
using Minio


function logImgArtifact(mlf,exprun,img;
  artifactName="$(Dates.datetime2unix(now())).png",
  s3cfg=MinioConfig("https://s3-api.lan.example.com", s3creds; region="us-east-1")
)
  save(artifactName, img)
  bucket = logartifact(mlf,exprun,artifactName)
  object_name = basename(bucket)
  open(artifactName, "r") do file
      file_data = read(file)
      Minio.s3_put(s3cfg, dirname(bucket)[6:end], object_name, file_data)
  end
  rm(artifactName)
  bucket
end


function logModelArtifact(mlf,exprun,model_state;
  artifactName="$(Dates.datetime2unix(now())).jld2",
  s3cfg=MinioConfig("https://s3-api.lan.example.com", s3creds; region="us-east-1")
)
  model_state = model_state |> cpu
  jldsave(artifactName, model_state=model_state)
  bucket = logartifact(mlf,exprun,artifactName)
  object_name = basename(bucket)
  open(artifactName, "r") do file
      file_data = read(file)
      Minio.s3_put(s3cfg, dirname(bucket)[6:end], object_name, file_data)
  end
  rm(artifactName)
  bucket
end

It just saves the model as an artifact that is in an s3 bucket. I'm currently exploring what needs to be done to make show up as an actual model in MLFlow.

[mashu]

Tieing to one commercial solution such as AWS might not be very generic. Artifacts can already be logged.

[mcamp-ata]

when I used the log function it just returns me a path, it doesn't save the files to the artifact location defined in the experiment (S3 bucket in my mlflow case). I wrote those two functions to make my life a little easier for pushing the artifacts to S3. And this doesn't tie you to AWS. I am actually using a self hosted Minio S3 instance.