Uploading to S3 buckets

[chris-martin]

NixOS/nixops#173 discusses creating S3 buckets.

Can nixops also upload things into the buckets?

[cessationoftime]

@edolstra I plan to try to build a solution for uploading to S3 buckets, but I am wondering if it is more appropriate to build it as part of the existing S3BucketState class, or as a separate resource/class (S3BucketFile) that depends on an existing bucket (whether the bucket is NixOps managed or not). I think the latter, but I'm not yet familiar enough with how NixOps works to be certain.

[edolstra]

I'm not sure. On the one hand, fine-grained resources are nice because they can be managed (created/deleted/...) separately. On the other hand, there could be a lot of files which would potentially clutter up the UI.

[cessationoftime]

I've made progress on this and I can upload files to S3 buckets. However I want to upload the output of a custom derivation.

So I have code like:

let
  mypkgs = import ./my-pkgs { };

  ec2Machine =
    { config, pkgs, resources, ... }:
    { deployment.targetEnv = "ec2";
...
...
    };

in
{
  ...
  ...
  resources.s3BucketUploads.frontend-upload =
    { inherit region accessKeyId;
      description = "web frontend";
      bucketName = "deploy.haste.us";
      source = "${mypkgs.frontend}/bin";
      #source = "/home/cessationoftime/workspace/frontend/dist/bin";
      bucketDestination = "frontendFolder";
    };

    web1 = ec2Machine;
}

When I use a source directory value of "/home/cessationoftime/workspace/frontend/dist/bin" the upload works fine, but if I use "${mypkgs.frontend}/bin" then the directory cannot be found. I need a way to trigger the build for ${mypkgs.frontend} before performing the upload and I'm not sure how best to go about that. I see the build_configs function, do I need to create a similar function in my resource or is there another way to trigger the build for ${mypkgs.frontend}?

[domenkozar]

I'll fix that shortly.

[cessationoftime]

I ended up calling nix-build within the resource. See: s3_bucket_upload.py. It is working well and I have a derivation that runs webpack and uploads to S3 which deploys alongside a website that references the S3 location. But I think it could be cleaner. @domenkozar

[glittershark]

@cessationoftime did you ever end up making more progress on this, or is your code available somewhere? This is something I'd like