Display bucket objects' metadata in Wing Console

[polamoros]

Feature Spec

see @Chriscbr comment https://github.com/winglang/wing/issues/1686#issuecomment-1456658687

Use Cases

see @ShaiBer comment https://github.com/winglang/wing/issues/1686#issuecomment-1735511301

Implementation Notes

No response

Component

Wing Console

Community Notes

• Please vote by adding a 👍 reaction to the issue to help us prioritize.
• If you are interested to work on this issue, please leave a comment.
• If this issue is labeled needs-discussion, it means the spec has not been finalized yet. Please reach out on the #dev channel in the Wing Slack.

[ainvoner]

both get and put uses "utf8" encoding. we should support mime-type and write/read files using the proper encoding

[staycoolcall911]

@Chriscbr / @hasanaburayyan - do you think it makes sense to add an optional mime_type parameter (default: utf8) to Bucket.get() and Bucket.put()? Any other solution?

[Chriscbr]

A mime type option sounds good to me.

We might also want to add this as a requirement for a more general Blob type (so put and get are not only operating on string types)

Here's an API sketch of what I'm thinking:

bring cloud;

let b = new cloud.Bucket();

let handler = inflight () => {
  let img = b.get("image.png"); // type: Blob
  print(img.mime_type); // "image/png"
  print(img.content); // raw bytes? not sure...
  
  let blob = Blob.from_file("data.json"); // auto-infer mime type from file metadata perhaps?
  let blob2 = Blob.from_file("image.jpg", { mime_type: "image/jpeg" }); // or just specify explicitly
  b.put(blob);
};

(What are some other use cases for Blob?)

[ainvoner]

@Chriscbr I like it, this is what I had in mind.

[staycoolcall911]

We decided this is not important for beta. Removing from the current sprint as well, of course

[github-actions[bot]]

Hi,

This issue hasn't seen activity in 60 days. Therefore, we are marking this issue as stale for now. It will be closed after 7 days. Feel free to re-open this issue when there's an update or relevant information to be added. Thanks!

[ShaiBer]

Hey, I think we should increase the priority of this one to P1 as people are starting to get blocked by it and there are no good workarounds. As an example, here is an email I got today:

... In short, I’m very excited about winglang, it feels like an ideal future for most Application Developers / Engineers who really just want to focus on their App without concern of their internal infrastructure mechanics.

With AWS as my target cloud given the coverage of implementation so far, the example I decided upon was “A cloud service (API Gateway + Lambda) that will receive a POST body json object and create a timeline image to store in a bucket (S3)”. I was mostly aiming to understand ‘the way’ of structuring a wing project beyond a singular main.w file. I have 2 files, main.w and timeline.js which I’ve included so you can see all the things I’m probably doing wrong 😊 For now I’m only generating a red square as I wanted to connect the everything before building it out.

I was able get everything working along the ‘happy path’ and deployed into AWS via terraform cli. The only issue I ran into (which I believe could be addressed as follows based on my own past AWS usage) was the ContentType: image/png is not available to configure, or automatically set based on the object key’s ‘file extension’.

I thought this was a reasonable stopping point to come back to you, I have a few work-arounds in mind (i.e. a download endpoint where I specify the ContentType via a Header) as I wanted to maintain a ‘cloud ubiquity’ and avoid integration via AWS SDK. I’m very open to your thoughts and feedback. What would you hope a developer do to address a gap like this?

Generally this was just amazing to get this far with so little code and so much of the traditional cloud Infra / Security / Networking / Source Code challenges just handled by winglang.

Thanks for building something for us futurists out there to enjoy today....

main.w

bring cloud;
let api = new cloud.Api();
let bucket = new cloud.Bucket();

class Timeline {
   init() {}
   extern "./helpers.js" pub static inflight uuid():str;
   extern "./timeline.js" pub static inflight load(message:str):str;
   extern "./timeline.js" pub static inflight generate():str;

   pub inflight save(id:str, body:str):str {
       Timeline.load(body);
       return Timeline.generate();
   }
}

let t = new Timeline();

api.post("/", inflight (request: cloud.ApiRequest): cloud.ApiResponse => {
  if let body = request.body  
  {
       let id = Timeline.uuid();
       let timelineData = t.save(id,body);
       log(timelineData);
       bucket.put(
           id + ".png",
           timelineData
       );

       return cloud.ApiResponse {
           status: 201,
           body: timelineData
       };
   }
});

timeline.js

const uuid = require('uuid')
import * as PImage from "pureimage"
import * as fs from 'fs'
id = uuid.v4()

data = {
   message: ""
}

const delay = ms => new Promise(res => setTimeout(res, ms));

exports.init = () =>
{
   console.log('Timeline init');
}

exports.load = (message) =>
{
   console.log('Timeline load');
   data.message = message
}

exports.createTimeline = async (cb) =>
{
   const img = PImage.make(100, 100)
   const ctx = img1.getContext('2d');

   ctx.fillStyle = 'red';
   ctx.fillRect(0,0,100,100);

   let i = await PImage.encodePNGToStream(img, fs.createWriteStream('/tmp/out.png')).then(() => {
       const contents = fs.readFileSync('/tmp/out.png', {encoding: 'base64'});
       return contents
   }).catch((e)=>{
       return undefined;
   });

   return i;
}

exports.generate = async () =>
{
   let image64 = await this.createTimeline( async(image) => {return image})
   return image64;
}

[staycoolcall911]

Thanks for the feedback @ShaiBer. Sure, I increased the priority to p1

[ShaiBer]

🙏

[skyrpex]

I get corrupted data in buckets when uploading files from the Console, too.

[polamoros]

I get corrupted data in buckets when uploading files from the Console, too.

Could you provide the file that is being corrupted once uploaded? Is it also corrupted after being downloaded?

Perhaps it's only when displayed as base64 in the file preview? I've created an issue to add a couple of improvements to the file previewer. https://github.com/winglang/wing/issues/4531

[polamoros]

I think we should go with @Chriscbr suggestion: https://github.com/winglang/wing/issues/1686#issuecomment-1456658687

[garysassano]

After #4515 gets merged, all uploaded objects will have metadata attached to them, so we can take that metadata and display it in the Wing Console when clicking/hovering over an object in a bucket.

[github-actions[bot]]

Hi,

This issue hasn't seen activity in 90 days. Therefore, we are marking this issue as stale for now. It will be closed after 7 days. Feel free to re-open this issue when there's an update or relevant information to be added. Thanks!