serverless-online-service

serverless-online-service copied to clipboard

Howdy folks – exploring using webhint for a new effort. As part of that effort exploring what it would take to run natively on AWS.

Looking through the code there are two places Azure specific services are utilized:

• azure-sb
• azure-storage

Would this group be open to accepting contributions that enable multiple vendors for these services?

Would this group be open to accepting contributions that enable multiple vendors for these services?

Definitely!

The architecture we have right now is to enable us scaling in case we have more load. Depending on your use case it might not be necessary.

This is what we are using those packages for:

• azure-sb for queues and communicate the services.
• azure-storage store the backups of the database.

@utsavized is in the process of migrating some of the current microservices to be serverless instead of containers so expect the code to change a bit in the next few weeks. Another change we are planning on soon-ish is to change the connector in here from jsdom to chrome, but that should just impact the worker image and the configuration file.

If you want we can set up a meeting to go through the current architecture.

Let us know if you need anything!

Would love to chat about this – especially on the serverless aspect of it. A vendor agnostic serverless implementation would be important for us to adopt this.

Let me know when you would like to catch up on a quick call.

What about Tuesday? Any time that works best for you? Right now my schedule is open morning and afternoon (I'm in PST).

Still up to chat today? Would say, 11am PT work?

If not when later in the week is best? Wednesday is very full for me, but I could do Thursday or Friday.

Today would be best though -- apologies for the delayed response.

I think we can make it happen today 😊 Hangouts OK? I just followed you on Twitter (@molant) so feel free to DM the emails for the meeting.

Updating with link to Hangouts: https://hangouts.google.com/call/2YU1Ci08iaTVJDnrwecHAEEE

Thanks all of you for attending! The next steps we agreed are:

• @utsavized: use serverless in our migration to Azure Functions so integration with any other cloud platform will be possible
• @indexzero: work in pkgcloud to add support for Service Bus and help in the migration to use that library

I'll leave this issue open while we work on this and link to webhintio/online-service#388 @utsavized feel free to create more tasks if needed!

We've created webhintio/online-service#442 for the serverless integration.

Awesome! Will keep an eye out for some PRs. Might be useful to break it up into multiple issues once you know which microservices will become serverless.

We'll make sure to ping you once we start having PRs. I think that initially all excep the worker and that one later once we investigate webhintio/online-service#386

FYI @molant & other webhint folks – released [email protected] in the past week. Would love to talk about how we could start moving towards using it for azure-storage

Awesome!

@utsavized has continue making progress during the last few days but had to slow down because of sickness 🤒 I think the current status is that most microservices are migrated to run as functions locally but when deployed to our staging environment they use about 1GB of space which doesn't make too much sense. We'll look into that as soon as he's back to work.

IIRC we are currently using azure-storage for the backups of the DB in storage.ts. That should be issue to migrate.

Do you know if there's been any progress about queues and such?

Thanks!

@indexzero we've created a new repo for serverless while we do all the migration:

https://github.com/webhintio/serverless-online-service

We're getting ready to kick off that work on our side. Is the goal to migrate all of the remaining services in ./src/lib/microservices into serverless-online-service?

Would love to hop on a quick call again and chat about how our plans to use webhint have evolved over time.

We're getting ready to kick off that work on our side. Is the goal to migrate all of the remaining services in ./src/lib/microservices into serverless-online-service?

@indexzero Yes, at least the one's that are applicable. The only one that is doubtful is the backup service that deals with a large data set, but the rest should be able to be converted. Currently, we have deployed the two converted functions into staging environment and are testing them.

Also, Serverless has some bugs on the Azure side, and looks like they have not updated their azure-functions package in a while. I will submit a PR and see how that goes. I expect the AWS side to be much smoother.

Great – I'd love to hop on a call next week and chat about it more. Have a few high level ideas I'd like to run by the group before opening a large proposal.

I'd love to hop on a call next week

Right now, any day next week is good for me for that call. Let see when @antross and @utsavized have time.

@indexzero Sure, next week sounds good to me. @antross do you want to set something up?

How about 1:30pm PT on Monday?

@indexzero That should work. I've scheduled a sync via zoom for 1:30pm PT on Monday. If you can DM me your email on our Gitter I'll forward you the invite.

@indexzero Is today still a good time to meet at 1:30pm PT? Here's a link to join the meeting: https://zoom.us/j/863328597

Yes that still works, speak with you then!