Case study proposal - Ecommerce web application using eShoponWeb

[atg-abhishek]

Discussed in https://github.com/Green-Software-Foundation/software_carbon_intensity/discussions/225

^{Originally posted by srini1978 January 14, 2022} As we are starting to identify case studies, one option we can use is to take a popular ecommerce open source code and calculate SCI for it. eShop is a popular one that is Asp.net core based. There are multiple variants - using containers, using Managed web services

https://github.com/dotnet-architecture/eShopOnWeb

SCI = (E*I) +M per R

Proposal

1. Deploy the application in Web based services on any cloud provider . For e.g App service in Azure with backend SQL server. For managed azure services, we will get the Vcore , RAM specs (# of cores , 3 GB/4 GB RAM). Calculate M based on spec sheet . Need to find out if we can get the thermal design power TDP value from the specs.
2. Run the application in Azure and calculate CPU consumption, Memory consumption and Disk consumption. Use the combination of these values to come up with E value.
3. Use Watt time API to get the marginal carbon intensity value of I for the data center in which the application is deployed.
4. R will be the functional unit here. In this case, we can consider R as "per number of users". For example SCI value of x per 100 users or 1000 users depending on the non functional requirements of the application.

@Henry-WattTime @atg-abhishek

[srini1978]

Please find attached the case study document for Ecommerce web application. Inferences and observations are also documented - https://docs.google.com/document/d/1i2ECykAXiQNwv0eeuavYMSYUtrob9HuY/edit?usp=sharing&ouid=114480843539813321551&rtpof=true&sd=true

[atg-abhishek]

Thank you for putting this together @srini1978 - maybe you can also share it on the Slack channel for those who don't get a chance to check the issue here?

[srini1978]

@atg-abhishek @Henry-WattTime Case study documented updated and complete

[srini1978]

Overview

The application is a web application that is used in an ecommerce scenario by customers. It is a sample ASP.NET Core reference application, powered by Microsoft, demonstrating a single-process (monolithic) application architecture and deployment model.

Architecture for the system under consideration

The architecture of the application is described in detail here -https://github.com/dotnet-architecture/eShopOnWeb. The architecture basically consists of a monolithic web application powered by a relational database.

Technical details of the components in the architecture

Technical details of the components in the architecture is here --https://github.com/dotnet-architecture/eShopOnWeb The eShopOnWeb reference application employs a number of software design patterns which are implemented either in the code itself, or with the assistance of various open source NuGet packages.

MVC

The front end of the application's web project implements the class Model-View-Controller pattern. It includes a number of controllers in its Controllers folder, which work with Views to return rendered HTML to the client, typically a browser. The sample also demonstrates the use of alternate flavors of the MVC pattern, including Razor Pages and API Endpoints, described in more detail below.

Sites for Software Sustainability Actions

Energy Efficiency

While energy efficiency is not in the scope of this use case, Energy efficiency can be achieved by optimizing at all the software boundary components defined below - App server, Database server, Network components etc by additional tuning. Energy efficiency techniques include but not limited to: adding caching, optimizing the queries being fired against the database etc. However tuning involves spending additional energy which is for now, not accounted in the E value as the tuning effort is done in a physical infrastructure that is outside the boundary of how we are measuring R. .

This is planned to be included in the future versions of the SCI specifications as per #223

Hardware Efficiency

Hardware efficiency is achieved by reducing the number of physical resources for the software to run. From the list of components that are part of the software boundary (below), this is the App server and Database server where such tuning techniques can be tried out. At the same time, as these tests demonstrate there is a minimum hardware that needs to be provisioned to ensure the overall SCI value is lowered.

Carbon Awareness

While carbon awareness is not in scope of this use case, we can make the application more green by shifting the application workload to a cloud region that has lower carbon intensity. The electricity grid in India has poor carbon intensity values as opposed to electricity grids in Europe and Americas. Hence a drastic SCI reduction value can be obtained in both runs by moving the workload to a different region (making the application carbon aware)

Procedure

(What) Software boundary

• App server for Web application

Excluded components

• Front end web application .This is the browser and the client that is displaying the application to the end users on their desktop/mobile/laptop etc. (Since the use case is focusing on hardware efficiency and hardware efficiency is not focused on reducing physical resources on the client desktop, this component is excluded)
• Network traffic between web application and servers ( The use case is not from an end to end standpoint and hence including the network traffic is not required)
• Network traffic between servers and database ( The intent of this case study is to analyze the impact of changing the infrastructure of the web servers alone (hardware efficiency) and hence the network traffic between application server and database is not considered and included).
• Databases (Not included in calculations as the database server configuration was kept constant between the test runs.)
• Test infrastructure . This is the load test resource that was used to simulate virtual users and http requests to the web server. This is a serverless resource and since this resource does not fit into the same functional unit scale as we had defined in the overall test plan, this infrastructure and the associated energy usage does not fit into the software boundary definition and hence is excluded.

(Scale) Functional unit

The functional unit R remained the same across all the tests i.e. Number of users as a metric but the values changed during the load test. There were load test times at which peak user load of 45 was attained but there are times at which there were hardly 10 users.

Hence when we define R as the number of users, we should call out that this value is during the peak usage of the application. For example, we can define SCI as a value per R, at the point when measurement is taken if we consider R as “number of concurrent users on the system”.

The choice of functional unit applies to all components in your software boundary.

For example in this case

• per 45 users

(How) Quantification method

The quantification is done by a combination technique - measurement of real world data for e.g CPU utilization of the app servers, Thermal design power (TDP) of the processors, number of cores etc and an estimation model for the GPU and Memory. Since the energy values for memory are much lower than the calculated energy values for processors or CPUs, we consider these values negligible.

We did multiple Load Test runs with different infrastructures and calculated the power consumed in these runs. For these tests, we used a functional unit reference of number of users . The energy value is calculated using models which take as input the CPU utilization of servers. Time component is constant, and all tests are run for the same time period.

The original intent of the test was to prove that SCI score can be reduced by making the application more hardware efficient. i.e., using fewer physical resources. As part of the test plan, we intended to prove that SCI score can be reduced by using Appserver with 1 core as opposed to 2 Core as compared to quad core.

(Quantify) SCI Value Calculation

The Energy value is calculated using the formula

P[kwH] = (Power consumed by CPU or Pc Number of cores + Power consumed by Memory or Pr + Power consumed by GPU or Pg Number of GPUs)/1000

There is no GPU. Hence Pg =0. Similarly in this case we are approximating PR=0 as we found that Power consumed by 4GB memory is close to 1.45 W and that by 8 GB memory is approximately 2.45 W and hence these values are much lower compared to that consumed by processors.

Detailed calculations are in the report below

(Report)

https://docs.google.com/document/d/1i2ECykAXiQNwv0eeuavYMSYUtrob9HuY/edit?usp=sharing&ouid=114480843539813321551&rtpof=true&sd=true

[atg-abhishek]

Network traffic between servers and database ( The intent of this case study is to analyze the impact of changing the infrastructure of the web servers alone (hardware efficiency) and hence the network traffic between application server and database is not considered and included).

On the above @srini1978 , isn't there a potential that the way the infrastructure is setup on the web server side, it will have impacts on the kind and size of network traffic that is emitted and hence the network traffic could potentially be included as well? Or conversely, the kind of network traffic being emitted from the database will have processing implications on the side of the server?

[srini1978]

There are couple of ways infrastructure can be setup on web server side. The database server can either be in the same datacenter or different data center or it could be in another cloud region. In all cases, if we assume that the traffic is going through internet backbone (the worst case scenario) rather than the cloud provider's own backbone, we have got some data from studies that energy cost of network traffic is approx 0.023 KwH /GB. Reference - https://medium.com/teads-engineering/evaluating-the-carbon-footprint-of-a-software-platform-hosted-in-the-cloud-e716e14e060c#3bf5.

We can calculate the data in and data out for both the web servers and database server and use the above approximation to come up with energy costs of network traffic. Based on preliminary calculations I did these numbers are not negligible and it comes to approx 0.092 KwH for a 24 hour period . Compare this to CPU energy cost of 1.8014 KwH for a 24 hour period.

However the above calculations hinge upon the reference value of 0.023 KwH/GB and hence as part of the SCI calculations we need to ratify this. @atg-abhishek

[davidmytton]

Note that figure of 0.023 kWh/GB is for the IP core network only and is from 2015. You'll need to check the network component is relevant and apply improvement factors for each year since then (improves by half every ~2 years)

[mrchrisadams]

Hey folks.

I've been watching this issue with interest, the last few comments bring up a a good point.

I know it's excluded from the initial model, and but since it came up it brings up an thorny issue - how to represent the emissions from transfer on a marginal basis for the SCI.

We might pay for bandwidth on a per gigabyte basis, but that's because of the business model behind cloud. It doesn't necessarily mean the energy usage and emissions scale the same way.

This peer reviewed paper, The real climate and transformative impact of ICT: A critique of estimates, trends, and regulations summarises it pretty succinctly:

Is data traffic independent of ICT emissions?

The amount of data traffic on the internet at a given time does not correspond with simultaneous increases in ICT's emissions. Instead, network operators plan capacity for peak data traffic,23 meaning emissions from ICT are fixed regardless of the amount of data traffic until growth in peak capacity is required.

This is a fairly accessible dive into how selling on a volumetric basis differs from the actual capacity being reserved - it's referring to the economics here, rather than the energy usage, but I think it gets across the idea that how you charge for transfer can map very differently to the underlying resource usage.

Charging for Stocks, Paying for Flows

AWS charges customers based on the amount of data delivered — 1 terabyte (TB) per month, for example. To visualize that, imagine data is water. AWS fills a bucket full of water and then charges you based on how much water is in the bucket. This is known as charging based on “stocks.”

On the other hand, AWS pays for bandwidth based on the capacity of their network. The base unit of wholesale bandwidth is priced as one Megabit per second per month (1 Mbps). Typically, a provider like AWS, will pay for bandwidth on a monthly fee based on the number of Mbps that their network uses at its peak capacity. So, extending the analogy, AWS doesn't pay for the amount of water that ends up in their customers' buckets, but rather the capacity based on the diameter of the “hose” that is used to fill them. This is known as paying for “flows.”

Source: AWS’s Egregious Egress by @cloudflare

I've also asked this on linkedin below, to get some numbers back.

https://www.linkedin.com/feed/update/urn:li:activity:6893929195408625664/

Given that network transfer is the thing continually coming up in the media, as well as coming up when looking at network traffic between servers as well, I wanted to ask what other valid approaches you've seen that would work for the SCI.

Pretty much every approach I've seen uses what I would associate as an attributional approach, of dividing total energy, by total transfer for a given system, usually on an annual basis.

Have you folks seen any useful guidance for using a marginal approach in this case for network transfer?

[Henry-WattTime]

WG comments: Question: does energy consumption of networking equipment scale with data throughput or is it constant regardless of throughput? Question: Is it different for cloud or on-prem? Asim hypothesis: Networking technology consumes data no matter what. for example, the network card in laptop consumes electricity no matter what.

Different networks types have different emissions? Cell vs. wifi network, and bias action to one.

Chris: It is possible to reduce the energy of networking equipment -deepening on when in use, or idle can use -can reduce power up to half [https://www.usenix.org/legacy/events/nsdi08/tech/full_papers/nedevschi/nedevschi_html/index.html]

Is this a SCI data question? Can the WG provide feedback.

Gadhu: NTT research project is to reduce energy consumption of networking equipment -biggest network providers researching. -Electronics to photonics, reduces energy use by 100x goal -moving from light in wires, to light in datacenters -https://www.rd.ntt/e/iown/

Daniel Schien - emissions data from streaming, might have feedback?

Gadhu: is this number important? Is the energy use/emissions of networking energy significant compared to CPUs? Is it like RAM and inconsequential? Srini: data is a part of delivering product. Network traffic is tracked.
Asim: network emissions is a hot topic of conversation. Shift institute number provided a few years ago. Hotly contested, claims too high, but no one has presented another number.

Navveen: https://www.cloudcarbonfootprint.org/docs/methodology - This takes in account networking and memory usage, might be a useful resource.

[atg-abhishek]

We could work with the CWG on how we applied the SCI to this case study. cc @Henry-WattTime

[atg-abhishek]

@seanmcilroy29 to reach out to Chris about SCI Open Data to calculate network emissions

[atg-abhishek]

We can also coordinate with Chris on the SCI Reporting once that spec is more well written out.

[jawache]

Excellent work @srini1978! So as I read this, the only component in the software boundary in this case is the "application server" is that correct? Which is fine, the whole point of documenting the software boundary is to make these distinctions clear.

1. I think there a couple of assumption in the doc which would like some clarity on. The M values seem incredibly high, which is why the SCI scores are so high. I suspect you are using the total LCA of the server and have not proportioned it to the slice of time you have rented it for. Can you elaborate?

EI =3185.9451 gms of Co2 eq M- 1205.52 kgCo2 eq. SCI = (EI) +M = 1208.71 KgCo2 per 45 users

2. Can you clarity what the "R" is in the calculation. You mention in the doc about R being the “number of concurrent users on the system” - so the SCI you are calculating is the carbon per "number of concurrent users on the system" which I'm struggling to wrap my head around. I think the SCI score you've calculated is the score for performing the whole test run, rather than per user, am I correct in my understanding? If so I would argue that your R is "test-run" and your SCI is carbon per test-run. Which is something we've discussed in the spec, it's fine to do and would be a great repeatable test to measure improvements in the app against.

3. How have you converted utilisation to energy consumption? It reads like you've assumed a linear relationship that crosses the origin, i.e. a server at 0% load will consume 0W and a server at 100% load will consume it's total max load and it's a straight line between the two. I.e. given this image: https://www.researchgate.net/figure/Energy-proportionality-curve-of-the-server-in-Table-2-and-an-ideal-energy-proportional_fig1_353042014 have you assumed the red line, or the green line?

This can be the start of a great case study, it's very well structured and written up in a format which others should follow I also think a case study using a very well known example app is an excellent idea. If we can expand it out to broaden the software boundary it would be far more useful.

Some thoughts on data in the SCI Data project that would be useful to expand this:

Networking:

• g/GB data-centre to front-end user.
• g/GB some-local-CDN-edge-node to front-end user (for local/regionally cached data)

Utilisation to Energy Consumption Curve:

• For a given %age utilization of a CPU what is the %age of the TDP we can reasonably assume.

Front End

• ??? What would be a good metric here?

[jawache]

@mrchrisadams my 2c is that overall reducing traffic regardless of how that traffic is paid for and block booked is the only lever that makes sense to pull. I mean data centers are the same in a way, they buy a fixed capacity but charge you a floating rate, just because the DC is only being 50% utilised doesn't mean 50% of the emissions go away, the other 50% is just sitting there unused, but overall we know that reducing the number of machines is better, reduced the number of servers bought over the long term.

[atg-abhishek]

@jawache @srini1978 looking at this exchange here, something comes to mind for me - we have Discourse which I think could become a good avenue for memorializing case studies and bringing in a broader community to work through the various points of a case study? The reason to propose that is 2-fold:

1. Discourse is more accessible than GitHub discussions
2. It has great search features which can help bring up similar discussions and points on the fly as someone types out an answer, say what has been the thought process of the GSF, and others, on the notion of how to go about calculating embodied emissions.

So, what I'm suggesting is that with every case study, we have a "topic" on Discourse which can be used as a venue for dissecting and improving each of them.

Thoughts?

[srini1978]

@atg-abhishek @jawache Great idea. I can create a post there. another thought is if we link to these discourse posts on our GSF linked in so that we can bring in that traffic. We can work with whoever is moderating our GSF linked in channel to do these posts ? Will check with Niloooka and/or @SaraEmilyBergman

[jawache]

Hia, currently people are engaging here via the comments on the doc and now we have this issue thread so it's already a little challenging to keep track of the conversation? What's the benefit of adding Discourse here as a 3rd channel for communication, esp. since with Discourse you need to register again and everyone already has an account with Google Docs and GitHub?

[srini1978]

Excellent work @srini1978! So as I read this, the only component in the software boundary in this case is the "application server" is that correct? Which is fine, the whole point of documenting the software boundary is to make these distinctions clear.

1. I think there a couple of assumption in the doc which would like some clarity on. The M values seem incredibly high, which is why the SCI scores are so high. I suspect you are using the total LCA of the server and have not proportioned it to the slice of time you have rented it for. Can you elaborate? @jawache This is a great catch. I should amortize the value of M through both the timeshare and resource share though I am a little hazy on resource share concept.

E_I =3185.9451 gms of Co2 eq M- 1205.52 kgCo2 eq. SCI = (E_I) +M = 1208.71 KgCo2 per 45 users

2. Can you clarity what the "R" is in the calculation. You mention in the doc about R being the “number of concurrent users on the system” - so the SCI you are calculating is the carbon per "number of concurrent users on the system" which I'm struggling to wrap my head around. I think the SCI score you've calculated is the score for performing the whole test run, rather than per user, am I correct in my understanding? If so I would argue that your R is "test-run" and your SCI is carbon per test-run. Which is something we've discussed in the spec, it's fine to do and would be a great repeatable test to measure improvements in the app against.

@jawache Agree to your observation here.

3. How have you converted utilisation to energy consumption? It reads like you've assumed a linear relationship that crosses the origin, i.e. a server at 0% load will consume 0W and a server at 100% load will consume it's total max load and it's a straight line between the two. I.e. given this image: https://www.researchgate.net/figure/Energy-proportionality-curve-of-the-server-in-Table-2-and-an-ideal-energy-proportional_fig1_353042014 have you assumed the red line, or the green line?

@jawache Great catch again. I believe I have assumed linear relationship which I realize now that it is not valid. I remember seeing a better formula to calculate energy consumption for cloud services but I dont seem to be able to find it. It would be great if you could point me to some of this and I can then rework the calculations.

This can be the start of a great case study, it's very well structured and written up in a format which others should follow I also think a case study using a very well known example app is an excellent idea. If we can expand it out to broaden the software boundary it would be far more useful.

Some thoughts on data in the SCI Data project that would be useful to expand this:

Networking:

• g/GB data-centre to front-end user.
• g/GB some-local-CDN-edge-node to front-end user (for local/regionally cached data)

Utilisation to Energy Consumption Curve:

• For a given %age utilization of a CPU what is the %age of the TDP we can reasonably assume.

Front End

• ??? What would be a good metric here?

[atg-abhishek]

Hia, currently people are engaging here via the comments on the doc and now we have this issue thread so it's already a little challenging to keep track of the conversation? What's the benefit of adding Discourse here as a 3rd channel for communication, esp. since with Discourse you need to register again and everyone already has an account with Google Docs and GitHub?

Agreed - I would like to keep all the discussions here so that they can be referenced later on tied to the work and discussions that have already taken place here. @jawache @srini1978 we should also try and figure out where we stand on having some of the ideas floating in the GDoc separate from this thread then?

[atg-abhishek]

#236 might help to resolve the above as well. As per the SWG call this week, @jawache is working on a case study and we might now have a PR template instead of the issue template as a way to standardize the case studies

[srini1978]

Excellent work @srini1978! So as I read this, the only component in the software boundary in this case is the "application server" is that correct? Which is fine, the whole point of documenting the software boundary is to make these distinctions clear.

1. I think there a couple of assumption in the doc which would like some clarity on. The M values seem incredibly high, which is why the SCI scores are so high. I suspect you are using the total LCA of the server and have not proportioned it to the slice of time you have rented it for. Can you elaborate? @jawache This is a great catch. I should amortize the value of M through both the timeshare and resource share though I am a little hazy on resource share concept.

E_I =3185.9451 gms of Co2 eq M- 1205.52 kgCo2 eq. SCI = (E_I) +M = 1208.71 KgCo2 per 45 users

2. Can you clarity what the "R" is in the calculation. You mention in the doc about R being the “number of concurrent users on the system” - so the SCI you are calculating is the carbon per "number of concurrent users on the system" which I'm struggling to wrap my head around. I think the SCI score you've calculated is the score for performing the whole test run, rather than per user, am I correct in my understanding? If so I would argue that your R is "test-run" and your SCI is carbon per test-run. Which is something we've discussed in the spec, it's fine to do and would be a great repeatable test to measure improvements in the app against.

@jawache Agree to your observation here.

3. How have you converted utilisation to energy consumption? It reads like you've assumed a linear relationship that crosses the origin, i.e. a server at 0% load will consume 0W and a server at 100% load will consume it's total max load and it's a straight line between the two. I.e. given this image: https://www.researchgate.net/figure/Energy-proportionality-curve-of-the-server-in-Table-2-and-an-ideal-energy-proportional_fig1_353042014 have you assumed the red line, or the green line?

@jawache Great catch again. I believe I have assumed linear relationship which I realize now that it is not valid. I remember seeing a better formula to calculate energy consumption for cloud services but I dont seem to be able to find it. It would be great if you could point me to some of this and I can then rework the calculations.

This can be the start of a great case study, it's very well structured and written up in a format which others should follow I also think a case study using a very well known example app is an excellent idea. If we can expand it out to broaden the software boundary it would be far more useful. Some thoughts on data in the SCI Data project that would be useful to expand this: Networking:

• g/GB data-centre to front-end user.
• g/GB some-local-CDN-edge-node to front-end user (for local/regionally cached data)

Utilisation to Energy Consumption Curve:

• For a given %age utilization of a CPU what is the %age of the TDP we can reasonably assume.

Front End

• ??? What would be a good metric here? I was thinking about this metric for the front end and here are my thoughts

1. Any web application has to be accessed on client devices and this could be either mobile devices or laptops or desktops. We would not be able to quantify the number of such people accessing the application. The reason this number is important: To be able to calculate the energy used on the front end browsers and client devices, we would need the total number of people accessing the application.
2. If we go with the approach of how we modeled this for the server components: we considered any one slice of time (1 hour) and counted the CPU utilization and memory utilization and calculated the energy spent. If we use the same approach , then we can count the total number of requests to the web application in a 1 hour period. Approximately this was around 494,373 . These many requests have emanated from 50 virtual users , each of whom can be assumed to have logged into this application from 50 laptops.

So would the right approach be

1. Calculate the energy spent on browsing the application from 50 laptops by modeling the CPU, Memory, screen resolution energy and GPU. For the embodied emissions, do the calculations for manufacturing emissions for 50 laptops prorated to this 1 hour period. Or
2. Consider this energy spent as out of scope for SCI calculation of web application

@atg-abhishek @jawache @Henry-WattTime @seanmcilroy29

[atg-abhishek]

I would lean towards option 2 since that seems to make more sense but we can discuss more during the WG call.

[Henry-WattTime]

Srini - created an updated version of case study, added the database server as well, pull request is available for review. Can see diff on old request. Link: https://github.com/Green-Software-Foundation/software_carbon_intensity/pull/247

How to calculate client side device emissions? Elmar-Much harder to measure multilayer systems, academic papers often simplify to single machine.

Can do different information/slices: -Geographic distribution, energy use and emissions intensity distribution might be different based on where it is deployed -Device distribution, mobile vs. laptop vs. desktop, may be dependent on application type

Should be modeled, don't have control over it, should be held constant. This type of information is not measured anywhere else works well for optimizing front end, using less

Front end calc, come up with some kind of approach Maybe mimic website approach Headless transaction for preliminary calc?

Certain things must be modeled, once you encounter it. Explain what baselines you are using

[srini1978]

@Henry-WattTime @jawache @atg-abhishek The client device calculations have been included and the case study updated. https://docs.google.com/document/d/1McS2-WOTtvubzM5aQV5TrNPye_R1E2yd/edit

[atg-abhishek]

I think there are some access issues with the doc, I just requested access, thanks!

[srini1978]

I think there are some access issues with the doc, I just requested access, thanks!

@atg-abhishek Access given

[njwalk]

In the "SCI Client device calculations" doc (https://docs.google.com/document/d/1McS2-WOTtvubzM5aQV5TrNPye_R1E2yd/edit), the calculation of CPU related energy uses the following formula:

E = Server utilization * Number of hours * Number of cores * TDP * TDP co-efficient

I am confused by the use of BOTH the "Server utilization" and the "TDP co-efficient" in the formula when the TDP co-efficient is already a function of utilization. Per the comments in SCI Data Project “[E] Energy Estimation from Utilization Model” model, the TDP co-efficient is a "model to convert utilization to energy consumption". Therefore, if we are using the TDP co-efficient in the formula then it seems like we should not include the "Server utilization" too. Or am I missing something?

[srini1978]

@njwalk Good catch . I will fix the calculations

[atg-abhishek]

Once PR #256 is completed, we will close out this issue.