cloud-pipeline icon indicating copy to clipboard operation
cloud-pipeline copied to clipboard

Published 20 hours ago • verified publisher icon epam

Autoscaled cluster costs

Open sidoruka opened this issue 4 years ago • 4 comments

Background At the moment, Cloud Pipeline allow to view a price estimation for the single instance jobs and static clusters. But the auto-scaled clusters do not provide such information. Users see a price only for a master node.

This feature shall be added, as the auto-scaled clusters are used at most.

Approach

  • Compute the estimated price for all the cluster (including static and auto-scaled) in the backend
  • Price is computed based on all the RUNNING instances, which are children of the particular master. Plus the mater's price (note: worker nodes can be of different instance size than a master)
  • This information shall be stored in the master run
  • Estimated price shall be updated on a scheduled basis (default: 30 seconds)
  • GUI shall use this new run's attribute to display estimated cost for the cluster (for the static as well)

sidoruka avatar Oct 04 '21 11:10 sidoruka

@rodichenko a new field workersPrice added to pipeline run output object. This value indicates the total price for all cluster workers.

ekazachkova avatar Oct 11 '21 12:10 ekazachkova

@sidoruka @mzueva @ekazachkova GUI part implemented (#2212)

rodichenko avatar Oct 11 '21 15:10 rodichenko

@sidoruka @mzueva backported to release/0.16 (79dfa1535c2c51263d805e18205ab2254172a35e)

rodichenko avatar Oct 12 '21 11:10 rodichenko

Docs were added via #3228 and located here.

NShaforostov avatar Oct 17 '23 17:10 NShaforostov