Add the ability to ramp-up the load

[domleb]

Proposal

Add the ability to ramp-up the load.

Background

I see this has already been requested here and closed in favor of invoking the command multiple times with different attack rates.

The trouble with this approach is that it creates connection spikes every time the load changes because the existing connections are not reused. This results in larger spikes as the load increases. At a certain point this will cause connection errors as most target systems have a limit on the number of instantaneous connections they can create. For example, this will cause connection errors with ELBs because the connection rate does not match the request rate for a given ELB instance. This is the problem I have.

Workarounds

This problem can be avoided by running multiple vegeta instances at the same time and staggering the start time.

I would be happy to implement this if we can get a design together.

[tsenart]

We already have all the machinery for this in here: https://github.com/tsenart/vegeta/blob/master/lib/pacer.go#L118

How would you see this exposed via the CLI?

[domleb]

I did see the pacer change. It's an unusual feature (for a load testing tool) and there isn't much documentation yet so I wasn't sure how to use it for a ramp-up. It's worth being careful with naming too, as historically tools use the term 'pacing' to control iterations and 'ramp-up' to ramp-up to a given pace.

Here's the DSL for gatling:

setUp(
  scn.inject(
    nothingFor(4 seconds), // 1
    atOnceUsers(10), // 2
    rampUsers(10) over (5 seconds), // 3
    constantUsersPerSec(20) during (15 seconds), // 4
    constantUsersPerSec(20) during (15 seconds) randomized, // 5
    rampUsersPerSec(10) to 20 during (10 minutes), // 6
    rampUsersPerSec(10) to 20 during (10 minutes) randomized, // 7
    splitUsers(1000) into (rampUsers(10) over (10 seconds)) separatedBy (10 seconds), // 8
    splitUsers(1000) into (rampUsers(10) over (10 seconds)) separatedBy atOnceUsers(30), // 9
    heavisideUsers(1000) over (20 seconds) // 10
  ).protocols(httpConf)
)

This is obviously a very rich API. The simplest CLI argument would be -ramp 60s like the simple ramp-up in earlier versions of LoadRunner.

Or something more elaborate like the suggestions in https://github.com/tsenart/vegeta/issues/163. Here's another example that could also be passed in as a csv file:

-plan "0,1000,60s|1000,1000,5m|1000,0,60s"

[tsenart]

I like -ramp 👍

[tsenart]

@fluffle: I'm trying to create a LinearPacer that describes a linear increase in rate. This is it's definition:

// LinearPacer paces an attack by starting at a given request rate
// and increasing linearly with the given slope.
type LinearPacer struct {
	StartAt Rate
	Slope   float64
}

I'm struggling with adapting your SinePacer method of integration to a linear equation. Isn't the integral of f(x) = ax + b analytically defined as 0.5x(ax + 2b) + C?

It'd be wonderful if you could help out with this code :-)

[caldempsey]

Hi @tsenart thank you for the service. I would like the ability to ramp up at intervals, rather than linear pacing. Is there a step interval function enabled in line with this PR?

[aweis89]

Not a replacement by any means, but for those running vegeta in k8s I've found StatefulSets to do a good job scheduling distributed tests and replicating ramp-up (without creating all new connection):

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: load-test
  namespace: test
  labels:
    app: load-test
spec:
  serviceName: load-test
  replicas: 4
  selector:
    matchLabels:
      app: load-test
  template:
    metadata:
      labels:
        app: load-test
    spec:
      containers:
      - name: vegeta
        image: peterevans/vegeta:6.9.1
        lifecycle:
          postStart:
            exec:
              command:
              - sh
              - -c
              - |
                sleep 60
                touch /ready
        readinessProbe:
          exec:
            command:
              - test
              - -f
              - /ready
        command:
          - sh
          - -c
          - |
             echo "GET .." | vegeta attack -rate 500/s ..

The sleep 60 is how long to wait between intervals, and each replica adds an additional 500 req/s as per the command.

[kazimir-malevich]

Could any please tell me whether ramp up/down is possible through the CLI?