Allow 0 primary replica during initialization

[sayap]

Describe the feature

We are trying to use Flagger to manage multiple deployments of the same service across multiple k8s clusters. As the new k8s clusters may not be well tested yet, ideally we would like to only send a small percentage of traffic to the new deployment.

However, it looks like https://github.com/fluxcd/flagger/commit/8b758fd61 from https://github.com/fluxcd/flagger/pull/107 would set the primary replica to be at least 1 during initialization. So, let's say the old deployment has 3 replicas, then the new deployment will receive 25% of traffic, which can be a bit too high.

Proposed solution

If there is no strong reason why the default primary replica has to be at least 1, maybe we can make this configurable and allow 0 replica when creating the primary deployment.

Any alternatives you've considered?

Currently we have some rather complicated workflow to workaround this initialization behavior.

[sayap]

Related to this, it looks like commit baeee62a might have introduced a bug.

Before the commit, there was this check that makes sure that primary has at least 1 replica:

func (c *Controller) getDeployment(r *flaggerv1.Canary, name string, namespace string) (*appsv1.Deployment, bool) {
	dep, err := c.kubeClient.AppsV1().Deployments(namespace).Get(name, v1.GetOptions{})
	... 
	if dep.Spec.Replicas == nil || *dep.Spec.Replicas == 0 {
		return nil, false
	}

	return dep, true
}

After the commit, the check now compares the pointer addresses, and will never evaluate to true:

func (c *CanaryDeployer) IsDeploymentHealthy(cd *flaggerv1.Canary) error {
	...
	primary, err := c.kubeClient.AppsV1().Deployments(cd.Namespace).Get(primaryName, metav1.GetOptions{})
	...
	if primary.Spec.Replicas == int32p(0) {
		return fmt.Errorf("halt %s.%s advancement %s",
			cd.Name, cd.Namespace, "primary deployment is scaled to zero")
	}
	return nil
}

For my feature request, this bug is actually helpful, because I do want the canary to succeed even when primary has 0 replica.