Ingester: support zone-awareness

[timonegk]

What this PR does:
Add zone awareness support to ingesters. Similar to #632, but with less duplication.
The remaining todo is the migration process. It has to be tested, documented and I have to decide which parts of the migration process should be possible via the Helm chart vs. manually. As documented by Grafana, the migration is not trivial.

Which issue(s) this PR fixes:
Fixes #203.

Checklist

• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX], [DEPENDENCY]

[kd7lxl]

Why would you modify the chart when the chart already supports another technique for zone awareness that doesn't require further modification? Is there something I'm missing?

[nschad]

another technique for zone awareness

That requires the admission controller, otherwise you won't get the env-vars. If that is something you are ok with running, then yes you are correct. Otherwise there is currently no true "stand-a-lone" way of deploying zone-aware ingesters

right?

[kd7lxl]

another technique for zone awareness

That requires the admission controller, otherwise you won't get the env-vars. If that is something you are ok with running, then yes you are correct. Otherwise there is currently no true "stand-a-lone" way of deploying zone-aware ingesters

right?

Yes, I would run the admission controller. There is no need for the admission controller to be deployed in the same chart as cortex (nor would I couple them), so this is possible today. A guide doc may be the only contribution needed.

A strong benefit of the admission controller is that is does not require prior knowledge of the available zones. In contrast, the configuration method requires first collecting information about the target cluster and what zones are available, then configuring cortex for those zones. This is significant increased deployment complexity.

[nschad]

another technique for zone awareness

That requires the admission controller, otherwise you won't get the env-vars. If that is something you are ok with running, then yes you are correct. Otherwise there is currently no true "stand-a-lone" way of deploying zone-aware ingesters
right?

Yes, I would run the admission controller. There is no need for the admission controller to be deployed in the same chart as cortex (nor would I couple them), so this is possible today. A guide doc may be the only contribution needed.

A strong benefit of the admission controller is that is does not require prior knowledge of the available zones. In contrast, the configuration method requires first collecting information about the target cluster and what zones are available, then configuring cortex for those zones. This is significant increased deployment complexity.

on the other hand, this PR allows you have to one Deployment/StatefulSet per zone which can be beneficial if you want to do per-zone rollouts. For example facialited by the grafana rollout operator

I think there is a case to be made for both. Also the admission controller (even though the code is not complicated) is not maintained.