feature:new helm-chart provding manifest handled by victoria-metrics-operator

[ccaillet1974]

Chart name and version, where you feel a lack of requested feature
chart: new chart proposal name victoria-stack
version: new

Is your feature request related to a problem? Please describe.
Currently using ArgoCD to manage apps in my kubernetes clusters, but when using manifest I don't have applications that represents the platforms deployed and managed by the operator. I could deploy directly as an application the manifest for each component deployed because there are under our own git but using Helm could permit the deployment of mutiple components of the same type in the same application

Describe the solution you'd like
Creation of a new Helm chart that produces manifest(s) used by victoria-metrics-operator without having the complex task of learning the API for the multiples obejcts managed by the operator. Could be more user-friendly

Sample values file for the vmagent component could be :

global:
  ...

vmagent:
  name: sample-vmagent
  # Directives to permit an override for kubernetes objects produced by the operator
  nameOverride: name_override
  fulNameOverride: 
  # Version for the component
  componentVersion: v1.137.0
  # Namespace followed by vmagent
  selectAllByDefault: true
  externalLabelName: vmagent_ha
  # Extra args used by the component
  extraArgs:
    envflag.enable: "true"
    envflag.prefix: VM_
    loggerFormat: json
    httpListenAddr: :8429
    license.forceOffline: "true"
    promscrape.dropOriginalLabels: "false"
  Permit override for using a specific image for the component defined here vmagent
  image:
    # Permit the override of componentVersion
    tag: v1.136.1-enterprise
  # If license defined add automatically the -enterprise suffix to componentVersion value
  license:
    forceOffline: true
    keyRef:
      name: lyo0-sfr-license
      key: license
  remoteWrite:
    - url: "http://lyo0-int-vminsert.kube-vmetrics.svc:8480/insert/1/prometheus"
  scrapeInterval: 10s
  # Replication:
  replicaCount: 2
  # Resources for the component
  resources:
    limits:
      cpu: 4
      memory: 8Gi
    requests:
      cpu: 256m
      memory: 1Gi
  # Security definitions for component
  securityContext:
    capabilities:
      drop:
      - ALL
    readOnlyRootFilesystem: true
    runAsNonRoot: true
    runAsUser: 1000
    fsGroup: 2000
  # Service definition and extend if needed
  serviceSpec:
    # useAsDefault: true
    spec:
      type: ClusterIP
  # StatefulMode:
  statefulMode: true
  # Storage parameters maybe it could be better if statefulStorage is defined
  # to force the statefulMode to true
  # In the same way maybe statefulStorage could renamed to storage to be
  # the same as other victoria components managed by operator that have only
  # storage as section name instead of statefulStorage
  statefulStorage:
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        storageClassName: "oc6810-ssd-100g"
        resources:
          requests:
            storage: 20Gi
        volumeMode: Filesystem

Describe alternatives you've considered
Workaround is to create application with manifest directly, but no coherency and no quality control of the manifest produced by user

Additional context
N/A

[AndrewChubatiuk]

without having the complex task of learning the API for the multiples obejcts managed by the operator

sorry, but don't get your idea how this approach should simplify learning
what is a difference between learning objects API and helm chart values?
what if multiple components of the same type are needed?
how to provide interconnection between these objects?

[ccaillet1974]

Andrew,
From my own experience, I begin with Helm static deployment for all my component currently running, because many people like or prefer to use Helm instead of operators because most of users doesn't have operators in their environment. It's also easier to fill values than composing from scratch a manifest.

I see like I said some advantages at least like :
1- manifest produced will be always compliant with your "best practices" and of course always correct in term of syntax, indentation and so on. Only values are changing, Helm could also check that a component could be deployed by checking if CRD is present and also if some dependencies needed are also present.
2- It is always easier for user to fill only values like how many replica you want or this kind of thing instead of searching into the API docs which is big and all informations are present but dispatched across many pages in many places.
3- I have a similar mechanism with my previous log platform (Elastic) and it has saved many time for deployment.
4- We could deploy directly many components and many of the same type if we use arrays inside component section. The relations between them are implemented by the user itself with the misc configuration he'll use on those components.

But it's just a proposal, I could use manifest directly inside my ArgoCD to produce applications just more complicated I think, even if I see than many sections from my Helm chart could be reused directly as-is inside component manifest for operator.

EDIT 1: I also add the fact that license is handle by the operator in Elastic so easier to manage license and license are applied directly by the operator instead of declare multiple time the secret containing the license. If a license is handle by the operator this license is automatically applied to deployed component without need to add the license section in each component. Don't sure to be clear but don't hesitate question me about that :)

[vrutkovs]

I'd prefer to have a distinct split between Helm-based and operator-based ones. See #2696 for more info.

Apart from the convenient syntax to define initial deployment its important to consider the further experience.

Do we need a complicated rollout procedure with multiple zones? This would mean the operator solution is preferred as it can define a complicated logic for rollouts.

If we want to focus on pure Helm-based resource management this would mean operating a complicated cluster would be more complicated - i.e. adding HPA would require a special knowledge which changes to apply (in operator its hidden behind spec.HPA: true).

From maintainer's point of view we are more convenient with operator support as it can be easily tested, released and supported at the expense of less customizability.

I begin with Helm static deployment for all my component currently running, because many people like or prefer to use Helm instead of operators because most of users doesn't have operators in their environment

I think this can be mitigated by using victoria-metrics-operator chart with extraObjects:

extraObjects:
  - apiVersion: v1
    kind: ConfigMap
    metadata:
        name: additional-config
    data:
        my-key: my-value
  - apiVersion: operator.victoriametrics.com/v1beta1
    kind: VMAuth
    metadata:
        name: vmauth-sample
    spec:
        selectAllByDefault: true

Helm could also check that a component could be deployed by checking if CRD is present and also if some dependencies needed are also present.

This would already be verified by operator - we ship an entire CRD every time

It is always easier for user to fill only values like how many replica you want or this kind of thing instead of searching into the API docs which is big and all informations are present but dispatched across many pages in many places.

Sure, however this would result into three different syntaxes we'll need to support, which is not sustainable.

It is always easier for user to fill only values like how many replica you want or this kind of thing instead of searching into the API docs which is big and all informations are present but dispatched across many pages in many places.

I agree that this is a major problem now, but adding yet another helm-chart will only make it worse very soon.

We could deploy directly many components and many of the same type if we use arrays inside component section

I think this is a good idea for a custom wrapper chart, which would template objects for operator. This would solve both interface complexity issue (much less options stored in a single place) and functionality (rely on operator to create objects). What do you think? It might be an interesting experiment

[ccaillet1974]

Vadim,

Helm is in first part a template engine it has been used for deployment but it is mainly a template engine, it could provide whatever we want, applications etc etc

Helm could provide with the correct templates the manifest(s) directly for the operator without any error syntax or at least minimal errors.

I'm working with Stephan because maybe it is due to my deployment but my Victoria operator has created objects with invalid syntax even if webhookadminission is enabled on the operator (joined the manifest below)

lyo0-int-test-wrong.yaml

As you said in last part of your last post : for me it is a "wrapper chart" not a chart on itself. I was not clear but this is the exact idea I had. For me this "meta" chart permit to produce manifest directly ingested by the operator, you could inspect the Helm from elastic which is called eck-stack which produce manifests for being processed by their eck-operator for multiple kind of objects like elasticsearch, kibana, logstash, elastic-agent, fleet-server who manages agents ...

The mnimal check could be to add the verification of resource available on the cluster with the correct if section like the below exemple used for checking if prometheus servicemonitor resource is available

{{- if and (.Capabilities.APIVersions.Has "monitoring.coreos.com/v1") .Values.monitoring.serviceMonitor.enabled }}

Thanks for clarifying my idea :)

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.