-
Notifications
You must be signed in to change notification settings - Fork 92
Document KHI job mode #709
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Haihan-Jiang
wants to merge
3
commits into
GoogleCloudPlatform:main
Choose a base branch
from
Haihan-Jiang:codex/khi-document-job-mode
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| # **Running KHI in Job Mode** | ||
|
|
||
| KHI usually starts a local web server and runs inspections from the browser. Job mode runs one inspection directly from the command line and writes the result to a `.khi` file. This is useful for scheduled jobs, alerting pipelines, or other automation that needs to generate a KHI file without starting the web UI. | ||
|
|
||
| ## **Required flags** | ||
|
|
||
| Job mode is enabled with `--job-mode`. When it is enabled, all of the following flags are required: | ||
|
|
||
| | Flag | Description | | ||
| | --- | --- | | ||
| | `--job-inspection-type` | Inspection type ID to run. | | ||
| | `--job-inspection-features` | Comma-separated feature IDs to enable. Use `ALL` to enable every feature available for the inspection type. | | ||
| | `--job-inspection-values` | JSON object containing the inspection parameters. These are the same form values used by the web UI. | | ||
| | `--job-export-destination` | Path where the generated `.khi` file is written. | | ||
|
|
||
| Available inspection type IDs include: | ||
|
|
||
| | Inspection type ID | Target | | ||
| | --- | --- | | ||
| | `gcp-gke` | Google Kubernetes Engine | | ||
| | `gcp-gke-on-aws` | GKE on AWS | | ||
| | `gcp-gke-on-azure` | GKE on Azure | | ||
| | `gcp-gdcv-for-vmware` | Google Distributed Cloud for VMware | | ||
| | `gcp-gdcv-for-baremetal` | Google Distributed Cloud for bare metal | | ||
| | `gcp-composer` | Cloud Composer | | ||
| | `oss-kubernetes-from-files` | OSS Kubernetes audit log files | | ||
|
|
||
| ## **Command format** | ||
|
|
||
| ```bash | ||
| ./khi \ | ||
| --job-mode \ | ||
| --job-inspection-type gcp-gke \ | ||
| --job-inspection-features <feature-id>,<feature-id> \ | ||
| --job-inspection-values '{"<text-field-id>":"<value>","<set-field-id>":["<value>"]}' \ | ||
| --job-export-destination ./inspection.khi | ||
| ``` | ||
|
|
||
| When the command succeeds, `inspection.khi` can be opened later with KHI. | ||
|
|
||
| If you use a released binary or run KHI from another directory, replace `./khi` with the path to that binary. | ||
|
|
||
| ## **Inspection values** | ||
|
|
||
| `--job-inspection-values` must be a JSON object. Each key is a KHI form field ID and each value is the value that would be submitted from the web UI for the same field. | ||
| Text fields use JSON strings. Set fields use JSON arrays of strings. | ||
|
|
||
| For example, a Google Cloud inspection usually includes common fields such as project, location, query end time, and duration, plus inspection-specific fields: | ||
|
|
||
| ```json | ||
| { | ||
| "cloud.google.com/common/input-project-id": "my-project", | ||
| "cloud.google.com/common/input-location": "us-central1", | ||
| "cloud.google.com/common/input-end-time": "2026-01-15T10:00:00Z", | ||
| "cloud.google.com/common/input-duration": "2h", | ||
| "<inspection-specific-text-field-id>": "<value>", | ||
| "<inspection-specific-set-field-id>": ["<value>"] | ||
| } | ||
| ``` | ||
|
|
||
| The exact set of required fields depends on the inspection type and enabled features. The most reliable way to build the JSON is to run the same inspection once in the web UI, open the browser developer tools, and check the request payload sent to `POST /api/v3/inspection/<inspection-id>/run` or `POST /api/v3/inspection/<inspection-id>/dryrun`. Copy the `parameters` object from that request as the `--job-inspection-values` JSON. It uses the same field IDs and values that job mode expects. | ||
|
|
||
| For a GKE inspection, the commonly required field IDs are: | ||
|
|
||
| | Field ID | Value | | ||
| | --- | --- | | ||
| | `cloud.google.com/common/input-project-id` | Google Cloud project ID. | | ||
| | `cloud.google.com/common/input-location` | Cluster location, such as `us-central1` or `us-central1-a`. | | ||
| | `cloud.google.com/common/input-end-time` | Query end time in RFC3339 format. | | ||
| | `cloud.google.com/common/input-duration` | Query duration, such as `2h`. | | ||
| | `cloud.google.com/k8s/input-cluster-name` | GKE cluster name. | | ||
| | `cloud.google.com/k8s/input-namespaces` | Namespace filter array. Use `["@all_cluster_scoped", "@all_namespaced"]` to include all scopes. | | ||
| | `cloud.google.com/k8s/input-kinds` | Kubernetes kind filter array. Use `["@default"]` for common kinds or `["@any"]` for every kind. | | ||
|
|
||
| Some optional GKE features add their own fields. Include these fields when the selected feature list needs them: | ||
|
|
||
| | Feature area | Field ID | Example value | | ||
| | --- | --- | --- | | ||
| | Node and serial port logs | `cloud.google.com/k8s/input/node-name-filter` | `[]` | | ||
| | Container logs | `cloud.google.com/log/k8s-container/input/query-namespaces` | `["@managed"]` | | ||
| | Container logs | `cloud.google.com/log/k8s-container/input/query-podnames` | `["@any"]` | | ||
| | Control plane component logs | `cloud.google.com/log/k8s-control-plane/input/component-names` | `["@any", "-apiserver"]` | | ||
| | CSM access logs | `cloud.google.com/log/csm-accesslog/input/response-flags` | `["@any", "-OK"]` | | ||
| | CSM resource audit logs | `cloud.google.com/log/csm-accesslog/input/fleet-project-id` | `"my-project"` | | ||
|
|
||
| When using `--job-inspection-features ALL` for GKE, start with a payload like this and adjust the values for the target cluster: | ||
|
|
||
| ```json | ||
| { | ||
| "cloud.google.com/common/input-project-id": "my-project", | ||
| "cloud.google.com/common/input-location": "us-central1", | ||
| "cloud.google.com/common/input-end-time": "2026-01-15T10:00:00Z", | ||
| "cloud.google.com/common/input-duration": "2h", | ||
| "cloud.google.com/k8s/input-cluster-name": "my-cluster", | ||
| "cloud.google.com/k8s/input-namespaces": ["@all_cluster_scoped", "@all_namespaced"], | ||
| "cloud.google.com/k8s/input-kinds": ["@default"], | ||
| "cloud.google.com/k8s/input/node-name-filter": [], | ||
| "cloud.google.com/log/k8s-container/input/query-namespaces": ["@managed"], | ||
| "cloud.google.com/log/k8s-container/input/query-podnames": ["@any"], | ||
| "cloud.google.com/log/k8s-control-plane/input/component-names": ["@any", "-apiserver"], | ||
| "cloud.google.com/log/csm-accesslog/input/response-flags": ["@any", "-OK"], | ||
| "cloud.google.com/log/csm-accesslog/input/fleet-project-id": "my-project" | ||
| } | ||
| ``` | ||
|
|
||
| ## **Selecting features** | ||
|
|
||
| Use `ALL` when the job should include every feature supported by the selected inspection type: | ||
|
|
||
| ```bash | ||
| --job-inspection-features ALL | ||
| ``` | ||
|
|
||
| `ALL` enables optional features that are disabled by default in the web UI. For GKE, this includes node logs, container logs, control plane component logs, CSM logs, and serial port logs, so include the extra fields listed above. | ||
|
|
||
| To enable only specific features, pass their IDs as a comma-separated list: | ||
|
|
||
| ```bash | ||
| --job-inspection-features feature-a,feature-b | ||
| ``` | ||
|
|
||
| You can inspect feature IDs from the web server API before switching the same configuration to job mode: | ||
|
|
||
| ```bash | ||
| curl http://localhost:8080/api/v3/inspection/types | ||
| curl -X POST http://localhost:8080/api/v3/inspection/types/gcp-gke | ||
| curl http://localhost:8080/api/v3/inspection/<inspection-id>/features | ||
| ``` | ||
|
|
||
| Use the `inspectionID` returned by the `POST` request in the final URL. | ||
|
|
||
| ## **Example** | ||
|
|
||
| The following example runs the default GKE feature set and writes the result to `./gke-inspection.khi`: | ||
|
|
||
| ```bash | ||
| ./khi \ | ||
| --job-mode \ | ||
| --job-inspection-type gcp-gke \ | ||
| --job-inspection-features khi.google.com/k8s-common-auditlog/k8s-auditlog-parser-tail#gcp,cloud.google.com/log/k8s-event/timeline-mapper#default,cloud.google.com/log/gke-api/timeline-mapper#default,cloud.google.com/log/compute-api/timeline-mapper#default,cloud.google.com/log/network-api/timeline-mapper#default,cloud.google.com/gke/log/autoscaler/timeline-mapper#default \ | ||
| --job-inspection-values '{ | ||
| "cloud.google.com/common/input-project-id": "my-project", | ||
| "cloud.google.com/common/input-location": "us-central1", | ||
| "cloud.google.com/common/input-end-time": "2026-01-15T10:00:00Z", | ||
| "cloud.google.com/common/input-duration": "2h", | ||
| "cloud.google.com/k8s/input-cluster-name": "my-cluster", | ||
| "cloud.google.com/k8s/input-namespaces": ["@all_cluster_scoped", "@all_namespaced"], | ||
| "cloud.google.com/k8s/input-kinds": ["@default"] | ||
| }' \ | ||
| --job-export-destination ./gke-inspection.khi | ||
| ``` | ||
|
|
||
| The command runs once, waits for the inspection to finish, and exits after writing the KHI file. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the most difficult part of this feature is building the
--job-inspection-valuesparameter.The current easiest way is using dev tool on browser to see what is sent from the frontend. It's a little hacky way but worth to be documented.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in
18e401e. The guide now keeps the browser dev tools/runor/dryrunpayload workflow, clarifies that set-form fields must be JSON arrays, and updates the GKE example to use the same field IDs/value shapes that job mode expects. I also added a table of extra fields required when optional GKE features are enabled.