Install kustomize

Kustomize is another tool to install applications on k8s beside Helm. Let's install it first.

curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
sudo mv kustomize /usr/local/bin/

Install Kubeflow

Clone the manifests repository. I did this step already for you so you don't have to redo this. I just want to show you what I did.

RELEASE=v1.7.0-rc.0
git clone -b $RELEASE --depth 1 --single-branch https://github.com/kubeflow/manifests.git
cd manifests
while ! kustomize build example | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done

Install KServe

KServe was already included in Kubeflow so we don't need to do it again, but make sure to run the following command before moving to the next section:

kubectl patch cm config-domain --patch '{"data":{"example.com":""}}' -n knative-serving

Quick start

Create a new namespace containing kserve services

kubectl create ns kserve-test

Deploy your first iris model using the following commands

kubens kserve-test
kubectl apply -f deployments/quickstart.yaml

Port-forward to access the service locally via istio ingress

kubectl port-forward svc/istio-ingressgateway 8000:80 -n istio-system

Test our newly created service by using Python client

python utils/quickstart/client.py

OR cURL as below (remember to replace authservice_session with your own session)

curl -v -H "Host: sklearn-iris.kserve-test.example.com" -H "Cookie: authservice_session=MTcxNDQ3Mjg3MHxOd3dBTkVKWE5VSXlTMUZUTlRKRlJFUkhWVWRNVVU1UldsUktXRkpXTjB3eVIxZElOVFpWVVVSSFFqWkZUa3MzUTBKWVNUZExVMEU9fAQuK4Wp7NIB2Ye5kTV54cjSDY8X_4yfaO3I1qzInblV" http://localhost:8000/v1/models/sklearn-iris:predict -d @./iris-input.json

Install pack CLI

Pack is a tool maintained by the Cloud Native Buildpacks project helps to build images by analyzing the source code.

sudo add-apt-repository ppa:cncf-buildpacks/pack-cli
sudo apt-get update
sudo apt-get install pack-cli

To build an image using pack, simple run the following command, then push it as usual.

Pack will read Procfile along with the python version mentioned in the runtime.txt and requirements.txt to build your image. For more information, please refer to the following documentation: https://github.com/heroku/cnb-builder-images?tab=readme-ov-file#usage

pack build --builder heroku/builder:22 quandvrobusto/kserve-intrusion-detection:0.0.1
docker push quandvrobusto/kserve-intrusion-detection:0.0.1

Deploy our instrusion-detection InferenceService

kubectl apply -f deployments/intrusion_detection.yaml

, and smoke test as follows

python utils/anomaly/anomaly_client.py

Wait for a few seconds, a pod intrusion-detection-predictor-default-* will be started and process your request.

Fix some bugs

kserve/kserve#2757

References

1. https://knative.dev/docs/serving/architecture/
2. https://github.com/knative/serving/blob/main/docs/scaling/SYSTEM.md
3. https://kserve.github.io/website/0.7/developer/debug/#debug-kserve-request-flow