Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 34 additions & 10 deletions docs/networking/ippool.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ To create a new IP pool:
1. Specify the **Name** of the IP pool.
1. Go to the **Range** tab to specify the **IP ranges** for the IP pool. You can add multiple IP ranges.
![](/img/v1.2/networking/multiple-ranges.png)
1. Go to the **Selector** tab to specify the **Scope** and **Priority** of the IP pool.
1. Go to the **Selector** tab to specify the **Selector**, **Scope** and **Priority** of the IP pool.
![](/img/v1.2/networking/ippool-scope.png)

When you operate from the Harvester UI, the `Scope` only includes `Namespace`. Click `Add Scope` to add new items.
Expand All @@ -44,20 +44,37 @@ If the Harvester cluster is imported to `Rancher Manager` from `Rancher Manager

The `Scope` includes `Project`, `Namespace` and `Guest Kubernetes Cluster`. For more information, see [Multi-Tenancy Example](../rancher/virtualization-management.md#multi-tenancy-example) and [Projects and Kubernetes Namespaces with Rancher](https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/projects-and-namespaces#about-projects).

When a pool has only one `Scope` and each selects `All`, then this IP Pool is marked as `global` automatically.
When a pool has only one `Scope` and each selects `All`, then this IP Pool is marked as **global** automatically.

## Selection policy

Each IP pool will have a specific range, and you can specify the corresponding requirements in the LB `annotations`. IP pools that meet the specified requirements will automatically assign IP addresses to LBs.

- LBs utilize the following annotations to express requirements (all annotations are optional):
- `loadbalancer.harvesterhci.io/network` specifies the VM network the guest cluster nodes use.
- `loadbalancer.harvesterhci.io/network` specifies the VM network the guest cluster nodes use. If this annotation is omitted, the mutator automatically selects the first Multus network defined in the guest cluster's virtual machine instances.
- `loadbalancer.harvesterhci.io/project` and `loadbalancer.harvesterhci.io/namespace` identify the project and namespace of the VMs that comprise the guest cluster.
Comment thread
w13915984028 marked this conversation as resolved.
- `loadbalancer.harvesterhci.io/cluster` denotes the name of the guest cluster.
- The IP pool has a selector, including network and scope, to match the requirements of the LB.
- Network is a hard condition. The optional IP pool must match the value of the LB annotation `loadbalancer.harvesterhci.io/network`.
- Every IP pool, except the global IP pool, has a unique scope different from others if its priority is `0`. The project, namespace, or cluster name of LBs should be in the scope of the IP pool if they want to get an IP from this pool.
- Network is a **hard condition**. The target IP pool must match the value of the LB annotation `loadbalancer.harvesterhci.io/network`.
- Every IP pool, except the **global** IP pool, has a unique scope different from others if its priority is `0`. The project, namespace, or cluster name of LBs should be in the scope of the IP pool if they want to get an IP from this pool.
- `spec.selector.priority` specifies the priority of the IP Pool. The larger the number, the higher the priority. If the priority is not `0`, the value should differ. The priority helps you to migrate the old IP pool to the new one.
- If the IP Pool has a scope that matches all projects, namespaces, and guest clusters, it's called a global IP pool, and only one global IP pool is allowed. If there is no IP pool matching the requirements of the LB, the IPAM will allocate an IP address from the global IP pool if it exists.
- If the IP Pool has a scope that matches all projects, namespaces, and guest clusters, it's called a **global** IP pool, and only one **global** IP pool is allowed under one **Network**. If there is no IP pool matching the requirements of the LB, the IPAM will allocate an IP address from the **global** IP pool if it exists and matches the **Network**.

:::caution

**Legacy Behavior (Harvester v1.8 and earlier):**

- The entire system was restricted to a maximum of one **global** IP pool.
- Network matching was not enforced, meaning a **global** IP pool could inadvertently allocate IPs to the incorrect network.

**Current Behavior (Harvester v1.9.0 and later):**

- You may define at most one **global** IP pool per specific VM Network.
- Network matching is strictly enforced. An IP pool with an empty `.spec.selector.network` will **not** be selected for allocation, even if it is marked as **global**.

- For guidance on resource allocation, see [IPPool Best Practices](#ippool-for-guest-cluster-type-loadbalancer).

:::

### Examples
- **Example 1:** You wish to set up an IP pool within the range `192.168.100.0/24` for the `default` namespace. In this scenario, all load balancers within the `default` namespace will receive an IP address from this designated IP pool:
Expand Down Expand Up @@ -119,12 +136,13 @@ Each IP pool will have a specific range, and you can specify the corresponding r
kind: IPPool
metadata:
name: global-ip-pool
labels:
loadbalancer.harvesterhci.io/global-ip-pool: 'true' # Added by the controller automatically
labels:
loadbalancer.harvesterhci.io/global-ip-pool: 'true' # Added automatically by Harvester
spec:
ranges:
- subnet: 192.168.20.0/24
selector:
network: default/vlan1
scope:
- project: "*"
namespace: "*"
Expand All @@ -145,7 +163,13 @@ Each IP pool will have a specific range, and you can specify the corresponding r

### IPPool for Guest Cluster type Loadbalancer

1. It is better to [Create IP Pool from Rancher Managery UI](#create-ip-pool-from-rancher-manager-ui), because it allows you to tune `Project` and `Guest Kubernetes Cluster` for better resource management and isolation.
1. It is better to [Create IP Pool from Rancher Managery UI](#create-ip-pool-from-rancher-manager-ui), because it allows you to tune `Project`, `Namespace` and `Guest Kubernetes Cluster` for better resource management and isolation.

1. Be careful when creating **global** IP pool as one guest cluster might allocate too many IPs and starve other clusters. The pool can't be deleted if any of the IPs is still in use.

1. Be careful when creating global IP pool as one guest cluster might allocate too many IPs and starve other clusters. The pool can't be deleted if any of the IPs is still in use.
1. Starting with v1.9.0, an IP pool with an empty `.spec.selector.network` will not be selected for allocation, even if it is marked as **global**. Always set the correct `.spec.selector.network`.

1. **Use project/namespace-scoped pools for better isolation:** Since each guest cluster is deployed within a specific project and namespace, you should create IP pools scoped to that project or namespace with defined IP ranges. This approach offers several advantages:
* **Granular Control:** A single network can support multiple non-global pools, allowing you to partition IP availability across different projects or namespaces.
* **Resource Protection:** The exhaustion of one pool does not affect others, even though they share the same underlying network.
* **Scalability:** It is easier to increase an IP pool's range than to decrease it. Shrinking a range is difficult when the currently used IPs are dispersed throughout the range (e.g., a pool of [1-10] where IPs 1, 5, and 9 are already in use). Consequently, a best practice is to start with a smaller, conservative range and expand as needed.
34 changes: 24 additions & 10 deletions versioned_docs/version-v1.5/networking/ippool.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ To create a new IP pool:
1. Specify the **Name** of the IP pool.
1. Go to the **Range** tab to specify the **IP ranges** for the IP pool. You can add multiple IP ranges.
![](/img/v1.2/networking/multiple-ranges.png)
1. Go to the **Selector** tab to specify the **Scope** and **Priority** of the IP pool.
1. Go to the **Selector** tab to specify the **Selector**, **Scope** and **Priority** of the IP pool.
![](/img/v1.2/networking/ippool-scope.png)

When you operate from the Harvester UI, the `Scope` only includes `Namespace`. Click `Add Scope` to add new items.
Expand All @@ -44,20 +44,29 @@ If the Harvester cluster is imported to `Rancher Manager` from `Rancher Manager

The `Scope` includes `Project`, `Namespace` and `Guest Kubernetes Cluster`. For more information, see [Multi-Tenancy Example](../rancher/virtualization-management.md#multi-tenancy-example) and [Projects and Kubernetes Namespaces with Rancher](https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/projects-and-namespaces#about-projects).

When a pool has only one `Scope` and each selects `All`, then this IP Pool is marked as `global` automatically.
When a pool has only one `Scope` and each selects `All`, then this IP Pool is marked as **global** automatically.

## Selection policy

Each IP pool will have a specific range, and you can specify the corresponding requirements in the LB `annotations`. IP pools that meet the specified requirements will automatically assign IP addresses to LBs.

- LBs utilize the following annotations to express requirements (all annotations are optional):
- `loadbalancer.harvesterhci.io/network` specifies the VM network the guest cluster nodes use.
- `loadbalancer.harvesterhci.io/network` specifies the VM network the guest cluster nodes use. If this annotation is omitted, the mutator automatically selects the first Multus network defined in the guest cluster's virtual machine instances.
- `loadbalancer.harvesterhci.io/project` and `loadbalancer.harvesterhci.io/namespace` identify the project and namespace of the VMs that comprise the guest cluster.
- `loadbalancer.harvesterhci.io/cluster` denotes the name of the guest cluster.
- The IP pool has a selector, including network and scope, to match the requirements of the LB.
- Network is a hard condition. The optional IP pool must match the value of the LB annotation `loadbalancer.harvesterhci.io/network`.
- Every IP pool, except the global IP pool, has a unique scope different from others if its priority is `0`. The project, namespace, or cluster name of LBs should be in the scope of the IP pool if they want to get an IP from this pool.
- Network is a **hard condition**. The target IP pool must match the value of the LB annotation `loadbalancer.harvesterhci.io/network`.
- Every IP pool, except the **global** IP pool, has a unique scope different from others if its priority is `0`. The project, namespace, or cluster name of LBs should be in the scope of the IP pool if they want to get an IP from this pool.
- `spec.selector.priority` specifies the priority of the IP Pool. The larger the number, the higher the priority. If the priority is not `0`, the value should differ. The priority helps you to migrate the old IP pool to the new one.
- If the IP Pool has a scope that matches all projects, namespaces, and guest clusters, it's called a global IP pool, and only one global IP pool is allowed. If there is no IP pool matching the requirements of the LB, the IPAM will allocate an IP address from the global IP pool if it exists.
- If the IP Pool has a scope that matches all projects, namespaces, and guest clusters, it's called a **global** IP pool, and only one **global** IP pool is allowed. If there is no IP pool matching the requirements of the LB, the IPAM will allocate an IP address from the **global** IP pool if it exists.

:::caution

- Network matching was not enforced for **global** IP pool, meaning a **global** IP pool could inadvertently allocate IPs to the incorrect network.

- For guidance on mitigating these risks and managing resource allocation, see [IPPool Best Practices](#ippool-for-guest-cluster-type-loadbalancer).

:::

### Examples
- **Example 1:** You wish to set up an IP pool within the range `192.168.100.0/24` for the `default` namespace. In this scenario, all load balancers within the `default` namespace will receive an IP address from this designated IP pool:
Expand Down Expand Up @@ -119,8 +128,8 @@ Each IP pool will have a specific range, and you can specify the corresponding r
kind: IPPool
metadata:
name: global-ip-pool
labels:
loadbalancer.harvesterhci.io/global-ip-pool: 'true' # Added by the controller automatically
labels:
loadbalancer.harvesterhci.io/global-ip-pool: 'true' # Added automatically by Harvester
spec:
ranges:
- subnet: 192.168.20.0/24
Expand All @@ -145,7 +154,12 @@ Each IP pool will have a specific range, and you can specify the corresponding r

### IPPool for Guest Cluster type Loadbalancer

1. It is better to [Create IP Pool from Rancher Managery UI](#create-ip-pool-from-rancher-manager-ui), because it allows you to tune `Project` and `Guest Kubernetes Cluster` for better resource management and isolation.
1. It is better to [Create IP Pool from Rancher Managery UI](#create-ip-pool-from-rancher-manager-ui), because it allows you to tune `Project`, `Namespace` and `Guest Kubernetes Cluster` for better resource management and isolation.

1. Be careful when creating global IP pool as one guest cluster might allocate too many IPs and starve other clusters. The pool can't be deleted if any of the IPs is still in use.
1. Be careful when creating **global** IP pool as one guest cluster might allocate too many IPs and starve other clusters. The pool can't be deleted if any of the IPs is still in use. And be aware that versions prior to v1.9.0 lacked mandatory network matching, which could lead to IP allocation across incorrect networks.

1. **Use project/namespace-scoped pools for better isolation:** Since each guest cluster is deployed within a specific project and namespace, you should create IP pools scoped to that project or namespace with defined IP ranges. This approach offers several advantages:
* **Granular Control:** A single network can support multiple non-global pools, allowing you to partition IP availability across different projects or namespaces.
* **Resource Protection:** The exhaustion of one pool does not affect others, even though they share the same underlying network.
* **Reduced Risk:** Because these pools are not **global**, they remain isolated from the risk of incorrect network selection inherent to **global** IP pools.
* **Scalability:** It is easier to increase an IP pool's range than to decrease it. Shrinking a range is difficult when the currently used IPs are dispersed throughout the range (e.g., a pool of [1-10] where IPs 1, 5, and 9 are already in use). Consequently, a best practice is to start with a smaller, conservative range and expand as needed.
Loading
Loading