Self-Managed Kubernetes (K8S) Provisioning Workflow on MTN Cloud

Overview

This guide covers the automated deployment of a custom, self-managed Kubernetes (MK8S) cluster on the MTN Cloud Platform. The cluster consists of control-plane nodes and worker nodes, provisioned using kubeadm, kubectl, and other core Kubernetes components to provide container orchestration and lifecycle management.

Why it matters:

Container Orchestration: Provides a production-ready Kubernetes environment for deploying, scaling, and managing containerized applications.
High Availability: Supports multi-master configurations with three or more control-plane nodes for fault-tolerant cluster operations.
Flexible Networking: Choose between Flannel or Calico as the Container Network Interface (CNI) provider to suit your networking requirements.
Scalability: Easily expand the cluster by adding master or worker nodes using either the Add Node action or Instance Type method.
Self-Managed Control: Offers full access to the Kubernetes API and cluster components, enabling custom configurations and integrations.

Prerequisites

Before you begin, ensure you have:

An active MTN Cloud Console login with the appropriate permissions (e.g., Customer Admin User role) to view the “Provisioning” and “Infrastructure” tabs.Tip: If you do not see required features and resources, please refresh the portal or contact support.
A created Resource Pool (Project). Refer to the Project Creation Workflow documentation.
A created Network. Refer to the Network and Router Creation Workflow documentation for detailed steps.
A created Router with the network attached. Refer to the Network and Router Creation Workflow documentation for detailed steps.
A pre-configured Security Group to manage inbound/outbound traffic. Refer to the Security Group Creation documentation for detailed steps.
A planned deployment strategy including:
- Cluster name and environment
- Number of master and worker nodes
- Network configuration for inter-node communication
- CNI provider selection (Flannel or Calico)

Important Note: Ports configured for the MK8S cluster (e.g., 6443 for API Server, 10250 for Kubelet, etc.) must be added as inbound rules in your security group and applied to the MK8S cluster nodes, alongside the default security group.

Phase 1: Deploy MTN Kubernetes (MK8S) Cluster

Step 1 – Select Catalog Item

Navigate to Provisioning > Catalog and select the MTN Kubernetes (MK8S) catalog item.

Figure 1: Accessing the Catalog from the Provisioning dashboard.

Figure 2: The Catalog view displaying the “MTN Kubernetes (MK8S)” option for ordering.

Step 2 – Configure Deployment

Fill in the following details in the order form:

General Configuration:

Cluster Name: Provide a unique name for your Kubernetes cluster.
Group & Cloud: Select your assigned group and cloud.
Name: Provide a unique name for the deployment.
Environment: Select the appropriate environment (e.g., Development, Staging, Production).
Labels: Optionally, add labels for organizational purposes.
Resource Pool: Choose the appropriate Resource Pool (Project).
Single Node Cluster?: If checked, the master taint is removed and worker node(s) will not be provisioned.

Master-Node Configuration:

Master Plan: Select a plan for master nodes. MK8S Master Node instances require at least 2GB of CPU and memory (i.e., at least the G2S2 Plan).
Master Volumes: Select the required volume capacity.
Master Security Group(s): Select both the Default security group and the custom Security Group created earlier.
Master Network: Assign your Network.
Master Floating IP: Assign a floating IP if required.
Master Node Count: Specify the number of master nodes (3 or more for HA).

Worker-Node Configuration:

Worker Plan: Select a plan for worker nodes.
Worker Volumes: Select the required volume capacity.
Worker Security Group(s): Select both the Default security group and the custom Security Group created earlier.
Worker Network: Assign your Network.
Worker Floating IP: Assign a floating IP if required.
Worker Node Count: Specify the number of worker nodes.

Cluster Configuration:

Container Network Interface Provider: Select either Flannel or Calico.
Virtual IP: (Coming soon)

Important Notes:

If the Master Node network is different from the Worker Node network, ensure reachability by appropriate routing (i.e., both networks on the same router) and ensure security groups for both allow communication between them.
For availability across availability zones, use Network Groups, which utilize a Round Robin algorithm.

Click Order Now to initiate the deployment.

Figure 3: The deployment order form for MK8S cluster configuration.

Step 3 – Verify and Manage

Navigate to Provisioning > Apps to view your MK8S cluster application.

Figure 4: Apps view showing the MK8S cluster application.

Clicking on the application provides additional details, including the estimated provisioning time (ETA) and a list of associated instances or virtual machines.

Figure 5: Application details showing ETA and associated instances.

Once provisioning is complete, you can verify the successful installation and configuration of the MK8S cluster from the master node(s):

Select the master node instance.
Open the Console tab.
Execute the required kubectl commands to verify or manage the cluster.

Figure 6: Select the Master Node.

Figure 7: Select Console Tab.

Figure 8: Console view of the master node showing kubectl commands.

Phase 2 – Expanding the Cluster

There are multiple methods to facilitate the expansion of an MTN Kubernetes (MK8S) cluster.

Method 1 – Add Node Action

When using Catalog Items to provision or extend an MK8S cluster, Cypher keys are automatically generated to facilitate the seamless addition of master or worker nodes.

Navigate to Tools > Cypher to access the Cypher keys.

Figure 9: Cypher keys view in Tools.

Important Note: Cypher keys are subject to lease expiration:

Certificates supporting master node addition are valid for two hours.
Worker node join commands are valid for 24 hours.
If credentials expire, new Cypher keys with updated join commands can be generated as needed.

To add a master or worker node, select an existing node.
Navigate to Actions and choose Add Node.

Figure 10: Add Node action from the Actions menu.

Click Execute.

Figure 11: Execute the Add Node action.

Confirm that the new node has joined the cluster by logging in to the master node’s CLI and running the appropriate kubectl commands.

Figure 12: Verify new node joining the cluster.

Method 2 – Instance Type

To expand the cluster using the Instance Type method:

Navigate to Provisioning > Instances.
Click +Add.
Select the appropriate instance type:
- MK8S Master for control-plane nodes
- MK8S Worker for worker nodes

Figure 13: Select the appropriate MK8S instance type.

Complete the provisioning workflow until you reach the step for entering the master or worker node join command.

Figure 14: Complete the workflow.

Important Notes:

To add a new node, the Cypher keys must be decrypted to retrieve the appropriate join commands for a master or worker node, with particular attention to master nodes.
If the credentials expire, new Cypher keys containing refreshed join commands can be generated as required.
When this method is used for cluster expansion, custom MK8S inputs (such as CNI provider, K8S VIP, etc.) are not applicable.

To connect or join the instance to the MK8S cluster application:
- Navigate to Provisioning > Apps.
- Select the cluster.
- Click Add Instance.

Figure 15: Add Instance to the cluster app.

Select Tier (Master or Worker) and the desired instance.

Figure 16: Select Tier.

Important Note: This method provisions the VM as an independent instance instead of associating it with the existing instance group. Although this behavior is expected and not problematic, the preferred approach is to use the Add Node action.

Confirm that the new node has joined the cluster by logging in to the master node’s CLI and running the appropriate kubectl commands.

Figure 17: Verify new node joining the cluster.

Regenerating Cypher Keys

The Cypher keys contain certificates and join commands with lease durations of two hours and twenty-four hours, respectively. Regeneration is necessary when performing cluster expansion.

Navigate to Provisioning > Apps.
Select the cluster.
Navigate to Instances and select the Master Node.
Click ACTIONS.
Select Run Workflow.

Figure 18: Run Workflow.

Select the workflow MK8S Cypher Regeneration Git.

Figure 19: Select the Cypher Regeneration workflow.

Click Execute when prompted.
Once executed, navigate to Tools > Cypher.

Figure 20: Access Cypher keys in Tools.

New Cypher keys can now be used to expand your cluster by adding extra master or worker nodes.

Additional Deployment Method – Instance Type Path

Provisioning can alternatively be performed via the Instances tab, using the same workflow typically applied when creating a new instance or virtual machine.

The Right Sequence:

Create Security Group: Establish a controlled network boundary. Refer to the Security Group Creation documentation for detailed steps.
Deploy MK8S Master Node: Deploy a cloud-optimized Kubernetes control plane with kubeadm, etcd, and API server.
Obtain Worker Node Join Command: Generate the required join commands and certificates to enable future addition of nodes.
Deploy MK8S Worker Node: Deploy a cloud-optimized Kubernetes worker node for workload execution and cluster scaling.

Step 1 – Deploy MK8S Master Node

Navigate to Provisioning > Instances.
Click +Add.

Figure 21: Navigate to Instances and click +Add.

Select MTN KUBERNETES (MK8S) MASTER instance type.

Figure 22: Select MK8S Master instance type.

Complete the provisioning workflow by providing all required inputs, covering both default inputs and custom MK8S-specific inputs.
Once provisioning is complete, verify the successful installation and configuration of the MK8S cluster via the Console tab.

Figure 23: Verify MK8S Master node deployment.

Figure 24: Console Tab.

Step 2 – Obtain Worker Node Join Command

Facilitate the generation of join commands and related certificates required for secure cluster expansion with additional nodes. Refer to the relevant section of this document (Section 3.3 – Regenerating Cypher Keys) for the process to obtain the join command and certificate. Alternatively, execute the necessary kubeadm commands on the provisioned master node.

Step 3 – Deploy MK8S Worker Node

Navigate to Provisioning > Instances.
Click +Add.

Figure 25: Navigate to Instances and click +Add.

Select MTN KUBERNETES (MK8S) WORKER instance type.

Figure 26: Select MK8S Worker instance type.

Complete the provisioning workflow by providing all required inputs, covering both default inputs and custom MK8S-specific inputs (Worker Join Command).
After provisioning is completed, validate the integration and configuration of the worker node within the MK8S cluster by navigating to the master node’s Console tab and executing the relevant kubectl commands.

Figure 27: Verify worker node joining the cluster.

Important Tips and Notes

Configuration Consistency:Ensure configuration consistency when expanding the cluster by selecting the correct node role, instance type, and matching the existing cluster’s CNI to prevent networking or registration issues.
Cypher Key Validity: Cypher keys have limited validity periods (2 hours for master certificates and 24 hours for worker join commands). Plan node additions accordingly and regenerate keys if credentials expire.
Recommended Expansion Method: Although alternative provisioning paths exist, the Add Node action is the recommended method for cluster expansion as it maintains proper association, visibility, and lifecycle management within the cluster environment.
Post-Provisioning Verification: After node provisioning, always verify cluster state from the master node using appropriate kubectl commands (for example, checking node registration and status) to confirm successful integration and readiness.
Network Security: Ensure that network routing and security group rules allow proper communication between master and worker nodes. If networks differ, verify routing and security group permissions.
High Availability: For production environments, deploy three or more master nodes to achieve high availability. Use the Virtual IP (VIP) feature when available for API server load balancing.