One of the most requested feature with previous versions of the VMware Container Service Extension (CSE) is to add a native UI to it. As of CSE 2.6 we have added a native UI to CSE, which is adding to the friendliness of CSE and will make it much more appealing to many of our cloud providers. At just few clicks, our customers can deploy a K8S clusters at our Cloud Providers with filling few easy to understand fields.

Kubeconfig file will be auto generated as well and can be handed out right away to the developer limiting the efforts required by the tenant operation team/cloud providers administrators. Here is a quick screenshot teaser of what CSE look like. You can find a nice demo of CSE 2.6.1 UI at my following blog post: Cloud Director Kubernetes as a Service with CSE 2.6.x Demo

For more info on what is new with CSE 2.6.x please check my following blog post: vCloud Director Container Service Extension 2.6.x is here

 

 

In this post, I am assuming you have an existing vCloud Director environment and AMQP already configured. To start the installation of CSE 2.6.1, you will need a supported OS. In my case, I have decided to go with CentOS 8.1. After installation CentOS 8.1, you will need to install the development tools and few libraries. Use the below few commands to achieve that.

 [root@vtcse01 ~]#  sudo yum -y groupinstall "Development Tools"
[root@vtcse01 ~]#  sudo yum -y install openssl-devel bzip2-devel libffi-devel
[root@vtcse01 ~]# gcc --version
gcc (GCC) 8.3.1 20190507 (Red Hat 8.3.1-4)
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[root@vtcse01 ~]# sudo yum -y install wget

************ Steps to Fix the SQLite Issue Start ******

The SQLite version including with CentOS is not adequate for the installation of CSE/vcd-cli, as it’s still using an older version of it. We will need to follow the below steps to upgrade it and to avoid hitting the issue I have documented in the following post:  Running vCD Cli fail with the following error: ModuleNotFoundError: No module named ‘_sqlite3’ . Please note that particular post go into more details in explaining the below steps, so if you want further understanding of it, please check it out.

 [root@vtcse01 Python-3.7.3]# wget https://www.sqlite.org/2020/sqlite-autoconf-3310100.tar.gz
[root@vtcse01 Python-3.7.3]# tar xvf sqlite-autoconf-3310100.tar.gz
[root@vtcse01 Python-3.7.3]# cd sqlite-autoconf-3310100/
[root@vtcse01 Python-3.7.3]# ./configure --prefix=/usr
[root@vtcse01 Python-3.7.3]# make install
Check your sqlite version is upgrade as expected
[root@vtcse01 Python-3.7.3]# sqlite3 –version
3.31.1 2020-01-27 19:55:54 3bfa9cc97da10598521b342961df8f5f68c7388fa117345eeb516eaa837bb4d6

************ Steps to Fix the SQLite Issue End ******

CSE will require Python 3.7.3 or higher. In here, I am going to install Python 3.7.3. Here is the steps to follow and install it from source code.

[root@vtcse01 ~]# wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tgz
[root@vtcse01 ~]# tar xvf Python-3.7.3.tgz
[root@vtcse01 ~]# cd Python-3.7*/
[root@vtcse01 Python-3.7.3]# ./configure --enable-optimizations
[root@vtcse01 Python-3.7.3]# make install
[root@vtcse01 Python-3.7.3]# python3.7 --version;
Python 3.7.3
[root@vtcse01 Python-3.7.3]# pip3.7 –version

Check below that Python3.7.3 is detecting your updated SQLite

[root@vtcse01 Python-3.7.3]# python3
Python 3.7.3 (default, May  4 2020, 15:36:31)
[GCC 8.3.1 20190507 (Red Hat 8.3.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.31.1'

Now we have all our pre-requisites satisfied, let’s start with the CSE installation.

[root@vtcse01 Python-3.7.3]# pip3 install container-service-extension

Test the CSE and vcd-cli commands

[root@vtcse01 Python-3.7.3]# cse version
CSE, Container Service Extension for VMware vCloud Director, version 2.6.1

[root@vtcse01 Python-3.7.3]# vcd cse version
Error: No such command 'cse'.
Note: The above error is OK, as we will need to add the CSE extension to vcd config files.

At first test that you can login to your vCD setup with the vCD CLI

[root@vtcse01 Python-3.7.3]# vcd login vcd.vt.com system administrator -i
Password:
administrator logged in, org: 'system', vdc: ''

Please note above I have used the -i flag to ignore my self-signed certificates. Not recommended for a production environment, but if you have a production environment, I assume you will have a valid certs on your vCD and you won’t need the -i flag.

Now we need to modify the ~/.vcd-cli/profiles.yaml file by adding the following lines at the end of the file

extensions:
- container_service_extension.client.cse

Here is what the file look like in my setup:

active: default
profiles:
- api_version: '33.0'
   disable_warnings: false
   host: vcd.doomdns.org
   is_jwt_token: true
   log_body: true
   log_header: true
   log_request: true
   name: default
   org: system
   org_href: https://vcd.vt.org/api/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9
   org_in_use: system
   token: eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbmlzdHJhdG9yIiwiaXNzIjoiYTkzYzlkYjktNzQ3MS0zMTkyLThkMDktYThmN2VlZGE4NWY5QDU3ZDIyMTY5LTcwYWEtNDNjNS04YzNkLWMyNGU5YzI3ZDc2NSIsImV4cCI6MTU4ODk0MTk2NCwidmVyc2lvbiI6InZjbG91ZF8xLjAiLCJqdGkiOiI2MGI3ZTU1N2MzZjA0MTk3OTk1YmM5NDQ0MGJhMWRlMCJ9.NFpVjmh0KpIpA43HcAbv7epovb7NZ_gm0Oz29ihMACnmTAl4rlMRx2VmFHjf004l0_TBYYZwv1FkB9khHKAbVqnQ_gl4C2O7v_odYg8WSB8AH6CqC54eAfXaQdFbk8zo3qHVu34xu3OkugfVYaH362AolGJ8O7e01Arf2hUctX3tTF2m7nARvk3CMyR9alCOnZofeoj7CTHGgmM6yqL87fEhTSq9s3FlcFc5M9HzeXqB6HNVHKi93g8WSB8AH6CqC54eAfXaQdFbk8zo3qHVu34xu3OkugfV
  user: administrator
  vapp_href: ''
  vapp_in_use: ''
  vdc_href: ''
  vdc_in_use: ''
  verify: false
extensions:
- container_service_extension.client.cse

Now let’s try the vcd cse command as the above modification of the profiles.yaml files should get it ready for us to use.

[root@vtcse01 Python-3.7.3]# vcd cse version
CSE, Container Service Extension for VMware vCloud Director, version 2.6.1

Create CSE Config file

Let’s start by creating a sample config.yaml file by running the following command:

[root@vtcse01 Python-3.7.3]# cse sample -o config.yaml

Here is a sample config file

amqp:
  exchange: cse-ext
  host: amqp.vmware.com
  password: guest
  port: 5672
  prefix: vcd
  routing_key: cse
  ssl: false
  ssl_accept_all: false
  username: guest
  vhost: /

vcd:
  api_version: ‘33.0’
  host: vcd.vt.com
  log: true
  password: my_secret_password
  port: 443
  username: administrator
  verify: true

vcs:
- name: vc1
  password: my_secret_password
  username: cse_user@vsphere.local
  verify: true
- name: vc2
  password: my_secret_password
  username: administrator@vsphere.local
  verify: true

service:
  enforce_authorization: false
  listeners: 10
  log_wire: false
  telemetry:
    enable: true

broker:
  catalog: cse
  default_template_name: ubuntu-16.04_k8-1.17_weave-2.6.0
  default_template_revision: 1
  ip_allocation_mode: pool
  network: mynetwork
  org: myorg
  remote_template_cookbook_url: https://raw.githubusercontent.com/vmware/container-service-extension-templates/master/template.yaml
  storage_profile: ‘*’
  vdc: myorgvdc

# [Optional] Template rule section
# Rules can be defined to override template definitions as defined by remote
# template cookbook.
# Any rule defined in this section can match exactly one template.
# Template name and revision must be provided for the rule to be processed.

Please note a good documentation on how to configure the above configuration file can be found at: https://github.com/vmware/container-service-extension/blob/master/docs/CSE_CONFIG.md. Please note that you will need to fill the default template field correctly as shown in the above sample file, else you will end up with the error I have documented in the following blog post: CSE 2.6.1 Error: Default template my_template with revision 0 not found. Unable to start CSE server.

CSE requires a valid SSH key, so before you can proceed any further, you will need to generate one. Use the below command to do so:

[root@vtcse01 Python-3.7.3]# ssh-keygen -t rsa -b 4096 -C “myemail@virtualizationteam.com”
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:YoATaF+IR9l5/o0pPtvSr5522nWkybNqwg0oRC6oI “myemail@virtualizationteam.com”
The key's randomart image is:
+---[RSA 4096]----+
| .+.= .          |
|.o O + o         |
|. * + + .     .  |
|.. o o +     + o |
|W .   = V =   T..|
| .   . * * o =.o.|
|      . = o 0o   |
|       o = .o .  |
|        . o. o.  |
+----[SHA256]-----+

Now let’s run the CSE installation with config.yaml file we have created. Please make sure that was updated to include your environment information

We need to encrypt the configuration file first in version 2.6 and higher, while there is options to skip that, it is highly recommended in production environment and help protect your configuration files

[root@vtcse01 Python-3.7.3]# cse encrypt config.yaml --output encrypted-config.yaml
Required Python version: >= 3.7.3
Installed Python version: 3.7.3 (default, May  4 2020, 15:36:31)
[GCC 8.3.1 20190507 (Red Hat 8.3.1-4)]
Password for config file encryption:
Encryption successful

[root@vtcse01 Python-3.7.3]# chmod 600 encrypted-config.yaml

Now, it is show time. We can now run the CSE install command to prepare the environment and download the desired Kubernetes templates and prepare them in your catalog.

[root@vtcse01 Python-3.7.3]#  cse install -c encrypted-config.yaml --ssh-key /root/.ssh/id_rsa.pub
Required Python version: >= 3.7.3
Installed Python version: 3.7.3 (default, May  4 2020, 15:36:31)
[GCC 8.3.1 20190507 (Red Hat 8.3.1-4)]
Password for config file decryption:
Decrypting 'encrypted-config.yaml'
Validating config file 'encrypted-config.yaml'
Connected to AMQP server (VTAMQP01.vt.com:5672)
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised.
Connected to vCloud Director (vcd.doomdns.org:443)
Connected to vCenter Server 'vtvc01' as 'administrator@vsphere.local' (vtvc01.vt.com:None)
Config file 'encrypted-config.yaml' is valid
Installing CSE on vCloud Director using config file 'encrypted-config.yaml'
………..

Preview(opens in a new tab)

Try to run use the following run command, and things should work unless if you have a mis-configuration issue:

[root@vtcse01 Python-3.7.3]# cse run --config encrypted-config.yaml

Download the CSE Plugin from the following source: https://github.com/vmware/container-service-extension/raw/master/cse_ui/1.0.1/container-ui-plugin.zip, then upload it into your vCloud Director under More ==> Customize Portal

Now you are ready to start using CSE, you can get to it from the “Kubernetes Container Clusters” under the More menu at the top. I have recorded a detailed Demo that can be found at: VMware Cloud Director Kubernetes as a Service with CSE 2.6.x Demo

Alright ideally you don’t want to have to start the service manually every-time you restart the VM, so you want your CSE to be restarted as a service each time the OS is rebooted. My colleague Stefan Berner had created a great blog post on that, which I don’t see a reason to duplicate the instruction of it here that can be found at: http://cloud-monkey.de/2020/05/20/cse-as-a-service-with-encrypted-configuration-files/

Just couple of extra tips on sitting up the service. You might need to disable or fix your SElinux permissions for the service to run.

Here is the command you need after you have setup the right cse.service and cse.sh files:

# systemctl enable cse.service
# systemctl start cse.service
# systemctl status cse.service