TBS: Install TBS on TKGm workload cluster

Eknath - 6 December 202114 December 2021

Reading Time: 4 mins

Contents

Overview

Tanzu Build Service uses the open-source Cloud Native Build packs project to turn application source code into container images. Build Service executes reproducible builds that align with modern container standards, and additionally keeps image resources up-to-date. It does so by leveraging Kubernetes infrastructure with kpack, a Cloud Native Build packs Platform, to orchestrate the image lifecycle.

Why Tanzu Build Service (TBS)?

Save time to re-build, re-test and re-deploy during patching hundreds of containers.
It auto scans source code configuration and language and inject dependencies into docker image that will be required to compile and/or run the app on container/K8s.
Maintain latest image in image registry.
OCI docker image support, build and run anywhere!

Refer to official page to know about supported build packs and click here to access vmware docs on TBS.

Prepare the setup:

Install Docker

Docker to be installed on bootstrap machine which can be your local workstation or any hosted vm of your choice. For this demo, I have used linux machine hosted on public cloud accessible through internet on port 22.
Steps to install Docker can be found in: https://docs.docker.com/get-docker/

Docker Hub or any Image Registry

Create a Docker Hub Account or you can use any image registry like Harbor or gcr or acr etc.. In this demo I have used Docker Hub.

Kubernetes Cluster

Install Kubernetes, I have used TKGM K8s cluster on Azure, you can also use KIND(K8s in Docker) with Docker Desktop or any kubernetes cluster of your choice. Ensure you have set the context to appropriate K8 cluster where you are planning to deploy TBS. Since I used TKGm on Azure, I have set the context to TKGm workload cluster.

$ kubectl config get-contexts
CURRENT   NAME                                                CLUSTER                 AUTHINFO                      NAMESPACE
          capv-tkg-mgmt-vsphere-admin@capv-tkg-mgmt-vsphere   capv-tkg-mgmt-vsphere   capv-tkg-mgmt-vsphere-admin
          tkg-captainv-demo-admin@tkg-captainv-demo           tkg-captainv-demo       tkg-captainv-demo-admin
*         tkg-workload-1-admin@tkg-workload-1                 tkg-workload-1          tkg-workload-1-admin

Install the Tanzu CLI and Kubectl on bootstrap machine where you manage the K8s cluster. Steps to Install can be found in post

Pivnet CLI

https://network.pivotal.io/products/build-service/ – Signup is required to download the executable from Tanzu Network

Install Pivnet CLI:

brew install pivotal/tap/pivnet-cli

https://network.pivotal.io/users/dashboard/edit-profile – To generate or view API token

Below pivnet CLI commands can be used or can be downloaded manually from Network pages to Install:

kp Install

# To login to Pivnet

pivnet login --api-token='<API Token>'

Note: In this demo, since I am using linux bootstrap machine, I have used kp-linux executables from network pages. You can use appropriate artifacts based on type of bootstrap machine OS. 

# To list the Tanzu build service release versions: 
pivnet releases --product-slug='build-service'

# Download specific version, 1.3.4 is the latest version at the time of writing this, can change over period of time. Please take a look at Tanzu network pages to know about latest releases. 

pivnet download-product-files --product-slug='build-service' --release-version='1.3.4' --product-file-id=1082452
2021/12/12 21:22:41 Downloading 'kp-linux-0.4.2' to 'kp-linux-0.4.2'
 43.89 MiB / 43.89 MiB [============================================] 100.00% 3s
2021/12/12 21:22:46 Verifying SHA256
2021/12/12 21:22:46 Successfully verified SHA256

# Rename the downloaded binary file to kp.

# On the command line, make the kp binary file executable:

chmod ugo+x kp

# Move the binary file into a directory that is on your PATH:

mv kp /usr/local/bin/kp

Note: https://network.tanzu.vmware.com/users/dashboard/eulas, refer to manually accept eula.

# Login to docker-hub. This step will save docker-hub credentials to your K8s cluster. Note: You can use other repository too.

docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: eknath009
Password:
Login Succeeded

docker login index.docker.io
Authenticating with existing credentials...
Login Succeeded

Additional tools: I know too many tools, but these are tiny and doesn’t take more than few mins.

Install below mentioned Carvel CLIs for your operating system. These can be found Tanzu Network pages:

kapp is a deployment tool that allows users to manage Kubernetes resources in bulk.

Install kapp

# Download using pivnet CLI or from network pages: 

pivnet download-product-files --product-slug='kapp' --release-version='0.41.0' --product-file-id=1047336

# Rename the downloaded binary file to kapp.

# On the command line, make the kapp binary file executable:

chmod ugo+x kapp

# Move the binary file into a directory that is on your PATH:

mv kapp /usr/local/bin/kapp

ytt is a templating tool that understands YAML structure.

Install ytt

# Download using pivnet CLI or from network pages: 

pivnet download-product-files --product-slug='ytt' --release-version='0.37.0' --product-file-id=1059871

# Rename the downloaded binary file to ytt.

# On the command line, make the ytt binary file executable:

chmod ugo+x ytt

# Move the binary file into a directory that is on your PATH:

mv ytt /usr/local/bin/ytt

kbld is tool that builds, pushes, and relocates container images

Install kbld

# Download using pivnet CLI or from network pages: 

pivnet download-product-files --product-slug='kbld' --release-version='0.31.0' --product-file-id=1047488

# Rename the downloaded binary file to kbld.

# On the command line, make the kbld binary file executable:

chmod +x kbld

# Move the binary file into a directory that is on your PATH:

mv kbld /usr/local/bin/kbld

imgpkg is tool that relocates container images and pulls the release configuration files. Note: imgpkg 0.12.0 or higher is required for installation.

Install imgpkg

# Download using pivnet CLI or from network pages: 

pivnet download-product-files --product-slug='imgpkg' --release-version='0.23.1' --product-file-id=1091314

# Rename the downloaded binary file to imgpkg.

# On the command line, make the imgpkg binary file executable:

chmod ugo+x imgpkg

# Move the binary file into a directory that is on your PATH:

mv imgpkg /usr/local/bin/imgpkg

https://network.pivotal.io/users/dashboard/eulas

Accept EULA:

copy build service bundle to Image Repo

$ docker login registry.tanzu.vmware.com
Username: <registered username for Tanzu networks>
Password:
Login Succeeded

$ docker login registry.pivotal.io
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

#Syntax: imgpkg copy -b "registry.pivotal.io/build-service/bundle:<version>" --to-repo <image registry>/<repo name>

$ imgpkg copy -b "registry.pivotal.io/build-service/bundle:1.3.4" --to-repo eknath009/tbs-demo

# Login to docker hub to see the list of images copied under repo mentioned above.

# Syntax: imgpkg pull -b "<registry name>/<repo name>:<version>" -o /tmp/bundle

$imgpkg pull -b "eknath009/tbs-demo:1.3.4" -o /tmp/bundle
Pulling bundle 'index.docker.io/eknath009/tbs-demo@sha256:82aff992c1dc02d7f7c1893add98b4920e141d2e52f04412eec27c0791846a0f'
Extracting layer 'sha256:825726164a5262599105db3ba5d6e40b67db045fe55496f9922cbe37e2b880e3' (1/1)
Locating image lock file images...
The bundle repo (index.docker.io/eknath009/tbs-demo) is hosting every image specified in the bundle's Images Lock file (.imgpkg/images.yml)

Updating all images in the ImagesLock file: /tmp/bundle/.imgpkg/images.yml
+ Changing all image registry/repository references in /tmp/bundle/.imgpkg/images.yml to index.docker.io/eknath009/tbs-demo

Succeeded

Deploy TBS

#Syntax:
$ ytt -f /tmp/bundle/values.yaml \
    -f /tmp/bundle/config/ \
	-v kp_default_repository='<registry account name>/<repo name>' \
	-v kp_default_repository_username='<registryaccount name>' \
	-v kp_default_repository_password='<password>' \
	-v pull_from_kp_default_repo=true \
	| kbld -f /tmp/bundle/.imgpkg/images.yml -f- \
	| kapp deploy -a tanzu-build-service -f- -y

#Example: 

$ ytt -f /tmp/bundle/values.yaml \
    -f /tmp/bundle/config/ \
	-v kp_default_repository='eknath009/tbs-demo' \
	-v kp_default_repository_username='eknath009' \
	-v kp_default_repository_password='<password>' \
	-v pull_from_kp_default_repo=true \
	| kbld -f /tmp/bundle/.imgpkg/images.yml -f- \
	| kapp deploy -a tanzu-build-service -f- -y

# You can see new pods getting created 

# Download the descriptor file from https://network.pivotal.io/products/tbs-dependencies/ and import using below command: 

$ kp import -f descriptor-100.0.231.yaml
Importing Lifecycle...
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:2b041ebbb343c9ceb4b0f4237eab69785fbeffb3f98693a190c60b6ee1cedb1c'
Importing ClusterStore 'default'...
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:9f06ae76e45310f2baa4abeb4d449b52423adb2143977aca15409c3cca22f3f9'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:619f92bb10c49acf2d67caceeb619f16ed7e3665ed35eb5fb128e14918a2b8ac'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:c489dda1e82d599da99fff2bed9a03498a5fde1b2cdd0a3bced6884163192eaf'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:520d891f921224054b806a5fcd959222b6f6a82529e3be09e202e4169a87e5f8'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:81569eedda0c65e2d46d443569e9fd7c025fed0a19048f44f498694ee49a7cb2'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:e1b6c20cbf98c8dae1b8e962f1ec1a2ca4061f6f4cbaebdaf211b1e40fbdd0f9'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:baf55402599aab20c52d1c819a9a63b0c6e861e18599b23b91d1871fc11b5d25'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:754d5bc97ebd0640e4d90b1bb72ab012c97f7ca7bf9aed810d15743c2b48860b'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:f6835353560fa4d60c5c7556e0783dd792014cf94c6ef44778a8618d43019400'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:0bbd75708ffbcba4f72723c6ed50932ea1149c0320216bc19d846bdf614e1ab0'
Importing ClusterStack 'tiny'...
Uploading to 'eknath009/tbs-demo'...
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:7504da18ce713342cc4151429cbc2034ef63e67715fc43145fdc1964abe81719'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:96af77e0a85029f588aa63524dd4f6f678c451c3d675472275510a93238c5d2f'
Importing ClusterStack 'base'...
Uploading to 'eknath009/tbs-demo'...
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:60137d9f6b4c602255d8263e34c5ed8f019b37e6c8c626fb0ce7a600f707d3cf'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:36d1b77e0d2220b1a3d6873fdeaf1e374edd56796b1d8be76845db53ffebfda4'
Importing ClusterStack 'full'...
Uploading to 'eknath009/tbs-demo'...
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:999eab7567ec796f4284b5b998b24952a2c977506c88f69dcf9b11299164d3a7'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:c7add890cc6d88fa12d2f1cd9864239aaac2f95c80a5a82c6d811c2ca0bee93d'
Importing ClusterStack 'default'...
Uploading to 'eknath009/tbs-demo'...
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:60137d9f6b4c602255d8263e34c5ed8f019b37e6c8c626fb0ce7a600f707d3cf'
	Uploading 'index.docker.io/eknath009/tbs-demo@sha256:36d1b77e0d2220b1a3d6873fdeaf1e374edd56796b1d8be76845db53ffebfda4'
Importing ClusterBuilder 'base'...
Importing ClusterBuilder 'full'...
Importing ClusterBuilder 'tiny'...
Importing ClusterBuilder 'default'...
Imported resources

# Command to check clusterbuilder list

$ kp clusterbuilder list
NAME       READY    STACK                          IMAGE
base       true     io.buildpacks.stacks.bionic    eknath009/tbs-demo:clusterbuilder-base@sha256:a38386db527cb4a7498af7d46f18f4482ca288d2e8faf83536558e09b3b890f9
default    true     io.buildpacks.stacks.bionic    eknath009/tbs-demo:clusterbuilder-default@sha256:a38386db527cb4a7498af7d46f18f4482ca288d2e8faf83536558e09b3b890f9
full       true     io.buildpacks.stacks.bionic    eknath009/tbs-demo:clusterbuilder-full@sha256:021b49afd5eafdbdb35dfdff86e54744f797d80b6c0e80e337cee37611c15331
tiny       true     io.paketo.stacks.tiny          eknath009/tbs-demo:clusterbuilder-tiny@sha256:0388e2a79c714dcab1a11c966e58bd9d1b2dcc9bf6f2f49d61833db384d99b3b

# Command to check clusterstack list
$ kp clusterstack list
NAME       READY    ID
base       True     io.buildpacks.stacks.bionic
default    True     io.buildpacks.stacks.bionic
full       True     io.buildpacks.stacks.bionic
tiny       True     io.paketo.stacks.tiny

Create a TBS image

Create Secrets and Image

# Here I have created namespace tbs-demo

$ kubectl create ns tbs-demo
namespace/tbs-demo created

$ kp secret create github-creds --git-url https://github.com --git-user eknathreddy09 -n tbs-demo
git password:
Secret "github-creds" created

$kp secret create docker-creds-1 --dockerhub eknath009 -n tbs-demo

$ kp secret list -n tbs-demo
NAME                   TARGET
default-token-2fc6q
docker-creds-1         https://index.docker.io/v1/
github-creds           https://github.com

$ kubectl get secrets -n tbs-demo
NAME                  TYPE                                  DATA   AGE
default-token-2fc6q   kubernetes.io/service-account-token   3      160m
docker-creds-1        kubernetes.io/dockerconfigjson        1      20m
github-creds          kubernetes.io/basic-auth              2      148m

#### Create Image
#Syntax: kp image create <Image name> --tag <Image registry>/<Repo account name>/<Repo>:<tag> --namespace <namespace> --git <git url> --git-revision <branch>


$ kp image create tbs-image --tag index.docker.io/eknath009/tbs-spring-image:3 --namespace tbs-demo --git https://github.com/eknathreddy09/spring-petclinic --git-revision main

# Check the build status using: kp build logs <Image name> -n <namespace>

kp build logs tbs-image -n tbs-demo

TBS will sync with two systems i.e., GitHub and Docker Hub (or any image registry).

Auto Build: After every commit done to git repo, TBS automatically triggers a new build job and create the image.
Manual Build: Here, we will have to execute the build command manually as given below.

KP commands


$ kp build list -n tbs-demo
BUILD    STATUS     BUILT IMAGE                                                                                                           REASON    IMAGE RESOURCE
1        SUCCESS    index.docker.io/eknath009/tbs-spring-image@sha256:04f4d558acc664b0abebe7db4859ae6caccc39d46b94da5be6e0598e4c4b81b1    CONFIG    tbs-image

$ kp build status tbs-image -n tbs-demo
Image:     index.docker.io/eknath009/tbs-spring-image@sha256:04f4d558acc664b0abebe7db4859ae6caccc39d46b94da5be6e0598e4c4b81b1
Status:    SUCCESS
Reason:    CONFIG
           resources: {}
           - source: {}
           + source:
           +   git:
           +     revision: d39fa7079473eb20c643caafa9667842ffebd53b
           +     url: https://github.com/eknathreddy09/spring-petclinic

Started:     2021-12-13 04:13:15
Finished:    2021-12-13 04:24:30

Pod Name:    tbs-image-build-1-build-pod

Builder:      eknath009/tbs-demo:clusterbuilder-default@sha256:a38386db527cb4a7498af7d46f18f4482ca288d2e8faf83536558e09b3b890f9
Run Image:    index.docker.io/eknath009/tbs-demo@sha256:36d1b77e0d2220b1a3d6873fdeaf1e374edd56796b1d8be76845db53ffebfda4

Source:      GitUrl
Url:         https://github.com/eknathreddy09/spring-petclinic
Revision:    d39fa7079473eb20c643caafa9667842ffebd53b

BUILDPACK ID                           BUILDPACK VERSION    HOMEPAGE
paketo-buildpacks/ca-certificates      2.4.2                https://github.com/paketo-buildpacks/ca-certificates
paketo-buildpacks/bellsoft-liberica    8.9.0                https://github.com/paketo-buildpacks/bellsoft-liberica
paketo-buildpacks/maven                5.7.0                https://github.com/paketo-buildpacks/maven
paketo-buildpacks/executable-jar       5.3.1                https://github.com/paketo-buildpacks/executable-jar
paketo-buildpacks/apache-tomcat        6.4.0                https://github.com/paketo-buildpacks/apache-tomcat
paketo-buildpacks/dist-zip             4.3.0                https://github.com/paketo-buildpacks/dist-zip
paketo-buildpacks/spring-boot          4.7.0                https://github.com/paketo-buildpacks/spring-boot

#Trigger manual Build: 

kp image trigger <image name> -n <name space>

Test Application with TBS image:

Application deployment

#Create deployment with spring image created using TBS: 

$ kubectl create deployment spring-deploy --port=8080 --image=eknath009/tbs-spring-image:3 --replicas=3 -n test-tbs-demo
deployment.apps/spring-deploy created
#Expose the deployment
$ kubectl expose deployment spring-deploy -n test-tbs-demo --port=8080 --type=LoadBalancer
service/spring-deploy exposed

#Get external IP 
$ kubectl get svc -n test-tbs-demo
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)          AGE
spring-deploy   LoadBalancer   10.96.237.164   10.212.195.9   8080:32111/TCP   69s