As in current days a new feature or a correction moves through several environments (Development, Staging, UAT) until is finally available to final users (Production environment). While the application moves throughout those environments, it will need to assume different configurations - connect to different databases, and get data from different resources. To allow this kind of flexibility K8s has specific resources like ConfigMap and Secrets to manage all that and this article will show how to use them in several ways.
ConfigMap vs Secrets
ConfigMap is the recommended place to put non-sensitive configurations like a link to a database but not things like the corresponding username and password, for that we have Secrets K8s object.
Setting ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
# this configmap name
name: myConfigmap
namespace: develop
data:
# property-like keys; each key maps to a simple value
mysql_connection: "jdbc:mysql://fake-mysql-db:3306/mydb"
oracle_connection: "jdbc:oracle:thin:@fake-oracle-db:4443:SID"
# file-like keys
kafka.yaml: |
spring:
kafka:
bootstrap-servers: 127.0.0.1:19091
consumer:
group-id: car-group
user-interface.properties: |
color.good=green
color.bad=red
allow.textmode=true
Fig. 1 - Example of a ConfigMap file
In the above example, it can be seen how to create a Configmap file. Section "metadata" provides the "name" by which its keys can be found. Section data gives the keys with values in two possible formats:
key-value pair:
mysql_connection: "jdbc:mysql://fake-mysql-db:3306/mydb"key-file pair:
kafka.yaml: | ....
In Fig. 2 we can see how those keys are used:
apiVersion: v1
kind: Pod
metadata:
name: mydemo-pod
spec:
containers:
- name: demo
image: alpine
command: ["sleep", "3600"]
env:
# Define the environment variable
- name: MYSQL_CONNECTION # Notice that the case is different here
# from the key name in the ConfigMap.
valueFrom:
configMapKeyRef:
name: myConfigmap # The ConfigMap this value comes from.
key: mysql_connection # The key to fetch.
#...unneeded code...
Fig. 2 - Example of usage of Configmap in Pod creation
It can be seen the values for the Mysql and Oracle databases are given in the valueFrom.configMapKeyRef section: we must provide the name given to the configmap (myConfigmap) and the key where it is located.
To apply/create a config map on K8s cluster using a file, open a console and performkubectl apply -f configmap_file_name.yaml
Setting Secrets
Secrets are the K8s object where sensitive data like passwords and tokens can be stored.
Secrets can be of several types [2], but here we will explore the Opaque type because is the one used for credentials like username and password for a Pod to use to connect to resources like a database.
apiVersion: v1
kind: Secret
metadata:
name: mysecret
namespace: develop
type: Opaque
data:
mysql_username: bXlzcWxfdXNlcm5hbWU=
mysql_password: bXlzcWxfcGFzc3dvcmQ=
Fig. 3 - Example of a Secret file with Mysql and Oracle credentials
In Fig. 3 shows how to create a Secret file. The username and passwords must be previously 64 bit encoded and the output pasted on the correspondent place:
echo -n 'mysql_username' | base64
#output bXlzcWxfdXNlcm5hbWU=
echo -n 'mysql_password' | base64
#output bXlzcWxfcGFzc3dvcmQ=
After the secret's file is created execute kubectl apply -f secret_file_name.yaml
Next, it's needed to setup a Pod configuration to make use of those secret values.
The next example shows how to setup a Pod to make use of the secrets we just created.
apiVersion: v1
kind: Pod
metadata:
name: mydemo-pod
spec:
containers:
- name: demo
image: alpine
command: ["sleep", "3600"]
env:
- name: MYSQL_USERNAME # Define the environment variable
valueFrom:
secretKeyRef:
name: mysecret # The Secret this value come from
key: mysql_username # The key to fetch.
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: mysql_password
Fig. 4 - Secrets applied to a Pod
Here we can see that it is created an environment variable in CAPS that will be used by the application and subsection valueFrom where we indicate the secret object name where to get the value and the key that has the value.
Setting configuration by mounting volumes
K8s allows making the configuration objects (both ConfigMap and Secrets) accessible by volume mounts. The configurations are created normally, then in the Pod configurations file they are defined as volume mount points and the keys are available inside those same mount points.
Here how it's done:
Create the Configmap
apiVersion: v1
kind: ConfigMap
metadata:
# this configmap name
name: my-configmap
data:
# property-like keys; each key maps to a simple value
oracle_connection: "jdbc:oracle:thin:@fake-oracle-db:4443:SID"
# file-like keys
kafka.yaml: |
spring:
kafka:
bootstrap-servers: 127.0.0.1:19091
consumer:
group-id: car-group
user-settings.properties: |
colors.color1=yellow
colors.color2=red
fruits.fruit1=banana
fruits.fruit2=strawberry
Fig. 5: ConfigMap to be used as a Volume
apiVersion: v1
kind: Secret
metadata:
name: my-secret
type: Opaque
data:
oracle_username: b3JhY2xlX3VzZXJuYW1l
oracle_password: b3JhY2xlX3Bhc3N3b3Jk
Fig. 6: Secret to be used as a Volume
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
containers:
- name: demo
image: alpine
resources:
limits:
memory: 600Mi
cpu: 1
command: ["sleep", "3600"]
volumeMounts:
- name: configmap-vol
mountPath: "/etc/config/configMap"
- name: secret-vol
mountPath: "/etc/config/secret"
volumes:
# You set volumes at the Pod level, then mount them into containers
# inside that Pod
- name: configmap-vol
configMap:
# Provide the name of the ConfigMap you want to mount.
name: my-configmap
# An array of keys from the ConfigMap to create as files
items:
- key: "kafka.yaml" # the key on configmap
path: "kafka_file.yaml" # the name to be given on the volume
- key: "user-settings.properties"
path: "user-settings_file.properties"
- name: secret-vol
secret:
secretName: my-secret
Fig. 7: Pod definition with the ConfigMap and Secret to be mounted as Volumes
After "applying" the K8s definition files it is possible to access the pod console and see how the artifacts were created.
Fig. 8 - Accessing the configuration as volumes inside a pod
Setting configuration as a Pod's environment variable
Another way to have configuration variables available is by setting them as environment variables. The way to set this up is simple. Create the ConfigMap with the variable names in upper case format:
apiVersion: v1
kind: ConfigMap
metadata:
# this configmap's name
name: my-configmap-posix
data:
# property-like keys; each key maps to a simple value
# the key must be UPPER CASE
ORACLE_CONNECTION: "jdbc:oracle:thin:@fake-oracle-db:4443:SID"
MYSQL_CONNECTION: "jdbc:mysql://localhost:3306/mydb"
Fig. 9 - ConfigMap defining 2 variables to be available as environment variables in a Pod
In the Pod definition file, set the ConfigMap's name to be used under spec.containers.envFrom.configMapRef.name:
apiVersion: v1
kind: Pod
metadata:
name: my-pod-posix
spec:
containers:
- name: demo
image: alpine
resources:
limits:
memory: 300Mi
cpu: 1
command: ["sleep", "3600"]
envFrom:
- configMapRef:
name: my-configmap-posix # the ConfigMap's name to be used
Fig. 10 - Pod definition file with the ConfigMap as environment variables placeholder
Apply both definition files and enter in the pod's console by issuing kubectl exec --stdin --tty my-pod-posix -- /bin/sh .
By issuing printenv it can be seen all the environment variables including the ones declared in the ConfigMap.
Fig. 11 - printenv command showing Pod's console showing environment variables
Using ConfigMap and Secrets as a file in a Pod
If you need a configuration file to setup a MySQL database and have those configurations available as a file in the proper place so the MySql engine can use it.
To create a ConfigMap:
Create the required file, for our example it will be mysql.conf.
To generate a ConfigMap from the file issue: kubectl create configmap mysql-conf-file --from-file mysql.conf . If you list the ConfigMaps in K8s cluster
> kubectl get configmap
NAME DATA AGE
kube-root-ca.crt 1 38d
my-configmap 3 4d20h
my-configmap-posix 2 4h7m
mysql-conf-file 1 07s #the configmap just created
To create a Secret is the same process:
Create the data in a file and issue the follwoing comand :
kubectl create secret <secret_name> --from-file <file_name>
In this example two certificates where created: my-certificate-1.crt and my-certificate-2.crt. To create the secrets input:
kubectl create secret generic certificate-1 --from-file my-certificate-1.crt
kubectl create secret generic certificate-2 --from-file my-certificate-2.crt
If you list the Secrets in K8s:
> kubectl get secret
NAME TYPE DATA AGE
certificate-1 Opaque 1 4m56s
certificate-2 Opaque 1 5m5s
Now we have to define in the Pod definition file how to associate the mysql-conf-file and my-certificate to the proper files.
As a note the secrets will be made available as a file but only inside of different folders.
apiVersion: v1
kind: Pod
metadata:
name: my-pod-conf-as-file
spec:
containers:
- name: demo
image: alpine
resources:
limits:
memory: 300Mi
cpu: 1
command: ["sleep", "3600"]
volumeMounts:
- name: mysql-config-volume # name of the volume that has the mysql.conf data
mountPath: /etc/mysql/my.cnf # the file name to mount the configmap
- name: my-certificate-1-volume # name of the volume that has the certificate
mountPath: /etc/certificate-folder-1 # folder where to put the secret
- name: my-certificate-2-volume
mountPath: /etc/certificate-folder-2
volumes:
- name: mysql-config-volume # the name to give to this volume
configMap:
name: mysql-conf-file # the name of the configmap
- name: my-certificate-1-volume # the name to give to this volume
secret:
secretName: certificate-1 # the name of the Secret
- name: my-certificate-2-volume # the name to give to this volume
secret:
secretName: certificate-2 # the name of the Secret
If we access the pod by issuing kubectl exec --stdin --tty my-pod-conf-as-file -- /bin/sh we can verify if the items were created:
> ls /etc/cert* # list all content starting with my
/etc/certificate-folder-1: # the folder where the 1st certificate was mounted
my-certificate-1.crt # the certificate inside the folder
/etc/certificate-folder-2: # the folder where the 2nd certificate was mounted
my-certificate-2.crt # the certificate inside the folder
References
[1] “Configuration,” Kubernetes. https://kubernetes.io/docs/concepts/configuration/ (accessed Nov. 19, 2023).
[2] “Secrets,” Kubernetes. https://kubernetes.io/docs/concepts/configuration/secret/
(accessed Nov. 19, 2023).