Stone Door Group

View Original

Deploying Quay and Clair in OpenShift on AWS - Part 3 (Quay and Clair)

Eric Archer

In our previous posts we discussed, in Part 1, the business case for using Red Hat Quay as our container registry and Red Hat Clair to scan the images in Quay for known security vulnerabilities. Then, in Part 2, we then deployed our Openshift cluster in AWS public cloud infrastructure to support Quay and Clair. 

In this, part 3 of our three part series, we will deploy Quay and Clair using the Quay operator in our Openshift cluster on AWS and push an image to the Quay registry. We will then monitor the image to see if it passes the Clair security scan

 

Series Overview

In Part 1 of this series. We prepared the resources required to install Openshift as a public cluster on the AWS Public cloud infrastructure including resources required to deploy Quay and Clair on that Openshift cluster. 

Part 2 of this series takes you through the steps required to install Openshift as a public cluster in AWS. A public Openshift cluster has internet accessible resources. Installing in AWS provides us the resources to lock down our Openshift environment to limit access to just the systems we specify. 

Part 3 of the series takes you through the steps required to deploy Quay Container Registry and the Clair security scanner in our Openshift Cluster running on AWS public cloud. 

 

Preparing for the Openshift Container Platform(OCP) cluster installation


The Openshift cluster installation will be done from your local workstation or laptop.

The local user account name used in the examples is "usera" with a home directory of /home/usera

You will need a user account on the Red Hat customer portal. (access.redhat.com) "yourname@redhat.com" is used as an example in the following steps.

Your Red Hat user account will be used to create and download a pull secret.


################### NOTE ###################

Use your own sshKey, AWS Access Key ID, AWS Secret Access Key, Base Domain, and Red Hat pullSecret.

The examples listed in the steps below WILL NOT WORK!!!

################# END NOTE ##################

Open a terminal session on your RHEL workstation or laptop


  1. Start in your home directory

[usera@local-workstation ~]$ cd ~


2. Make a directories for the kubernettes configuration file:

[usera@local-workstation ~]$ mkdir ~/.kube


3. Copy the kubeconfig file to the .kube directory:

[usera@local-workstation ~]$ cp \
/home/usera/aws-ocp/install/auth/kubeconfig ~/.kube/config


4. Run the following command to configure your CLI session to connect to the Openshift cluster

[usera@local-workstation ~]$ export \
KUBECONFIG=/home/usera/aws-ocp/install/auth/kubeconfig


5. Make a directory for the Quay installation

[usera@local-workstation ~]$ mkdir -p \
aws-ocp-quay-clair/ocp-quay


6. Create a new project for Quay in the openshift cluster

[usera@local-workstation ~]$ oc new-project quay-enterprise


7. Create an htpasswd file with the user who will be the local administrator for Quay:

[usera@local-workstation ~]$ htpasswd -c -B -b \
/home/usera/aws-ocp-quay-clair/ocp-quay/htshad \
quay-admin QuayAdministrator


8. Create an OpenShift Container Platform Secret that contains the HTPasswd users file.

[usera@local-workstation ~]$ oc create secret generic \
htpass-secret \
--from-file=htpasswd=/home/usera/aws-ocp-quay-clair/ocp-quay/htshad -n openshift-config

9. Create a Custom Resource (CR) yaml file for the HTPasswd provider

[usera@local-workstation ~]$ vi htpasswd-cr.yaml
apiVersion: config.openshift.io/v1
kind: OAuth
metadata:
  name: cluster
spec:
  identityProviders:
  - name: Local
    mappingMethod: claim
    type: HTPasswd
    htpasswd:
      fileData:
        name: htpass-secret


10. Apply the defined CR(Custom Resource) to the Openshift cluster:

[usera@local-workstation ~]$ oc apply -f htpasswd-cr.yaml



Install the Red Hat Quay Operator in the Openshift webUI


  1. Log into the Openshift webUI  using the kubeadmin account and password listed when the Openshift cluster finished installing at the end of part 2 of this series.


2. You may have to accept one or two security exceptions because the new Openshift cluster is using a self signed certificate.

    In the OpenShift webUI, select Operators → OperatorHub

    Search for Quay

    Select "Red Hat Quay"

    Click "Install"


3. Choose the following then select Subscribe:

          Update Channel: Choose the update channel (I used quay 3.3 for this series)

          Installation Mode: Select a specific namespace (quay-enterprise)

          Approval Strategy: Choose to approve automatic or manual updates (automatic)


Create the secrets for Quay and Clair


Return to the command line.


  1. Credentials are required for Accessing Quay.io registry. Create a new file with the required credentials.

[usera@local-workstation ~]$ vi docker_quay.json
{
  "auths":{
    "quay.io": {
        "auth": "cmVkaGF0K3F1YXk6TzgxV1NIUlNKUjE0VUFaQks1NEdRSEpTMFAxVjRDTFdBSlYxWDJDNFNEN0tPNTlDUTlOM1JFMTI2MTJYVTFIUg==",
        "email": ""
    }
  }
}


2. Create a Red Hat pull secret:

[usera@local-workstation ~]$ oc create secret generic \
redhat-pull-secret \
--from-file=".dockerconfigjson=docker_quay.json" \
--type='kubernetes.io/dockerconfigjson'



3. Create a Quay Superuser secret:

[usera@local-workstation ~]$ oc create secret generic \
quay-admin --from-literal=superuser-username=quay-admin \
--from-literal=superuser-password=QUAYadminPW \
--from-literal=superuser-email=network.operations@your.aws.rt53.dns.domain.com


4. A dedicated deployment of Quay Enterprise is used to manage the configuration of Quay. Access to the configuration interface is secured and requires authentication in order to gain access.

Create a Quay Configuration Secret:

[usera@local-workstation ~]$ oc create secret generic \
quay-config --from-literal=superuser-username=quay-config \
--from-literal=config-app-password=CONFIGadminPW


5. Create a Quay Configuration App Secret:

[usera@local-workstation ~]$ oc create secret generic \
quay-config-app \
--from-literal=config-app-password=QUAYCONFIGAPPadminPW



6. Create a Database credentials secret – PostgreSQL:

[usera@local-workstation ~]$ oc create secret generic \
quay-database-credential --from-literal=database-username=quay \
--from-literal=database-password=quay \
--from-literal=database-root-password=quayAdmin \
--from-literal=database-name=quay-enterprise


7. Create a Redis Password Secret

By default, the operator managed Redis instance is deployed without a password. A password can be specified by creating a secret containing the password:


[usera@local-workstation ~]$ oc create secret generic \
redis-password --from-literal=password=REDISPW


8. Create an S3 secret

[usera@local-workstation ~]$ oc create secret generic \
S3-credentials \
--from-literal=accessKey=ANKIYOURACCESSKEYHEREVES2W \
--from-literal=secretKey=A1IqkYourSecretKeyHere/%bksiujrwgQZrc


9. Create the Quay Ecosystem yaml file:

[usera@local-workstation ~]$ vi quay-ecosystem.yaml
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
  name: quay-ecosystem
spec:
  clair:
    enabled: true
    imagePullSecretName: redhat-pull-secret
    updateInterval: "60m"
  quay:
    superusers:
      - quay-admin
      - quay-config
    externalAccess:
      hostname: registry.apps.ocp-quay.your.aws.rt53.dns.domain.com
    registryBackends:
      - name: s3
        s3:
             accessKey: ANKIYOURACCESSKEYHEREVES2W
          bucketName: your-s3-bucket
          secretKey: A1IqkYourSecretKeyHere/%bksiujrwgQZrc
    imagePullSecretName: redhat-pull-secret
    superuserCredentialsSecretName: quay-admin
    configSecretName: quay-config-app
    deploymentStrategy: RollingUpdate
    skipSetup: false
    redis:
      credentialsSecretName: redis-password
    database:
      volumeSize: 10Gi
      credentialsSecretName: quay-database-credential
    registryStorage:
      persistentVolumeSize: 20Gi
      persistentVolumeAccessModes:
        - ReadWriteMany
    livenessProbe:
      initialDelaySeconds: 120
      httpGet:
        path: /health/instance
        port: 8443
        scheme: HTTPS
    readinessProbe:
      initialDelaySeconds: 10
      httpGet:
        path: /health/instance
        port: 8443
        scheme: HTTPS


10. Modify the file to fit the requirements for your environment. When done making the changes, apply the configuration to the Openshift cluster to start the Quay deployment:

[usera@local-workstation ~]$ oc apply -f quay-ecosystem.yaml


Verify the Quay deployment and create the first Quay repository


Return to the Openshift webUI


  1. In the list of options along the far left side of the page, click the arrow to expand “Home” and select “Projects”.

  2. Search for “quay” and select quay-enterprise.

  3. Scroll down the page until you can see the “Inventory” section on the left and verify you have at least 4 pods running.
  4. In the list of options along the far left side of the page, click the arrow to expand “Networking” and select “Routes”.

  5. Double click on the URL for the quay-ecosystem-quay route.

6. A new web page will open where you can log into the Quay webUI with the credentials from the Quay Superuser secret you created earlier. You may have to accept one or two security exceptions to access the Quay webUI due to the Openshift self signed certificate. 

Quay-admin

QUAYadminPW


7. In the Quay webUI, click on the plus sign on the left side of the page next to “Create New Repository”.

8. Name your new repository, “thanos” and click the button to create it as a private repository.

9. Click the icon along the left side of the page that looks like a diagonal price tag.

The page that opens will show the image based on it’s tag when we push it from the command line session.



Push an image to the Quay Repository.


Return to the command line session.

 

  1. Login to registry.redhat.io with your Red Hat account so you can pull down an existing image:

[usera@local-workstation ~]$ podman login --tls-verify=false \
registry.redhat.io


2. Login to your new Quay registry:

[usera@local-workstation ~]$ podman login --tls-verify=false \
Registry.apps.ocp-quay.your.aws.rt53.dns.domain.com 
Username: quay-admin
Password: QUAYadminPW


3. Pull an older image that may have security issues.

[usera@local-workstation ~]$ podman pull \
registry.redhat.io/rhacm2/thanos-rhel7:v2.1.0-5


4. List the images in your local registry:

[usera@local-workstation ~]$ podman image ls


5. Apply a tag to the image so it can be pushed to the new Quay registry:

[usera@local-workstation ~]$ podman tag \
registry.redhat.io/rhacm2/thanos-rhel7:v2.1.0-5 \
registry.apps.ocp-quay.your.aws.rt53.dns.domain.com/quay-admin/thanos:1.0


6. List the images in your local registry. 

[usera@local-workstation ~]$ podman image ls




7. Push the newly tagged image to the Quay repository:

[usera@local-workstation ~]$ podman push --tls-verify=false \
docker://registry.apps.ocp-quay.your.aws.rt53.dns.domain.com/quay-admin/thanos:1.0



Verify the image was successfully pushed to the Quay registry and was scanned by Clair.

Return to the Quay webUi and refresh the page to see the image and the results of the Clair security scan. 

Conclusion

You have completed part 3 of this series and have successfully deployed Quay and Clair to the Openshift cluster you created on AWS in part 2. With the completion of part 3, you have learned the skills to deploy Quay and Clair in the Openshift cluster running on AWS public cloud but that is just the beginning. You now have the basic knowledge to deploy more operators and expand the uses of Openshift for your organization. I hope you found this series to be informative.

About the Author

Eric Archer is a Senior Red Hat Consultant for Stone Door Group, a Hybrid Cloud and DevOps consulting company that helps enterprises successfully complete digital transformation projects. Stone Door Group offers rapid adoption of Red Hat Hybrid Cloud technologies with their OpenShift Container Platform Accelerator. To speak to Eric, drop us a line at letsdothis@stonedoorgroup.com

About Stone Door Group

Stone Door Group is a Hybrid Cloud and DevOps consulting company that delivers successful digital transformation projects in the private and public sectors. Stone Door Group is a team of leading experts in Hybrid Cloud and DevOps technologies. To speak with Eric and our team, send us an email at letsdothis@stonedoorgroup.com