Install Che on Amazon Elastic Kubernetes Service

To provide cloud development environments on AWS, deploy Che on an Amazon EKS cluster by configuring DNS, TLS certificates, and Keycloak as the OIDC identity provider.

Prerequisites

Configuring environment variables for Amazon EKS

Follow these instructions to define environment variables and update your kubeconfig to connect to Amazon EKS.

Prerequisites
Procedure

  1. Find the AWS account ID:

    AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)

  2. Define the cluster name:

    CHE_EKS_CLUSTER_NAME=che

  3. Define the region:

    CHE_EKS_CLUSTER_REGION=eu-central-1

  4. Update kubeconfig:

    aws eks update-kubeconfig --region $CHE_EKS_CLUSTER_REGION --name $CHE_EKS_CLUSTER_NAME

  5. Make sure that you have the default storage class set:

    kubectl get storageclass

    The output should display a storage class with default next to its name:

    NAME            PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
gp2 (default)   kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  126m
Additional resources

Installing Ingress-Nginx Controller on Amazon EKS

Follow these instructions to install the Ingress-Nginx Controller on Amazon EKS.

Procedure

  1. Install the Ingress-Nginx Controller using Helm:

    helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

helm install ingress-nginx ingress-nginx/ingress-nginx \
  --wait \
  --create-namespace \
  --namespace ingress-nginx \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-backend-protocol"=tcp \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-cross-zone-load-balancing-enabled"="true" \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-type"=nlb

  2. Verify that you can access the load balancer externally. It may take a few minutes for the load balancer to be created:

    until curl $(kubectl get service -n ingress-nginx ingress-nginx-controller -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}'); do sleep 5s; done

    You should receive the output similar to:

    <html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
Additional resources

Configuring DNS on Amazon EKS

Follow these instructions to configure DNS on Amazon EKS.

Prerequisites
Procedure

  1. Define the registered domain name:

    CHE_DOMAIN_NAME=eclipse-che-eks-clould.click

  2. Define domain name for Keycloak OIDC provider:

    KEYCLOAK_DOMAIN_NAME=keycloak.$CHE_DOMAIN_NAME

  3. Find out the hosted zone ID for the domain:

    HOSTED_ZONE_ID=$(aws route53 list-hosted-zones-by-name --dns-name $CHE_DOMAIN_NAME --query "HostedZones[0].Id" --output text)

  4. Find out the Canonical Hosted Zone ID for the load balancer:

    CANONICAL_HOSTED_ZONE_ID=$(aws elbv2 describe-load-balancers --query "LoadBalancers[0].CanonicalHostedZoneId" --output text)

  5. Find out the DNS name for the load balancer:

    DNS_NAME=$(kubectl get service -n ingress-nginx ingress-nginx-controller -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

  6. Create a DNS record set:

    aws route53 change-resource-record-sets \
  --hosted-zone-id $HOSTED_ZONE_ID \
  --change-batch '
  {
    "Comment": "Ceating a record set",
     "Changes": [{
      "Action"              : "CREATE",
       "ResourceRecordSet"  : {
         "Name"              : "'"$CHE_DOMAIN_NAME"'",
         "Type"             : "A",
         "AliasTarget"      : {
            "HostedZoneId"            : "'"$CANONICAL_HOSTED_ZONE_ID"'",
            "DNSName"                 : "'"$DNS_NAME"'",
            "EvaluateTargetHealth"    : false
        }
      }
    }]
  }
  '

  7. Verify that you can access Che domain externally:

    until curl $CHE_DOMAIN_NAME; do sleep 5s; done

  8. Create a DNS record set:

    aws route53 change-resource-record-sets \
  --hosted-zone-id $HOSTED_ZONE_ID \
  --change-batch '
  {
    "Comment": "Ceating a record set",
     "Changes": [{
      "Action"              : "CREATE",
       "ResourceRecordSet"  : {
         "Name"              : "'"$KEYCLOAK_DOMAIN_NAME"'",
         "Type"             : "A",
         "AliasTarget"      : {
            "HostedZoneId"            : "'"$CANONICAL_HOSTED_ZONE_ID"'",
            "DNSName"                 : "'"$DNS_NAME"'",
            "EvaluateTargetHealth"    : false
        }
      }
    }]
  }
  '

  9. Verify that you can access the Keycloak domain externally:

    until curl $KEYCLOAK_DOMAIN_NAME; do sleep 5s; done

Installing cert-manager on Amazon EKS

Follow these instructions to install the cert-manager on Amazon EKS.

Procedure

  1. Install cert-manager using Helm:

    helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install cert-manager jetstack/cert-manager \
  --wait \
  --create-namespace \
  --namespace cert-manager \
  --set crds.enabled=true
Additional resources

Creating Let’s Encrypt certificate for Che on Amazon EKS

Follow these instructions to create a Let’s Encrypt certificate for Che on Amazon EKS.

Procedure

  1. Create an IAM OIDC provider:

    eksctl utils associate-iam-oidc-provider --cluster $CHE_EKS_CLUSTER_NAME --approve

  2. Create a service principal:

    aws iam create-policy \
     --policy-name cert-manager-acme-dns01-route53 \
     --description "This policy allows cert-manager to manage ACME DNS01 records in Route53 hosted zones. See https://cert-manager.io/docs/configuration/acme/dns01/route53" \
     --policy-document file:///dev/stdin <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "route53:GetChange",
      "Resource": "arn:aws:route53:::change/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "route53:ChangeResourceRecordSets",
        "route53:ListResourceRecordSets"
      ],
      "Resource": "arn:aws:route53:::hostedzone/*"
    },
    {
      "Effect": "Allow",
      "Action": "route53:ListHostedZonesByName",
      "Resource": "*"
    }
  ]
}
EOF

  3. Create an IAM role and associate it with a Kubernetes Service Account:

    eksctl create iamserviceaccount \
  --name cert-manager-acme-dns01-route53 \
  --namespace cert-manager \
  --cluster $CHE_EKS_CLUSTER_NAME \
  --role-name cert-manager-acme-dns01-route53 \
  --attach-policy-arn arn:aws:iam::$AWS_ACCOUNT_ID:policy/cert-manager-acme-dns01-route53 \
  --approve

  4. Grant permission for cert-manager to create Service Account tokens:

    kubectl apply -f - << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cert-manager-acme-dns01-route53-tokenrequest
  namespace: cert-manager
rules:
  - apiGroups: ['']
    resources: ['serviceaccounts/token']
    resourceNames: ['cert-manager-acme-dns01-route53']
    verbs: ['create']
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cert-manager-acme-dns01-route53-tokenrequest
  namespace: cert-manager
subjects:
  - kind: ServiceAccount
    name: cert-manager
    namespace: cert-manager
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cert-manager-acme-dns01-route53-tokenrequest
EOF

  5. Create the Issuer:

    kubectl apply -f - << EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: che-letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: <email_address> (1)
    privateKeySecretRef:
      name: che-letsencrypt-production
    solvers:
    - dns01:
        route53:
          region: $CHE_EKS_CLUSTER_REGION
          role: arn:aws:iam::${AWS_ACCOUNT_ID}:role/cert-manager-acme-dns01-route53
          auth:
            kubernetes:
              serviceAccountRef:
                name: cert-manager-acme-dns01-route53
EOF
    1 Replace <email_address> with your email address.

  6. Create the eclipse-che namespace:

    kubectl create namespace eclipse-che

  7. Create the Certificate:

    kubectl apply -f - << EOF
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: che-tls
  namespace: eclipse-che
spec:
  secretName: che-tls
  issuerRef:
    name: che-letsencrypt
    kind: ClusterIssuer
  commonName: '$CHE_DOMAIN_NAME'
  dnsNames:
  - '$CHE_DOMAIN_NAME'
  - '*.$CHE_DOMAIN_NAME'
  usages:
    - server auth
    - digital signature
    - key encipherment
    - key agreement
    - data encipherment
EOF

  8. Wait for the che-tls secret to be created:

    until kubectl get secret -n eclipse-che che-tls; do sleep 5s; done
Additional resources

Installing Keycloak on Amazon Elastic Kubernetes Service

Follow these instructions to install Keycloak as the OpenID Connect (OIDC) provider.

Procedure

  1. Install Keycloak:

    While this guide provides a development configuration for deploying Keycloak on {kubernetes}, remember that production environments might require different settings, such as external database configuration.
    		 
    kubectl apply -f - <<EOF
---
apiVersion: v1
kind: Namespace
metadata:
  name: keycloak
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: keycloak
  namespace: keycloak
  labels:
    app: keycloak
spec:
  secretName: keycloak.tls
  issuerRef:
    name: che-letsencrypt
    kind: ClusterIssuer
  commonName: '$KEYCLOAK_DOMAIN_NAME'
  dnsNames:
  - '$KEYCLOAK_DOMAIN_NAME'
  usages:
    - server auth
    - digital signature
    - key encipherment
    - key agreement
    - data encipherment
---
apiVersion: v1
kind: Service
metadata:
  name: keycloak
  namespace: keycloak
  labels:
    app: keycloak
spec:
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  selector:
    app: keycloak
  type: LoadBalancer
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: keycloak
  namespace: keycloak
  labels:
    app: keycloak
spec:
  replicas: 1
  selector:
    matchLabels:
      app: keycloak
  template:
    metadata:
      labels:
        app: keycloak
    spec:
      containers:
      - name: keycloak
        image: quay.io/keycloak/keycloak:18.0.2
        args: ["start-dev"]
        env:
        - name: KEYCLOAK_ADMIN
          value: "admin"
        - name: KEYCLOAK_ADMIN_PASSWORD
          value: "admin"
        - name: KC_PROXY
          value: "edge"
        ports:
        - name: http
          containerPort: 8080
        readinessProbe:
          httpGet:
            path: /realms/master
            port: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keycloak
  namespace: keycloak
  annotations:
    nginx.ingress.kubernetes.io/proxy-connect-timeout: '3600'
    nginx.ingress.kubernetes.io/proxy-read-timeout: '3600'
    nginx.ingress.kubernetes.io/ssl-redirect: 'true'
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - $KEYCLOAK_DOMAIN_NAME
      secretName: keycloak.tls
  rules:
  - host: $KEYCLOAK_DOMAIN_NAME
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: keycloak
            port:
              number: 8080
EOF

  2. Wait until the Keycloak pod is ready:

    kubectl wait --for=condition=ready pod -l app=keycloak -n keycloak --timeout=120s

  3. Wait for the keycloak.tls secret to be created:

    until kubectl get secret -n keycloak keycloak.tls; do sleep 5s; done

  4. Configure Keycloak to create the realm, client, and user:

    kubectl exec deploy/keycloak -n keycloak -- bash -c \
    "/opt/keycloak/bin/kcadm.sh config credentials \
        --server http://localhost:8080 \
        --realm master \
        --user admin  \
        --password admin && \
    /opt/keycloak/bin/kcadm.sh create realms \
        -s realm='che' \
        -s displayName='che' \
        -s enabled=true \
        -s registrationAllowed=false \
        -s resetPasswordAllowed=true && \
    /opt/keycloak/bin/kcadm.sh create clients \
        -r 'che' \
        -s clientId=k8s-client \
        -s id=k8s-client \
        -s redirectUris='[\"*\"]' \
        -s directAccessGrantsEnabled=true \
        -s secret=eclipse-che && \
    /opt/keycloak/bin/kcadm.sh create users \
        -r 'che' \
        -s username=test \
        -s email=\"test@test.com\" \
        -s enabled=true \
        -s emailVerified=true &&  \
    /opt/keycloak/bin/kcadm.sh set-password \
        -r 'che' \
        --username test \
        --new-password test"
Additional resources

Associate keycloak as OIDC identity provider on Amazon EKS

Follow these instructions to associate Keycloak an OIDC identity provider on Amazon EKS.

Procedure

  1. Associate Keycloak an identity provider using eksctl:

    eksctl associate identityprovider \
  --wait \
  --config-file - << EOF
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: $CHE_EKS_CLUSTER_NAME
  region: $CHE_EKS_CLUSTER_REGION
identityProviders:
  - name: keycloak-oidc
    type: oidc
    issuerUrl: https://$KEYCLOAK_DOMAIN_NAME/realms/che
    clientId: k8s-client
    usernameClaim: email
EOF
Additional resources

Installing Che on Amazon EKS

Follow these instructions to install Che on Amazon EKS.

Procedure

  1. Prepare a CheCluster patch YAML file:

    cat > che-cluster-patch.yaml << EOF
spec:
  networking:
    auth:
      oAuthClientName: k8s-client
      oAuthSecret: eclipse-che
      identityProviderURL: "https://$KEYCLOAK_DOMAIN_NAME/realms/che"
      gateway:
        oAuthProxy:
          cookieExpireSeconds: 300
        deployment:
          containers:
          - env:
             - name: OAUTH2_PROXY_BACKEND_LOGOUT_URL
               value: "http://$KEYCLOAK_DOMAIN_NAME/realms/che/protocol/openid-connect/logout?id_token_hint={id_token}"
            name: oauth-proxy
  components:
    cheServer:
      extraProperties:
        CHE_OIDC_USERNAME__CLAIM: email
EOF

  2. Deploy Che:

    chectl server:deploy \
    --platform k8s \
    --domain $CHE_DOMAIN_NAME \
    --che-operator-cr-patch-yaml che-cluster-patch.yaml \
    --skip-cert-manager \
    --k8spodreadytimeout 240000 \
    --k8spoddownloadimagetimeout 240000

  3. Navigate to the Che cluster instance:

    chectl dashboard:open