Kubernetes setup

Prerequisites

1. Download the latest version of the Snowflake Profiler from the Delphix download page.

2. Ensure that the host running the profiler has Helm, kubectl and Docker installed. Also ensure it is set up to communicate to the Kubernetes cluster using either MicroK8s or Amazon’s EKS.

3. Set up the Hyperscale Snowflake Connector and add the required ConnectorInfo details.

4. For accessing Snowflake warehouse, we will require a service account user with key pair authentication for enhanced authentication security as an alternative to basic authentication.

   1. To configure key-pair authentication for a snowflake service account, follow the steps provided in the Snowflake’s configuring key-pair authentication.

   2.  Use the following commands to generate base64 encoded value of encrypted private key and passphrase.
      To generate base64 encoded value of encrypted private key:

      CODE

      echo -n `cat /home/delphix/keys/MKK_TEST_SFHSC_ADMIN_key.p8 | base64`

      To generate base64 encode value of passphrase:

      CODE

      echo -n <passphrase> | base64 -w 0

5. Similarly, the profiler needs the Controller API Key to authenticate to the controller which also needs to be provided in base64 encoded format. To generate base64 encode value of Controller API key:

   CODE

   echo –n “<controller_api_key>” | base64 –w 0

Procedure

1. Untar the profiler downloaded from Delphix download page. It should contain the Docker images for the profiler and the snowflake-profiler-helm.tar.

   CODE

   tar -xf snowflake-profler.tar.gz

2. Load the delphix-snowflake-profiler.tar Docker image.

   CODE

   cd snowflake-profiler
   docker load --input delphix-snowflake-profiler.tar

3. Push the image to a registry which can be accessed by the Kubernetes cluster.

   CODE

   
   docker tag delphix-snowflake-profiler-service-app:1.0.0 image.registry.com/delphix-snowflake-profiler-service-app:1.0.0
   docker push image.registry.com/delphix-snowflake-profiler-service-app:1.0.0
   (‘docker login <registry>' may be required)

4. Untar the helm repository and change the directory to snowflake-profiler-helm

   CODE

   tar -xf snowflake-profiler-helm.tar.gz
   cd snowflake-profiler-helm

5. Edit the values.yaml file

   1. Configure the controllerURL with the controller URL, IP or hostname.

      CODE

      controllerURL: controller.delphix.com

   2. If required, configure the registry credentials.

      CODE

      imageCredentials:
        username: <registry-username>
        password: <registry-password>
        email: <registry-user-email>

   3.  Configure the image details.

      CODE

      image:
        repository: image.registry.com/delphix-snowflake-profiler-service-app
        tag: 1.0.0
        pullPolicy: Always

   4. Configuring the credentials required to connect to Snowflake instance, similar to how it is configured for the Hyperscale Snowflake connector. If the Kubernetes secret is already configured in the same namespace (.Values.namespace), you can use the same.

      CODE

      secret:
        storedSnowflakeSecretName: stored-secret-name

      Or you can configure the required values for snowflakePrivateKey, snowflakePassphrase and controllerApiKey

      CODE

      secret:
        snowflakePrivateKey: user_encrypted_private_key_base64_encoded
        snowflakePassphrase: passhrase_base64_encoded
        controllerApiKey: controller_api_key_base64_encoded

   5. Optionally, see additional configurations for further details.

6. Start the snowflake profiler service.

   CODE

   
   helm install snowflake-profiler-helm <path_to_snowflake_profiler_helm_chart> -f values.yaml

7. Create the Kubernetes Ingress to allow connection to the profiler.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hyperscale-snowflake-profiler-ingress
  namespace: hyperscale-snowflake-profiler-service
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
spec:
  ingressClassName: nginx
  rules:
http:
        paths:
path: /profiler
          pathType: Prefix
          backend:
            service:
              name: snowflake-profiler-k8s-service
              port:
                 number: 8080

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hyperscale-snowflake-profiler-ingress
  namespace: hyperscale-snowflake-profiler-service
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":8080}]'
    alb.ingress.kubernetes.io/backend-protocol: HTTP
spec:
  rules:
http:
       paths:
path: /profiler
          pathType: Prefix
          backend:
            service:
              name: snowflake-profiler-k8s-service
              port:
                 number: 8080

8. Apply the Ingress configuration:

   CODE

   kubectl apply –f ingress.yaml

9. Access the profiler Swagger UI at http://<host-ip>/profiler