Google Container Registry (GCR) with Minikube or K8s
Category : Kubernetes
When you use Google Container Registry (GCR) and seeing the dreaded ImagePullBackoff status on your pods in minikube/K8s Then this article can help you to solve that error.
Error :
(base) saurabhkumar@Saurabhs-MacBook-Pro ~ % kubectl describe pod airflow-postgres-694899d6fd-lqp2c -n airflow
Events:
Type Reason Age From Message
—- —— —- —- ——-
Normal Scheduled 56s default-scheduler Successfully assigned airflow/airflow-postgres-694899d6fd-lqp2c to minikube
Warning Failed 29s (x2 over 48s) kubelet Failed to pull image “gcr.io/<gcp_project>/postgres-airflow”: rpc error: code = Unknown desc = Error response from daemon: unauthorized: You don’t have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
Warning Failed 29s (x2 over 48s) kubelet Error: ErrImagePull
Normal BackOff 13s (x2 over 48s) kubelet Back-off pulling image “gcr.io/<gcp_project>/postgres-airflow”
Warning Failed 13s (x2 over 48s) kubelet Error: ImagePullBackOff
Normal Pulling 1s (x3 over 55s) kubelet Pulling image “gcr.io/<gcp_project>/postgres-airflow”
Solution :
************************ this is to create secrate for image pull *****************************************
Stpe 1: Create the Kubernetes secret using the kubectl create secret docker-registry command. The docker-registry secret subcommand makes it easy to create secrets that work with ImagePullSecrets
kubectl -n airflow create secret docker-registry spark-pullimage –docker-server=https://gcr.io –docker-username=_json_key –docker-email=airflow-gcsfuse@<gcp_project>.iam.gserviceaccount.com –docker-password=”$(cat /Users/saurabhkumar/Downloads/<gcp_project>-edae8efc166c.json)”
Stpe 2:Now update the default service account for the namespace with ImagePullSecrets
kubectl –namespace=airflow patch serviceaccount default -p ‘{“imagePullSecrets”: [{“name”: “spark-pullimage”}]}
Stpe 3: Deploy your pod (.yaml) file
(base) saurabhkumar@Saurabhs-MacBook-Pro ~ % kubectl get pods -n airflow
No resources found in airflow namespace.
(base) saurabhkumar@Saurabhs-MacBook-Pro ~ % kubectl –namespace=airflow apply -f /Users/saurabhkumar/Downloads/postgres.yaml
service/airflow-postgres-svc created
deployment.apps/airflow-postgres created
(base) saurabhkumar@Saurabhs-MacBook-Pro ~ % kubectl get pods -n airflow
NAME READY STATUS RESTARTS AGE
airflow-postgres-6d85bf7599-s6bdv 0/1 ContainerCreating 0 3s
(base) saurabhkumar@Saurabhs-MacBook-Pro ~ % kubectl get pods -n airflow
NAME READY STATUS RESTARTS AGE
airflow-postgres-6d85bf7599-s6bdv 1/1 Running 0 72s
Stpe 4: Login to Pod and test it.
(base) saurabhkumar@Saurabhs-MacBook-Pro ~ % kubectl -n airflow exec -ti airflow-postgres-6d85bf7599-s6bdv /bin/sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] — [COMMAND] instead.
# psql -U postgres
psql (13.2 (Debian 13.2-1.pgdg100+1))
Type “help” for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
———–+———-+———-+————+————+———————–
airflow | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =Tc/postgres +
| | | | | postgres=CTc/postgres+
| | | | | airflow=CTc/postgres
celery | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =Tc/postgres +
| | | | | postgres=CTc/postgres+
| | | | | celery=CTc/postgres
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(5 rows)
postgres=# \c airflow
You are now connected to database “airflow” as user “postgres”.
airflow=# \dt
Did not find any relations.
airflow=# \q
# exit
***************************************************************************************************************************