Arango Pod Uses Incorrect Image Hash
Problem
A cluster health-check returns a RED status with the following arango pod error:
root@node1:~# health-check
Health status checking...
< output snipped>
======================================================
Overall cluster status
======================================================
RED
======================================================
Please review the statuses below and check for any failures
======================================================
======================================================
Check Kubernetes PODS status
======================================================
NOT OK
Errors found in pods.
arango foghorn-agnt-tpac7woa-176b40 0/1 Init:ImagePullBackOff 0 164m
arango foghorn-prmr-y6jyitgj-176b40 0/1 Init:ImagePullBackOff 0 164m
routingbotdb routingbot-arango-agnt-9g8nwqir-5cfb29 0/1 Init:ImagePullBackOff 0 123m
routingbotdb routingbot-arango-prmr-ughdun2a-5cfb29 0/1 Init:ImagePullBackOff 0 163m
< output snipped >
======================================================
Check Kubernetes Containers status
======================================================
NOT OK
NAMESPACE POD NAME CONTAINER NAME STATE
arango foghorn-agnt-tpac7woa-176b40 init-lifecycle null
arango foghorn-agnt-tpac7woa-176b40 uuid null
arango foghorn-agnt-tpac7woa-176b40 version-check null
arango foghorn-agnt-tpac7woa-176b40 server UNKNOWN
arango foghorn-prmr-y6jyitgj-176b40 init-lifecycle null
arango foghorn-prmr-y6jyitgj-176b40 uuid null
arango foghorn-prmr-y6jyitgj-176b40 version-check null
arango foghorn-prmr-y6jyitgj-176b40 server UNKNOWN
routingbotdb routingbot-arango-agnt-9g8nwqir-5cfb29 init-lifecycle null
routingbotdb routingbot-arango-agnt-9g8nwqir-5cfb29 uuid null
< output snipped >
2025-07-15 08:40:54 Health status checking completed!
root@node1:~# Cause
Sometimes after a Kubernetes failure, the arango-operator assigns an incorrect wrong image tag or wrong image hash to the arango pods.
Use the describe command to determine the image hash that does not
exist in the registry.
root@node1:~# kubectl describe pod -n arango foghorn-agnt-tpac7woa-176b40 ... Normal BackOff 9s (x44 over 10m) kubelet Back-off pulling image "paragon-registry.local/arangodb/arangodb@sha256:af579e4bdfcd985cf7c8f78edc4d2017ac4c84285b90be51eaf14b68c899870a"
Verify that the registry does not have that particular image hash. The following output displays the image hash that exists in the registry.
root@node1:~# paragon-get-image-tag paragon-registry.local/arangodb/arangodb 3.11.8 sha256:91d8c51c92767b6c5e36f753e62ae00eadfbe1ba0dd1c6ccbf41356f3756e67e
Solution
Perform the following steps:
Restart the arango-operator.
root@node1:~# kubectl rollout restart deploy -n arango arango-arango-operator
Restart the pods that are stuck.
root@node1:~# kubectl rollout restart deploy -n routingbotdb arango-routingbot-arango-operator-operator