Category Archives: Security

Security Magic

AKS Private Cluster, kubenet, UDR, and Azure Firewall

This configuration will be covered the following requirement:

  • AKS Private Cluster with kubenet
  • Bring your own subnet with route table
  • limit egress with Azure Firewall
  • Access to ACR via public endpoint
  • Todo:
    • Access to ACR via private endpoint
    • Access to Azure Database for PostgreSQL via private endpoint
    • Monitoring via private link
    • Traffic flow query
    • Adding Azure Front Door/Application Gateway for public access


Creating AKS Private Cluster with Azure CLI

Sample output

a@myVM001:~$ ip address | grep "inet 10.42"
    inet metric 100 brd scope global eth0
a@myVM001:~$ kubectl get pods,svc
NAME                                   READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-676579fdc-68b6d   1/1     Running   0          6m30s
pod/nginx-deployment-676579fdc-7kdwg   1/1     Running   0          6m30s
pod/nginx-deployment-676579fdc-q7q9f   1/1     Running   0          6m30s

NAME                 TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/fuju-nginx   LoadBalancer     80:31373/TCP   6m30s
service/kubernetes   ClusterIP       <none>        443/TCP        63m
a@myVM001:~$ curl -s | grep title
<title>Welcome to nginx!</title>

Related Error Messages:

The following is “Error messages” when AKS Node unable to access ACR

a@myVM001:~$ az aks check-acr --resource-group $RG --name $AKSNAME --acr $
Merged "aks-egress" as current context in /tmp/tmpaacrqofd
WARNING: version difference between client (1.26) and server (1.23) exceeds the supported minor version skew of +/-1
[2022-12-11T09:27:52Z] Checking host name resolution ($ SUCCEEDED
[2022-12-11T09:27:52Z] Canonical name for ACR ($
[2022-12-11T09:27:52Z] ACR location: eastasia
[2022-12-11T09:27:52Z] Checking managed identity...
[2022-12-11T09:27:52Z] Kubelet managed identity client ID: 000-0000-0000-0000-0000c
[2022-12-11T09:27:53Z] Validating managed identity existance: SUCCEEDED
[2022-12-11T09:27:53Z] Validating image pull permission: FAILED
[2022-12-11T09:27:53Z] ACR rejected token exchange: failed to send token exchange request: Post "https://$": EOF

The following is “Error message” when creating AKS cluster but $IDENTITY_ID doesn’t have permission to create entry in route table (UDR) – Additional information for built-in-roles (Network Contributor) can be found @ And ” Microsoft.Network/routeTables/*” provider operations can be found details under

 (CustomRouteTableMissingPermission) Managed identity or service principle must be given permission to read and write to custom route table /subscriptions/0000-0000-0000-0000-300a/resourceGroups/101-aks-egress-rg/providers/Microsoft.Network/routeTables/aks-egress-fwrt. Please see for more information

Sample Route Table after adding AKS ( Kubenet ) in VNET

Sample Route Table

See Also: