Category Archives: Azure

AKS Private Cluster, kubenet, UDR, and Azure Firewall

This configuration will be covered the following requirement:

  • AKS Private Cluster with kubenet
  • Bring your own subnet with route table
  • limit egress with Azure Firewall
  • Access to ACR via public endpoint
  • Todo:
    • Access to ACR via private endpoint
    • Access to Azure Database for PostgreSQL via private endpoint
    • Monitoring via private link
    • Traffic flow query
    • Adding Azure Front Door/Application Gateway for public access


Creating AKS Private Cluster with Azure CLI

Sample output

a@myVM001:~$ ip address | grep "inet 10.42"
    inet metric 100 brd scope global eth0
a@myVM001:~$ kubectl get pods,svc
NAME                                   READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-676579fdc-68b6d   1/1     Running   0          6m30s
pod/nginx-deployment-676579fdc-7kdwg   1/1     Running   0          6m30s
pod/nginx-deployment-676579fdc-q7q9f   1/1     Running   0          6m30s

NAME                 TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/fuju-nginx   LoadBalancer     80:31373/TCP   6m30s
service/kubernetes   ClusterIP       <none>        443/TCP        63m
a@myVM001:~$ curl -s | grep title
<title>Welcome to nginx!</title>

Related Error Messages:

The following is “Error messages” when AKS Node unable to access ACR

a@myVM001:~$ az aks check-acr --resource-group $RG --name $AKSNAME --acr $
Merged "aks-egress" as current context in /tmp/tmpaacrqofd
WARNING: version difference between client (1.26) and server (1.23) exceeds the supported minor version skew of +/-1
[2022-12-11T09:27:52Z] Checking host name resolution ($ SUCCEEDED
[2022-12-11T09:27:52Z] Canonical name for ACR ($
[2022-12-11T09:27:52Z] ACR location: eastasia
[2022-12-11T09:27:52Z] Checking managed identity...
[2022-12-11T09:27:52Z] Kubelet managed identity client ID: 000-0000-0000-0000-0000c
[2022-12-11T09:27:53Z] Validating managed identity existance: SUCCEEDED
[2022-12-11T09:27:53Z] Validating image pull permission: FAILED
[2022-12-11T09:27:53Z] ACR rejected token exchange: failed to send token exchange request: Post "https://$": EOF

The following is “Error message” when creating AKS cluster but $IDENTITY_ID doesn’t have permission to create entry in route table (UDR) – Additional information for built-in-roles (Network Contributor) can be found @ And ” Microsoft.Network/routeTables/*” provider operations can be found details under

 (CustomRouteTableMissingPermission) Managed identity or service principle must be given permission to read and write to custom route table /subscriptions/0000-0000-0000-0000-300a/resourceGroups/101-aks-egress-rg/providers/Microsoft.Network/routeTables/aks-egress-fwrt. Please see for more information

Sample Route Table after adding AKS ( Kubenet ) in VNET

Sample Route Table

See Also:

AKS – Private Cluster and PostgreSQL

Sample CLI:
* Creating ” AKS – Private Cluster with v-net peering ”

* Creating AKS with ” Bring your own subnet and route table with kubenet”

AKS  - Private Cluster - Bring your own subnet and route table with kubenet
AKS – Private Cluster – Bring your own subnet and route table with kubenet

See Also

Azure Application Insights customEvents and .NET 6

Getting customEvents with Azure Application Insights customEvents and .NET 6

dotnet new mvc -o videowebapp
cd videowebapp
dotnet add package Microsoft.ApplicationInsights.AspNetCore

Program.cs: Adding builder.Services.AddApplicationInsightsTelemetry();

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
IServiceCollection serviceCollection = builder.Services.AddApplicationInsightsTelemetry();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
    // The default HSTS value is 30 days. You may want to change this for production scenarios, see




    name: "default",
    pattern: "{controller=Home}/{action=Index}/{id?}");


Getting InstrumentationKey

az resource show \
    --resource-group <resource_group_name> \
    --name <resource_name> \
    --resource-type "Microsoft.Insights/components" \
    --query properties.InstrumentationKey

appsettings.json : adding InstrumentationKey

  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
  "AllowedHosts": "*",
  "ApplicationInsights": {
    "InstrumentationKey": "0000000000000000000000000000000",
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Error"

HomeController.cs : Adding ” this.aiClient.TrackEvent(“CommentSubmitted”); ”

using System.Diagnostics;
using Microsoft.AspNetCore.Mvc;
using videowebapp.Models;
using Microsoft.ApplicationInsights;

namespace videowebapp.Controllers;

public class HomeController : Controller
    private TelemetryClient aiClient;
 //   private readonly ILogger<HomeController> _logger;

public HomeController(TelemetryClient aiClient)
    this.aiClient = aiClient;
/*    public HomeController(ILogger<HomeController> logger)
        _logger = logger;
    public IActionResult Index()
       // _logger.LogWarning("fujuTE: An example of a Warning trace..");
       // _logger.LogError("fujuTE: An example of an Error level message");
        return View();

    public IActionResult Privacy()
        // _logger.LogInformation("fujuTE: An example of a Information trace..");
        // Track an event

        // Track an event with properties
        this.aiClient.TrackEvent("VideoUploaded", new Dictionary<string, string> {{"Category", "Sports"}, {"Format", "mp4"}});

        return View();

    [ResponseCache(Duration = 0, Location = ResponseCacheLocation.None, NoStore = true)]
    public IActionResult Error()
        return View(new ErrorViewModel { RequestId = Activity.Current?.Id ?? HttpContext.TraceIdentifier });
Application Insight – Transaction Search
Getting customEvents vai Log Analytics Workspace

See Also:

Terraform: Create Virtual Network Peering and VMs

Sample “Virtual network peering” with 2 VMs with terraform deployment

TF file:


Building VNET peering with terraform

terraform plan -out main-vnet.tfplan
terraform apply "main-vnet.tfplan"

Sample screenshot to access fx1-vm1 in fx1-network1 via jump host fx1-vm2

ssh azureuser@

adminuser@fx1-vm2:~$ hostname
adminuser@fx1-vm2:~$ ssh azureuser@
azureuser@'s password:
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-1080-azure x86_64)

 * Documentation:
 * Management:
 * Support:

  System information as of Thu Jun  2 10:38:38 UTC 2022

  System load:  0.33              Processes:           131
  Usage of /:   6.5% of 28.90GB   Users logged in:     0
  Memory usage: 3%                IP address for eth0:
  Swap usage:   0%

0 updates can be applied immediately.

New release '20.04.4 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Thu Jun  2 10:37:25 2022 from
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.


delete the deployment with terraform destroy

terraform plan -destroy -out main-vnet.destroy.tfplan
terraform apply main-vnet.destroy.tfplan

Optional Tasks:

  • Adding Private Endpoint for Azure Blob Storage and Private DNS Zone

Reference command:

az vm image list-skus --location eastasia --offer WindowsServer --publisher MicrosoftWindowsServer
az vm image list-skus --location eastasia --offer UbuntuServer --publisher Canonical
az vm list-skus -l southeastasia  --resource-type virtualMachines  --output table | grep Standard_D2ds_v4

az account set --subscription "xxxxxxxxxxxxxxx"
az vm list-usage --location southeastasia -o table | grep -E -w -i  'DSv4|FSv2|ESv4'
declare -a subscrptionsID=(

echo "${subscrptionsID[@]}"

for mySubscrptionsID in "${subscrptionsID[@]}" 
   az account set --subscription  “$mySubscrptionsID”
   az vm list-usage --location southeastasia -o table | grep -E -w -i  'DSv4|FSv2|ESv4|DSv3'

Deploy an ASP.NET Core and Azure SQL Database app to Azure App Service with Bicep,CLI and VS Code

Deploy software infrastructure with Bicep ( more details can be found from

// based on
az group create --name $RESOURCE_GROUP_NAME=23ExampleRG --location "eastasia"
az deployment group create --resource-group $RESOURCE_GROUP_NAME=23ExampleRG --template-file 01-main-web-app-sqldb.bicep  --parameters sqlAdministratorLogin='azuresqladmin' sqlAdministratorLoginPassword='MYSECRTEPASSWORD' location='eastasia'

Deploy sample ASP.NET Core Application to App Service with VS code


git clone
cd msdocs-app-service-sqldb-dotnetcore
dotnet build .
dotnet publish -c Release
#Right-click on the generated publish folder in the Visual Studio Code explorer and #select Deploy to Web App.

Update database connection string and create database table with entity framework core


az sql db show-connection-string \
    --client \
    --name sampledb \
    --server $
az webapp config connection-string set \
    -t SQLServer \
    --settings MyDbConnection="Server=tcp:$,1433;Database=sampledb;User ID=azuresqladmin;Password='MYSECRETPASSWORD';Encrypt=true;Connection Timeout=30;"

# az sql server firewall-rule create --resource-group $RESOURCE_GROUP_NAME --server $ --name LocalAccess --start-ip-address <your-ip> --end-ip-address <your-ip>

# Change database connection string in "appsettings.json" 
#"ConnectionStrings": {
#    "MyDbConnection": "Server=tcp:$,1433;
#        Initial Catalog=sampledb;
#        Persist Security Info=False;
#        User ID=azuresqladmin;Password=MYSECRETPASSWORD;
#        Encrypt=True;
#        TrustServerCertificate=False;"
# }
# create database table in target database
dotnet tool install -g dotnet-ef
dotnet ef migrations add InitialCreat --project DotNetCoreSqlDb
dotnet ef database update --project DotNetCoreSqlDb

Access web app logs and delete resources group

az webapp log config \
    --web-server-logging 'filesystem' \
    --name $APP_SERVICE_NAME \
    --resource-group $RESOURCE_GROUP_NAME
az webapp log tail \
    --name $APP_SERVICE_NAME \
    --resource-group $RESOURCE_GROUP_NAME

az group delete --name $RESOURCE_GROUP_NAME

Azure App Service and Bicep

List of sample deployments of Azure App Service with various scenario by using Bicep

  • Azure App service with Blob
  • Azure App Service + SQLDB + App Insight
  • Azure App Service (Webapp + private endpoint (WebApp and SQL Server) + VNET integration.
  • Application Gateway (private and public listeners) -> Azure App Service (via PE) -> SQL Server (via PE)

Azure App service with Blob:

az group create --name $RESOURCE_GROUP--location westus3
az deployment group create --resource-group $RESOURCE_GROUP --template-file main.bicep --parameters environmentType=nonprod

az deployment group list --output table

Azure App Service + SQLDB + App Insight :

az group create --name $RESOURCE_GROUP --location westus3
az deployment group create --resource-group $RESOURCE_GROUP --template-file 01-main-web-app-sqldb.bicep  --parameters sqlAdministratorLogin='azuresqladmin' sqlAdministratorLoginPassword='PLEASECHANGEtoSECRETPASSWORD' location='westus3'

Azure App Service (Webapp + private endpoint (WebApp and SQL Server) + VNET integration:
Deploys two web apps (frontend and backend) and SQL Server securely connected together with VNet integration (webAppFrontend) and Private Endpoint(webAppBackend,SQLServer)
m18h/02-main-webapp-pe-vnet-integration.bicep at master · fujute/m18h (

az group create --name $RESOURCE_GROUP--location westus3
az deployment group create --resource-group $RESOURCE_GROUP--template-file 02-main-webapp-pe-vnet-integration.bicep \
--parameters virtualNetworkName='m6290cvnet' sqlAdministratorLoginPassword='PLEASECHANGETOSECUREPASSWORD'


Integrate your app with an Azure virtual network


picture: Regional virtual network integration

Application Gateway integration: Integration with App Service (multi-tenant)

Continue reading Azure App Service and Bicep

DevOps: Azure DevOps Release Gate (Query work items, SonarQube, Azure monitor)

A Sample of “Azure DevOps Release Gate ” the Gates with the following items

  • Query work items: Query Active Bug
  • SonarQube: ” Invoke REST API: POST”
  • Azure monitor: Azure monitor Alert

Software infra:

Based on ” Controlling Deployments using Release Gates | Azure DevOps Hands-on-Labs ( ” and ” Managing technical debt with SonarQube and Azure DevOps | Azure DevOps Hands-on-Labs (

  • 2 Web app for sample canary and production
  • sonarqube in Azure Container Instance

az group create -n $AprRG -l eastasia
az appservice plan create -g $AprRG -n MyPlan --sku S1
az webapp create -g $AprRG -p MyPlan -n "PartsUnlimited-$RNUMBER-Canary"
az webapp create -g $AprRG -p MyPlan -n "PartsUnlimited-$RNUMBER-Prod"

RG_ID=$(az group create --name $AprRG  --location "eastasia" --query "id" --output tsv)
PASSWORD=$(az ad sp create-for-rbac --name $SERVICE_PRINCIPAL_NAME --role contributor --scopes $RG_ID --query "password" --output tsv)
USER_NAME=$(az ad sp list --display-name $SERVICE_PRINCIPAL_NAME --query "[].appId" --output tsv)

az container create -g $AprRG --name sonarqubeaci180422 --image sonarqube --ports 9000 --dns-name-label mysonarqube200422 --cpu 2 --memory 3.5

#curl -u ea---fe:

Hint: Azure DevOps Release Gate with Azure DevOps Starter

To build quick demo for “Release Gate” with condition from Azure Board, Azure Monitor and SonarQube

  1. DevOps Starter ” .NET Core -> App Service ”
  2. Add SonarQube in “build” Pipeline
  3. Add Release ” UAT” Stage (Then, we have Dev & UAT)
  4. Add “Pre-deployment approvals”
  5. Add “Pre-deployment Gates” ” Query work items”
    Azure DevOps -> Boards -> Queries -> Active Bugs -> … -> Security -> ReleaseGate Build Service(myOrg) -> Read ( Allow )
  6. Add “monitoring” gate in “Dev” Stage
  7. Add Agentless Job in Tasks( manual Intervention )
  8. Add ” Post-deployment approvals” Gate “Query Azure Monitor Alerts”
  9. Add ” Post-deployment approvals” Gate “Invoke REST API:POST”

#URL suffix:

#success critirial:

Sample of Canary releases (ref: What are deployment patterns? – Learn | Microsoft Docs )

DevTest and DevOps for microservice solutions


See Also:

A user delegation SAS for blob with the Azure CLI & azcopy

Sample of using A user delegation SAS for blob with the Azure CLI and copy the blob file via azcopy with the SAS token

RG_ID=$(az group create --name $MY_RG  --location southeastasia --query "id" --output tsv)
PASSWORD=$(az ad sp create-for-rbac --name $SERVICE_PRINCIPAL_NAME --role "Storage Blob Data Contributor" --scopes $MY_SCOPES  --query "password" --output tsv)
USER_NAME=$(az ad sp list --display-name $SERVICE_PRINCIPAL_NAME --query "[].appId" --output tsv)

az login --service-principal -u $USER_NAME -p $PASSWORD --tenant <mytenant>

END=$(date -u -d "30 minutes" '+%Y-%m-%dT%H:%MZ')
SAS4BLOB=$(az storage blob generate-sas \
    --account-name m14storage \
    --container-name data0314 \
    --name "kblob-file-001.txt" \
    --permissions acdrw \
    --expiry $END \
    --auth-mode login \
    --as-user \
    --full-uri )
azcopy copy $SAS4BLOB  . 

See Also:


  • SP’s Password protection ?

Create Azure Service Principal and “az login”

Sample of creating Resource Group and then create Service Principle with contributor role under the RG for DevOps Pipeline

#az group create --name $MY_RG --location southeastasia
#RG_ID=$(az group show --name $MY_RG --query "id" --output tsv)
# or 
RG_ID=$(az group create --name $MY_RG  --location southeastasia --query "id" --output tsv)
PASSWORD=$(az ad sp create-for-rbac --name $SERVICE_PRINCIPAL_NAME --role contributor --scopes $RG_ID  --query "password" --output tsv)
USER_NAME=$(az ad sp list --display-name $SERVICE_PRINCIPAL_NAME --query "[].appId" --output tsv)

az login --service-principal -u $USER_NAME -p $PASSWORD --tenant <mytenant>

Connect to SQL Database from App Service by using a managed identity – System-assigned.

Running App Service with Azure SQL based

Fix list:
The steps in this guide has been tested with “.NET 6.0”

dotnet tool install -g dotnet-ef
dotnet ef migrations add InitialCreate
dotnet ef database update
dotnet add package Microsoft.Data.SqlClient --version 4.0.1
dotnet add package Azure.Identity --version 1.5.0

appsettings.json :

  "Logging": {
    "LogLevel": {
      "Default": "Information"
  "AllowedHosts": "*",
  "ConnectionStrings": {
    "CoredBConnection": ";Authentication=Active Directory Default; Database=CoredB;"
az webapp identity assign --resource-group my-002-rg --name webapp-core-sql-018
ALTER ROLE db_datareader ADD MEMBER [webapp-core-sql-018];
ALTER ROLE db_datawriter ADD MEMBER [webapp-core-sql-018];
ALTER ROLE db_ddladmin ADD MEMBER [webapp-core-sql-018];

Program.cs :

public void ConfigureServices(IServiceCollection services)
   services.AddDbContext<MyDatabaseContext>(options =>

See Also:

Web App & Azure SQL Templates: