All posts by fujute

About fujute

Fukiat Julnual's role as Technical Evangelist is to support awareness for Microsoft Azure , engages with Technical Community and help IT Professionals to discover the technical value of innovations in Microsoft platform.

AKS Private Cluster, kubenet, UDR, and Azure Firewall

This configuration will be covered the following requirement:

  • AKS Private Cluster with kubenet
  • Bring your own subnet with route table
  • limit egress with Azure Firewall
  • Access to ACR via public endpoint
  • Todo:
    • Access to ACR via private endpoint
    • Access to Azure Database for PostgreSQL via private endpoint
    • Monitoring via private link
    • Traffic flow query
    • Adding Azure Front Door/Application Gateway for public access


Creating AKS Private Cluster with Azure CLI

Sample output

a@myVM001:~$ ip address | grep "inet 10.42"
    inet metric 100 brd scope global eth0
a@myVM001:~$ kubectl get pods,svc
NAME                                   READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-676579fdc-68b6d   1/1     Running   0          6m30s
pod/nginx-deployment-676579fdc-7kdwg   1/1     Running   0          6m30s
pod/nginx-deployment-676579fdc-q7q9f   1/1     Running   0          6m30s

NAME                 TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/fuju-nginx   LoadBalancer     80:31373/TCP   6m30s
service/kubernetes   ClusterIP       <none>        443/TCP        63m
a@myVM001:~$ curl -s | grep title
<title>Welcome to nginx!</title>

Related Error Messages:

The following is “Error messages” when AKS Node unable to access ACR

a@myVM001:~$ az aks check-acr --resource-group $RG --name $AKSNAME --acr $
Merged "aks-egress" as current context in /tmp/tmpaacrqofd
WARNING: version difference between client (1.26) and server (1.23) exceeds the supported minor version skew of +/-1
[2022-12-11T09:27:52Z] Checking host name resolution ($ SUCCEEDED
[2022-12-11T09:27:52Z] Canonical name for ACR ($
[2022-12-11T09:27:52Z] ACR location: eastasia
[2022-12-11T09:27:52Z] Checking managed identity...
[2022-12-11T09:27:52Z] Kubelet managed identity client ID: 000-0000-0000-0000-0000c
[2022-12-11T09:27:53Z] Validating managed identity existance: SUCCEEDED
[2022-12-11T09:27:53Z] Validating image pull permission: FAILED
[2022-12-11T09:27:53Z] ACR rejected token exchange: failed to send token exchange request: Post "https://$": EOF

The following is “Error message” when creating AKS cluster but $IDENTITY_ID doesn’t have permission to create entry in route table (UDR) – Additional information for built-in-roles (Network Contributor) can be found @ And ” Microsoft.Network/routeTables/*” provider operations can be found details under

 (CustomRouteTableMissingPermission) Managed identity or service principle must be given permission to read and write to custom route table /subscriptions/0000-0000-0000-0000-300a/resourceGroups/101-aks-egress-rg/providers/Microsoft.Network/routeTables/aks-egress-fwrt. Please see for more information

Sample Route Table after adding AKS ( Kubenet ) in VNET

Sample Route Table

See Also:

AKS – Private Cluster and PostgreSQL

Sample CLI:
* Creating ” AKS – Private Cluster with v-net peering ”

* Creating AKS with ” Bring your own subnet and route table with kubenet”

AKS  - Private Cluster - Bring your own subnet and route table with kubenet
AKS – Private Cluster – Bring your own subnet and route table with kubenet

See Also

Azure Application Insights customEvents and .NET 6

Getting customEvents with Azure Application Insights customEvents and .NET 6

dotnet new mvc -o videowebapp
cd videowebapp
dotnet add package Microsoft.ApplicationInsights.AspNetCore

Program.cs: Adding builder.Services.AddApplicationInsightsTelemetry();

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
IServiceCollection serviceCollection = builder.Services.AddApplicationInsightsTelemetry();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
    // The default HSTS value is 30 days. You may want to change this for production scenarios, see




    name: "default",
    pattern: "{controller=Home}/{action=Index}/{id?}");


Getting InstrumentationKey

az resource show \
    --resource-group <resource_group_name> \
    --name <resource_name> \
    --resource-type "Microsoft.Insights/components" \
    --query properties.InstrumentationKey

appsettings.json : adding InstrumentationKey

  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
  "AllowedHosts": "*",
  "ApplicationInsights": {
    "InstrumentationKey": "0000000000000000000000000000000",
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Error"

HomeController.cs : Adding ” this.aiClient.TrackEvent(“CommentSubmitted”); ”

using System.Diagnostics;
using Microsoft.AspNetCore.Mvc;
using videowebapp.Models;
using Microsoft.ApplicationInsights;

namespace videowebapp.Controllers;

public class HomeController : Controller
    private TelemetryClient aiClient;
 //   private readonly ILogger<HomeController> _logger;

public HomeController(TelemetryClient aiClient)
    this.aiClient = aiClient;
/*    public HomeController(ILogger<HomeController> logger)
        _logger = logger;
    public IActionResult Index()
       // _logger.LogWarning("fujuTE: An example of a Warning trace..");
       // _logger.LogError("fujuTE: An example of an Error level message");
        return View();

    public IActionResult Privacy()
        // _logger.LogInformation("fujuTE: An example of a Information trace..");
        // Track an event

        // Track an event with properties
        this.aiClient.TrackEvent("VideoUploaded", new Dictionary<string, string> {{"Category", "Sports"}, {"Format", "mp4"}});

        return View();

    [ResponseCache(Duration = 0, Location = ResponseCacheLocation.None, NoStore = true)]
    public IActionResult Error()
        return View(new ErrorViewModel { RequestId = Activity.Current?.Id ?? HttpContext.TraceIdentifier });
Application Insight – Transaction Search
Getting customEvents vai Log Analytics Workspace

See Also:

Terraform: Create Virtual Network Peering and VMs

Sample “Virtual network peering” with 2 VMs with terraform deployment

TF file:


Building VNET peering with terraform

terraform plan -out main-vnet.tfplan
terraform apply "main-vnet.tfplan"

Sample screenshot to access fx1-vm1 in fx1-network1 via jump host fx1-vm2

ssh azureuser@

adminuser@fx1-vm2:~$ hostname
adminuser@fx1-vm2:~$ ssh azureuser@
azureuser@'s password:
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-1080-azure x86_64)

 * Documentation:
 * Management:
 * Support:

  System information as of Thu Jun  2 10:38:38 UTC 2022

  System load:  0.33              Processes:           131
  Usage of /:   6.5% of 28.90GB   Users logged in:     0
  Memory usage: 3%                IP address for eth0:
  Swap usage:   0%

0 updates can be applied immediately.

New release '20.04.4 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Thu Jun  2 10:37:25 2022 from
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.


delete the deployment with terraform destroy

terraform plan -destroy -out main-vnet.destroy.tfplan
terraform apply main-vnet.destroy.tfplan

Optional Tasks:

  • Adding Private Endpoint for Azure Blob Storage and Private DNS Zone

Reference command:

az vm image list-skus --location eastasia --offer WindowsServer --publisher MicrosoftWindowsServer
az vm image list-skus --location eastasia --offer UbuntuServer --publisher Canonical
az vm list-skus -l southeastasia  --resource-type virtualMachines  --output table | grep Standard_D2ds_v4

az account set --subscription "xxxxxxxxxxxxxxx"
az vm list-usage --location southeastasia -o table | grep -E -w -i  'DSv4|FSv2|ESv4'
declare -a subscrptionsID=(

echo "${subscrptionsID[@]}"

for mySubscrptionsID in "${subscrptionsID[@]}" 
   az account set --subscription  “$mySubscrptionsID”
   az vm list-usage --location southeastasia -o table | grep -E -w -i  'DSv4|FSv2|ESv4|DSv3'

Deploy an ASP.NET Core and Azure SQL Database app to Azure App Service with Bicep,CLI and VS Code

Deploy software infrastructure with Bicep ( more details can be found from

// based on
az group create --name $RESOURCE_GROUP_NAME=23ExampleRG --location "eastasia"
az deployment group create --resource-group $RESOURCE_GROUP_NAME=23ExampleRG --template-file 01-main-web-app-sqldb.bicep  --parameters sqlAdministratorLogin='azuresqladmin' sqlAdministratorLoginPassword='MYSECRTEPASSWORD' location='eastasia'

Deploy sample ASP.NET Core Application to App Service with VS code


git clone
cd msdocs-app-service-sqldb-dotnetcore
dotnet build .
dotnet publish -c Release
#Right-click on the generated publish folder in the Visual Studio Code explorer and #select Deploy to Web App.

Update database connection string and create database table with entity framework core


az sql db show-connection-string \
    --client \
    --name sampledb \
    --server $
az webapp config connection-string set \
    -t SQLServer \
    --settings MyDbConnection="Server=tcp:$,1433;Database=sampledb;User ID=azuresqladmin;Password='MYSECRETPASSWORD';Encrypt=true;Connection Timeout=30;"

# az sql server firewall-rule create --resource-group $RESOURCE_GROUP_NAME --server $ --name LocalAccess --start-ip-address <your-ip> --end-ip-address <your-ip>

# Change database connection string in "appsettings.json" 
#"ConnectionStrings": {
#    "MyDbConnection": "Server=tcp:$,1433;
#        Initial Catalog=sampledb;
#        Persist Security Info=False;
#        User ID=azuresqladmin;Password=MYSECRETPASSWORD;
#        Encrypt=True;
#        TrustServerCertificate=False;"
# }
# create database table in target database
dotnet tool install -g dotnet-ef
dotnet ef migrations add InitialCreat --project DotNetCoreSqlDb
dotnet ef database update --project DotNetCoreSqlDb

Access web app logs and delete resources group

az webapp log config \
    --web-server-logging 'filesystem' \
    --name $APP_SERVICE_NAME \
    --resource-group $RESOURCE_GROUP_NAME
az webapp log tail \
    --name $APP_SERVICE_NAME \
    --resource-group $RESOURCE_GROUP_NAME

az group delete --name $RESOURCE_GROUP_NAME

Azure App Service and Bicep

List of sample deployments of Azure App Service with various scenario by using Bicep

  • Azure App service with Blob
  • Azure App Service + SQLDB + App Insight
  • Azure App Service (Webapp + private endpoint (WebApp and SQL Server) + VNET integration.
  • Application Gateway (private and public listeners) -> Azure App Service (via PE) -> SQL Server (via PE)

Azure App service with Blob:

az group create --name $RESOURCE_GROUP--location westus3
az deployment group create --resource-group $RESOURCE_GROUP --template-file main.bicep --parameters environmentType=nonprod

az deployment group list --output table

Azure App Service + SQLDB + App Insight :

az group create --name $RESOURCE_GROUP --location westus3
az deployment group create --resource-group $RESOURCE_GROUP --template-file 01-main-web-app-sqldb.bicep  --parameters sqlAdministratorLogin='azuresqladmin' sqlAdministratorLoginPassword='PLEASECHANGEtoSECRETPASSWORD' location='westus3'

Azure App Service (Webapp + private endpoint (WebApp and SQL Server) + VNET integration:
Deploys two web apps (frontend and backend) and SQL Server securely connected together with VNet integration (webAppFrontend) and Private Endpoint(webAppBackend,SQLServer)
m18h/02-main-webapp-pe-vnet-integration.bicep at master · fujute/m18h (

az group create --name $RESOURCE_GROUP--location westus3
az deployment group create --resource-group $RESOURCE_GROUP--template-file 02-main-webapp-pe-vnet-integration.bicep \
--parameters virtualNetworkName='m6290cvnet' sqlAdministratorLoginPassword='PLEASECHANGETOSECUREPASSWORD'


Integrate your app with an Azure virtual network


picture: Regional virtual network integration

Application Gateway integration: Integration with App Service (multi-tenant)

Continue reading Azure App Service and Bicep

Dynamically create and use a persistent volume with Azure Files in Azure Kubernetes Service (AKS)

How to dynamically create an Azure Files share for use by multiple pods in an Azure Kubernetes Service (AKS) cluster

This sample has been tested with ” AKS, Azure File, PV,PVC, Deployment, Multi-Container ”

Sample file can be accessed from at “

az aks get-credentials --admin --name MyManagedCluster --resource-group MyResourceGroup
kubectl apply -f storageclass-and-pvc.yaml
kubectl apply -f fuju-nginx-azfile.yaml

kubectl get pod

kubectl exec fuju-nginx-azfile-5c55fddc9f-6tgk  -c 1st-c -- /bin/cat /usr/share/nginx/html/index2.html
kubectl exec fuju-nginx-azfile-5c55fddc9f-6tgk6 -c 1st-c -- /bin/cat /mnt/config-1st/my-config-file.txt
kubectl exec fuju-nginx-azfile-5c55fddc9f-6tgk6 -c 2nd-c -- /bin/cat /mnt/config-2nd/my-config-file.txt

DevOps: Azure DevOps Release Gate (Query work items, SonarQube, Azure monitor)

A Sample of “Azure DevOps Release Gate ” the Gates with the following items

  • Query work items: Query Active Bug
  • SonarQube: ” Invoke REST API: POST”
  • Azure monitor: Azure monitor Alert

Software infra:

Based on ” Controlling Deployments using Release Gates | Azure DevOps Hands-on-Labs ( ” and ” Managing technical debt with SonarQube and Azure DevOps | Azure DevOps Hands-on-Labs (

  • 2 Web app for sample canary and production
  • sonarqube in Azure Container Instance

az group create -n $AprRG -l eastasia
az appservice plan create -g $AprRG -n MyPlan --sku S1
az webapp create -g $AprRG -p MyPlan -n "PartsUnlimited-$RNUMBER-Canary"
az webapp create -g $AprRG -p MyPlan -n "PartsUnlimited-$RNUMBER-Prod"

RG_ID=$(az group create --name $AprRG  --location "eastasia" --query "id" --output tsv)
PASSWORD=$(az ad sp create-for-rbac --name $SERVICE_PRINCIPAL_NAME --role contributor --scopes $RG_ID --query "password" --output tsv)
USER_NAME=$(az ad sp list --display-name $SERVICE_PRINCIPAL_NAME --query "[].appId" --output tsv)

az container create -g $AprRG --name sonarqubeaci180422 --image sonarqube --ports 9000 --dns-name-label mysonarqube200422 --cpu 2 --memory 3.5

#curl -u ea---fe:

Hint: Azure DevOps Release Gate with Azure DevOps Starter

To build quick demo for “Release Gate” with condition from Azure Board, Azure Monitor and SonarQube

  1. DevOps Starter ” .NET Core -> App Service ”
  2. Add SonarQube in “build” Pipeline
  3. Add Release ” UAT” Stage (Then, we have Dev & UAT)
  4. Add “Pre-deployment approvals”
  5. Add “Pre-deployment Gates” ” Query work items”
    Azure DevOps -> Boards -> Queries -> Active Bugs -> … -> Security -> ReleaseGate Build Service(myOrg) -> Read ( Allow )
  6. Add “monitoring” gate in “Dev” Stage
  7. Add Agentless Job in Tasks( manual Intervention )
  8. Add ” Post-deployment approvals” Gate “Query Azure Monitor Alerts”
  9. Add ” Post-deployment approvals” Gate “Invoke REST API:POST”

#URL suffix:

#success critirial:

Sample of Canary releases (ref: What are deployment patterns? – Learn | Microsoft Docs )

DevTest and DevOps for microservice solutions


See Also:

A user delegation SAS for blob with the Azure CLI & azcopy

Sample of using A user delegation SAS for blob with the Azure CLI and copy the blob file via azcopy with the SAS token

RG_ID=$(az group create --name $MY_RG  --location southeastasia --query "id" --output tsv)
PASSWORD=$(az ad sp create-for-rbac --name $SERVICE_PRINCIPAL_NAME --role "Storage Blob Data Contributor" --scopes $MY_SCOPES  --query "password" --output tsv)
USER_NAME=$(az ad sp list --display-name $SERVICE_PRINCIPAL_NAME --query "[].appId" --output tsv)

az login --service-principal -u $USER_NAME -p $PASSWORD --tenant <mytenant>

END=$(date -u -d "30 minutes" '+%Y-%m-%dT%H:%MZ')
SAS4BLOB=$(az storage blob generate-sas \
    --account-name m14storage \
    --container-name data0314 \
    --name "kblob-file-001.txt" \
    --permissions acdrw \
    --expiry $END \
    --auth-mode login \
    --as-user \
    --full-uri )
azcopy copy $SAS4BLOB  . 

See Also:


  • SP’s Password protection ?

Connect to SQL Database from App Service by using a managed identity – System-assigned.

Running App Service with Azure SQL based

Fix list:
The steps in this guide has been tested with “.NET 6.0”

dotnet tool install -g dotnet-ef
dotnet ef migrations add InitialCreate
dotnet ef database update
dotnet add package Microsoft.Data.SqlClient --version 4.0.1
dotnet add package Azure.Identity --version 1.5.0

appsettings.json :

  "Logging": {
    "LogLevel": {
      "Default": "Information"
  "AllowedHosts": "*",
  "ConnectionStrings": {
    "CoredBConnection": ";Authentication=Active Directory Default; Database=CoredB;"
az webapp identity assign --resource-group my-002-rg --name webapp-core-sql-018
ALTER ROLE db_datareader ADD MEMBER [webapp-core-sql-018];
ALTER ROLE db_datawriter ADD MEMBER [webapp-core-sql-018];
ALTER ROLE db_ddladmin ADD MEMBER [webapp-core-sql-018];

Program.cs :

public void ConfigureServices(IServiceCollection services)
   services.AddDbContext<MyDatabaseContext>(options =>

See Also:

Web App & Azure SQL Templates: