Expert · Incident · 4 of 180:00 / 12:00

Monitoring lost both app servers

Problem

The dashboards show both app servers as down.

Your job
  • Get vm-monitor reaching vm-app1 and vm-app2 on 9100
Without breaking
  • Keep vm-user blocked from 9100 on both serversbroken
  • Keep UFW running on both serversbroken
  • Change only what is actually brokenbroken
Background

"Since the security team's hardening run on Monday, the dashboards show vm-app1 and vm-app2 as down. Users say the app itself works fine." vm-monitor in subnet-monitor collects metrics from both servers on port 9100. Only the monitoring subnet should reach that port. The security baseline stays in place.

How to work it
  1. 1See it fail

    Prove the problem yourself before you change anything. Reports are often wrong.

    VM commands forVMs: Linux
    1. ssh vm-monitor
      What's this?

      Opens a shell on vm-monitor, so the next commands run on that VM. In this lab it's a shortcut. In real Azure you'd connect through something like Azure Bastion, or SSH to an address you can reach.

    2. nc -zv 10.1.1.4 9100
      What's this?

      netcat, a standard Linux tool. Tries to open a TCP connection and reports whether it worked.

      • -z only check whether the port answers. Send no data
      • -v print the result in words
      • 10.1.1.4 where to connect
      • 9100 the port to try

    Ticks off when the test runs on vm-monitor.

  2. 2Find where it stops
  3. 3Name the cause
  4. 4Fix it and test again
Sources on Microsoft Learn (4)
0/2 changes
Network map
?
  • Resource group, the outer dashed frame
  • Virtual network, with its address space top right
  • Subnet, with range and usable IPs (5 reserved)
  • Network security group. A small shield on a VM's NIC means the NIC has one too
  • Route table on the subnet
  • NICVM Appliance Private endpoint
  • Peering Connected Initiated or access off
  • Goal working Goal broken, X marks where it stops
  • Private DNS zone, dotted lines go to linked VNets
  • Orange pulse: what your last command changed. Drag to pan, Ctrl and scroll to zoom.
Resource grouprg-labvnet-app: tap to inspectVirtual networkvnet-app10.1.0.0/16subnet-app: tap to inspectsubnet-app10.1.1.0/24 · 251 usablevm-app1 10.1.1.4: tap to inspectNICvm-app110.1.1.4vm-app2 10.1.1.5: tap to inspectNICvm-app210.1.1.5subnet-users: tap to inspectsubnet-users10.1.2.0/24 · 251 usablevm-user 10.1.2.4: tap to inspectNICvm-user10.1.2.4subnet-monitor: tap to inspectsubnet-monitor10.1.5.0/24 · 251 usablevm-monitor 10.1.5.4: tap to inspectNICvm-monitor10.1.5.4
  • vm-monitor → vm-app1:9100should work: not working
  • vm-monitor → vm-app2:9100should work: not working
  • vm-user → vm-app1:9100should stay blocked: blocked
Goals 0/4
Cloud Shell · bashVMs: Linuxrg-lab · eastus2
cloudshell:~$