Expert · Incident · 6 of 180:00 / 15:00

Checkout times out for everyone

Problem

Checkout times out for everyone.

Your job
  • Get vm-client reaching lb-shop on 443
  • Get vm-web1 and vm-web2 both passing the health probe
Without breaking
  • Keep SSH from vm-client to the web servers blockedbroken
Background

"Checkout has timed out for every customer since this morning's certificate change." The shop runs on vm-web1 and vm-web2 behind lb-shop, an internal load balancer at 10.1.1.10 on 443. Clients sit in subnet-client, like vm-client. Both servers have to be back in rotation, and clients never get SSH to them.

How to work it
  1. 1See it fail

    Prove the problem yourself before you change anything. Reports are often wrong.

    VM commands forVMs: Linux
    1. ssh vm-client
      What's this?

      Opens a shell on vm-client, so the next commands run on that VM. In this lab it's a shortcut. In real Azure you'd connect through something like Azure Bastion, or SSH to an address you can reach.

    2. nc -zv 10.1.1.10 443
      What's this?

      netcat, a standard Linux tool. Tries to open a TCP connection and reports whether it worked.

      • -z only check whether the port answers. Send no data
      • -v print the result in words
      • 10.1.1.10 where to connect
      • 443 the port to try

    Ticks off when the test runs on vm-client.

  2. 2Find where it stops
  3. 3Name the cause
  4. 4Fix it and test again
Sources on Microsoft Learn (3)
0/2 changes
Network map
?
  • Resource group, the outer dashed frame
  • Virtual network, with its address space top right
  • Subnet, with range and usable IPs (5 reserved)
  • Network security group. A small shield on a VM's NIC means the NIC has one too
  • Route table on the subnet
  • NICVM Appliance Private endpoint
  • Peering Connected Initiated or access off
  • Goal working Goal broken, X marks where it stops
  • Private DNS zone, dotted lines go to linked VNets
  • Orange pulse: what your last command changed. Drag to pan, Ctrl and scroll to zoom.
Resource grouprg-labvnet-shop: tap to inspectVirtual networkvnet-shop10.1.0.0/16subnet-web: tap to inspectsubnet-web10.1.1.0/24 · 251 usablensg-webvm-web1 10.1.1.4: tap to inspectNICvm-web110.1.1.4vm-web2 10.1.1.5: tap to inspectNICvm-web210.1.1.5lb-shop-fe 10.1.1.10: tap to inspectLBlb-shop-fe10.1.1.10subnet-client: tap to inspectsubnet-client10.1.2.0/24 · 251 usablevm-client 10.1.2.4: tap to inspectNICvm-client10.1.2.4
  • vm-client → lb-shop-fe:443should work: not working
  • vm-client → vm-web1:22should stay blocked: blocked
Goals 0/3
Cloud Shell · bashVMs: Linuxrg-lab · eastus2
cloudshell:~$