Expert · Incident · 5 of 180:00 / 15:00

Uploads keep failing

Problem

The app in Spoke 1 can't upload to stuploads01.

Your job
  • Get the app reaching stuploads01 on 443 over its private endpoint
Without breaking
  • Keep public network access on stuploads01 offbroken
  • Keep Spoke 1 using the company DNS serverbroken
Background

"Since security locked down storage last week, the app in Spoke 1 can't upload to stuploads01." The app runs on vm-app. stuploads01 is a blob storage account reached through a private endpoint in the hub, and its public network access stays off. Spoke 1 uses the company DNS server, dns-hub at 10.0.4.4 in the hub, and keeps using it.

How to work it
  1. 1See it fail

    Prove the problem yourself before you change anything. Reports are often wrong.

    VM commands forVMs
    1. ssh vm-app
      What's this?

      Opens a shell on vm-app, so the next commands run on that VM. In this lab it's a shortcut. In real Azure you'd connect through something like Azure Bastion, or SSH to an address you can reach.

    2. nslookup stuploads01.blob.core.windows.net
      What's this?

      Asks the VM's DNS server what IP address a name points to. Run it on the VM that has the problem, because each VM can get a different answer.

    3. nc -zv stuploads01.blob.core.windows.net 443
      What's this?

      netcat, a standard Linux tool. Tries to open a TCP connection and reports whether it worked.

      • -z only check whether the port answers. Send no data
      • -v print the result in words
      • stuploads01.blob.core.windows.net where to connect
      • 443 the port to try

    Ticks off when the test runs on vm-app.

  2. 2Find where it stops
  3. 3Name the cause
  4. 4Fix it and test again
Sources on Microsoft Learn (3)
0/2 changes
Network map
?
  • Resource group, the outer dashed frame
  • Virtual network, with its address space top right
  • Subnet, with range and usable IPs (5 reserved)
  • Network security group. A small shield on a VM's NIC means the NIC has one too
  • Route table on the subnet
  • NICVM Appliance Private endpoint
  • Peering Connected Initiated or access off
  • Goal working Goal broken, X marks where it stops
  • Private DNS zone, dotted lines go to linked VNets
  • Orange pulse: what your last command changed. Drag to pan, Ctrl and scroll to zoom.
Resource grouprg-labvnet-spoke1: tap to inspectVirtual networkvnet-spoke110.1.0.0/16subnet-app: tap to inspectsubnet-app10.1.1.0/24 · 251 usablevm-app 10.1.1.4: tap to inspectNICvm-app10.1.1.4vnet-hub: tap to inspectVirtual networkvnet-hub10.0.0.0/16subnet-pe: tap to inspectsubnet-pe10.0.3.0/24 · 251 usablepe-stuploads01 10.0.3.8: tap to inspectpe-stuploads…10.0.3.8subnet-dns: tap to inspectsubnet-dns10.0.4.0/24 · 251 usabledns-hub 10.0.4.4: tap to inspectNICdns-hub10.0.4.4Connected, VNet access off: tap to inspectno accessprivatelink.blob.core.windows.net: tap to inspectDNSPrivate DNS zone · privatelink.blob.core.windows.net · 1 link · stuploads01 → 10.0.3.8
  • vm-app → stuploads01.blob.core.windows.net:443should work: not working
Goals 0/3
Cloud Shell · bashVMsrg-lab · eastus2
cloudshell:~$