Web Server

A web server is a program that answers

Strip away the racks and the cloud logos and a web server is one program doing one thing in a loop: wait for someone to connect, read what they asked for, send text back, and go back to waiting.

The word server means two things and people mix them up constantly. There is the machine sitting in a data centre, and there is the program listening on a port. Here we mean the program — and it runs just as happily on the laptop in front of you.

One visit, four steps

1. Listen The program claims a port — 8084 in our example — and waits. A port is just a numbered door on the machine, so two programs cannot hold the same one.
2. Read the request A browser connects and sends a few lines of text. The first line says which method and which path it wants, for example a GET of the path /.
3. Decide The server works out what to send: read a file from disk, run some code, query a database. Our example always sends the same file, which is the only reason it fits in twenty lines.
4. Respond and close It writes a status line, a few headers, one blank line and the content, then closes the connection and loops round for the next visitor.

A web server in one shell script

Here is a complete web server written in bash. It opens a port, reads the request line, logs it, and sends index.html back as a proper HTTP response. Twenty lines, no libraries, and a browser cannot tell the difference.

Two commands do the heavy lifting. netcat (nc) speaks the network, and mkfifo makes a named pipe so the answer we write can be fed back into the connection nc is holding open.

#!/bin/bash

PORT=8084
PIPE=simple-web-server-pipe

rm -f "$PIPE"
mkfifo "$PIPE"

response() {
    read -r method path _
    echo "[$(date)] $method $path" >&2
    index=$(cat index.html)
    printf 'HTTP/1.1 200 OK\r\nContent-Type: text/html; charset=utf-8\r\nConnection: close\r\n\r\n%s' \
      "$index"
}

while true; do
    response < "$PIPE" | nc -l "$PORT" > "$PIPE"
done

Line by line

1. PORT=8084 The port to listen on. Anything above 1024 works without administrator rights, and 8084 is unlikely to collide with something you already run.
2. PIPE=simple-web-server-pipe The name of the pipe file. It appears in the folder like an ordinary file, but nothing is ever stored in it — it only passes data between two commands.
3. mkfifo "$PIPE" Creates the named pipe (a FIFO). The line above deletes any leftover from a previous run, because mkfifo refuses to overwrite an existing file.
4. read -r method path _ Reads the first line of the request and splits it into method, path and the rest. Everything after it — the browser's headers — is ignored by this server.
5. echo "[$(date)] $method $path" >&2 Prints a log line. The redirection sends it to standard error so it appears on your terminal instead of being sent to the browser as part of the page.
6. index=$(cat index.html) Reads index.html from the current folder into a variable. This is why the server must be started from the folder that holds the file.
7. printf 'HTTP/1.1 200 OK\r\n…' Writes the response: a status line, the content type, a header saying the connection ends here, then a blank line and the page. The blank line is what separates headers from content, and HTTP demands carriage-return-and-newline on every one of them.
8. response < "$PIPE" | nc -l "$PORT" > "$PIPE" The trick that makes it work. nc reads our response from the pipe and writes the browser's request back into it, so the two halves feed each other.
9. while true; do … done The loop. nc handles one connection and exits, so without this the server would answer a single visitor and stop.

Run it yourself

Save the script as server.sh, put an index.html next to it, and start it. Leave that terminal running and use a second one to make requests.

# 1. create an index.html in the same folder
echo '<h1>Hello</h1>' > index.html

# 2. save the script above as server.sh and start it
bash server.sh

# 3. open it in a browser (or check from a second terminal)
open http://localhost:8084
curl http://localhost:8084

Watch the first terminal while you reload the browser: one log line appears per request. Stop the server with Control-C, and delete the pipe file afterwards if it is still there. On Linux the nc command may need an extra -p flag before the port; on macOS the version shipped with the system works as written.

What actually travels down the wire

Open the connection and you find plain text you could have typed yourself. This is the whole reason the shell server is possible — and the reason you can debug a web problem by reading, rather than guessing.

The request the browser sends

GET / HTTP/1.1
Host: localhost:8084
User-Agent: Mozilla/5.0
Accept: text/html

• The first line is the only part our server reads: the method (GET means fetch, POST means send something), the path, and the protocol version.
• Host says which site is wanted. One server can hold hundreds of domains on one address, and this header is how it tells them apart.
• The rest are the browser introducing itself: what software it is, what formats it can read, what languages it prefers. A server may use them or ignore them.

The response the server sends

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Connection: close

<h1>Hello</h1>

• The status line: the protocol version and a code saying how it went. 200 means the request succeeded.
• Content-Type tells the browser how to read what follows. Send HTML with the wrong type and the browser will show your tags as text instead of a page.
• The charset part is not optional in practice. Without it, any non-English text is likely to arrive as garbled characters.
• Then one blank line, and everything after it is the content. That blank line is the entire border between the headers and the page.

Status codes worth knowing

What this server cannot do

The example is honest teaching material and a poor service. Knowing exactly why it falls short is what a real server has to solve for you.

• One visitor at a time. nc handles a single connection, so while it is busy everyone else waits.
• It ignores the path. Ask for anything at all and you still get index.html — there is no routing and no second page.
• It sends everything as HTML. Images, stylesheets and downloads all need their own content type, and this server has exactly one.
• No HTTPS. Everything travels in the clear, so any network in between can read it or change it on the way.
• No safety at all. It never checks the size of a request, never times out, and would happily read a file path someone else chose if you extended it carelessly.

Caddy: a real web server that stays readable

Caddy is a modern web server in a single executable file. It is worth learning right after the shell example because it keeps the same feeling — a few readable lines and it is running — while solving everything the shell version cannot.

Its headline feature is automatic HTTPS. Give it a real domain name and it obtains a certificate, installs it, renews it before it expires, and redirects visitors from http to https. On other servers that is a chore you repeat every few months.

Installing it

# macOS
brew install caddy

# Ubuntu / Debian
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
sudo apt update && sudo apt install caddy

# Docker alone is enough
docker run -p 8080:80 -v "$PWD":/usr/share/caddy caddy

Serving a folder in one command

No configuration file needed. Go to the folder holding your index.html and run one command — this is the fastest way to show a page to someone on the same network.

# serve this folder's index.html on port 8080
caddy file-server --listen :8080

# add a directory listing for folders without an index.html
caddy file-server --listen :8080 --browse

The Caddyfile

For anything permanent you write a Caddyfile. It is a plain text file, and a whole site can be four lines long.

example.com {
	root * /var/www/html
	file_server
	encode gzip zstd
}

1. example.com { The site address. Write a real domain here and Caddy arranges the HTTPS certificate on its own; write localhost and it serves plain http for local work.
2. root * /var/www/html Where the files live on disk. The star means it applies to every path of this site.
3. file_server Actually serve those files. Without this line Caddy answers the request with nothing.
4. encode gzip zstd Compress responses before sending them. One word, and pages arrive noticeably faster on slow connections.

Reverse proxy: passing the request on

Most sites are not files on disk; they are an application listening on some local port. A reverse proxy is a server that takes the public request and hands it to that application, then returns its answer. One line does it.

getpes.com {
	reverse_proxy pes-web:3000
}

dl.getpes.com {
	root * /srv/files
	file_server browse
}

This is also how one machine hosts many sites. Only one program may hold port 443, so Caddy holds it, reads which domain each visitor asked for, and forwards each one to a different application behind it.

The commands you need