The program that boots your server, starts your services, and quietly brings them back every time they die. One mental model + four commands = daily ops.
← / → to navigate · EN/VI switch below
When a server powers on, the kernel's very first job is to start exactly one program: systemd. Everything else on the machine waits for it.
the machine boots
loads from disk, inits hardware
PID 1 — the first process, the boss
runs forever
nginx · database · your app — started by systemd
no one starts systemd — the kernel starts it, and it starts everyone else
systemd is the init system — the process that owns everything after boot. It wears three hats at once.
The first program the kernel starts. Every other process on the machine is, in the end, its descendant.
Reads unit files and starts or stops each service exactly according to its instructions.
Watches every service it started — and restarts it when it crashes, whether that's 3 p.m. or 3 a.m.
Before 'just run a program on a server' works, two things must be true. systemd exists to guarantee both.
Someone reboots the machine — an update, a power blip. Your site must be back on the air without anyone logging in.
→ solved by: ENABLE
A bug, out of memory, a bad deploy — the process dies. It must come back by itself, immediately.
→ solved by: Restart=always
Your server has to work while you sleep. systemd is the colleague who never sleeps: it starts the whole team at boot and revives anyone who drops.
What a 'service' really is, and why the program in your terminal is not one.
Every process on Linux has a parent. Follow the parents up and you always end at systemd — the root of the process tree.
the supervisor · runs for the whole life of the machine
serves the web
your Python app
the database
kill a service and its processes die with it — then systemd can bring it right back
A 'service' is nothing magical: it's a small file (a unit) that tells systemd how to run one program and keep it alive.
The file's name is the service name. systemd reads it and gets the whole job description.
The exact command, the user it runs as, the working directory, the environment variables.
Whether it starts automatically at boot (WantedBy=multi-user.target).
The restart policy: come back always, only on failure, or stay down.
systemd becomes PID 1 — the first process, the root of everything on the machine.
It scans /etc/systemd/system/ and /lib/systemd/system/ for .service files.
Every unit whose [Install] section says 'start me at boot' comes up, in the right order.
systemd watches each service for the rest of the machine's life — restarting any that crash.
python server.py in your terminal and a systemd service can run the exact same code — with very different fates.
Tied to your session: close the terminal or the SSH drops, and it dies. It crashes? Dead until someone re-runs it by hand.
Great for development — fatal on a server.
Detached from any terminal. Survives you logging out and the machine rebooting — and restarts itself when it crashes.
This is what 'running in production' actually means.
You don't 'install and run' software on a Linux box — you add services. The whole machine is a team of services.
The web server — a service that serves your site.
The database — a service that keeps the data.
Your Python/Flask app — the service you write the unit file for.
SSH — the very door you use to administer the server.
Scheduled jobs — the 3 a.m. backups.
The container runtime itself — also just another service.
One command talks to systemd. Four verbs cover 90% of what you'll ever type.
All typed the same way: sudo systemctl <verb> <service>.
Start the service now — for this boot only. The next reboot won't remember, unless you enable it.
Stop it now. It will still start at the next boot, unless you disable it.
Stop + start, in one move. The daily go-to after a deploy or a config change.
Alive? Since when? What did it log recently? The first command of every single incident.
One command: the service's state, how long it's been up, and its most recent log lines.
Two switches, two different times: one is about 'now', the other about 'every boot'. They work independently.
sudo systemctl start app turns the process on in the current session. Reboot the machine and it's gone — unless it's also enabled.
Think: pressing the power button on an AC.
sudo systemctl enable app tells systemd: start this automatically at every boot. It does NOT start it right now.
Think: plugging the AC into the wall socket.
is-active asks 'now?', is-enabled asks 'at boot?'. Check both — a service can be any of the four combinations.
Everything else is rare. These become muscle memory within a week.
| Goal | Command |
|---|---|
| Is it running? what's wrong? | systemctl status app |
| Restart after a deploy | sudo systemctl restart app |
| Start it right now | sudo systemctl start app |
| Start at every boot | sudo systemctl enable app |
| Never auto-start again | sudo systemctl disable app |
| Yes/no checks for scripts | systemctl is-active app |
Stops the service, then starts it fresh. There's a brief gap — a few in-flight requests may drop. That's expected and normal.
The process gets a new PID; old connections don't survive it.
After a deploy (new code), after a config change, or when status looks odd even though the service is still 'active'.
If the fix is a config file the app only reads at startup, restart IS the apply button.
Plain text, three sections, a handful of lines that matter. You'll write them by hand or with AI — but you must be able to read them.
This is the entire unit file for our Flask app. Every line you'll ever need is in it.
[Unit]
Description=My Flask App
After=network.target
[Service]
User=www-data
WorkingDirectory=/opt/app
ExecStart=/usr/bin/python3 /opt/app/server.py
Restart=always
RestartSec=2
[Install]
WantedBy=multi-user.target[Unit] = who I am · [Service] = how to run me · [Install] = when to start me
Metadata and ordering: how systemd talks about this service, and when it may start.
A human label shown in every status output. The first thing you'll read in every incident.
Ordering: start me after these (After=network.target = the network is up). It's order, not a hard dependency.
For our purposes, Description + After=network.target is the whole section. Copy it from a working example.
The heart of the file: the actual command and the environment it runs in.
The exact command, with absolute paths. Relative paths don't work — a server has no 'current folder'.
Who runs it. Never root for an app — www-data or a dedicated user limits the blast radius of any bug.
Where the process starts — matters if your code opens relative paths or writes local files.
Variables the process sees: FLASK_ENV=production, DATABASE_URL=… AI loves generating these — you must know what they do.
One line does all the work. It's what 'enable' reads.
'Start me whenever the system reaches normal multi-user state' = at boot. That's the whole line you'll ever use.
enable app reads WantedBy and creates a symlink inside multi-user.target.wants/ — that symlink IS 'start at boot'.
systemd keeps the unit files in memory. After every edit, you must tell it to re-read — forget that and your edit is invisible.
sudo nano /etc/systemd/system/app.service
sudo systemctl daemon-reload — without this, systemd still runs the old instructions.
sudo systemctl enable --now app — enable for every boot, --now starts it immediately.
systemctl status app — green 'active (running)' is the goal.
Yours. Admin-edited files, including the app.service you wrote. This is where you create and modify services.
If a name exists in both folders, this one wins.
Theirs. Ships with every installed package — nginx.service, sshd.service, postgresql.service live here, pre-written.
Usually you only read these. Overriding one = copying it to /etc and editing the copy.
Every service's stdout and stderr lands in one system-wide journal. One command opens the window to any of them.
Your app prints to stdout or stderr? It's in the journal. So is every other service — no scattered log files to hunt down.
journalctl -u app filters to one service — as if each had its own log file, without any file existing.
A service is down? Its last words are in the journal, a few keystrokes away from you.
-f = follow: new lines stream in as they happen. The ops equivalent of tail -f — while you deploy, watch it come up.
| What you want | The flag |
|---|---|
| Only one service | -u app |
| The last N lines, not the whole history | -n 50 |
| Follow new lines live | -f |
| Since a moment in time | --since "10 min ago" |
| Errors only | -p err |
| Jump straight to the end, page with arrows | -e |
The single line that separates 'died at 3 a.m.' from 'back up at 3 a.m. + 2 seconds'.
By default, a crashed process stays dead — exactly like your terminal program. One line changes that.
Always bring it back, no matter how it died. The default choice for anything that should be up.
Bring it back only if it died badly (error exit, crash signal). Cleaner — but 'always' is the safe habit.
Only on signals or timeouts. You'll rarely need the fine-grained levels.
The default. Stays down. For a service that should be up, that's an outage waiting to happen.
Wait 2 seconds between resurrection attempts, so a crash loop can't spin the CPU.
Kill the process (simulating a crash). systemd notices within a second and starts it again. The journal tells the whole story.
Something is down. You have five commands and about a minute. This is the entire job, nine times out of ten.
systemctl status app — 'failed' or 'inactive' confirms it.
journalctl -u app -n 50 — the traceback, the error, the reason.
Bad deploy? Roll back. Missing env var? Add it to the unit file, then daemon-reload.
sudo systemctl restart app
systemctl status app, then journalctl -u app -f while you send a test request.
Two ways a service can die. systemd handles both, automatically — so the site stays up.
update, power blip — the machine restarts
a bug kills the process
enabled → starts it at boot
Restart=always → revives it on crash
up · up · up — 24/7
you deploy code — systemd keeps it breathing across reboots and crashes
systemd is a deep toolbox. These are real features — and not your next step.
The formal dependency graph (multi-user.target, Wants/Requires). After= + WantedBy already covers what you need.
systemd timers (a cron replacement) and socket activation — powerful, and cron is fine for now.
Managing the network stack and resource limits from systemd. Real and deep — not the mental model you need first.
Containers have their own restart policy — the mental model is identical, the commands differ. Covered in the Docker topic.
You write the code. You deploy the code. systemd makes sure the code keeps running — through reboots, crashes, and 3 a.m. You ship; it supervises.
status → journalctl → restart: the daily loop is in your hands. Next: putting the whole stack together in one deployment.