How to Use Python to Monitor Files and Processes With psutil

Parsing ps aux and df output with awk works until a column shifts, a name contains a space or the script runs on macOS. The psutil library offers the same data as Python objects on every platform. It is a good base for a small monitor, and it is a cleaner alternative to the shell checks in monitoring and restarting a process. The examples assume Python 3.13 and psutil 6.

Find processes

import psutil

for proc in psutil.process_iter(["pid", "name", "memory_info", "cmdline"]):
    try:
        if proc.info["name"] == "nginx":
            rss_mb = proc.info["memory_info"].rss / 1_048_576
            print(proc.info["pid"], f"{rss_mb:.1f} MiB")
    except (psutil.NoSuchProcess, psutil.AccessDenied):
        continue

1204 8.4 MiB
1205 22.9 MiB
1206 23.1 MiB

Requesting attributes up front makes process_iter fetch them in one pass, which is faster than touching each property later. A process can exit between listing and reading, and others belong to another user, which is why the two exceptions are caught. Match on cmdline when the name is generic, like python3.

System load, memory and disk

print("cpu %:", psutil.cpu_percent(interval=1))
mem = psutil.virtual_memory()
print(f"memory available: {mem.available / 1_073_741_824:.1f} GiB ({mem.percent}% used)")
disk = psutil.disk_usage("/")
print(f"disk used: {disk.percent}%")
print("load:", psutil.getloadavg())

cpu %: 6.4
memory available: 5.3 GiB (31.2% used)
disk used: 64.0%
load: (0.21, 0.35, 0.40)

The sample numbers are only illustrative. cpu_percent needs an interval to compute a rate: the first call without one returns 0.0. Use available for memory pressure, not free, because the kernel uses spare memory for cache, as explained in the page cache.

Open files and connections of one process

proc = psutil.Process(1204)
print(proc.num_fds(), "file descriptors")
for f in proc.open_files()[:3]:
    print(f.path)
for c in proc.net_connections(kind="inet")[:3]:
    print(c.laddr, c.status)

This replaces lsof -p and shows descriptor leaks: a number that only grows is a leak. Reading another user's process needs root. The method was named connections before psutil 6; use net_connections now. See lsof for hung processes for the shell-side tool.

A small alert loop

import time

while True:
    if psutil.disk_usage("/").percent > 90:
        print("ALERT: disk above 90%", flush=True)
    time.sleep(60)

Run it under systemd or as a cron job that exits after one check, and send alerts through the channels in Slack notifications or ntfy. Alert on conditions that persist for several checks, not a single sample, or one spike pages someone at night.

What psutil gives you that ps does not

The psutil library presents processes, memory, disks, network interfaces and sensors through one Python interface that works on Linux, macOS and Windows. For a script, the advantage over parsing ps, free and df output is stability and type safety: a process is an object with attributes that return numbers and lists, not columns that vary between versions and locales. A monitor built on it needs no regular expressions and survives an upgrade of the operating system tools.

The library reads from the kernel interfaces, /proc on Linux, so its cost is small, but calls are not free. Reading attributes of thousands of processes in a tight loop takes real time. Use process_iter with a list of attribute names, which fetches them in one pass and tolerates processes that exit mid-scan, rather than creating objects and querying them one by one.

Handling processes that disappear

The main source of bugs in process monitoring is a race: a process listed a moment ago may be gone by the time you read its attributes. psutil raises NoSuchProcess in that case, and AccessDenied when the account lacks permission to read another user's details. A loop over all processes must catch both. Treat a vanished process as normal, skip it silently, and count denied reads, because a monitor running as an ordinary user sees less than one running as root and the report should say so.

Also keep in mind that process ids are reused. Holding a pid across several seconds and then acting on it can hit a different program. Create the Process object once and use it, since psutil checks the creation time and raises an error if the pid now refers to something else.

CPU percentages and sampling

CPU usage is a rate, so a single reading of it is meaningless. cpu_percent(interval=None) compares against the previous call, and the first call returns 0.0 as a baseline. Call it once to prime, wait, and read again, or pass an interval so the function blocks and measures over that period. For per-process values the same applies, and a value above 100 is normal on a multi-core machine, since 100 means one full core.

Memory figures need the same care. Resident set size counts shared pages in every process that maps them, so summing it over processes overstates use. The proportional set size, available as part of the full memory info on Linux, divides shared pages fairly and gives a better total. For system-wide pressure, look at available memory and swap activity, not the free figure, since the kernel uses spare memory for the page cache by design.

Alerting without noise

An alert loop should trigger on conditions that last, not on a single reading. Require a threshold to be crossed for several consecutive samples before sending anything, and send one message when it clears. Add a cooldown so a flapping metric does not produce a stream of notifications. Write the state to a small file so a restart of the monitor does not forget an open alert. For anything beyond a handful of hosts, a metrics agent and a time-series database give history, graphs and alerting rules without custom code, and the script is best kept for checks that are specific to your application. Run the monitor under systemd with a restart policy and send a heartbeat so its own failure is noticed.

Package the monitor properly. Pin the psutil version in a requirements file, because attribute names and behaviours have changed across major releases, and run the script in a virtual environment. Add a unit test that feeds fake process objects to the threshold logic, so the alert rules can be checked without waiting for a real overload. Keep the sampling interval and thresholds in a configuration file, not in the code, so tuning does not need a deployment.

Finally, remember that a monitor on the same host as the failure shares its fate. If the machine locks up, nothing runs. Pair local checks with an external probe that notices silence.

Where this does not apply

For serious monitoring, use a collector such as Prometheus node exporter, covered in node exporter basics, which keeps history and handles alerting. psutil is a library for scripts and quick tools, and it adds a dependency that must be installed in each environment. Inside containers, many values reflect the host unless the cgroup files are read. See the psutil documentation.

For the underlying data, the kernel exposes the same information under /proc, documented in the proc manual page, and psutil is a portable wrapper around it. Reading the manual page explains fields such as resident set size and why they differ from virtual size.