Skip to main content

Command Palette

Search for a command to run...

Linux File System Hunting

What I Found When I Actually Explored Linux

Updated
10 min read
Linux File System Hunting
A

Hi, I’m Abdul Samad. A web development learner and tech enthusiast. I write about what I learn, share practical coding tips, and publish in-depth blogs on programming and modern web development.

Check out my full collection of blogs on Hashnode: https://abdulsamad30.hashnode.dev/

Connect with me on X for quick updates and insights: @abdul_sama60108

So our cohort give this assignment explore the Linux file system and write a blog about what you find.

Not a "run these commands" tutorial. An actual investigation.

Honestly, my first reaction was okay but where do I even start? I knew the basics ls, cd, the usual surface-level stuff. But this task was asking me to go deeper. To actually understand what Linux is doing under the hood, through its own files.

So I opened google , stackoverflow and GPT and start experimenting with the commands…


/etc The Brain of the System

The first place I went was /etc. I'd seen it mentioned a hundred times but never really looked inside it properly.

Turns out, /etc is where Linux stores all its persistent configuration. Not programs, not user data just settings. It's the place the system reads from to decide how to behave every time it boots up.

One file that immediately caught my eye was /etc/hostname. It's literally just one line the name of your machine. Simple, but the system reads this at boot to know what to call itself on the network.

Then I opened /etc/hosts and things got more interesting:

127.0.0.1   localhost
127.0.1.1   my-machine

This file is checked before any DNS query goes out to the internet. So if you put a domain name here with a custom IP, the system will follow it no questions asked. Developers use this to test domains locally. But it also means that if someone malicious gets into your system, this is one of the first places they'd plant a redirect.

Why it matters: /etc is the system's long-term memory. When something behaves weirdly on Linux, this is the first place an investigator looks.


DNS Is Not Just One File It's a Chain

I assumed DNS was just one config file somewhere. It's not. It's a whole pipeline, and each part does something different.

/etc/resolv.conf is where the nameservers are listed the servers your machine asks when it needs to look up a domain:

nameserver 8.8.8.8
nameserver 1.1.1.1

But when I ran this on my machine, I discovered something unexpected:

ls -la /etc/resolv.conf
# lrwxrwxrwx ... /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

It's a symlink. On modern systems using systemd-resolved, this file is auto-generated and managed by systemd. You're not editing it directly systemd owns it.

Then there's /etc/nsswitch.conf, which controls the order of resolution:

hosts:  files dns

"Files" means check /etc/hosts first. "dns" means then go ask the nameserver. This one line decides the priority. Flip the order and the system's entire lookup behavior changes.

Why it matters: DNS on Linux isn't a single switch. It's a chain hosts file, nsswitch, resolv.conf, systemd-resolved and each link can be customized independently.


The Routing Table How the System Decides Where Packets Go

I didn't expect to find the routing table as a readable file, but there it is:

cat /proc/net/route

The output is in hex, so more useful is:

ip route
default via 192.168.1.1 dev eth0
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.5

This table is how every outgoing packet gets directed. The "default" route is your gateway where packets go when nothing more specific matches. The second line says anything on the local network goes straight through the network interface.

What blew my mind is that this table is live. Connect a VPN and new routes appear here automatically. Disconnect and they vanish. It's a real-time map of how your machine thinks about the network at any given moment.

Why it matters: If traffic isn't going where you expect wrong interface, VPN leaking, packets going nowhere this table is the first thing to check, not the application.


/proc A Filesystem That Generates Itself

This one genuinely surprised me. /proc looks like a normal directory. You can browse it, open files, read them. But nothing in it is stored on disk. The kernel generates all of it in real time whenever you read from it.

Every running process gets its own folder inside /proc, named after its PID. So process 1234 lives at /proc/1234/. Inside:

  • cmdline the exact command that launched the process

  • environ every environment variable it has access to

  • fd/ every file, socket, or device it currently has open

  • maps its memory layout, showing loaded libraries

cat /proc/1/cmdline | tr '\0' ' '
# systemd --system --deserialize=28

The fd/ directory was the most revealing part. It's a folder of symlinks one per open file descriptor. You can see exactly what resources any running process is holding onto right now. This is literally how lsof works internally it reads /proc.

/proc/meminfo and /proc/cpuinfo are the raw source for everything htop, free, and top show you. Those tools are just reading from here and formatting it nicely.

Why it matters: /proc is the kernel speaking directly to you. When monitoring tools give you conflicting data, this is the ground truth.


/etc/passwd and /etc/shadow Passwords Aren't Where You Think

I opened /etc/passwd expecting to find passwords. There are none. It looks like this:

username:x:1001:1001:Full Name:/home/username:/bin/bash

Each colon-separated field means: username, password placeholder, UID, GID, display name, home directory, default shell. That x in the password slot means the actual password hash is elsewhere in /etc/shadow, which is only readable by root.

/etc/shadow doesn't just store the hash. It stores the whole password policy when it was last changed, when it expires, how many days of warning before expiry. All of that lives here quietly.

Meanwhile, /etc/group tells you which groups each user belongs to. And group membership determines a lot who can use sudo, who can access disk devices, who can modify network settings.

Why it matters: User access in Linux is spread across three files. If they fall out of sync even slightly permissions can silently break in ways that are really hard to diagnose.


File Permissions and the setuid Bit Where Security Gets Interesting

The standard Linux permission model owner, group, others, each with read/write/execute is something most people learn early. But there's a layer beyond that most tutorials skip over.

The setuid bit lets an executable run as its owner, not the person who launched it. The classic example is passwd the command regular users run to change their own password. To do its job, it needs to write to /etc/shadow, which only root can touch.

ls -la /usr/bin/passwd
# -rwsr-xr-x 1 root root ... /usr/bin/passwd

See that s where the x should be in the owner's execute column? That's the setuid bit. The program temporarily runs as root so it can update the shadow file then drops that privilege when done.

It's a brilliant design. It's also a historically popular target for exploits, because any bug in a setuid binary is potentially a path to root access.

Why it matters: Permissions in Linux aren't just about who can read a file. The setuid mechanism shows that security and usability are sometimes in direct tension and the system makes a deliberate tradeoff.


/var/log The System's Diary

/var/log is where the system writes everything it does. Not always in a way that's obvious at first glance, but once you start reading, it's like a detailed diary.

/var/log/auth.log is specifically for authentication events every SSH login attempt, every sudo command, every failed password. Spending five minutes reading this on any public-facing server is a wake-up call. Failed login attempts from random IPs are constant, quiet background noise.

On systems using systemd-journald, logs go to /run/log/journal/ in binary format, and you query them with:

journalctl -u ssh --since "1 hour ago"

The binary format means you can't just cat the file, but it enables something useful structured, queryable logs with full metadata on every entry: timestamp, process name, PID, unit, and more.

Why it matters: Logs are a forensic timeline. If something went wrong a failed deployment, an unauthorized login, a service crash the answer is almost certainly in here. You just have to know which log to look at.


/dev Hardware as Files

/dev is where Linux turns physical hardware into files you can interact with directly.

/dev/sda is your first hard drive. /dev/sda1 is its first partition. You can read from and write to them like files which is exactly how disk imaging tools work.

Then there are the non-hardware "devices" that are genuinely useful:

  • /dev/null anything written here disappears. Useful for silencing output you don't care about.

  • /dev/zero outputs endless zero bytes. Used for zeroing out drives or generating test files.

  • /dev/urandom outputs cryptographically random bytes. This is where secure random number generation comes from on Linux.

head -c 16 /dev/urandom | xxd
# 16 random bytes shown in hex

Why it matters: /dev is where Linux makes good on its "everything is a file" philosophy. The same interface you use to read a text file can read from hardware but the stakes are very different.


/boot What Runs Before Linux Runs

/boot is the directory that exists before the rest of the system is available. The kernel image itself lives here the actual binary that becomes the running operating system.

Next to it is initrd.img the initial RAM disk. When your machine starts, the bootloader loads this compressed mini-filesystem into memory and uses it to handle early boot tasks before the real filesystem is mounted. It's a temporary operating environment that exists purely to bootstrap the real one.

/boot/grub/grub.cfg is the GRUB bootloader config it lists what kernels are installed, boot parameters, and timeouts. Whatever kernel parameters GRUB passes at startup are visible later at:

cat /proc/cmdline

Why it matters: /boot is the most critical path in the system. A misconfiguration here doesn't break a service it breaks the entire machine. It's also the first place to investigate when a system refuses to start.


/etc/systemd The Startup Dependency Graph

The last thing I explored was how systemd manages services, and it changed how I think about Linux startup entirely.

Each service is defined by a unit file in /etc/systemd/system/. These files declare not just how a service starts, but what it depends on:

[Unit]
After=network.target
Restart=on-failure

After=network.target means this service waits for networking before starting. Restart=on-failure means systemd will automatically bring it back if it crashes. The entire boot process is encoded in these dependency declarations.

What systemd actually does is build a dependency graph from all these unit files and then parallelize as much as possible. That's why modern Linux boots noticeably faster than older systems it's not running things one by one anymore.

systemctl list-dependencies multi-user.target

This shows the full tree of everything that must be running before the system reaches its normal operational state.

Why it matters: The boot process isn't a sequential script. It's a graph. Understanding this is the difference between efficiently debugging a boot failure and randomly trying things until something works.


What This Assignment Actually Taught Me

Going into this, I thought Linux was mostly opaque something you learned commands for, not something you understood. Coming out of it, I see it completely differently.

Linux explains itself. The kernel exposes its own state through /proc. Services document themselves in /etc/systemd. The routing table is a readable file. Authentication history is in /var/log/auth.log. Every configuration decision the system has made is stored somewhere you can look at.

The real skill isn't memorizing commands. It's knowing where the system keeps its information. Once you know that, the commands are just the means to get there.

Next time something breaks and something will break don't Google the error first. Go look at the files. The answer is almost always already there.