Study on the go with our IT certification podcast
Tune in to Linux tips, security hardening walkthroughs, and exam strategies while commuting or working out. New episodes weekly.
Listen on SpotifyCourse Modules
01
Linux Foundations & Boot Process
5 lessons · ~4 hours
Linux+ never asks "what is a kernel" outright — it asks where a file belongs, or which command family a distro uses. Both are the same question in disguise: do you hold the kernel / shell / userspace split and the FHS tree in your head? Learn the layout once and half the "which path?" and "which tool?" answers become mechanical.
- The Linux kernel manages hardware resources: CPU scheduling, memory management, device drivers, and system calls
- The shell (bash, zsh, sh) is the user-facing interpreter that parses commands and communicates with the kernel via system calls
- Userspace contains all processes running outside the kernel — daemons, applications, and libraries (glibc)
- The Filesystem Hierarchy Standard (FHS) defines the directory tree:
/bin(essential binaries),/sbin(system binaries),/etc(config files),/var(variable data),/usr(user programs),/tmp(temporary),/proc(kernel/process virtual FS),/sys(device/driver info) - Major distro families: Red Hat (RHEL, CentOS, Fedora, Rocky, AlmaLinux) — RPM/DNF; Debian (Ubuntu, Mint) — DEB/APT; SUSE (openSUSE, SLES) — RPM/Zypper; Arch — Pacman
billingd to a Rocky Linux fleet and have to decide where each file goes. The binary is not needed to boot single-user, so it goes to /usr/sbin/billingd, not /sbin. Its config goes to /etc/billingd/billingd.conf — everything under /etc is host-specific and gets backed up. Its runtime data (spool, queue, growing state) goes to /var/lib/billingd/, because /var is the branch that is expected to change size. Its logs go to /var/log/billingd/, or straight to the journal if the unit just writes to stdout. What you must not do is drop it in /usr/bin with state files beside it: that breaks the read-only-/usr assumption many hardened builds rely on, and RPM packaging will reject it in a lint check.
/etc = configuration, /var = data that grows, /usr = shipped software, /proc and /sys = kernel views that exist only in memory. Distro family then tells you the tool family: Red Hat → RPM/DNF, Debian → DEB/APT, SUSE → RPM/Zypper.
A machine that will not boot is the highest-stakes scenario on the exam and in the job, and the fix always depends on how far it got. No firmware screen is hardware; a grub rescue> prompt means GRUB lost its modules; a kernel panic means GRUB did its job and the kernel could not mount root. This lesson pins each symptom to its link in the chain.
- BIOS/UEFI performs POST (Power-On Self-Test), initializes hardware, then loads the bootloader from MBR (BIOS) or EFI partition (UEFI)
- GRUB2 (GRand Unified Bootloader v2) is the standard Linux bootloader — loads the kernel and initramfs into memory
- GRUB2 config location:
/boot/grub2/grub.cfg(RHEL/Fedora) or/boot/grub/grub.cfg(Debian/Ubuntu) - Never edit
grub.cfgdirectly — edit/etc/default/gruband regenerate withgrub2-mkconfig -o /boot/grub2/grub.cfg - Key
/etc/default/grubparameters:GRUB_TIMEOUT(menu delay),GRUB_CMDLINE_LINUX(kernel parameters),GRUB_DEFAULT(default entry)
GRUB2 Rescue Mode
- If GRUB fails to find its modules, you drop to
grub rescue>prompt grub rescue> ls— list detected partitions (e.g.,(hd0,gpt1),(hd0,gpt2))grub rescue> ls (hd0,gpt2)/— check for/boot/grub2/directorygrub rescue> set root=(hd0,gpt2)— set the root partitiongrub rescue> set prefix=(hd0,gpt2)/boot/grub2— point to GRUB modulesgrub rescue> insmod normalthengrub rescue> normal— load normal GRUB mode
/boot/grub2/grub.cfg on RHEL/CentOS systems. Always regenerate with grub2-mkconfig after editing /etc/default/grub. Editing grub.cfg directly is wrong — changes are overwritten on update.grub rescue> prompt. Recovery sequence: (1) ls to list detected partitions — look for the one containing /boot/grub2/ (e.g. (hd0,gpt2)); (2) set root=(hd0,gpt2); (3) set prefix=(hd0,gpt2)/boot/grub2; (4) insmod normal then normal — GRUB loads its full menu; (5) boot into the OS, then fix permanently: edit /etc/default/grub as needed and run grub2-mkconfig -o /boot/grub2/grub.cfg. Verify: grep menuentry /boot/grub2/grub.cfg shows the boot entries.
root (which partition) and prefix (where GRUB's modules live) — set both, insmod normal, and you are back. Persist nothing by editing grub.cfg: edit /etc/default/grub and regenerate.
Targets are where "boot into a smaller system" happens — the recovery lever you pull when a service, a bad /etc/fstab line, or a lost root password blocks a normal boot. The exam's trap is the difference between now and next boot: isolate switches the running system, set-default changes what comes back after a reboot.
poweroff.target— runlevel 0, system haltrescue.target— runlevel 1 / single-user mode; minimal services, root shell for recoverymulti-user.target— runlevel 3; full multi-user, no GUI; standard for serversgraphical.target— runlevel 5; multi-user with desktop environmentreboot.target— runlevel 6, system restartsystemctl get-default— view current default targetsystemctl set-default multi-user.target— change default target persistentlysystemctl isolate rescue.target— switch to rescue mode immediately (non-persistent)
systemd.unit=rescue.target to the kernel command line in GRUB. This is the standard recovery technique for forgotten root passwords alongside rd.break./etc/fstab and reboots; the host now stops at "Give root password for maintenance". What happened: the mount unit failed and local-fs.target never completed, so systemd dropped to emergency.target. Recovery: (1) reboot, press e in GRUB and append systemd.unit=rescue.target to the linux line — rescue mounts the local filesystems it can and gives you a root shell; (2) journalctl -xb | grep -i mount or systemctl --failed names the failing unit; (3) fix the /etc/fstab entry, or add nofail,x-systemd.device-timeout=10 so a dead NFS server can never again block boot; (4) systemctl daemon-reload && mount -a proves it before you trust it; (5) systemctl default continues into the normal target without a reboot.
systemctl isolate = now, systemctl set-default = every boot, systemd.unit=… on the kernel line = this boot only. Any fstab entry for a remote or optional device should carry nofail.The initramfs exists to solve one chicken-and-egg problem: the kernel needs a driver to reach the root filesystem, and that driver lives on the root filesystem. Every "worked fine until we changed the storage / added encryption / moved the disk" failure traces back to an initramfs that no longer carries the module or the key it needs.
- The initramfs (initial RAM filesystem) is a compressed cpio archive loaded into memory at boot before the real root filesystem is mounted
- It provides the minimal tools needed to: load kernel modules for storage controllers, set up LVM/RAID, unlock encrypted volumes, and mount the real root filesystem
- Lives at
/boot/initramfs-$(uname -r).img(RHEL) or/boot/initrd.img-$(uname -r)(Debian)
Rebuilding initramfs
- RHEL/Fedora:
dracut --force /boot/initramfs-$(uname -r).img $(uname -r) - Debian/Ubuntu:
update-initramfs -u -k $(uname -r) - Necessary after: adding kernel modules to initramfs, changing storage drivers, modifying
/etc/crypttab lsinitrd /boot/initramfs-$(uname -r).img | less— inspect initramfs contents (RHEL)
dracut --force (RHEL) or update-initramfs -u (Debian) is often the fix. The exam tests which tool to use per distro family.virtio. First boot ends in "Kernel panic — not syncing: VFS: Unable to mount root fs". Why: the original initramfs was built on hardware with a MegaRAID controller, so it contains megaraid_sas and no virtio_blk — the kernel simply cannot see the new disk. Fix: (1) boot the VM from the RHEL ISO in Troubleshooting → Rescue; (2) chroot /mnt/sysroot; (3) dracut --force --add-drivers "virtio_blk virtio_pci virtio_scsi" /boot/initramfs-$(uname -r).img $(uname -r); (4) confirm before rebooting with lsinitrd /boot/initramfs-*.img | grep virtio; (5) exit and reboot. Same class of failure, same fix, after adding a LUKS volume — the initramfs needs the crypt modules and the /etc/crypttab entry baked in.
dracut --force on RHEL, update-initramfs -u on Debian — and verify with lsinitrd rather than hoping.
Modules are how a single kernel image supports hardware it has never met. The exam splits this into two halves that look alike but are not: loading a module right now (modprobe, rmmod) versus making that choice survive a reboot (files under /etc/modprobe.d/ and /etc/modules-load.d/). Blacklisting is the classic hardening question.
lsmod— list currently loaded kernel modules and their dependenciesmodinfo MODULE— display module metadata: description, author, parameters, filenamemodprobe MODULE— load a module and its dependencies automaticallymodprobe -r MODULE— remove (unload) a module and unused dependenciesrmmod MODULE— remove a module directly (does not handle dependencies)insmod /path/to/module.ko— insert a module by file path (no dependency resolution)
Persistent Module Configuration
/etc/modprobe.d/— directory for module configuration files (e.g., aliases, options, blacklisting)- Blacklist a module: create
/etc/modprobe.d/blacklist-MODULE.confwithblacklist MODULE - Set module options:
options MODULE param=valuein a conf file under/etc/modprobe.d/ - Modules to load at boot: list names in
/etc/modules-load.d/*.conffiles
modprobe over insmod in almost all cases — modprobe resolves dependencies automatically. insmod requires the full path and won't load required dependencies first.modprobe -r usb_storage (fails if a stick is mounted; unmount first); (2) make it stick — create /etc/modprobe.d/cis-usb-storage.conf containing install usb_storage /bin/true and blacklist usb_storage: blacklist alone only stops automatic alias-based loading, while the install … /bin/true line also defeats an explicit modprobe usb_storage; (3) rebuild the initramfs (dracut --force) so an early-boot copy of the module cannot reintroduce it. Verify: lsmod | grep usb_storage is empty and modprobe usb_storage && lsmod | grep usb_storage still shows nothing.
modprobe resolves dependencies, insmod does not. Runtime changes die at reboot — persistence lives in /etc/modprobe.d/*.conf (options, blacklists) and /etc/modules-load.d/*.conf (load at boot).- BIOS/UEFI hands control to GRUB2, which loads
vmlinuz+initramfs; never editgrub.cfgby hand — change/etc/default/grubthen rungrub2-mkconfig -o /boot/grub2/grub.cfg. - SysV runlevels are gone — systemd targets replace them (
multi-user.target,graphical.target,rescue.target). Switch withsystemctl isolate; persist withsystemctl set-default. - After any storage / encryption change, rebuild the initramfs —
dracut --forceon RHEL,update-initramfs -uon Debian — and prefermodprobeoverinsmodfor kernel modules (dependency resolution included).
02
Package Management & Software
5 lessons · ~4 hours
./configure && make in five answer choices. The trick is to learn each package manager as a small verb table — install · remove · search · update · query · verify — and memorise the one or two flags that diverge per family. Module 02 builds that table for RHEL/Fedora, Debian/Ubuntu, SUSE, and source compiles.
rpm is not how you install software day to day — dnf is. What rpm gives you is the local database: which package owns this file, what did this package ship, and has anything on disk drifted from what the vendor signed. Those are forensic questions, and they are exactly the ones Linux+ asks.
rpm -ivh package.rpm— install a package (-i) with verbose output (-v) and progress bar (-h)rpm -Uvh package.rpm— upgrade a package (installs if not present)rpm -e PACKAGENAME— erase (remove) an installed packagerpm -qa— query all installed packages; combine withgrepto searchrpm -qi PACKAGENAME— detailed info about an installed packagerpm -ql PACKAGENAME— list files owned by an installed packagerpm -qf /path/to/file— which package owns a given filerpm -V PACKAGENAME— verify package integrity (checks checksums, permissions, ownership)rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release— import a GPG signing key
rpm -V output codes: S=file size changed, M=mode changed, 5=MD5 checksum mismatch, U=user ownership changed. A dot (.) means no change. This is a frequently tested command on the Linux+ exam.sshd on one host behaves differently from the rest of the fleet. Investigate with the RPM database: (1) rpm -qf /usr/sbin/sshd → openssh-server-8.7p1-38.el9, so the file is vendor-owned; (2) rpm -V openssh-server prints S.5....T. /usr/sbin/sshd — size changed, MD5 mismatch, mtime changed: the shipped binary is not the one on disk; (3) rpm -qi openssh-server confirms the signature and vendor of what should be there; (4) restore from the signed package with dnf reinstall openssh-server; (5) re-run rpm -V openssh-server — all dots, clean. Contrast the harmless case: rpm -V openssh-server showing S.5....T. c /etc/ssh/sshd_config, where the leading c marks a config file you are expected to have edited.
rpm -qf (who owns this file), rpm -ql (what did this package ship), rpm -V (has it drifted) are the three query verbs worth memorising. In -V output a dot is "unchanged", 5 is a checksum mismatch, and a c flag means the file is config, so drift is expected.
DNF is rpm plus two things the exam cares about: dependency resolution and a transaction log you can reverse. That log is what turns "the update broke production" from an outage into a two-minute rollback, and dnf history undo is the answer to a whole family of scenario questions.
dnf install PACKAGE— install a package and resolve dependenciesdnf remove PACKAGE— remove a packagednf update— update all packages to latest available versionsdnf update PACKAGE— update a specific packagednf search KEYWORD— search for packages by name or descriptiondnf info PACKAGE— show detailed package metadatadnf provides /path/to/file— find which package provides a file or commanddnf history— show transaction history;dnf history undo Nreverses transaction Ndnf group install "Development Tools"— install a package groupdnf repolist— list enabled repositories
Repository Configuration
- Repo files live in
/etc/yum.repos.d/with.repoextension - Key fields:
[repo-id],name,baseurlormirrorlist,enabled=1,gpgcheck=1,gpgkey= dnf config-manager --add-repo URL— add a new repositorydnf config-manager --enable REPO_ID/--disable REPO_ID— toggle repos
dnf update refreshes and installs newer package versions; dnf upgrade is an alias. dnf check-update lists available updates without installing them. Also: yum is the legacy name — on modern RHEL 8+ systems it is a symlink to dnf./usr/bin/iostat, install it, verify, then roll it back. Walk: (1) dnf provides /usr/bin/iostat → shows sysstat; (2) dnf install sysstat; (3) rpm -ql sysstat | grep iostat confirms the binary is installed; (4) dnf history → note the transaction ID (e.g. 47); (5) dnf history info 47 to inspect what was changed; (6) dnf history undo 47 removes the package cleanly. This sequence also tests repos: if dnf provides returns nothing, dnf repolist and check that the right repo is enabled.
dnf provides. When it says "undo last night's update", the answer is dnf history then dnf history undo <id> — never a manual rpm -e, which leaves dependencies stranded.
The Debian side maps one-to-one onto the RHEL side — dpkg is the local database like rpm, apt is the resolver like dnf — with one asymmetry the exam leans on hard: apt update installs nothing. It only refreshes metadata. Half the trick questions in this domain hinge on that single fact, and on remove versus purge.
dpkg -i package.deb— install a .deb package filedpkg -r PACKAGENAME— remove a package (keeps config files)dpkg -P PACKAGENAME— purge a package (removes config files too)dpkg -l— list all installed packages with status codesdpkg -L PACKAGENAME— list files installed by a packagedpkg -S /path/to/file— which package owns a given filedpkg --get-selections | grep PACKAGENAME— check package installation status
APT Package Manager
apt update— refresh the local package index (downloads metadata from repos)apt upgrade— install available package updatesapt install PACKAGE— install a package with dependenciesapt remove PACKAGE— remove package, keep config;apt purge PACKAGEremoves config tooapt autoremove— remove packages that were installed as dependencies but are no longer neededapt search KEYWORD— search packages;apt-cache search KEYWORD(older syntax)apt show PACKAGE— show package details- Repo sources:
/etc/apt/sources.listand/etc/apt/sources.list.d/*.list
apt update only refreshes the local package cache — it does NOT install any updates. apt upgrade installs the available updates. This two-step pattern is a classic exam question. Always run apt update before apt install on a freshly started system.dpkg -i agent_3.2_amd64.deb. It exits with "dependency problems — leaving unconfigured", and every later apt install now refuses to run. Why: dpkg unpacks a single file and does not fetch dependencies; the package is registered but unconfigured, and APT will not proceed while the database is inconsistent. Fix: (1) apt --fix-broken install — APT reads what dpkg recorded and pulls the missing libraries in; (2) confirm the state with dpkg -l agent — the leading code should now be ii (installed/installed), not iU; (3) if the agent later gets removed, use apt purge agent rather than apt remove agent so its /etc/agent/ config and API token do not linger. Better next time: apt install ./agent_3.2_amd64.deb — the leading ./ makes APT handle the local file with dependency resolution.
apt update refreshes, apt upgrade installs — always a pair. dpkg -i never resolves dependencies; apt install ./file.deb does. remove keeps config files, purge deletes them.
Zypper is worth perhaps one or two marks, so treat it as a translation exercise rather than a new subject: SUSE ships RPM packages like RHEL but drives them with a different front end. Learn the verbs against the DNF column you already know, and the short aliases (in, rm, up, se, lr) that answer options like to show up in.
zypper install PACKAGE(orzypper in PACKAGE) — install a packagezypper remove PACKAGE(orzypper rm PACKAGE) — remove a packagezypper update(orzypper up) — update installed packageszypper search KEYWORD(orzypper se KEYWORD) — search for packageszypper info PACKAGE— display detailed package informationzypper repos(orzypper lr) — list configured repositorieszypper addrepo URL ALIAS— add a new repositoryzypper refresh(orzypper ref) — refresh repository metadata
in, rm, up, se, lr) as they appear in practical scenarios.dnf config-manager --add-repo https://vendor/rhel.repo, dnf repolist, dnf install nginx. Debian — drop the source into /etc/apt/sources.list.d/vendor.list, apt update, apt install nginx. SUSE — zypper addrepo https://vendor/sle vendor, zypper ref, zypper lr to check the GPG Check column reads Yes, then zypper in nginx. Note where the refresh sits: APT needs an explicit apt update, Zypper does it with ref, DNF refreshes metadata on demand. That "which step refreshes?" difference is the exam's favourite way to test cross-distro fluency.
zypper in/rm/up/se = dnf install/remove/update/search, zypper lr = dnf repolist, zypper ref = apt update. Same RPM payload underneath, different front end.
Compiling is what you do when no package exists — and the reason it stays on the syllabus is that it exposes the whole dependency model. ./configure is a dependency checker; its error message names the missing piece, and translating that name into a -devel (RHEL) or -dev (Debian) package is the actual skill being tested.
- Download source tarball:
wget https://example.com/app-1.0.tar.gzthentar -xzf app-1.0.tar.gz ./configure— checks for required build dependencies, sets compile options, generatesMakefile./configure --prefix=/usr/local— install to a custom directory (default is/usr/local)make— compiles the source code using the generated Makefilemake install— installs compiled binaries to the prefix directorymake uninstall— remove installed files (if the Makefile supports it)
Build Dependencies
- Common required packages:
gcc,make,autoconf,automake,libtool,kernel-devel - On RHEL:
dnf groupinstall "Development Tools"installs the full toolchain - On Debian:
apt install build-essential - Missing header errors during
./configuremean a-devel/-devpackage is not installed
./configure fails with "missing library", install the corresponding -devel package (RHEL) or -dev package (Debian). The configure script reports the exact missing dependency in its error output../configure stops with "error: OpenSSL headers not found". Walk it: (1) the toolchain first — dnf group install "Development Tools" (Debian: apt install build-essential); (2) headers are shipped separately from the runtime library, so the fix is dnf install openssl-devel, not openssl, which is already installed — that distinction is the whole point of the question; (3) when the name is not obvious, ask the package manager: dnf provides '*/ssl.h'; (4) re-run ./configure --prefix=/usr/local, then make -j$(nproc), then make install as root; (5) confirm with which app → /usr/local/bin/app. Keep it uninstallable: the RPM database knows nothing about these files, so keep the unpacked source tree around for make uninstall — otherwise removal means deleting files by hand.
-devel/-dev package, never the runtime one. ./configure → make → make install, default prefix /usr/local, and nothing you build this way appears in rpm -qa or dpkg -l.- RHEL family:
dnf install/remove/update/search; query the installed DB withrpm -qa,rpm -ql,rpm -V;dnf history+dnf history undo <id>rolls back transactions. - Debian family:
apt updateonly refreshes the cache,apt upgradeinstalls it — always run them as a pair.dpkg -ifor local.debfiles,apt --fix-broken installrepairs dependency hell. - SUSE uses
zypperwith short aliases (in,rm,up,se,lr); source compiles always follow./configure → make → make installand need a-devel/-devpackage per missing header.
03
User, Group & Permission Management
6 lessons · ~5 hours
/etc/passwd, /etc/shadow, /etc/group, /etc/gshadow) — and then layered on top: special bits (setuid, setgid, sticky), POSIX ACLs for extra entries, and PAM for the login policy. Module 03 covers everything from useradd to setfacl, plus the password-policy levers the exam loves to test.
Accounts are three flat text files (/etc/passwd, /etc/shadow, /etc/group) and a handful of commands that edit them safely. Knowing the field order in each file is what lets you read a broken account at a glance — and the single most damaging flag in the whole domain is the -a you forget on usermod -G.
useradd USERNAME— create a new user; add-mto create home dir,-s /bin/bashto set shell,-u UIDfor specific UIDusermod -aG GROUP USERNAME— add user to a supplementary group (-ais critical — appends instead of replacing)usermod -s /sbin/nologin USERNAME— disable login shell for a service accountuserdel USERNAME— delete a user;userdel -r USERNAMEalso removes home directory and mail spoolid USERNAME— display UID, GID, and all supplementary groups for a user
Group Commands & Key Files
groupadd GROUPNAME— create a new group;groupmod -n NEWNAME OLDNAME— rename;groupdel GROUPNAME— deletenewgrp GROUPNAME— switch active primary group in the current session without logging out/etc/passwd— format:username:x:UID:GID:comment:home:shell/etc/shadow— format:username:hashed_password:last_change:min:max:warn:inactive:expire/etc/group— format:groupname:x:GID:member1,member2
-a flag with usermod -G is critical. Running usermod -G GROUP USER without -a REPLACES all supplementary groups with only the specified group — this is a classic misconfiguration that locks users out of shared resources.svc-monitor with UID 3001, no interactive login, added to the monitoring group without disrupting other memberships. Walk: (1) useradd -u 3001 -s /sbin/nologin svc-monitor; (2) groupadd monitoring (if the group doesn't exist); (3) usermod -aG monitoring svc-monitor — the -a is mandatory, omitting it silently wipes all other supplementary groups; (4) verify: id svc-monitor shows UID=3001 and includes the monitoring GID; (5) grep svc-monitor /etc/passwd confirms the shell is /sbin/nologin. Attempting su - svc-monitor as root should say "This account is currently not available."
usermod -aG appends, usermod -G replaces — the missing -a is the classic lockout. Service accounts get -s /sbin/nologin, and id user is how you prove membership rather than trusting the command exited 0.
Password rules land in two different places and the exam expects you to tell them apart: aging (how long a password lives) is per-account metadata in /etc/shadow, edited with chage; strength (how hard a password is) is enforced at the moment of change by a PAM module. Set the wrong one and the policy silently does nothing.
passwd USERNAME— set or change a user's password;passwd -l USERNAMElocks,passwd -u USERNAMEunlockschage -M 90 USERNAME— set maximum password age to 90 dayschage -m 7 USERNAME— set minimum days before password can be changedchage -W 14 USERNAME— warn user 14 days before password expirationchage -E 2026-12-31 USERNAME— set account expiration datechage -l USERNAME— list all aging information for a user
PAM & Password Complexity
- PAM (Pluggable Authentication Modules) controls authentication via
/etc/pam.d/config files pam_pwqualitymodule enforces password complexity (length, uppercase, digits, special characters)- Configure in
/etc/security/pwquality.conf:minlen=12,ucredit=-1,dcredit=-1,ocredit=-1 /etc/login.defs— system-wide defaults:PASS_MAX_DAYS,PASS_MIN_DAYS,PASS_WARN_AGE,UID_MIN,UID_MAX
required (must pass, continues), requisite (must pass, stops on fail), sufficient (if pass, no further required checks), optional (result ignored unless only module).PASS_MAX_DAYS 90, PASS_MIN_DAYS 7, PASS_WARN_AGE 14 in /etc/login.defs — but note this applies only to accounts created after the edit. Existing accounts: must be updated one by one, e.g. for u in $(awk -F: '$3>=1000 && $3<65534 {print $1}' /etc/passwd); do chage -M 90 -m 7 -W 14 "$u"; done, then spot-check with chage -l alice. Forgetting this second step is the single most common way the policy "does not apply". Complexity: set minlen = 12 and ucredit = -1, dcredit = -1, ocredit = -1 in /etc/security/pwquality.conf — a negative credit means "require at least one of this class". Prove it: as an ordinary user, passwd with summer2026 is rejected for length and missing classes; as root it is only warned about, because root bypasses pwquality.
/etc/login.defs sets defaults for future accounts; chage fixes the ones that already exist. Complexity comes from pam_pwquality via /etc/security/pwquality.conf, and root is exempt from it — always test policy as a normal user.
Nine bits, three commands, one arithmetic trick. The exam almost never asks "what does 755 mean" directly — it gives you a symptom (a user cannot cd into a directory, a script will not run, a new file came out world-readable) and expects you to work backwards to the bit or the umask that caused it.
- Permission bits: owner (u), group (g), others (o); each has read (r=4), write (w=2), execute (x=1)
chmod 755 file— numeric: owner rwx (7), group r-x (5), others r-x (5)chmod u+x,g-w file— symbolic: add execute to owner, remove write from groupchmod -R 750 /dir— recursive permission changechown USER:GROUP file— change owner and group;chown USER file— change owner onlychgrp GROUP file— change group ownership only
umask
umaskdefines default permissions by masking bits from 666 (files) and 777 (directories)- Default umask
022: files get 644 (rw-r--r--), directories get 755 (rwxr-xr-x) umask 027: files get 640, directories get 750 — more restrictive, suitable for shared servers- Set persistently in
/etc/bashrcor~/.bashrc
alice is in the finance group and /srv/reports/q3.csv is -rw-r----- root finance, yet cat /srv/reports/q3.csv fails. Diagnose upward, not at the file: ls -ld /srv/reports shows drwxr----- root finance — read on a directory lets you list names, but you need execute to traverse into it and open anything inside. Fix: chmod 750 /srv/reports (or chmod g+x), which is why directories are 750/755 while their files are 640/644. Second half: reports alice creates there come out rw-r--r--, readable by the whole box, because her shell's umask is 022. Set umask 027 in /etc/profile.d/finance-umask.sh and new files land at 640 — 666 minus 027. Verify: sudo -u alice cat /srv/reports/q3.csv succeeds, and sudo -u alice touch /srv/reports/x && ls -l /srv/reports/x shows -rw-r-----.
x means "traverse" — without it nothing inside is reachable no matter how the file itself is set. New-file defaults come from umask: 666 minus the mask for files, 777 minus it for directories.Three extra bits exist because the owner/group/other triad cannot express everything: SUID lets an unprivileged user run something as its owner, SGID on a directory forces group inheritance, and sticky stops people deleting each other's files in a shared writable directory. Each solves one specific problem — the exam tests whether you pick the right one.
- SUID (Set User ID) — on an executable: the process runs as the file owner, not the invoking user. Example:
/usr/bin/passwdruns as root. Set withchmod 4755 fileorchmod u+s file - SGID (Set Group ID) — on an executable: process runs with the file's group. On a directory: new files/directories inherit the directory's group (not the creator's primary group). Set with
chmod 2755 fileorchmod g+s dir - Sticky Bit — on a directory: only the file owner, directory owner, or root can delete/rename files within it, even if others have write permission. Classic use:
/tmp. Set withchmod 1777 dirorchmod +t dir - Display:
ls -lshowssin place ofxfor SUID/SGID,tin place ofxfor sticky bit in others position
devs needs /srv/project where everyone can read and edit each other's files, but nobody can delete a colleague's work. Build it: (1) chgrp devs /srv/project; (2) chmod 2770 /srv/project — the leading 2 is SGID, so every file created inside is owned by group devs instead of the creator's private group, which is what stops the "I made it, only I can read it" problem; (3) chmod +t /srv/project adds the sticky bit → mode 3770, so a user can only delete files they own; (4) setfacl -d -m g:devs:rwx /srv/project if you also want inherited permissions, since SGID only inherits the group, not the mode. Verify: ls -ld /srv/project shows drwxrws--T; as bob, touch f && ls -l f shows group devs, and rm on alice's file is refused. Security note: hunt for the opposite mistake with find / -perm -4000 -type f 2>/dev/null — an unexpected SUID root binary is a privilege-escalation path.
/tmp. Read them out of ls -l as s and t replacing x.
ACLs are the escape hatch for the case standard permissions cannot express: one extra user or a second group on a file that already has an owner and a group. The exam's favourite trap is the mask — you grant rwx, getfacl still reports the user as effectively read-only, and nothing about the setfacl command was wrong.
getfacl file— display the full ACL for a file or directorysetfacl -m u:USERNAME:rwx file— grant a specific user rwx on a filesetfacl -m g:GROUPNAME:r-- file— grant a group read-only accesssetfacl -x u:USERNAME file— remove a user's ACL entrysetfacl -b file— remove all ACL entries (except base permissions)setfacl -m mask::r-- file— set the effective rights mask (limits maximum ACL permissions)
Default ACLs for Directories
setfacl -d -m u:USERNAME:rwx /dir— set a default ACL (-d): new files/directories created inside inherit this ACL- A
+sign inls -loutput indicates ACL entries beyond standard permissions are set - Filesystem must be mounted with ACL support; ext4 and XFS support ACLs natively; check with
tune2fs -l /dev/sdX | grep "Default mount"
getfacl to see effective permissions after mask application./srv/project belongs to group devs at mode 2770. An external auditor needs read access for a month — adding them to devs would grant write, and changing the group would break the team. Use an ACL: (1) setfacl -m u:auditor:rx /srv/project — rx, because without x they cannot traverse the directory; (2) setfacl -R -m u:auditor:r /srv/project for the existing files; (3) setfacl -d -m u:auditor:r /srv/project so files created later inherit the grant — without -d the auditor loses access to everything written next week. The mask trap: getfacl may show user:auditor:r-x followed by #effective:r--, meaning the mask is clipping it; setfacl -m mask::rx /srv/project lifts the ceiling. Verify: ls -ld /srv/project now ends in +, and sudo -u auditor ls /srv/project works while sudo -u auditor touch /srv/project/x is denied. Cleanup after the engagement: setfacl -R -x u:auditor /srv/project.
-m modifies, -x removes one entry, -b wipes them all, -d sets the default that new files inherit. The mask caps every named user and group (never the owner), so always read #effective: in getfacl before believing a grant. A trailing + in ls -l is the only hint an ACL exists.- User state lives in
/etc/passwd+/etc/shadow+/etc/group; mutate it throughuseradd,usermod,passwd,chage— never by hand-editing those files in production. - Standard mode (
rwxrwxrwx) covers 90% of cases; reach for special bits (SUIDfor privileged binaries,SGIDon directories for shared-group inheritance, sticky on/tmp) and POSIX ACLs (setfacl) only when the owner/group/other triad is too narrow. - Password policy is set through PAM (
/etc/pam.d/stacks) plus/etc/login.defs;umaskdrives default permissions on new files (file base 666, dir base 777, minus umask).
chmod math, special bits on dirs, ACL mask behaviour, and PAM stack order.
04
Storage & Filesystems
6 lessons · ~5 hours
fdisk/parted/gdisk for partitions, mkfs.* + mount + /etc/fstab for filesystems, pvcreate/vgcreate/lvcreate for LVM, mdadm for RAID, cryptsetup for LUKS, and mkswap/swapon for swap.
Picking a partitioning tool is really picking a partition table, and that choice is forced by two facts about the machine: disk size and firmware. Over 2 TB, or booting UEFI, means GPT — and GPT is what gdisk and parted are for. Reach for fdisk only on small legacy MBR disks.
- MBR (Master Boot Record) — legacy; max 4 primary partitions or 3 primary + 1 extended (with logical partitions); max disk size 2 TB
- GPT (GUID Partition Table) — modern; supports up to 128 partitions per disk; required for disks over 2 TB; used with UEFI
fdisk /dev/sdX— interactive MBR partition editor;n(new),d(delete),t(change type),w(write),q(quit)gdisk /dev/sdX— interactive GPT partition editor (same command letters as fdisk)parted /dev/sdX— supports both MBR and GPT; non-interactive mode:parted /dev/sdX mklabel gptlsblk— list block devices and partition layout;blkid— show UUIDs and filesystem types
fdisk for MBR disks up to 2 TB. For GPT or disks larger than 2 TB, use gdisk or parted. The exam will present scenarios requiring you to identify the correct partitioning tool based on disk size and UEFI vs BIOS context.fdisk, but the resulting partition tops out around 2 TB. Why: MBR addresses sectors with 32 bits — at 512-byte sectors that ceiling is 2 TiB, and the rest of the disk is simply unaddressable. Redo it as GPT: (1) lsblk and parted /dev/sdb print to confirm the current label reads msdos; (2) parted /dev/sdb mklabel gpt — this discards the existing table, so confirm the disk is empty first; (3) parted -a optimal /dev/sdb mkpart primary xfs 0% 100%, where -a optimal aligns the start to the device's I/O boundary instead of an arbitrary sector; (4) partprobe /dev/sdb so the kernel re-reads the table without a reboot; (5) lsblk /dev/sdb now shows a ~4 TB sdb1, and blkid gives you the UUID for /etc/fstab. The tell in exam wording: "the server boots UEFI" or any capacity above 2 TB rules fdisk out before you read the answers.
lsblk shows the layout, blkid gives the UUIDs you will need in /etc/fstab, and partprobe re-reads the table without a reboot.
A partition is just reserved space until you put a filesystem on it and attach it to the tree. Two details carry most of the exam weight here: mount by UUID, never by /dev/sdb1 (device names shuffle when disks are added), and test every /etc/fstab edit with mount -a before you reboot — a typo there is an unbootable machine.
mkfs.ext4 /dev/sdX1— format a partition as ext4;-L LABELto add a volume labelmkfs.xfs /dev/sdX1— format as XFS (default on RHEL 7+)tune2fs -L NEWLABEL /dev/sdX1— change ext2/3/4 volume label;tune2fs -c 50— set max mount count before fsckxfs_admin -L NEWLABEL /dev/sdX1— change XFS volume labele2fsck -f /dev/sdX1— check and repair an ext filesystem (must be unmounted)xfs_repair /dev/sdX1— check and repair an XFS filesystem (must be unmounted)
Mounting & /etc/fstab
mount /dev/sdX1 /mnt/data— mount temporarily;mount -o ro /dev/sdX1 /mnt— mount read-onlyumount /mnt/data— unmount; uselsof /mnt/dataorfuser /mnt/dataif "device busy"/etc/fstabformat:UUID=... /mountpoint fstype options dump pass- Use UUIDs (from
blkid) in fstab, not device names — device names can change across reboots - Common mount options:
defaults,noatime,nosuid,noexec,ro mount -a— mount all entries in fstab that aren't already mounted (tests fstab syntax)
e2fsck first). The exam will present a scenario asking about resizing, and the correct answer depends on the filesystem type./, so /var/log moves to its own disk with noexec,nosuid,nodev hardening. Walk: (1) mkfs.xfs -L varlog /dev/sdb1; (2) mount it somewhere neutral first — mount /dev/sdb1 /mnt — and copy the existing data across with rsync -aXS /var/log/ /mnt/ (-X preserves SELinux and extended attributes, which a plain cp would drop and SELinux would then block); (3) umount /mnt; (4) blkid /dev/sdb1 → add UUID=<value> /var/log xfs defaults,noexec,nosuid,nodev 0 0; (5) mount -a, then findmnt /var/log to confirm the options actually applied; (6) restorecon -Rv /var/log and systemctl restart rsyslog. If umount says "target is busy": lsof +D /var/log or fuser -vm /var/log names the process holding it — stop that, do not reach for umount -l, which hides the problem rather than fixing it.
mount -a after editing /etc/fstab, and always use UUID=. XFS grows online but never shrinks; ext4 shrinks only offline and only after e2fsck -f. "Device busy" is a job for lsof/fuser, not -l.LVM exists so "the disk is full" stops being an outage. It inserts a pool between disks and filesystems, which means capacity can be added while the service keeps running. The exam tests the order of operations more than the commands: growing goes LV first then filesystem, and shrinking goes filesystem first then LV — get it backwards and you truncate live data.
- PV (Physical Volume) — raw disks or partitions initialized for LVM use
- VG (Volume Group) — pool of storage combining one or more PVs
- LV (Logical Volume) — virtual partitions carved from a VG; flexible and resizable
LVM Commands
pvcreate /dev/sdX— initialize a physical volumevgcreate myvg /dev/sdX— create a volume group;vgextend myvg /dev/sdY— add a PV to VGlvcreate -L 20G -n mylv myvg— create a 20G logical volumelvextend -L +10G /dev/myvg/mylv— increase LV size by 10G- After
lvextendon ext4:resize2fs /dev/myvg/mylvto grow the filesystem - After
lvextendon XFS:xfs_growfs /mountpointto grow the filesystem (XFS grow is online) lvreduce -L -5G /dev/myvg/mylv— decrease LV size (ext4 only, must unmount first)pvs/vgs/lvs— brief display of PV/VG/LV informationpvdisplay/vgdisplay/lvdisplay— detailed outputlvcreate -L 5G -s -n snap /dev/myvg/mylv— create a snapshot of an LV
e2fsck -f → resize2fs to new smaller size → lvreduce. XFS cannot be shrunk at all. The exam frequently tests this order-of-operations for both grow and shrink scenarios.pvcreate /dev/sdb /dev/sdc; (2) vgcreate datavg /dev/sdb /dev/sdc; (3) lvcreate -L 10G -n datalv datavg; (4) mkfs.xfs /dev/datavg/datalv; (5) mkdir /data && mount /dev/datavg/datalv /data; (6) get the UUID with blkid /dev/datavg/datalv, then add to /etc/fstab: UUID=<value> /data xfs defaults 0 0; (7) mount -a to test. To extend: lvextend -r -L +5G /dev/datavg/datalv — the -r flag grows the LV and the XFS filesystem in one step. Confirm with df -h /data showing ~15G total.
pvcreate → vgcreate → lvcreate going down, and lvextend -r (or lvextend then resize2fs/xfs_growfs) going up. If vgs shows no free extents, the answer is vgextend with a new PV first. Shrinking is ext4-only, offline, filesystem before LV.
RAID questions are arithmetic wearing a costume: given N disks and a level, how much usable capacity, and how many failures survived? Learn the two columns (usable = N−1 for RAID 5, N−2 for RAID 6, N/2 for RAID 1 and 10) and the rest is mdadm syntax plus one habit — checking /proc/mdstat before you believe an array is healthy.
- RAID 0 (striping) — performance, no redundancy; min 2 disks; if one disk fails, all data is lost
- RAID 1 (mirroring) — full redundancy, 50% usable capacity; min 2 disks; can lose N-1 disks
- RAID 5 (distributed parity) — min 3 disks; can lose 1 disk; usable capacity = (N-1) disks
- RAID 6 (double parity) — min 4 disks; can lose 2 disks; usable = (N-2) disks
- RAID 10 (stripe of mirrors) — min 4 disks; high performance + redundancy; can lose 1 disk per mirror set
mdadm Commands
mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sd{b,c,d}— create RAID 5cat /proc/mdstat— view RAID status and rebuild progressmdadm --detail /dev/md0— detailed RAID array infomdadm --add /dev/md0 /dev/sde— add a hot spare or replacement drivemdadm --fail /dev/md0 /dev/sdb— mark a drive as failed;mdadm --remove /dev/md0 /dev/sdb— remove it/etc/mdadm.confor/etc/mdadm/mdadm.conf— persist RAID config across reboots
cat /proc/mdstat shows [3/2] [U_U] — three members, two up, the middle one gone; (2) mdadm --detail /dev/md0 names the faulty device as /dev/sdc; (3) if it has not already been ejected, mdadm --fail /dev/md0 /dev/sdc then mdadm --remove /dev/md0 /dev/sdc; (4) swap the disk, copy the partition layout onto the replacement with sfdisk -d /dev/sdb | sfdisk /dev/sdc so the new member matches; (5) mdadm --add /dev/md0 /dev/sdc starts the rebuild — cat /proc/mdstat shows a progress bar and an ETA; (6) once it reads [UUU], persist the config with mdadm --detail --scan >> /etc/mdadm.conf and rebuild the initramfs so the array assembles at boot. The exam's real question: during that rebuild the array has no remaining redundancy — a second failure loses everything, which is precisely the argument for RAID 6 or RAID 10 on large disks.
/proc/mdstat is the status view; --fail → --remove → --add is the replacement sequence, and the config must be written to mdadm.conf to survive a reboot.Swap is the pressure-relief valve: it buys the kernel time to reclaim memory instead of invoking the OOM killer. Two practical points get tested — a swap file can be added to a running system with no repartitioning, and swappiness tunes how eagerly the kernel uses it, which matters on database hosts where paging is worse than a smaller cache.
mkswap /dev/sdX2— format a partition as swap spaceswapon /dev/sdX2— activate swap;swapoff /dev/sdX2— deactivateswapon -s(orswapon --show) — list active swap devices with priority and usage- Persist in
/etc/fstab:UUID=... none swap sw 0 0 - Swap file:
fallocate -l 2G /swapfile→chmod 600 /swapfile→mkswap /swapfile→swapon /swapfile cat /proc/sys/vm/swappiness— view swappiness (default 60); lower values reduce swap aggressivenesssysctl vm.swappiness=10— set swappiness temporarily; persist in/etc/sysctl.d/99-swap.conf
0600 and owned by root — swapon refuses or warns otherwise, because a world-readable swap file exposes whatever memory was paged out. Create it with fallocate, or with dd if the filesystem does not support fallocate.free -h and swapon --show confirm swap is 0; (2) fallocate -l 4G /swapfile; (3) chmod 600 /swapfile — skip this and swapon warns about insecure permissions; (4) mkswap /swapfile; (5) swapon /swapfile, then swapon --show lists it immediately; (6) persist with /swapfile none swap sw 0 0 in /etc/fstab; (7) since this is a database host, lower the paging pressure: echo 'vm.swappiness=10' > /etc/sysctl.d/99-swap.conf && sysctl --system. Confirm the fix worked: journalctl -k | grep -i "out of memory" stays quiet through the next batch window.
fallocate → chmod 600 → mkswap → swapon, plus an fstab line to survive reboot. swapon --show and free -h verify it; vm.swappiness (default 60) is lowered toward 10 on latency-sensitive hosts and persisted under /etc/sysctl.d/.
LUKS protects data at rest — a stolen or RMA'd disk is unreadable without a key. The mental model is a two-step device stack: the raw partition holds a LUKS header, and unlocking it produces a /dev/mapper/ device that you then format and mount like anything else. Everything the exam asks follows from that layering, including why the initramfs has to be rebuilt.
cryptsetup luksFormat /dev/sdX1— initialize a LUKS encrypted container (destroys data)cryptsetup luksOpen /dev/sdX1 cryptdata— unlock the container; creates/dev/mapper/cryptdatamkfs.ext4 /dev/mapper/cryptdata— create filesystem on the unlocked devicemount /dev/mapper/cryptdata /mnt/secure— mount the encrypted filesystemcryptsetup luksClose cryptdata— lock/close the containercryptsetup luksDump /dev/sdX1— display LUKS header info (slots used, cipher)/etc/crypttab— maps LUKS devices to mapper names for auto-unlock at boot (with keyfile or passphrase prompt)
/etc/crypttab and /etc/fstab, you must rebuild the initramfs (dracut --force on RHEL) so the initramfs includes the cryptsetup tools needed to unlock the device early in the boot process./secure encrypted at rest, but the host must reboot without someone typing a passphrase. Build the stack bottom-up: (1) cryptsetup luksFormat /dev/sdb1 and set a strong passphrase — this destroys any existing data; (2) add a keyfile so boot can be unattended: dd if=/dev/urandom of=/root/.luks-secure bs=512 count=8, chmod 400 /root/.luks-secure, cryptsetup luksAddKey /dev/sdb1 /root/.luks-secure — LUKS has 8 key slots, so passphrase and keyfile coexist; (3) cryptsetup luksOpen /dev/sdb1 securedata; (4) mkfs.xfs /dev/mapper/securedata — note you format the mapper device, never /dev/sdb1; (5) /etc/crypttab: securedata UUID=<luks-uuid> /root/.luks-secure luks, where the UUID comes from cryptsetup luksUUID /dev/sdb1; (6) /etc/fstab: /dev/mapper/securedata /secure xfs defaults,_netdev 0 0; (7) dracut --force so the crypt tooling is present early; (8) reboot and confirm with lsblk that sdb1 carries a crypt child that is mounted. Caveat worth stating: a keyfile on the same machine protects against a stolen disk, not against a stolen server — that is the trade-off the scenario is really about.
luksFormat → luksOpen → mkfs on /dev/mapper/<name> → mount. /etc/crypttab handles the unlock and /etc/fstab the mount, in that order, and any crypt change must be followed by an initramfs rebuild.- Provisioning order: partition (
fdisk/gdisk/parted) →mkfs.xfs/mkfs.ext4→ mount +/etc/fstabwith UUID;fdiskonly for ≤2 TB MBR disks,gdisk/partedfor GPT. - LVM =
pvcreate → vgcreate → lvcreate; grow online withlvextend -r; shrink only ext4 (offline, afterresize2fs) — XFS cannot shrink at all. - RAID with mdadm for software arrays (RAID 1 / 5 / 6 / 10 — capacity and resiliency tables matter); LUKS for at-rest encryption (
cryptsetup luksFormat+/etc/crypttab); always rebuild initramfs after a crypt change.
05
Networking
6 lessons · ~5 hours
ip suite, lost on reboot), persistent (NetworkManager via nmcli/nmtui or config files), and filter (firewalld, ufw, raw iptables). Module 05 maps each layer to its command set, covers DNS via /etc/resolv.conf + /etc/nsswitch.conf, and walks SSH key-based auth — the most-tested authentication topic on the exam.
ip is the tool for seeing and testing, not for configuring. Everything it writes lives in the kernel and dies at the next reboot — which is exactly what makes it safe for a temporary route while you diagnose, and exactly why the answer to "make this permanent" is never ip addr add.
- The
ipcommand (from theiproute2package) replaces deprecated tools:ifconfig,route,arp,netstat ip addr show(orip a) — display IP addresses on all interfacesip addr add 192.168.1.10/24 dev eth0— assign an IP address (non-persistent)ip addr del 192.168.1.10/24 dev eth0— remove an IP addressip link show— display network interface state;ip link set eth0 up/down— bring interface up/downip route show(orip r) — display the routing tableip route add default via 192.168.1.1— add a default gateway (non-persistent)ip route add 10.0.0.0/8 via 192.168.1.254 dev eth0— add a static routeip neigh show— display the ARP/neighbor cache
ip command are not persistent across reboots. For persistent configuration, use NetworkManager (nmcli) or edit interface config files in /etc/NetworkManager/system-connections/.ip link show — is the interface even up? state DOWN or missing LOWER_UP means a cable, a VLAN, or a driver problem, and nothing above it matters; (2) ip -br addr show — the brief form gives one line per interface; an address of 169.254.x.x means DHCP never answered; (3) ip route get 10.20.0.5 — better than reading the whole table, it tells you which interface and gateway the kernel would actually use for that destination, which is how you catch a missing static route; (4) ip neigh show — an entry stuck in FAILED for the gateway means ARP is unanswered, so the problem is layer 2, not routing. Test a hypothesis without committing to it: ip route add 10.20.0.0/16 via 192.168.1.254, confirm connectivity, then make it permanent with nmcli connection modify … +ipv4.routes — the temporary route disappears on reboot either way, so a wrong guess cannot strand the host.
ip a / ip r / ip link / ip neigh replace ifconfig, route, and arp; ip route get <dest> answers "which path would this actually take". Nothing ip writes survives a reboot — persistence belongs to NetworkManager.
NetworkManager owns the persistent side: profiles on disk that get reapplied at every boot. The one behaviour to internalise is that nmcli connection modify only edits the stored profile — nothing changes on the wire until you bring the connection back up, which is why a "correct" config can appear to do nothing.
nmcli connection show— list all configured network connectionsnmcli connection show --active— list only active connectionsnmcli connection up CONNECTION_NAME— activate a connectionnmcli connection down CONNECTION_NAME— deactivate a connectionnmcli connection modify CONNECTION_NAME ipv4.addresses 192.168.1.50/24— set static IPnmcli connection modify CONNECTION_NAME ipv4.gateway 192.168.1.1— set gatewaynmcli connection modify CONNECTION_NAME ipv4.dns "8.8.8.8 8.8.4.4"— set DNS serversnmcli connection modify CONNECTION_NAME ipv4.method manual— switch from DHCP to staticnmcli device wifi list— list available Wi-Fi networksnmtui— text-based interactive UI for NetworkManager; useful when no GUI is available
Config File Location
- Connection profiles:
/etc/NetworkManager/system-connections/(keyfile format in RHEL 8+) - Restart networking:
nmcli connection reloadthennmcli connection up CONNECTION
ens3 with static IP 10.0.1.50/24, gateway 10.0.1.1, DNS 1.1.1.1, and set hostname to server2.lab.local. Walk: (1) hostnamectl set-hostname server2.lab.local; (2) nmcli con mod ens3 ipv4.addresses 10.0.1.50/24 ipv4.gateway 10.0.1.1 ipv4.dns 1.1.1.1 ipv4.method manual; (3) nmcli con up ens3 to activate — the mod writes to disk but changes are only applied when the connection is brought up; (4) verify: ip a show ens3 (IP assigned), ip route show default (gateway), cat /etc/resolv.conf (DNS), hostname (hostname). Critical: anything set via ip addr add instead of nmcli vanishes at the next reboot.
nmcli con mod writes the profile, nmcli con up applies it — both steps, every time. Switching off DHCP needs ipv4.method manual alongside the address, and profiles live in /etc/NetworkManager/system-connections/. On a headless box with no muscle memory for the syntax, nmtui does the same job.
Name resolution is a chain of three files and the exam probes each link separately: /etc/nsswitch.conf decides which sources are consulted and in what order, /etc/hosts is the static source, and /etc/resolv.conf names the DNS servers. "Ping works by IP but not by name" is always a failure somewhere in that chain.
/etc/hosts— static hostname-to-IP mappings; checked before DNS by default/etc/resolv.conf— specifies DNS servers (nameserver 8.8.8.8) and search domains (search example.com)/etc/nsswitch.conf— controls lookup order; thehosts:line (typicallyfiles dns) determines whether/etc/hostsis checked before DNSdig DOMAIN— detailed DNS query;dig @8.8.8.8 DOMAIN— query specific server;dig -x IP— reverse lookupnslookup DOMAIN— simple DNS query (older tool); interactive mode:nslookupthenserver 8.8.8.8host DOMAIN— quick DNS lookup;host IP— reverse DNSsystemd-resolve --status— show DNS configuration used by systemd-resolved
/etc/nsswitch.conf, not /etc/resolv.conf. If the exam asks how to make /etc/hosts take precedence over DNS, the answer is to check the hosts: line in nsswitch.conf — it must list files before dns.ping app.lab.local reports Name or service not known, yet dig app.lab.local returns the right A record. That contradiction is the diagnosis: dig talks to a DNS server directly, while ping goes through the NSS chain — so DNS is fine and the chain is broken. Walk it: (1) grep ^hosts /etc/nsswitch.conf — if dns is missing from the line, no ordinary program will ever consult DNS no matter how healthy it is; (2) cat /etc/resolv.conf — on a systemd-resolved host this should point at 127.0.0.53, and the real upstream servers are shown by resolvectl status; (3) if the file was hand-edited, NetworkManager overwrites it on the next connection change — set the servers with nmcli con mod ens3 ipv4.dns "10.0.1.1" instead; (4) short names failing while FQDNs work points at a missing search lab.local line, set via ipv4.dns-search. Prove the fix through the same path the application uses: getent hosts app.lab.local — that goes through NSS exactly as ping does, unlike dig.
dig and host query DNS directly; getent hosts tests the resolution path applications actually take. Order lives in /etc/nsswitch.conf, servers in /etc/resolv.conf — and on a NetworkManager host, that file is generated, so edit the connection profile instead.Three front ends, one kernel underneath. What unites them on the exam is first match wins and the split between the running ruleset and the saved one — the two facts behind almost every firewall question, and behind the classic disaster of locking yourself out of a remote host with a rule you were about to fix.
- Three main chains: INPUT (packets destined for the local system), OUTPUT (packets originating from local system), FORWARD (routed packets)
iptables -L -n -v --line-numbers— list all rules with line numbers and packet countsiptables -A INPUT -p tcp --dport 22 -j ACCEPT— append rule to allow SSHiptables -I INPUT 1 -p tcp --dport 80 -j ACCEPT— insert rule at position 1iptables -D INPUT 3— delete rule by line numberiptables -A INPUT -s 10.0.0.5 -j DROP— drop all traffic from a source IPiptables -P INPUT DROP— set default policy to DROP (deny-all baseline)- Persist:
iptables-save > /etc/iptables/rules.v4; restore:iptables-restore < /etc/iptables/rules.v4
ufw & firewalld
ufw enable— enable the UFW firewall (Ubuntu/Debian);ufw allow 22/tcp— allow SSH;ufw deny 23— block telnetfirewalld— zone-based firewall on RHEL/Fedora;firewall-cmd --list-all— show active zone configfirewall-cmd --permanent --add-service=http— allow HTTP;--reloadapplies permanent changes
--line-numbers and -I (insert) for targeted rule placement.firewall-cmd --get-active-zones and firewall-cmd --list-all — establish which zone the interface is in before changing anything, since a rule added to the wrong zone does nothing at all; (2) firewall-cmd --add-port=8443/tcp — no --permanent, so it applies immediately and evaporates on reload; test the app now; (3) once it works, commit it: firewall-cmd --permanent --add-port=8443/tcp followed by firewall-cmd --reload. That two-step order is the safety net — the runtime rule proves the change is right, and if a permanent edit ever locks you out, a reboot reverts to the last known-good permanent set. Verify from outside: nc -zv server 8443 from another host, not from the server itself, where loopback traffic bypasses the filter entirely. Same job on Ubuntu: ufw allow 8443/tcp, then ufw status numbered.
--permanent plus --reload is what persists. iptables evaluates top-down, first match wins, so -I (insert) places a rule ahead of a broad DROP where -A (append) would land uselessly behind it. Always test a port from another machine.
Key-based SSH is the one hardening step that makes a server both safer and easier to automate. The detail that trips people up is not the cryptography — it is the permissions: sshd silently refuses a key if ~/.ssh or authorized_keys is group-writable, and the client-side error says nothing useful about why.
ssh-keygen -t ed25519 -C "comment"— generate an Ed25519 key pair (recommended over RSA for new keys)ssh-keygen -t rsa -b 4096— generate 4096-bit RSA key pair- Private key:
~/.ssh/id_ed25519(protect withchmod 600); Public key:~/.ssh/id_ed25519.pub ssh-copy-id user@host— copy public key to remote host's~/.ssh/authorized_keys~/.ssh/authorized_keyson the server: must bechmod 600and owned by the user~/.ssh/directory: must bechmod 700and owned by the user
sshd_config Hardening
PasswordAuthentication no— disable password auth (force key-based only)PermitRootLogin no— prevent direct root login via SSHAllowUsers alice bob— whitelist specific users; all others deniedPort 2222— change SSH port (security through obscurity; adjust firewall accordingly)ClientAliveInterval 300/ClientAliveCountMax 2— disconnect idle sessions after 10 minutes- After editing
/etc/ssh/sshd_config:systemctl restart sshd
ssh-keygen -t ed25519 -C "ops@laptop"; (2) ssh-copy-id ops@server while password auth is still enabled; (3) open a second terminal and confirm ssh ops@server now succeeds with no password prompt — keep the first session open as your lifeline; (4) only now edit /etc/ssh/sshd_config: PasswordAuthentication no, PermitRootLogin no; (5) sshd -t validates the syntax — a typo here plus a restart is how servers become unreachable; (6) systemctl restart sshd, then test from the second terminal before you close the first. When the key is ignored and it falls back to a password prompt, the cause is almost always permissions: chmod 700 ~/.ssh, chmod 600 ~/.ssh/authorized_keys, correct ownership — and journalctl -u sshd on the server states it plainly ("Authentication refused: bad ownership or modes"), while ssh -v on the client only shows the key being offered and declined. On RHEL, a home directory relabelled or restored from backup may also need restorecon -R -v ~/.ssh.
sshd -t before restarting. ~/.ssh is 700 and authorized_keys is 600 — anything looser is silently ignored. ed25519 for new keys; the server's side of the story is in journalctl -u sshd.Troubleshooting is not a list of commands, it is an order of elimination: is the interface up, does the name resolve, does the route exist, is the port listening, does the packet arrive. Each rung has one tool, and picking the right rung first is what separates a two-minute diagnosis from an afternoon of guessing.
ping -c 4 HOST— test basic connectivity;ping -I eth0 HOST— send from specific interfacetraceroute HOST— show path packets take (hop-by-hop);tracepath HOST— similar but no root requiredss -tulnp— list listening TCP (t) and UDP (u) sockets with process names; replacesnetstat -tulnptcpdump -i eth0 port 80— capture HTTP traffic;tcpdump -i eth0 -w capture.pcap— write to filetcpdump -i eth0 host 10.0.0.5 and tcp— filter by host and protocolcurl -I https://example.com— fetch HTTP headers only;curl -v URL— verbose output showing TLS handshakewget -O /dev/null URL— test download speed;wget --spider URL— check URL without downloadingnc -zv HOST PORT— test if a TCP port is open (netcat);nc -l 8080— listen on port 8080
ss -tulnp is the modern replacement for netstat -tulnp. The flags: -t TCP, -u UDP, -l listening only, -n show numbers not names, -p show process info. The Linux+ exam may test either command.https://api.lab.local. Climb the ladder, stopping at the first rung that fails: (1) Does the name resolve? getent hosts api.lab.local — if not, it is a DNS problem and nothing below matters; (2) Is anything listening? on the server, ss -tulnp | grep 443 — 127.0.0.1:443 instead of 0.0.0.0:443 means the service is bound to loopback only, which looks healthy locally and is unreachable from anywhere else; (3) Does the port answer from outside? nc -zv api.lab.local 443 from a client — refused means nothing is listening, while a timeout points at a firewall silently dropping instead of rejecting; (4) Do packets even arrive? tcpdump -i any -n port 443 on the server while the client retries — SYNs with no SYN/ACK back means the host received them and the filter dropped them; total silence means they never got there, so look at routing or an upstream firewall; (5) Is it TLS rather than the network? curl -vI https://api.lab.local shows exactly where the handshake stops. Diagnosis: a SYN arriving with no reply, plus 0.0.0.0:443 in ss, is firewalld — check the zone.
ss -tulnp answers "is it listening, and on which address"; nc -zv distinguishes refused (nothing there) from timeout (dropped); tcpdump proves whether packets arrive at all; curl -v separates a TLS failure from a network one. Work the ladder in order rather than guessing.- Inspect with
ip a,ip r,ss -tulnp(the modernnetstat); change persistently withnmcli/nmtui—ip-suite changes vanish on reboot. - Resolver order is set by
/etc/nsswitch.conf, NOT/etc/resolv.conf; firewalls follow first-match:firewallduses zones,ufwis the Ubuntu-friendly wrapper,iptablesis the raw kernel layer underneath. - SSH hardening hits in three places:
/etc/ssh/sshd_config(PermitRootLogin no,PasswordAuthentication no),~/.ssh/authorized_keys(key-based auth), andssh-keygen -t ed25519for modern key pairs.
ip vs nmcli, nsswitch order, firewalld zones, and SSH key auth.
06
Security & Hardening
6 lessons · ~5 hours
audit2allow/audit2why when something denies. Module 06 covers all three areas with the production patterns.
SELinux is the answer to "the permissions are right and it still won't work". It adds a second, label-based check on top of Unix modes, so a process can own a file and still be refused. The exam wants three reflexes: read the denial in the audit log, know that chcon is temporary while semanage fcontext is permanent, and never answer a scenario with "disable SELinux".
- Enforcing — SELinux policy is enforced; violations are blocked and logged
- Permissive — violations are logged but NOT blocked; useful for troubleshooting and policy development
- Disabled — SELinux is completely turned off; requires reboot + relabeling to re-enable
getenforce— display current mode (Enforcing/Permissive/Disabled)setenforce 0— switch to Permissive (temporary, survives only until reboot)setenforce 1— switch to Enforcing (temporary)- Persistent mode: edit
/etc/selinux/config→ setSELINUX=enforcing/permissive/disabled
File Context Management
ls -Z /var/www/html/— show SELinux file context labelsrestorecon -Rv /var/www/html/— restore default SELinux contexts recursively (fixes "wrong context" denials)chcon -t httpd_sys_content_t /new/file— change file context temporarily (overridden by restorecon)semanage fcontext -a -t httpd_sys_content_t "/newpath(/.*)?"— add a persistent context rulerestorecon -Rv /newpath— apply the newly added context rulegetsebool -a | grep httpd— list all SELinux booleans related to httpdsetsebool -P httpd_can_network_connect on— enable a boolean persistently (-P)
Analyzing Denials
audit2why < /var/log/audit/audit.log— explain why actions were deniedaudit2allow -M mypolicy < /var/log/audit/audit.log— generate a custom allow policy module from denialssemodule -i mypolicy.pp— install a custom SELinux policy module
chcon to change a file context, the change is temporary. A subsequent restorecon will reset it back to the default policy label. The correct permanent approach is semanage fcontext followed by restorecon. The exam tests this two-step workflow./data/web/ even though the Unix permissions are correct. Walk: (1) getenforce → Enforcing; (2) grep AVC /var/log/audit/audit.log | tail -5 → shows a denial for httpd_t trying to read /data/web; (3) audit2why < /var/log/audit/audit.log explains the cause: wrong SELinux file context; (4) fix permanently: semanage fcontext -a -t httpd_sys_content_t "/data/web(/.*)?"; (5) restorecon -Rv /data/web/ applies the rule; (6) reload nginx and confirm the 403 is gone. If you had used chcon -t httpd_sys_content_t /data/web instead, the next restorecon run would silently revert it — permanent fix requires semanage fcontext.
/var/log/audit/audit.log — find them with ausearch -m AVC -ts recent and explain them with audit2why. Wrong label → semanage fcontext then restorecon; blocked behaviour (outbound connections, non-standard ports) → a boolean, setsebool -P. Reach for audit2allow last, and read what it generates before installing it.
AppArmor solves the same problem as SELinux on the Debian side, with a simpler model: profiles list paths a program may touch, rather than labels attached to inodes. Translate the vocabulary once — complain is permissive, enforce is enforcing — and your SELinux troubleshooting instincts carry straight across.
- AppArmor is the MAC (Mandatory Access Control) framework used on Debian/Ubuntu systems (vs SELinux on RHEL)
aa-status— show AppArmor status: profiles loaded, enforcement mode per profile- Profile modes: enforce (blocks violations), complain (logs but does not block — like SELinux permissive)
aa-enforce /etc/apparmor.d/usr.sbin.nginx— put a profile into enforce modeaa-complain /etc/apparmor.d/usr.sbin.nginx— put into complain (logging) mode- Profile files live in:
/etc/apparmor.d/ apparmor_parser -r /etc/apparmor.d/PROFILE— reload a profile after editingaa-genprof /path/to/binary— generate a new profile interactively by watching program behavior
/srv/sites/app and immediately returns 403 despite correct ownership. Walk: (1) aa-status confirms /usr/sbin/nginx is loaded in enforce mode; (2) dmesg | grep -i apparmor or journalctl -k | grep DENIED shows apparmor="DENIED" operation="open" name="/srv/sites/app/index.html" — the path is simply not in the profile; (3) confirm the diagnosis without editing anything: aa-complain /usr/sbin/nginx, retry, and the page loads while the denial is still logged; (4) fix it properly by adding /srv/sites/app/** r, to /etc/apparmor.d/usr.sbin.nginx (or to the local/ include so a package update does not overwrite it); (5) apparmor_parser -r /etc/apparmor.d/usr.sbin.nginx reloads it; (6) aa-enforce /usr/sbin/nginx puts the profile back under enforcement and the site still works. The parallel to hold onto: complain mode is the diagnostic step, exactly like setenforce 0 — and in both worlds leaving it there is the wrong answer.
/etc/apparmor.d/; aa-status shows modes, aa-complain diagnoses, apparmor_parser -r reloads after an edit, aa-enforce restores enforcement.
GPG covers two different guarantees and the exam expects you to keep them apart: encryption uses the recipient's public key so only they can read it, while signing uses your private key so anyone can prove it came from you unmodified. Package repositories rely on the second one, which is why gpgcheck=1 belongs in this lesson too.
gpg --gen-key— generate a new GPG key pair interactivelygpg --list-keys— list all keys in the public keyringgpg --export -a "User Name" > public.key— export public key to ASCII-armored filegpg --import public.key— import a public key from a filegpg --keyserver keyserver.ubuntu.com --recv-keys KEY_ID— download a key from a keyserver
Encryption, Signing & Verification
gpg --encrypt -r "Recipient" file— encrypt file for recipient (producesfile.gpg)gpg --decrypt file.gpg > file— decrypt a filegpg --sign file— create a signed version of a file (embedded signature)gpg --detach-sign file— create a separatefile.sigsignature filegpg --verify file.sig file— verify a detached signature- RPM uses GPG to sign packages —
rpm -K package.rpmverifies the package signature
gpg --sign --encrypt). If a question says "prove it has not been altered", the answer is a signature, not encryption.agent-3.2.rpm plus a detached signature and a public key. Verify before you trust: (1) gpg --import vendor-public.key, then gpg --list-keys and check the fingerprint against the one published on the vendor's site over HTTPS — importing a key that arrived with the file proves nothing on its own; (2) gpg --verify agent-3.2.rpm.sig agent-3.2.rpm → "Good signature from Vendor Ops"; a BAD signature means the file is corrupt or tampered with, and stop there; (3) for RPM specifically, wire it into the package manager instead of checking by hand: rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-vendor, then rpm -K agent-3.2.rpm → digests signatures OK; (4) keep gpgcheck=1 in the .repo file so every future update from that repo is verified automatically — turning it off to "make the install work" is the wrong fix for an unimported key. The other direction: to send a config file containing secrets to that vendor, gpg --encrypt -r "Vendor Ops" config.yml — encrypted with their public key, so only their private key opens it.
gpg --verify sig file. For packages, import the vendor key and keep gpgcheck=1; rpm -K checks a single file.
Hardening on this exam is mostly one idea applied repeatedly: grant the narrowest privilege that still lets the job get done. In practice that means sudoers entries scoped to specific commands rather than blanket ALL, resource limits that stop one runaway process taking the host down, and lockout policies on failed logins.
/etc/pam.d/— service-specific PAM configuration;system-authandpassword-authare key files on RHELpam_pwquality.so— enforces password complexity rules configured in/etc/security/pwquality.conffaillock— PAM module that locks accounts after N failed login attempts; check withfaillock --user USERNAME; reset withfaillock --user USERNAME --reset/etc/security/limits.conf— set per-user/group resource limits:nofile(open files),nproc(processes),memlock(locked memory)ulimit -n— show current shell's open file descriptor limit;ulimit -n 65536— set for current session
sudoers Configuration
- Always edit sudoers with
visudo— validates syntax before saving, preventing lockouts - Format:
USER HOST=(RUNAS) COMMANDS— e.g.,alice ALL=(ALL) ALL - Group syntax:
%admins ALL=(ALL) ALL - NOPASSWD:
bob ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart nginx - Restrict to specific commands:
carol ALL=(root) /usr/bin/dnf install, /usr/bin/dnf remove - Include drop-in files:
/etc/sudoers.d/— add separate files here rather than editing/etc/sudoersdirectly
systemctl restart myapp unattended. The lazy answer, deploy ALL=(ALL) NOPASSWD: ALL, hands out full root. Do it narrowly: (1) visudo -f /etc/sudoers.d/deploy — a drop-in file, so a distro upgrade to /etc/sudoers cannot clobber it, and visudo refuses to save a syntax error that would otherwise break sudo for everyone; (2) the rule: deploy ALL=(root) NOPASSWD: /usr/bin/systemctl restart myapp, /usr/bin/systemctl status myapp; (3) use absolute paths and no wildcard — systemctl restart * would let the user restart any unit, and an argument-less /usr/bin/systemctl would permit systemctl mask sshd too; (4) chmod 0440 /etc/sudoers.d/deploy, the mode sudo requires; (5) verify with sudo -l -U deploy, which lists exactly what is permitted, then confirm sudo -u deploy sudo systemctl restart sshd is refused. Pair it with a limit: deploy hard nproc 512 in /etc/security/limits.conf keeps a runaway build from fork-bombing the host — check the running value with ulimit -u.
visudo, always a drop-in under /etc/sudoers.d/, always absolute paths with explicit arguments — wildcards give away more than they look like they do. sudo -l -U user is how you audit the result; faillock and /etc/security/limits.conf cover lockouts and resource caps.
Logs answer "what happened"; the audit subsystem answers "who touched this file, when, and from where". They are separate systems with separate tools — journalctl and /var/log/secure on one side, auditctl and ausearch on the other — and the exam tests whether you reach for the right one for the question being asked.
/var/log/auth.log(Debian) or/var/log/secure(RHEL) — authentication events: logins, sudo usage, SSH attempts/var/log/messages(RHEL) or/var/log/syslog(Debian) — general system messagesjournalctl -u sshd --since "1 hour ago"— filter systemd journal by unit and timejournalctl _COMM=sshd— all journal entries from the sshd processjournalctl -p err— show only error-level messageslastb— list bad (failed) login attempts from/var/log/btmplast— list successful logins from/var/log/wtmpwho/w— currently logged-in users
auditd
auditd— the Linux audit daemon; writes security events to/var/log/audit/audit.logauditctl -w /etc/passwd -p wa -k passwd_changes— watch /etc/passwd for write and attribute changesausearch -k passwd_changes— search audit log by keyausearch -ua USERNAME— search audit events by useraureport --summary— summary of audit events by category- Persistent rules:
/etc/audit/rules.d/audit.rules
audit2allow generates allow rules from denial messages but always review the output — it may create overly permissive rules. Use the minimum necessary permissions and apply the principle of least privilege. The exam may test whether you know audit2why (explains denials) vs audit2allow (generates allow policy).svc-deploy is locked out most mornings and nobody knows why. Investigate: (1) faillock --user svc-deploy lists the failure times and the source addresses — a burst from one unfamiliar IP looks very different from three failures from the CI runner; (2) lastb | head -20 corroborates from /var/log/btmp; (3) narrow it in the journal: journalctl -u sshd --since "yesterday 22:00" --until "yesterday 23:59" | grep -i "failed\|invalid"; (4) if the source is your own automation, the cause is usually a stale key or an expired password rather than an attack — chage -l svc-deploy settles that in one line; (5) clear the lock with faillock --user svc-deploy --reset. Then make the next occurrence answerable: auditctl -w /etc/ssh/sshd_config -p wa -k sshd_cfg flags any write to the SSH config, ausearch -k sshd_cfg retrieves those events with the real user behind a sudo (the auid field, which the journal does not give you), and the same rule in /etc/audit/rules.d/audit.rules makes it survive a reboot.
/var/log/secure (RHEL) or /var/log/auth.log (Debian); last and lastb read the login databases. For file-level accountability use auditctl -w … -k key and ausearch -k key, persist the rule under /etc/audit/rules.d/, and remember auid is what identifies the human behind a sudo.- SELinux = label-based; modes are
enforcing/permissive/disabled;chconis temporary,semanage fcontext+restoreconis permanent; troubleshoot withausearch+audit2why/audit2allow. - AppArmor = path-based profiles in
/etc/apparmor.d/, modes areenforceandcomplain; manage withaa-enforce,aa-complain,aa-status. - Hardening = principle of least privilege everywhere — disable unused services (
systemctl mask), tightensshd_config, sign packages with GPG, and audit withauditd+ journalctl filters; correlate failed logins vialastband/var/log/secure.
07
Scripting, Containers & Troubleshooting
5 lessons · ~7 hours
grep/sed/awk pipelines), scheduling (cron, anacron, systemd timers), containers (Docker vs Podman, the daemonless distinction the exam loves), configuration management (Ansible, agentless, idempotent), and troubleshooting (the systematic flow: status → logs → kernel → resources). Module 07 walks each in production order.
Linux+ does not ask you to write elegant code — it shows you a short script and asks what it does, or which line is wrong. That makes three things worth memorising cold: the special variables ($?, $#, $@, $1), the test operators (-f vs -d, -eq vs =), and why unquoted variables break the moment a path contains a space.
- Always start with a shebang:
#!/bin/bash(or#!/usr/bin/env bashfor portability) - Make executable:
chmod +x script.sh; run with./script.shorbash script.sh - Variables:
NAME="Alice"(no spaces around=); reference with$NAMEor${NAME} - Command substitution:
DATE=$(date +%Y-%m-%d)— stores command output in variable - Special variables:
$?(exit code of last command),$#(number of arguments),$@(all arguments as separate strings),$0(script name),$1-$9(positional arguments)
Control Flow
- If/elif/else:
if [ "$VAR" = "value" ]; then ... elif [ condition ]; then ... else ... fi - For loop:
for FILE in /etc/*.conf; do echo "$FILE"; done - While loop:
while [ $COUNT -lt 10 ]; do ((COUNT++)); done - Until loop:
until ping -c1 HOST >/dev/null 2>&1; do sleep 5; done - Functions:
function check_service() { systemctl is-active "$1" || return 1; }
test Operators
- File tests:
-f(regular file),-d(directory),-e(exists),-r(readable),-w(writable),-x(executable),-s(non-empty) - String tests:
-z(zero length / empty),-n(non-zero length / not empty),=(equal),!=(not equal) - Integer comparison:
-eq,-ne,-lt,-le,-gt,-ge
$? must be checked immediately after the command it refers to — the very next command overwrites it. A common pattern: command; RC=$?; if [ $RC -ne 0 ]; then .... Also: set -e at the top of a script causes it to exit immediately on any non-zero return code.nginx and sshd are running, starts any that aren't, and logs results. Walk: Start with #!/bin/bash and set -e removed (we want the loop to continue on failure). Declare an array: SERVICES=(nginx sshd). Loop: for SVC in "${SERVICES[@]}"; do — always quote array expansions. Check status: if ! systemctl is-active --quiet "$SVC"; then — is-active --quiet exits 0 (active) or non-zero (not), so ! triggers on failure. Inside: systemctl start "$SVC" && echo "$(date '+%Y-%m-%d %T'): started $SVC" >> /var/log/svc-check.log. Close with fi; done. Schedule via cron: add */5 * * * * root /usr/local/bin/svc-check.sh to /etc/cron.d/svc-check.
"$SVC", "${ARR[@]}") — unquoted, a value with a space becomes two arguments. Test exit status with the command itself (if ! systemctl is-active --quiet …) rather than parsing its output, and capture $? on the very next line if you need it later. String comparison uses =, integers use -eq.
Every one of these tools does one thing to a stream of lines, and the skill being tested is composition: grep selects, awk and cut project columns, sed rewrites, sort and uniq aggregate. Read a pipeline left to right as a sentence and the exam's "what does this output?" questions stop being guesswork.
grep -E "pattern" file— extended regex search;-icase-insensitive;-rrecursive;-vinvert match;-lfilenames only;-ccount matchesgrep -P "\d{3}-\d{4}" file— Perl-compatible regex for complex patternssed 's/old/new/g' file— substitute all occurrences;-iflag edits file in-place;sed -n '5,10p' file— print lines 5–10awk '{print $1, $3}' file— print fields 1 and 3;awk -F: '{print $1}' /etc/passwd— use colon as delimiterawk '$3 > 1000 {print $1}' /etc/passwd— conditional: print username if UID > 1000cut -d: -f1,3 /etc/passwd— cut fields 1 and 3 from colon-delimited filesort -k3 -n file— sort numerically by field 3;sort -rreverse order;sort -uunique linesuniq -c— count duplicate consecutive lines; always pipe throughsortfirsttr 'a-z' 'A-Z'— translate lowercase to uppercase;tr -d '\r'— remove carriage returnswc -l file— count lines;wc -wwords;wc -cbyteshead -n 20 file— first 20 lines;tail -n 20 file— last 20 lines;tail -f /var/log/syslog— follow a file live
cat /etc/passwd | awk -F: '$3 >= 1000 {print $1}' | sort — list all regular users sorted alphabetically. Know how to chain grep | awk | sort | uniq pipelines.grep "Failed password" /var/log/secure selects the interesting lines; | awk '{print $(NF-3)}' projects the IP — counting fields from the end with NF is what makes this robust, since the leading timestamp shifts column numbers around; | sort | uniq -c aggregates, and the sort is mandatory because uniq only collapses adjacent duplicates — the single most common mistake in this pipeline; | sort -rn | head -10 ranks them. Full line: grep "Failed password" /var/log/secure | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head -10. Two variations worth knowing: awk '$1=="Aug" && /Failed/ {c++} END {print c}' does the select-and-count in one process, and sed -i.bak 's/PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config shows the rewrite side — note -i.bak, which keeps a backup, because a bare sed -i with a wrong pattern edits the original irreversibly.
sort before uniq -c, then sort -rn to rank. awk beats cut when the delimiter is whitespace of varying width, and NF counts fields from the end. Use sed -i.bak rather than bare -i when editing files in place.Containers appear on Linux+ as an operations topic, not a development one: run an image, map a port, persist data in a volume, read the logs. The one architectural fact that carries marks is daemonless and rootless — Podman runs containers as the invoking user with no background daemon, which is why RHEL ships it in place of Docker.
docker run -d -p 8080:80 --name webserver nginx— run nginx in background, map port 8080→80docker ps— list running containers;docker ps -a— all containers including stoppeddocker images— list local imagesdocker pull IMAGE:TAG— pull an image from registrydocker exec -it CONTAINER bash— interactive shell in a running containerdocker logs CONTAINER— view container logs;docker logs -f CONTAINER— followdocker stop CONTAINER— gracefully stop;docker rm CONTAINER— remove stopped containerdocker rmi IMAGE— remove an imagedocker run -v /host/path:/container/path IMAGE— bind mount a host directory into a containerpodman— drop-in Docker replacement; rootless and daemonless by design; commands are identical to Docker in most cases
Writing a Dockerfile
FROM ubuntu:22.04— base imageRUN apt-get update && apt-get install -y nginx— execute commands during image buildCOPY ./app /var/www/html/— copy files from build context into imageENV APP_ENV=production— set environment variableEXPOSE 80— document which port the container listens on (does not publish)CMD ["nginx", "-g", "daemon off;"]— default command to run (can be overridden)ENTRYPOINT ["/entrypoint.sh"]— executable that always runs (CMD becomes its arguments)docker build -t myapp:1.0 .— build image from Dockerfile in current directory
sudo). Docker requires the docker daemon running as root. The Linux+ exam specifically tests this architectural difference./srv/site on port 8080 as an unprivileged user on RHEL. Walk: (1) podman run -d --name web -p 8080:80 -v /srv/site:/usr/share/nginx/html:Z docker.io/library/nginx — the :Z suffix tells Podman to relabel the bind mount for SELinux, and without it the container gets permission denied on files whose Unix permissions look perfectly fine; (2) note the port: rootless containers cannot bind below 1024, which is why this is 8080 and not 80 — publishing 80 would need net.ipv4.ip_unprivileged_port_start lowered or a reverse proxy in front; (3) podman logs -f web and curl -I localhost:8080 confirm it serves; (4) make it persistent the systemd way rather than with a restart policy: podman generate systemd --name web --files --new, move the unit into ~/.config/systemd/user/, then systemctl --user enable --now container-web; (5) loginctl enable-linger $USER so the user's services start at boot without a login session. Why this shape: no daemon, no root, and the container is supervised by the same init the rest of the host uses.
:Z; rootless containers cannot bind ports under 1024. -d detaches, -p host:container publishes, -v host:container persists, podman logs is the first place to look.Ansible turns "SSH into forty boxes and run these commands" into one declarative file. Two properties carry the exam questions: it is agentless (SSH plus Python on the target, nothing installed), and it is idempotent — you describe the desired end state, so running the same playbook twice changes nothing the second time.
- Inventory file (
/etc/ansible/hostsor custom file with-i): groups of hosts in INI or YAML format ansible all -m ping— test connectivity to all hosts in inventoryansible webservers -m shell -a "df -h"— run shell command on webservers groupansible all -m copy -a "src=/etc/hosts dest=/tmp/hosts"— copy file to all hostsansible all -m service -a "name=nginx state=started"— ensure nginx is runningansible all -b -m dnf -a "name=httpd state=present"— install httpd (become=sudo)
Playbook Structure
- Playbooks are YAML files defining plays (which hosts to target) and tasks (what to do)
- Key modules:
apt/dnf(package management),copy(copy files),template(Jinja2 templates),service(manage services),user(manage users),file(manage file permissions/ownership) ansible-playbook site.yml— run a playbookansible-playbook site.yml --check— dry run (shows what would change without changing it)ansible-playbook site.yml -v/-vvv— verbose output for debuggingansible-playbook site.yml --limit webservers— run only against a specific group
PermitRootLogin no must be set on every web server. Walk: (1) prove reachability first — ansible webservers -m ping — a failure here is SSH or inventory, never the playbook; (2) write the play with a module rather than a shell command, because lineinfile is idempotent while shell: sed -i … is not:
- hosts: webservers / become: yes / tasks: / - name: disable root SSH login / lineinfile: { path: /etc/ssh/sshd_config, regexp: '^#?PermitRootLogin', line: 'PermitRootLogin no', validate: 'sshd -t -f %s' } / notify: restart sshd;
(3) the validate option is the safety catch — Ansible tests the candidate file with sshd -t and refuses to install it if the syntax is bad, so a typo cannot lock you out of forty hosts at once; (4) a handler restarts sshd only when the line actually changed, which is what keeps the run idempotent; (5) rehearse with ansible-playbook harden.yml --check --diff — it shows the exact diff without touching anything; (6) roll it out gradually with --limit web01 before running the whole group. Read the recap line: changed=1 on the first run and changed=0 on the second is the proof that the play is idempotent.
lineinfile, package, service) over shell, since only modules can report changed honestly. --check --diff is the dry run, --limit stages the blast radius, -b/become is privilege escalation.This lesson is the one that ties the other thirty-six together. Scenario questions rarely need an obscure command — they need the order: service state, then its logs, then the kernel's view, then resources. Following that sequence turns a vague "the server is slow" into a specific, provable cause, and it is exactly how the exam expects you to reason.
- Step 1 — Service status:
systemctl status servicename— shows active/failed state, last 10 log lines, and exit code - Step 2 — Journal:
journalctl -xe -u servicename— full journal with explanations;journalctl --boot -1— previous boot logs - Step 3 — Kernel messages:
dmesg | tail -50— recent kernel messages;dmesg | grep -i error - Step 4 — Resources: check disk (
df -h,du -sh /*), memory (free -h), CPU (top,vmstat 1 5), I/O (iostat -x 1 5) - Step 5 — OOM killer:
dmesg | grep -i "out of memory"orgrep -i oom /var/log/kern.log— identifies memory-killed processes
Common Failure Scenarios
- Service fails to start: check
systemctl statusandjournalctl -xefor the exact error; often a config file syntax error or missing dependency - Disk full:
df -hto identify full filesystem;du -sh /var/log/*to find large log files;journalctl --vacuum-size=500Mto trim journal - High CPU:
topsorted by CPU (Pkey);ps aux --sort=-%cpu | head - High memory / swap usage:
free -h;vmstat 1to watch memory pressure; consider increasing swap or identifying memory leaks withps aux --sort=-%mem - Cannot SSH to host: check firewall (
iptables -Lorfirewall-cmd --list-all), sshd status, SELinux (getenforce),ss -tlnp | grep 22
dmesg or /var/log/kern.log, not just /var/log/messages.systemctl status postgresql → failed (Result: exit-code), and the excerpt shows "could not write to file: No space left on device". That single line has already redirected the whole investigation away from the database. Step 2 — logs: journalctl -xeu postgresql confirms the write failures started at 03:14. Step 3 — resources: df -h shows /var at 100%. Step 4 — locate it: du -xh /var --max-depth=2 | sort -rh | head — the -x keeps du on one filesystem so it does not wander into mounts and mislead you — and points at /var/log/journal at 14 GB. Step 5 — the trap: if someone already ran rm on a log that rsyslog still holds open, df stays full while du shows the space as free; lsof +L1 lists deleted-but-open files and a service restart is what actually releases them. Step 6 — fix and prevent: journalctl --vacuum-size=500M reclaims immediately, SystemMaxUse=500M in /etc/systemd/journald.conf stops the recurrence, then systemctl restart postgresql. Step 7 — confirm: systemctl is-active postgresql and df -h /var together, because fixing the symptom without re-checking the cause is how the same ticket comes back tomorrow.
df full while du disagrees means deleted-but-open files (lsof +L1); an Out of memory: Killed process line in dmesg means the OOM killer, not an application crash. Always verify the fix with the same command that showed the problem.- Bash scripts use shebang
#!/bin/bash, exit codes from$?, andset -efor fail-fast;grep/sed/awkchained through pipes (awk -F:,sed -i 's///') cover most parsing scenarios. - Schedule with
cron(* * * * *minute / hour / dom / month / dow),anacronfor irregular runs, and systemd timers (the modern alternative — atomic with their service units). Persistent rules live in/etc/crontaband/etc/cron.d/. - Podman is daemonless and rootless-capable — the key architectural difference from Docker the exam tests. Ansible is agentless (SSH + Python on target) and idempotent. Troubleshoot in the order
systemctl status→journalctl -xe→dmesg→top/free/ss.