Differences

This shows you the differences between two versions of the page.

--- processes:list_all_non-kernel_processes [2016/10/17 13:57] – peter
+++ processes:list_all_non-kernel_processes [2019/12/01 22:37] (current) – removed peter
@@ Line 1: / Line 1: @@
-====== Processes - List all non-kernel processes ======
-Usually kernel processes are safe and clean.  For kernel processes, either PID (process id) is 2 or PPID (parent process id) is 2.  Here is how to get all non-kernel processes.
-<code bash>
-ps --ppid 2 -p 2 -p 1 \
-   --deselect -o uid,pid,rss,%cpu,command
-</code>
-Returns:
-<code>
-UID   PID   RSS %CPU COMMAND
-   411  1848  0.0 /lib/systemd/systemd-
-   572  2904  0.0 dhclient -1 -v -pf /r
-   902  1244  0.0 dbus-daemon --system
-   912  1948  0.0 /lib/systemd/systemd-
-  5869   388  0.0 upstart-socket-bridge
-  1953   904  0.0 /usr/sbin/apache2 -k
-  3463  3700  0.0 /usr/sbin/apache2 -k
-  ...  ...
-  ...  ...
-  5098  4224  0.0 sshd: ubuntu [priv]
-  5139  1748  0.0 /usr/bin/python /usr/
-  5140  3484  0.0 /usr/bin/python /usr/
-  5176  1904  0.0 sshd: ubuntu@pts/3
-  5177  3860  0.0 -bash
-  5193  1200  0.0 tmux attach -t denny
-  5297  4224  0.0 sshd: ubuntu [priv]
-  ...  ...
-  ...  ...
-</code>
-**NOTE**:
-  * **rss** (resident set size): real RAM usage.
-  * **-deselect**: rule out matched processes.
-===== Rule out trusted procsses =====
-We may have many processes running, which are expected and trusted. e.g apache2, tomcat7, mysqld, etc. To avoid distraction, build a white list especial for your project.
-===== Sort processes by memory and cpu =====
-We’re more concerned about suspicious processes using noticeable resource.
-<code bash>
-# Sort by memory first, then cpu
-ps --ppid 2 -p 2 -p 1 --deselect \
-  -o uid,pid,rss,%cpu,command, \
-  --sort -rss,-cpu
-</code>
-===== Automate Detection Process and Get Alerts =====
-We hide all the complexities and white list configuration in a python script (detect_suspicious_process.py).  If you issue the python command, you may see output like "**Identified processes count: XXX.**"  Define a scheduled task to run periodical check and confirm the number.
-If the number is not 0 or it changes, send alerts.  It might take a while to build a suitable white list.  Once it’s done, your servers are always more secured and managed!
-<code bash>
-wget -O /tmp/detect_suspicious_process.py \
-https://raw.githubusercontent.com/\
-DennyZhang/devops_public/tag_v2/python/\
-detect_suspicious_process/\
-detect_suspicious_process.py
-# Detect suspicious process
-python /tmp/detect_suspicious_process.py
-# Detect by customized whitelist
-python /tmp/detect_suspicious_process.py \
-   --whitelist_file /tmp/whitelist.txt
-</code>
-===== detect_suspicious_process.py =====
-<code python>
-# -*- coding: utf-8 -*-
-#!/usr/bin/python
-##-------------------------------------------------------------------
-## @copyright 2015 DennyZhang.com
-## File : detect_suspicious_process.py
-## Author : DennyZhang.com <denny@dennyzhang.com>
-## Description : http://www.dennyzhang.com/suspicious_process/
-##        python ./detect_suspicious_process.py
-##        python ./detect_suspicious_process.py --whitelist_file /tmp/whitelist.txt
-## --
-## Created : <2016-01-15>
-## Updated: Time-stamp: <2016-08-20 13:32:55>
-##-------------------------------------------------------------------
-import argparse
-import subprocess
-import os, sys
-################################################################################
-# TODO: move to common library
-def string_in_regex_list(string, regex_list):
-    import re
-    for regex in regex_list:
-        regex = regex.strip()
-        if regex == "":
-            continue
-        if re.search(regex, string) is not None:
-            # print "regex: %s, string: %s" % (regex, string)
-            return True
-    return False
-################################################################################
-DEFAULT_WHITE_LIST = '''
-/sbin/getty -.*
-dbus-daemon .*
- acpid -c /etc/acpi/events -s /var/run/acpid.socket$
- atd$
- cron$
- /lib/systemd/systemd-udevd --daemon$
- /lib/systemd/systemd-logind$
- dbus-daemon --system --fork$
- /usr/sbin/sshd -D$
- rsyslogd$
- /usr/sbin/mysqld$
- /usr/sbin/apache2 -k start$
-'''
-COMMAND_GET_NONKERNEL = '''
-sudo ps --ppid 2 -p 2 -p 1 --deselect \
--o uid,pid,rss,%cpu,command \
---sort -rss,-cpu
-'''
-def get_nonkernel_process():
-    process_list = subprocess.check_output(COMMAND_GET_NONKERNEL, shell=True)
-    return process_list
-def load_whitelist(fname):
-    white_list = ""
-    if fname is None:
-        print "No white list file is given. Use default value."
-        white_list = DEFAULT_WHITE_LIST
-    else:
-        print "load white list from %s" % (fname)
-        with open(fname) as f:
-            white_list = f.readlines()
-    return white_list
-def list_process(process_list, white_list):
-    import re
-    l = []
-    for line in process_list.split("\n"):
-        line = line.strip()
-        if line == "":
-            continue
-        if not string_in_regex_list(line, white_list.split("\n")):
-            l.append(line)
-    return l
-################################################################################
-if __name__=='__main__':
-    parser = argparse.ArgumentParser()
-    parser.add_argument('--whitelist_file', required=False,
-                        help="config file for whitelist", type=str)
-    args = parser.parse_args()
-    white_list = load_whitelist(args.whitelist_file)
-    nonkernel_process_list = get_nonkernel_process()
-    process_list = list_process(nonkernel_process_list, white_list)
-    # Remove header
-    print "Identified processes count: %d." % (len(process_list) - 1)
-    print "\n".join(process_list)
-## File : detect_suspicious_process.py ends
-</code>