Ady Wicaksono Daily Activities

investigate a process on Linux server using lsof

leave a comment »

lsof – list open files is a very good tool for Linux Administrator

For example, you found a weird process

# ps aux

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     14668 48.2  0.1 243504  7448 ?        Sl   Jul10 3319:53 ./bin/StoreGrid

Hmmm… what is that, then you could simply check it using lsof

#  lsof  -p 14668
COMMAND     PID USER   FD   TYPE   DEVICE      SIZE     NODE NAME
StoreGrid 14668 root  cwd    DIR    253,0     32768 10977505 /home/storegrid/StoreGrid
StoreGrid 14668 root  rtd    DIR    253,0      4096        2 /
StoreGrid 14668 root  txt    REG    253,0  11812748 10977512 /home/storegrid/bin/StoreGrid
StoreGrid 14668 root  mem    REG    253,0     21704 38502449 /lib/libnss_dns-2.3.6.so
StoreGrid 14668 root  mem    REG    253,0     76272 38502417 /lib/libresolv-2.3.6.so
StoreGrid 14668 root  mem    REG    253,0    126576 38502420 /lib/ld-2.3.6.so
StoreGrid 14668 root  mem    REG    253,0   1481808 38502422 /lib/libc-2.3.6.so
StoreGrid 14668 root  mem    REG    253,0     16244 38502435 /lib/libdl-2.3.6.so
StoreGrid 14668 root  mem    REG    253,0    201032 38502439 /lib/libm-2.3.6.so
StoreGrid 14668 root  mem    REG    253,0    101680 38502429 /lib/libpthread-2.3.6.so
StoreGrid 14668 root  mem    REG    253,0     46640 38502452 /lib/libnss_files-2.3.6.so
StoreGrid 14668 root  mem    REG    253,0    733520 37828423 /usr/lib/libstdc++.so.5.0.7
StoreGrid 14668 root  mem    REG    253,0     40112 38502443 /lib/libgcc_s-4.0.2-20051126.so.1
StoreGrid 14668 root    0r   CHR      1,3               1938 /dev/null
StoreGrid 14668 root    1w   CHR      1,3               1938 /dev/null
StoreGrid 14668 root    2w   CHR      1,3               1938 /dev/null
StoreGrid 14668 root    3u  IPv4 34761557                TCP *:32005 (LISTEN)
StoreGrid 14668 root    4u  IPv4 34761558                UDP *:58304
StoreGrid 14668 root    5u  IPv4 34761559                UDP *:17254
StoreGrid 14668 root    6u  IPv4 34761560                UDP *:6363
StoreGrid 14668 root    7u  IPv4 69369170                TCP xxxxxx:37402->yyyyy:32004 (ESTABLISHED)
StoreGrid 14668 root    8u  IPv4 34761564                UDP *:6364
StoreGrid 14668 root   12u  IPv4 34761590                UDP *:32006

Got it, I found binary executables, socket information, and many other information on this process. Actually this lsof tool read and process information from “/proc” pseudo filesystem on Linux, for pid 14668, then this tool will investigate /proc/14668 directory content.

# ls -l /proc/14668
total 0
dr-xr-xr-x    5 root root 0 Jul 10 04:15 ./
dr-xr-xr-x  172 root root 0 Jul  5 04:36 ../
dr-xr-xr-x    2 root root 0 Jul 14 23:04 attr/
-r--------    1 root root 0 Jul 14 23:04 auxv
-r--r--r--    1 root root 0 Jul 14 21:19 cmdline
-r--r--r--    1 root root 0 Jul 14 23:04 cpuset
lrwxrwxrwx    1 root root 0 Jul 14 22:47 cwd -> /home/storegrid/StoreGrid/
-r--------    1 root root 0 Jul 14 23:04 environ
lrwxrwxrwx    1 root root 0 Jul 14 08:50 exe -> /home/storegrid/bin/StoreGrid*
dr-x------    2 root root 0 Jul 14 22:47 fd/
-rw-r--r--    1 root root 0 Jul 14 23:04 loginuid
-r--------    1 root root 0 Jul 14 22:47 maps
-rw-------    1 root root 0 Jul 14 23:04 mem
-r--r--r--    1 root root 0 Jul 14 23:04 mounts
-r--------    1 root root 0 Jul 14 23:04 mountstats
-rw-r--r--    1 root root 0 Jul 14 23:04 oom_adj
-r--r--r--    1 root root 0 Jul 14 23:04 oom_score
lrwxrwxrwx    1 root root 0 Jul 14 22:47 root -> //
-r--r--r--    1 root root 0 Jul 14 23:04 schedstat
-r--------    1 root root 0 Jul 14 23:04 smaps
-r--r--r--    1 root root 0 Jul 14 21:19 stat
-r--r--r--    1 root root 0 Jul 14 08:50 statm
-r--r--r--    1 root root 0 Jul 14 21:19 status
dr-xr-xr-x   23 root root 0 Jul 10 04:15 task/
-r--r--r--    1 root root 0 Jul 14 23:04 wchan

lsof is also able to detect port UDP/TCP activity, for example from “netstat” you get weird information

#netstat -nl

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 0.0.0.0:1                   0.0.0.0:*                   LISTEN

Hey, what server running on port TCP number 1???

Find it

#  lsof -i TCP:1
COMMAND    PID USER   FD   TYPE DEVICE SIZE NODE NAME
portsentr 3646 root    0u  IPv4  10777       TCP *:tcpmux (LISTEN)

# lsof -p 3646
COMMAND    PID USER   FD   TYPE DEVICE    SIZE     NODE NAME
portsentr 3646 root  cwd    DIR  253,0    4096        2 /
portsentr 3646 root  rtd    DIR  253,0    4096        2 /
portsentr 3646 root  txt    REG  253,0   30128 37818458 /usr/sbin/portsentry
portsentr 3646 root  mem    REG  253,0  126576 38502420 /lib/ld-2.3.6.so
portsentr 3646 root  mem    REG  253,0 1481808 38502422 /lib/libc-2.3.6.so
portsentr 3646 root    0u  IPv4  10777              TCP *:tcpmux (LISTEN)
portsentr 3646 root    1u  IPv4  10779              TCP *:sunrpc (LISTEN)

Got it….. nice tool right?

Written by adywicaksono

July 14, 2007 at 1:09 pm

Posted in Linux

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: