2015-05 SSH, File Servers, Operational Awareness
SSH
SSH Servers
On CentOS, the SSH server is already running and already allowed through the firewall; it is running version 5.3p1.
On Ubuntu systems (Desktop or Server), the server is not present. For a Desktop system, grab the package openssh-server_5.9p1-1ubuntu7_i386.deb from the labshare or online. Install it in the usual way:
zathras@achilles:~$ sudo dpkg -i /home/zathras/Desktop/openssh- server_5.9p1-5ubuntu1_i386.deb [sudo] password for zathras: Selecting previously unselected package openssh-server. (Reading database ... 141338 files and directories currently installed.) Unpacking openssh-server (from .../openssh-server_5.9p1-5ubuntu1_i386.deb) ... Setting up openssh-server (1:5.9p1-5ubuntu1) ... Creating SSH2 RSA key; this may take some time ... Creating SSH2 DSA key; this may take some time ... Creating SSH2 ECDSA key; this may take some time ... ssh start/running, process 2855 Processing triggers for ureadahead ... ureadahead will be reprofiled on next reboot Processing triggers for ufw ... Processing triggers for man-db ...
As noted earlier, the Ubuntu firewall is off by default; if you have enabled it, then you must open the necessary port.
If you want to know what services are running on your Ubuntu system, you might try a command like chkconfig, however it is not installed by default. The Ubuntu alternative is to use the command
zathras@achilles:~$ service --status-all
Getting the SSH server onto the Ubuntu Server system is complicated by the fact that it has no GUI, so the usual trick of dragging a file from the host and dropping it on the desktop of the guest will not allow us to copy the files. We could set up a web server or an ftp server, and use client software like wget on the server to get the files; we have already seen how to do that with Python. We could also take the server out out of the lab to an Internet connection and use ftp and/or wget.
There is another approach. From VMWare, select VM, then Settings. Under the Options tab, select Shared Folders. Select Add; then you can choose a folder on the host that will be mapped to a folder on the guest.
In this example, I mapped the host directory d:\Mike\Desktop\New Software to the directory "Software". That directory will now appear in the guest, in the location /mnt/hgfs/Software
[Did he just say "hgfs"? Yep. Host-Guest-File-System. hgfs. It does make sense. Sort of.]
To actually install the SSH server, we need two packages- the server itself (openssh-server_5.9p1-5ubuntu1_amd64.deb) and the package libwrap0_7.6.q-18_amd64.deb. Install them both via dpkg.
Mint 13 Cinnamon also does not come with an SSH server by default; however since it is a variant of Ubuntu, it can be installed using the same method and package as on Ubuntu.
The Mint 13 KDE system does have an SSH server already started.
An SSH server can also be installed on a Windows system. My preferred method for doing so is to use Cygwin, which provides Windows ports for a large number of Linux applications. The catch though, is that though it is possible to install and use Cygwin on systems not connected to the Internet, it is difficult, especially for beginners to Cygwin.
Rather than go through that trouble, let me offer as an alternative FreeSSHD. It is older, but it is free and (mostly) works on Windows systems. Installation is standard, though I should note that I have had nothing but bad luck when I installed the tool as a service.
The icon on the Desktop starts a tool to control the server; this controller lives in the System tray. Another place I have run into trouble is starting another instance of the controller when one is already running; because the system tray hides the FreeSSHd icon by default, this has happened embarrassingly often.
Another catch is that if you connect to the system using a Linux SSH client, it assumes that you are at the top of the screen; it also does not clear the screen. Thus, you may be staring at the bottom of the screen waiting for a response, when it has already authenticated you and moved your cursor to the top of the screen:
Note also that you need to manually specify which users are allowed to use the SSH server. They can be domain users if you wish, but you can also create new users that are not connected to either a domain account or a local Windows account.
SSH Clients
On Linux systems, simply use the built-in command line client to connect to a remote SSH server.
The best client for SSH from a Windows System in PuTTY. It is available on the lab share and online.
Its general use is straightforward- enter the required connection information in the box and select Open. You can save the profile settings by simply giving it a name and choosing Save.
SSH PKI
SSH can use public keys to authenticate users; this is often more convenient than passwords. To set up the PKI in CentOS 6.2, modify /etc/ssh/sshd_config to uncomment out the following lines
RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile .ssh/authorized_keys
The situation in Mint is similar, though it writes the location of the Authorized Keys File as
#AuthorizedKeysFile %h/.ssh/authorized_keys
Restart sshd; on CentOS 6.2 you can use
[root@acheron ~]# service sshd restart Stopping sshd: [ OK ] Starting sshd: [ OK ]
while on Mint 11 & Ubuntu 11.04 you can use
zathras@alexander /etc/ssh $ sudo service ssh restart [sudo] password for zathras: ssh stop/waiting ssh start/running, process 3992
Each user that wants to log in to this server using SSH PKI needs to set the proper permissions on the ~/.ssh directory, create the file ~/.ssh/authorized_keys, and ensure that file also has the proper permissions. Indeed, we require
-
~/.sshis chmod 700 (rwx — —) -
~/.ssh/authorized_keysis chmod 644 (rw- r– r–)
We can do this manually now, but there is a method to do this at the same time the keys are copied to the server.
A user that wants to connect to a client via PKI must first generate the keys that will be used for the connection via ssh-keygen.
zathras@alexander ~ $ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/zathras/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/zathras/.ssh/id_rsa. Your public key has been saved in /home/zathras/.ssh/id_rsa.pub. The key fingerprint is: 0f:3c:53:1a:95:17:f1:1c:0e:28:0f:14:d2:d0:9e:cc zathras@alexander.cosc.tu The key's randomart image is: +--[ RSA 2048]----+ | o=o..o+o. | | .=.o .= . | | +.=.. + | | .E+. | | S | | = | | . | | | | | +-----------------+
Note that zathras did not use a passphrase to secure the keys.
What results are a pair of keys, a private key and a public key, stored in the directory /home/seldon/.ssh:
+-----------------+ zathras@alexander ~ $ ls -l .ssh/ total 8 -rw------- 1 zathras zathras 1675 Feb 14 14:30 id_rsa -rw-r--r-- 1 zathras zathras 407 Feb 14 14:30 id_rsa.pub
To use SSH PKI, we need to copy the public key (ONLY) to the server; the content of that key needs to then be appended to the file ~/.ssh/authorized_keys on the server for that user. Note that the username on the client and the username on the server can be different. We can do this manually a number of ways, including using SSH with a password. However, there is a tool that will do this automatically for us, and will even ensure that the server’s directories and files have the correct permissions.
zathras@alexander ~ $ ssh-copy-id valen@acheron.cosc.tu The authenticity of host 'acheron.cosc.tu (10.0.2.100)' can't be established. RSA key fingerprint is 5a:b1:0d:b1:a0:ad:05:6c:28:51:ff:15:df:53:9e:1e. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'acheron.cosc.tu,10.0.2.100' (RSA) to the list of known hosts. valen@acheron.cosc.tu's password: Now try logging into the machine, with "ssh 'valen@acheron.cosc.tu'", and check in: ~/.ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. zathras@alexander ~ $ ssh valen@acheron.cosc.tu [valen@acheron ~]$
Once this is complete, zathras on alexander can SSH into acheron as valen without using a password.
You can do a lot of surprising thing with SSH. Suppose you want to just execute a single command on a remote system- say cat /etc/hostname. Well, run it as follows:
[seldon@proclas ~]$ ssh nexon cat /etc/hostname nexon.cosc.tu
Since PKI has been setup already, no password was required, and the result of the command is simply dropped to our screen. If we had not set up PKI, then we would need to enter our password each time we wanted to execute a command.
Suppose you wanted to copy the file /home/seldon/text to the remote host. Then you can use scp, which used ssh as its transport mechanism:
zathras@alexander ~/Documents $ scp text valen@acheron.cosc.tu:/home/valen/ copied_text text 100% 27 0.0KB/s 00:00
Let’s run ls on the remote host to see if the file arrived:
zathras@alexander ~/Documents $ ssh valen@acheron.cosc.tu "ls -al /home/valen" total 36 drwx------. 5 valen valen 4096 Feb 14 11:49 . drwxr-xr-x. 4 root root 4096 Feb 14 11:40 .. -rw-r--r--. 1 valen valen 18 Dec 2 2011 .bash_logout -rw-r--r--. 1 valen valen 176 Dec 2 2011 .bash_profile -rw-r--r--. 1 valen valen 124 Dec 2 2011 .bashrc -rw-r--r--. 1 valen valen 27 Feb 14 11:49 copied_text drwxr-xr-x. 2 valen valen 4096 Nov 11 2010 .gnome2 drwxr-xr-x. 4 valen valen 4096 Jan 4 01:56 .mozilla drwx------. 2 valen valen 4096 Feb 14 11:40 .ssh
And now let’s bring the file back
zathras@alexander ~/Documents $ scp valen@acheron.cosc.tu:/home/valen/ copied_text ./copied_text copied_text 100% 27 0.0KB/s 00:00 zathras@alexander ~/Documents $ ls -l total 8 -rw-r--r-- 1 zathras zathras 27 Feb 14 14:53 copied_text -rw-r--r-- 1 zathras zathras 27 Feb 14 14:49 text
Suppose that there is a graphical program like firefox on your remote host. You can run it remotely, provided you pass the "-Y" flag. Here is a screenshot of the user zathrad on the Mint 13 system alexander logging in to the CentOS system acheron as valen, and then calling Firefox from the command prompt.
We can also use PKI with PuTTY. Start by running the PUTTYGEN key generator program; be sure you select SSH2-RSA for the key type, as there are weaknesses with the other approaches.
Save both the private key and the public key. We don’t have the simple technique to add the public key to the remote server that we used before, however we can use SSH with a password, then copy and paste the public key on to the server. Unfortunately, the format that PuTTY uses is not one well suited to copying.
For example, here is the public key file generated in this example
---- BEGIN SSH2 PUBLIC KEY ---- Comment: "rsa-key-20140214" AAAAB3NzaC1yc2EAAAABJQAAAIB+u/75GJgycyGe0tcwHzS6cckE+1skZbG1Me5y FfJbtaXYDbY+JURdZ5DsDwAGxacROnUF1f/hwJxWkyglK7lo7xWoM2KJRE1WW+4D nXofLY6I3HvkVN/N+Hvx2QFIsg+0cS2rW0tx8PSpsonnqCnpFC/kvT2/YVNAJq/O U078CQ== ---- END SSH2 PUBLIC KEY ----
If you copy this to the file ~/.ssh/authorized_keys file for a user, it will fail. Why? Suppose that we have an existing authorized_keys file generated by OpenSSH; it has permissions 600, and has contents similar to the following
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA3b9Hp1k4lqRUNIbx4jg1nrXJw1CkchKIgdtKrTSQi52 72tC84mg74XNlxxlNkzVs8cxxJTxfimXOhZUHWbaYHoMPius9o9yQk2U+2/CHMx6JS90pTaHkMvEgyM aZNnSVZhIg2/v0n/aLOM2ite2iaTQKN3LNZ39rTdcejLSo/DGWJACR6JJS5Af3dD1AWiIyNhDELXCsJ FMuQqhrIzIfNT52qJ3Yd8gmNbWctATSQKNxBKK1zoBCTVlYjZ/IkRhoANjpCsnFV20VOU9xFrQ2aQ6S kXzp7NzsIk+DlGV1FefbnhNKiMGglzgPNLXWZ24Bjq/XsSeikFsCLLHOSLQeFw== valen@alexande r.cosc.tu ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6y+v/2XXCk/694UzQ3egHOE0jWzAM/gwkLTbud371A6 TbcRS5+chfLwmDl1aO5+T/dz9ydkHKOOln4QbFs/JtCtJecpE9DlLC9jFQf9H1+4Vbv5eYYmPKlbkJa GLoSPs4Vk1EVg75kx4pBGRdfR839t9axiDwIrku7HdGapoGRmFOzvcNmNbagx8Rz7/74GRmpdYCHbSL dQ7BtcRHX3XV2NtdQIifCiUje5KSp+RT08JT82q7qZ++Hsbk6OnTJDBJ5TeJXXU4WgNikKlKn+gAYSW DcjmYTfZLvnzkte6OFPNQXktvivI27y3lPRPWQXNQ7RdzL3hgDD/ro2bDlXJ+w== root@alexander .cosc.tu
Here we see two RSA keys, one for the user valen on the host alexander.cosc.tu, and a second for the user root on the same host. We need to modify our PuTTY generated key so that it has the same format; is needs to start with the text ssh-rsa and should end with the user and host that has the corresponding private key; the commented lines need to be removed. Further, the result needs to be on a single line with no newlines.
Even better, if you open the public key with a simple text editor like Notepad, you won’t even realize that the key is split over multiple lines, so and copying it becomes even more trouble.
Here is one way to proceed. Open the public key in Wordpad. Under View, make sure that lines are not being wrapped. Delete the first two lines and the last line; then make sure that all that remains is on a single line. Be sure that the line starts with the magic text "ssh-rsa" and ends with the user name and host name where the private key is stored; you should end with something like this
Add that single line into the file ~/.ssh/authorized_keys on the remote server; I did it by using SSH and a password to login, then used vi to open the file, and just pasted the result to the end of the file by right-clicking. You should obtain something like the following:
Save the result. To log in using this key, specify the user name in the Data dialog box (beneath Connection), while the location of the private key is specified in the Auth dialog box (beneath Connection, beneath SSH).
Given the settings in different places, I recommend that you save your resulting configuration (in Session).
Opening the connection will then give you access to the remote server without a password.
SSH can also be used to create tunnels, but we don’t have time to discuss SSH tunneling in the detail it deserves.
Securing SSH
check authorized keys
There are a number of things that we can do to try to secure our SSH server. First, we can prevent root from logging in directly. If someone needs root privileges from a remote site, they can log in as a regular user and then use either sudo or su. To make this change, add the line
PermitRootLogin no
to the sshd_config file. Note that the default state is to allow root logins; in the CentOS 6.2 default configuration file a commented out line allowing root logins is present, while on the Mint 13 / Ubuntu 12.04 default configuration files the line is present and explicitly allows root logins.
Speaking of users, you may want to allow only certain users to be allowed to use SSH to log in to the box. The line
AllowUsers zathras valen
will allow only the listed users to be able to SSH into the machine; other users (e.g.mcole) would not be able to log in. Though using a white list of allowed users is generally preferred, you could create a black list of users not allowed to log in via the DenyUsers directive.
Be sure to use only SSH protocol 2, as the older SSH-1 has a number of vulnerabilities and should be considered obsolete. Though SSH-2 is the default, your sshd_config file should contain the line
Protocol 2
to ensure that the system does not fall back to SSH-1.
You can also use the files /etc/hosts.allow and /etc/hosts.deny to restrict access. This is a three step process-
- If a service / host combination is in the file
hosts.allow, then access to the service is granted - If the combination is not in
hosts.allow, thenhosts.denyis checked; if the service / host combination matches then access is denied - If neither has occurred, then access is granted
Each line has the form
service : host(s)
For our purpose the service can be either sshd or ALL. The host(s) can be specified by name, by IP address, or by IP Address Range. Multiple hosts can be separated by commas. Suppose we want to allow SSH from only the hosts 10.0.2.3 and 10.0.2.13; then we can set up the hosts.allow file as
# /etc/hosts.allow sshd : 10.0.2.3, 10.0.2.13
while the hosts.deny file becomes
# /etc/hosts.deny sshd: ALL
Then any ssh connection attempt from other than 10.0.2.3 or 10.0.2.13 will not even receive a response.
You are encouraged to read the manual page for hosts_access to learn more about the syntax for these files, and to learn other options.
Windows File Sharing
Microsoft Windows operating systems are designed to allow users to share files and folders.
Creating a (SMB) file share (Windows 8)
Creating a file share for a Windows 8 system connected to a domain is actually quite simple. Suppose that the domain user Marcus Cole has a folder called "Shared Folder" in the Documents Library. To share that folder, simply navigate to that folder and select Share, then Specific People. You can then grant permissions to user(s) and group(s) defined on the domain controller. So, for example we can give particular domain members permission to read the directory, and all members of the sales admin group (which we created in our discussion of group policy) both read and write permissions.
Creating a (SMB) file share (Windows 7)
Create a folder; navigate Folder (right click) → Properties → Sharing → Sharing. Specify the Share name, and the permissions of the users allowed to access the share. One apparent difference is that an administrator password was required for Windows 7 but not for Windows 8 to enable the share.
Setting up a file server (Windows 2012)
In most large organizations however, file shares are provided centrally as part of the overall network infrastructure and run from a file server.
In this example, we will set up a Windows 2012 system as a file server. This system will be a domain member, but will not be a domain controller. In this example, the system will be named minbari.corp.class.tu; it will be a member of the domain corp.class.tu, which has brakiri.corp.class.tu as the domain controller; the system will live at the IP address 10.0.6.50.
With the basics covered, go to initial configuration tasks, select Add Roles → File Services. In this example, we install both the File Server and the File Server Resource Manager.
Installation may take a bit of time.
Once the installation completes, we need to create the shared directory that the approved domain users can use. From the Server Manager, select File and Storage Services, then select Shares.
From Tasks, select New Share. Two kinds of shares are available, SMB shares, which are a Windows standard, and NFS shares, which are an older *nix standard. For simplicity, let’s select a quick SMB share.
Once selected, you first select the volume that will be used for the shares; the default c: is appropriate here. Moving forward, we name the share "CommonShare" and accept the default local path and remote paths.
Leave the other settings in their default settings; we will now enable access-based enumeration and we will not encrypt data access.
Next, and more important we need to specify the permissions that govern access. This process is greatly simplified from the settings in Windows 2008. Simply select Customize permissions from the menu. You can then customize permissions as needed from the Permissions tab. If we want to set up a common share for all domain users, we select Add, then select domain users, and grant Modify permissions, which includes read, write, and execute.
After the settings are confirmed, the share will be created.
The shares can be accessed from a Windows system by navigating to minbari\Commonshare
They can be mapped to a drive letter via the net use command
Microsoft Windows [Version 6.2.9200]
(c) 2012 Microsoft Corporation. All rights reserved.
C:\Users\zack>net use z: \\minbari\CommonShare
The command completed successfully.
C:\Users\zack>z:
Z:\>dir
Volume in drive Z has no label.
Volume Serial Number is E627-301D
Directory of Z:\
02/14/2014 09:31 PM .
02/14/2014 09:31 PM ..
02/14/2014 09:31 PM 18 Test.txt.txt
1 File(s) 18 bytes
2 Dir(s) 52,669,808,640 bytes free
You can also access the file share from Linux systems. Suppose that you are on a CentOS system joined to the domain; then from Places, select Connect to Server. Add the name of the server and the name of the share; then select Connect.
A network shared folder will appear on the Desktop, and you can browse the folder quite simply.
The same process works on a Mint Cinnamon system. More generally, most Linux systems accept an address of smb://minbari/CommonShare in their file browser. In Ubuntu, you can startby browsing folders, then press ctrl+L to enter the name of the place.
This will bring the user to the shared folder.
Automatic Drive Mappings via Group Policy
We would like each user to have a drive letter mapped to the common file share just created; this is done through Group Policy.
On a domain controller, as a domain administrator go to group policy management and create a new group policy object- say “Common Drive Mapping”. Edit that policy: Common Drive Mapping → User Configuration → Preferences → Windows Settings → Drive Maps.
In the Drive Maps window, right click and select New → Mapped Drive. For Action, select “Create”; for Location, select the file share created previously. Be sure that you include both the host name and the directory for your share. Select a label for the drive share; select a drive letter- say P:\. Apply the result.
Exit the group policy editor (with our policy now saved).
Return to Group Policy Management; select Group Policy Management → Forest → Domains → “YourDomainNameWhateverItIs” (Right click) → Link an existing GPO → Select your new GPO, which this example calls “Common Drive Mapping”. Apply this policy.
You do not need to apply this policy to the entire domain; you could simply apply it to one or more OUs.
Log onto a machine in your domain; verify that the drive P:\ is now mapped to the file share.
Individual file shares
We can also set up a directory to provide an individual file share for each domain user. We start by creating the file share itself, so we head back to the file server, and from Server Manager make our way back to Shares. From Tasks, select New Share, and once again we will use the profile SMB Share - Quick. We’ll keep the same share location on the C: drive, but now the share name will be UserData; then the local path to the shared data is c:\Shares\UserData while the remote path will be \\minbari\UserData. Again, we leave the other settings in their default values, so we don’t enable access-based enumeration or encrypt data access.
Once we get to the permissions section, we will diverge from our experience with the CommonShare. We Customize Permissions, and select Disable inheritance; convert all inherited permissions into explicit ones.
Next, we remove both sets of permissions for MINBARI\Users, but add permissions for Domain Admins, giving them Full Control. We end up with a permission set like the following:
With these changes completed, create the share.
With the share complete, we now move to the individual users. From Server Manager on a domain controller, choose Tools, then navigate to Active Directory Users and Computers. Select a domain user, then select Properties. From the dialog box that results, choose the Profile tab. Under Home Folder, choose a drive letter and then provide the path to the file share we just created- with one additional twist. We will actually provide a link to a subdirectory based on the account user name; we can even simply specify %username in the directory name:
We repeat this process for each user.
When complete, we can simply login as a user, and verify that both drives are mapped to the correct places:
When you complete these tasks, you should also verify that you are unable to access the file shares of other users.
I would love to say that this process is painless, but it is not, as for some unknown reason it takes some time for information about the new file shares to propagate out. If you log in and the shared drives are not there, you may need to reboot the system.
Remote Desktop
If you want to gain graphical access to a Windows system, one approach is to use the Remote Desktop Protocol. Enabling an RDP server is quite simple. On a Windows 8 system as an example, from the Control Panel, navigate to System and Security, then to System. Select Remote Settings and pass through UAC. From the Remote tab, simply allow remote connections to the computer.
One nice feature is that this will automatically open the proper ports in the firewall, at least for users on the domain and on private networks. In case you forgot, RDP uses TCP/3389 and UDP/3389.
The situation on server systems is essentially the same.
To connect to a remote Windows system, you can use the Microsoft Terminal Services Client; and you can launch it by running mstsc.exe. Simply enter the name of the computer to which you want to connect, then provide the appropriate credentials when asked.
A very simple client for Mint and Ubuntu systems is available, called rdesktop; I have made it available on the lab share. Install it via dpkg:
valen@hyach:~$ sudo dpkg -i /home/valen/Desktop/rdesktop_1.7.0-1ubuntu2_i386.deb
To use it, simply provide the hostname on the command line and then authenticate:
A client is available for CentOS systems; you can install it via yum in the usual fashion
[root@gaim ~]# yum --disablerepo=\* --enablerepo=c6-media install rdesktop
where the CentOS system has presumably been configured as we did for BIND to use the CD as a repo. If you are online, a simple yum install rdesktop will work.
Operational Awareness in Linux
By running a service like SSH, you are now explicitly allowing other people to access your system. To do so safely, you need to know what those users are doing on your system; this is the concept of operational awareness.
Linux provides a number of tools to give you, the system administrator, information about what is occurring on your system.
psacct
There is a package that allows for much more detailed logging of what is being done on a Linux system called psacct or acct depending on the particular distribution
The tool is already installed on your CentOS virtual machine, though it is not enabled. Called psacct it is controlled in the usual fashion. The command
[root@gaim ~]# /etc/init.d/psacct start Starting process accounting: [ OK ]
will start and stop the service, and we can ensure it starts on boot either via chkconfig or by navigating System → Administration → Services.
For Ubuntu or Mint systems the package is called acct, and for Ubuntu 12.04 / Mint 13 we use version 6.5.5-1ubuntu1. You can get the software from the labshare or directly from the Ubuntu software repository.
Install it in the usual fashion.
valen@markab ~ $ sudo dpkg -i /home/valen/Desktop/acct_6.5.5-1ubuntu1_amd64.deb [sudo] password for valen: Selecting previously unselected package acct. (Reading database ... 141706 files and directories currently installed.) Unpacking acct (from .../acct_6.5.5-1ubuntu1_amd64.deb) ... Setting up acct (6.5.5-1ubuntu1) ... Turning on process accounting, file set to '/var/log/account/pacct'. * Done. Processing triggers for doc-base ... Processing 30 changed doc-base files, 1 added doc-base file... Registering documents with scrollkeeper... Processing triggers for install-info ... Processing triggers for man-db ... Processing triggers for ureadahead ... ureadahead will be reprofiled on next reboot
Ubuntu and Mint set up the package so that it will start and will start on the system’s next boot.
Using psacct
The psacct package keeps track of a number of pieces of system information. For example, the command ac can be used to determine how may hours of connect time have been used.
CORP\mcole@markab ~ $ ac total 10.30
This shows that users have been connected to the system for 10.30 hours. If we want to see that number broken down by user, we can run
CORP\mcole@markab ~ $ ac -p zathras 0.39 valen 5.56 CORP\zack 0.04 CORP\mcole 0.44 CORP\ivanova 3.87 total 10.31
to see that essentially all of the time has been from the domain users valen amd ivanova, with the other users contributing essentially nothing to the total. We can use ac -d
CORP\mcole@markab ~ $ ac -d Feb 1 total 6.17 Feb 7 total 2.88 Feb 14 total 0.04 Today total 1.31
to get a breakdown by date; we can even combine them as ac -dp
CORP\mcole@markab ~ $ ac -dp valen 1.91 CORP\ivanova 3.87 zathras 0.39 Feb 1 total 6.17 valen 2.88 Feb 7 total 2.88 CORP\zack 0.04 Feb 14 total 0.04 valen 0.77 CORP\mcole 0.55 Today total 1.32
Another useful feature of this package is the ability to determine the commands that have been run on the system. For example, to find out each time the command su has been run, you can execute
CORP\mcole@markab ~ $ sudo lastcomm su [sudo] password for CORP\mcole: su S CORP\zac __ 0.00 secs Sat Feb 15 09:37 su S valen __ 0.00 secs Sat Feb 15 09:34 su CORP\mco pts/1 0.00 secs Sat Feb 15 09:37 su CORP\mco pts/0 0.00 secs Sat Feb 15 09:34
Note that this command requires root privileges or su. Running
CORP\mcole@markab ~ $ sudo lastcomm CORP\\mcole bash F CORP\mco pts/0 0.00 secs Sat Feb 15 09:51 python CORP\mco pts/0 0.05 secs Sat Feb 15 09:51 sudo S CORP\mco pts/0 0.00 secs Sat Feb 15 09:51 sudo S CORP\mco pts/0 0.00 secs Sat Feb 15 09:51 bash F CORP\mco pts/0 0.00 secs Sat Feb 15 09:51 python CORP\mco pts/0 0.07 secs Sat Feb 15 09:51 sudo S CORP\mco pts/0 0.00 secs Sat Feb 15 09:49 lastcomm CORP\mco pts/0 0.00 secs Sat Feb 15 09:49
we find the command recently executed by the domain user mcole; note the use of a pair of backslashes (which are normally used as escapes in bash). You can read the man page for lastcomm to determine the significance of the flags (e.g. the S that occasionally appears.)
If you want to find out information about previously run commands, you can use the sa tool; it is able to give you information on the cpu usage of all of the commands run on the system. See the man page for more details.
Who is Logged on the System?
The command w can be used to determine who is logged into the system.
CORP\mcole@markab ~ $ w 09:59:10 up 52 min, 5 users, load average: 0.31, 0.25, 0.18 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT CORP\mco tty9 :0 09:39 52:08 2.55s 0.27s gnome-session - CORP\mco pts/0 :0.0 09:39 0.00s 0.14s 0.00s w CORP\zac pts/1 hyach.corp.class 09:56 3:08 0.09s 0.09s -bash CORP\mco pts/2 :0.0 09:57 1:13 0.09s 0.09s bash valen pts/3 10.0.6.101 09:59 4.00s 0.08s 0.08s -bash
Here we see three users logged in (CORP\zack, CORP\mcole) and valen, with two SSH logins from two different hosts.
Similar information is provided by the command who:
CORP\mcole@markab ~ $ who CORP\mcole tty9 2014-02-15 09:39 (:0) CORP\mcole pts/0 2014-02-15 09:39 (:0.0) CORP\zack pts/1 2014-02-15 09:56 (hyach.corp.class.tu) CORP\mcole pts/2 2014-02-15 09:57 (:0.0) valen pts/3 2014-02-15 09:59 (10.0.6.101)
without the truncation of the user names.
The data these commands rely on is stored in the file /var/run/utmp.
You may also want to know not who is currently logged in, but rather the users that have logged in recently. This information is available through the command last
CORP\mcole@markab ~ $ last valen pts/3 10.0.6.101 Sat Feb 15 09:59 still logged in CORP\mco pts/2 :0.0 Sat Feb 15 09:57 still logged in CORP\zac pts/1 hyach.corp.class Sat Feb 15 09:56 still logged in CORP\mco pts/0 :0.0 Sat Feb 15 09:39 still logged in CORP\mco tty9 :0 Sat Feb 15 09:39 still logged in CORP\mco pts/1 :0.0 Sat Feb 15 09:37 - 09:38 (00:01) CORP\mco pts/1 :0.0 Sat Feb 15 09:36 - 09:37 (00:00) CORP\mco pts/0 :0.0 Sat Feb 15 09:34 - 09:38 (00:04) CORP\mco tty9 :0 Sat Feb 15 09:34 - 09:38 (00:04) valen pts/0 :0.0 Sat Feb 15 09:11 - 09:34 (00:22) ....
This data is stored in the file /var/log/wtmp, which is why attackers often clobber that file before departing a system.
Remember that if commands are executed via a bash shell, then the list of past commands for a user is stored in the file ~/.bash_history. Thus, if you want to see the last 15 bash commands executed by the user , you can simply check the list
CORP\mcole@markab ~ $ sudo tail -n4 /home/local/CORP/zack/.bash_history sudo ls pwd vi TextFile ifconfig
Remember though, that this file is written when the user exits their shell, so it won’t have the history of commands in a currently running shell.
What Processes are Running on the System?
If you want a real-time list of processes running on your system you can use the command top; here is the output of that program at a particular moment.
top - 10:07:34 up 1:00, 4 users, load average: 0.09, 0.10, 0.13
Tasks: 141 total, 1 running, 140 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.7%us, 0.3%sy, 0.0%ni, 99.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 1012348k total, 785476k used, 226872k free, 52424k buffers
Swap: 1046524k total, 4832k used, 1041692k free, 410880k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4469 root 20 0 176m 41m 10m S 1.3 4.2 0:03.66 Xorg
4792 CORP\mco 20 0 556m 19m 12m S 1.3 1.9 0:02.36 gnome-terminal
4491 CORP\mco 20 0 381m 10m 7840 S 0.3 1.1 0:00.40 gnome-session
4617 CORP\mco 20 0 590m 22m 12m S 0.3 2.3 0:01.17 gnome-panel
4628 CORP\mco 20 0 586m 16m 11m S 0.3 1.7 0:00.35 nm-applet
4632 CORP\mco 20 0 465m 12m 8852 S 0.3 1.3 0:00.30 bluetooth-apple
4635 CORP\mco 20 0 978m 27m 17m S 0.3 2.7 0:00.81 nautilus
4648 CORP\mco 20 0 479m 15m 10m S 0.3 1.6 0:00.51 indicator-apple
4992 root 20 0 0 0 0 S 0.3 0.0 0:00.28 kworker/0:0
1 root 20 0 36132 2856 1972 S 0.0 0.3 0:00.78 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:00.57 ksoftirqd/0
6 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
7 root RT 0 0 0 0 S 0.0 0.0 0:00.01 watchdog/0
The processes are listed in order, with the processes using most CPU listed at the top. If your system is slow or sluggish due to heavy load, this is the place to start diagnosing the problem.
If you want to know more about what your processes are up to, then you want to use the ps command. This is full of command switches that can give you some very different behaviors. If you want to know all of the processes running on a system, run
CORP\mcole@markab ~ $ ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.2 36132 2856 ? Ss 09:06 0:00 /sbin/init root 2 0.0 0.0 0 0 ? S 09:06 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? S 09:06 0:00 [ksoftirqd/0] root 6 0.0 0.0 0 0 ? S 09:06 0:00 [migration/0] root 7 0.0 0.0 0 0 ? S 09:06 0:00 [watchdog/0] root 8 0.0 0.0 0 0 ? S< 09:06 0:00 [cpuset] root 9 0.0 0.0 0 0 ? S< 09:06 0:00 [khelper] root 10 0.0 0.0 0 0 ? S 09:06 0:00 [kdevtmpfs] root 11 0.0 0.0 0 0 ? S< 09:06 0:00 [netns] root 12 0.0 0.0 0 0 ? S 09:06 0:00 [sync_supers] root 13 0.0 0.0 0 0 ? S 09:06 0:00 [bdi-default] ...
This gives so much information, it is often hides what you want to know. Suppose you want to find out if there are any SSH processes running; then you can pipe the output of the ps command directly to a grep to to some filtering:
CORP\mcole@markab ~ $ ps aux | grep sshd root 5272 0.0 0.2 49948 2868 ? Ss 09:55 0:00 /usr/sbin/sshd -D root 5556 0.0 0.4 99760 4972 ? Ss 09:59 0:00 sshd: valen [priv] valen 5575 0.0 0.2 100808 2764 ? S 09:59 0:00 sshd: valen@pts/3 372770881 5907 0.0 0.0 13576 932 pts/0 S+ 10:12 0:00 grep -- colour=auto sshd
Now we see SSH related processes from root, seldon, mallow, and hardin.
If you want to see the structure, showing which process spawned another, you can use either the -H or --forest switches:
CORP\mcole@markab ~ $ ps aux --forest USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND .... 372770881 4792 0.1 1.9 569576 19664 ? Sl 09:39 0:02 gnome-terminal 372770881 4797 0.0 0.1 30704 1992 ? S 09:39 0:00 \_ gnome-pty-h 372770881 4798 0.0 0.5 41352 5700 pts/0 Ss 09:39 0:00 \_ bash 372770881 5921 0.0 0.2 38524 2456 pts/0 R+ 10:13 0:00 | \_ ps aux 372770881 5501 0.0 0.5 41256 5496 pts/2 Ss+ 09:57 0:00 \_ bash 372770881 4863 0.0 0.6 304640 6340 ? Sl 09:39 0:00 /usr/bin/zeitge 372770881 4872 0.0 1.0 259544 10292 ? Sl 09:39 0:00 /usr/lib/zeitge 372770881 4879 0.0 0.0 11372 612 ? S 09:39 0:00 \_ /bin/cat 372770881 5112 0.0 0.2 46140 2156 ? S 09:55 0:00 /usr/lib/gvfs/g root 5272 0.0 0.2 49948 2868 ? Ss 09:55 0:00 /usr/sbin/sshd root 5556 0.0 0.4 99760 4972 ? Ss 09:59 0:00 \_ sshd: valen valen 5575 0.0 0.2 100808 2764 ? S 09:59 0:00 \_ sshd: v valen 5576 0.0 0.4 27368 4352 pts/3 Ss+ 09:59 0:00 \_ -ba
Notice that many users are replaced by their UID; this is typical for users on a Linux system connected to a Windows domain. To find the username that corresponds to a uid, you can try
CORP\mcole@markab ~ $ getent passwd 372770881 CORP\mcole:x:372770881:372769281:Marcus Cole:/home/local/CORP /mcole:/bin/bash
while if you want to go the other way, use id
CORP\mcole@markab ~ $ id CORP\\mcole uid=372770881(CORP\mcole) gid=372769281(CORP\domain^users) groups=372769281 (CORP\domain^users),372769280(CORP\domain^admins),372769340(CORP\denied^rod c^password^replication^group)
What Ports are Open on the System?
The command to determine what ports are open on the system is called netstat. When using netstat though, you should remember that Linux and Unix systems have two kinds of ports- network ports and Unix sockets. Unix sockets are used for communication by different processes on the same system, so in general we are uninterested in those.
The tool has a number of useful flags, including
- -v Be verbose
- -n Use numeric values for ports, rather than names
- -A inet (or –inet) Show only IP (v4) connections
- -A inet6 (or –inet6) Show only IPv6 connections
- -x Show only Unix sockets
- -t Show only TCP (v4/v6)
- -u Show only UDP (v4/v6)
- -p Show the PID for that connection
- -l Show listening sockets (not shown by default)
- -a Show listening and open sockets
- -r Show routing table
If you want to find out what is listening on your system, a good set of flags is
CORP\mcole@markab ~ $ sudo netstat -nlpv --inet Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:139 0.0.0.0:* LISTEN 564/smbd tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN 842/dnsmasq tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 5272/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 652/cupsd tcp 0 0 0.0.0.0:445 0.0.0.0:* LISTEN 564/smbd udp 0 0 127.0.0.1:53 0.0.0.0:* 842/dnsmasq udp 0 0 0.0.0.0:44614 0.0.0.0:* 632/avahi-daemon: r udp 0 0 10.0.255.255:137 0.0.0.0:* 764/nmbd udp 0 0 10.0.6.103:137 0.0.0.0:* 764/nmbd udp 0 0 0.0.0.0:137 0.0.0.0:* 764/nmbd udp 0 0 10.0.255.255:138 0.0.0.0:* 764/nmbd udp 0 0 10.0.6.103:138 0.0.0.0:* 764/nmbd udp 0 0 0.0.0.0:138 0.0.0.0:* 764/nmbd udp 0 0 0.0.0.0:5353 0.0.0.0:* 632/avahi-daemon: r
Here we were able both TCP and UDP ports in numeric form, and we also know the name & PID of the process that opened the port.
What Resources are Opened by What Process?
You can ask a broader question, namely what resources are being used by the system. These resources can be network sockets, but they can also be devices (like your USB drive) or files. The tool here is called lsof, and it can do a lot.
Suppose we wanted to get the same kind of information we saw with netstat; we can then try
CORP\mcole@markab ~ $ sudo lsof -i 4 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME smbd 564 root 29u IPv4 8429 0t0 TCP *:microsoft-ds (LISTEN) smbd 564 root 30u IPv4 8431 0t0 TCP *:netbios-ssn (LISTEN) avahi-dae 632 avahi 12u IPv4 8627 0t0 UDP *:mdns avahi-dae 632 avahi 14u IPv4 8629 0t0 UDP *:44614 cupsd 652 root 10u IPv4 17213 0t0 TCP markab.corp.class.tu:ipp (LISTEN) nmbd 764 root 9u IPv4 9015 0t0 UDP *:netbios-ns nmbd 764 root 10u IPv4 9016 0t0 UDP *:netbios-dgm nmbd 764 root 11u IPv4 9018 0t0 UDP markab.corp.class.tu:netbios-ns nmbd 764 root 12u IPv4 9019 0t0 UDP 10.0.255.255:netbios-ns nmbd 764 root 13u IPv4 9020 0t0 UDP markab.corp.class.tu:netbios-dgm nmbd 764 root 14u IPv4 9021 0t0 UDP 10.0.255.255:netbios-dgm dnsmasq 842 nobody 4w IPv4 9088 0t0 UDP markab.corp.class.tu:domain dnsmasq 842 nobody 5u IPv4 9089 0t0 TCP markab.corp.class.tu:domain (LISTEN) lwsmd 1428 root 64u IPv4 32658 0t0 TCP markab.corp.class.tu:38540->brakiri.corp.class.tu:ldap (ESTABLISHED) ubuntu-ge 4703 CORP\mcole 9u IPv4 25078 0t0 TCP markab.corp.class.tu:37864->mistletoe.canonical.com:http (CLOSE_WAIT) sshd 5272 root 3r IPv4 28471 0t0 TCP *:ssh (LISTEN) sshd 5556 root 3r IPv4 29166 0t0 TCP markab.corp.class.tu:ssh->drazi.corp.class.tu:34104 (ESTABLISHED) sshd 5575 valen 3u IPv4 29166 0t0 TCP markab.corp.class.tu:ssh->drazi.corp.class.tu:34104 (ESTABLISHED)
Here the flag -i 4 asks for information about all IPv4 connections; we can ask for IPv6 with -i 6
CORP\mcole@markab ~ $ sudo lsof -i 6 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME smbd 564 root 27u IPv6 8424 0t0 TCP *:microsoft-ds (LISTEN) smbd 564 root 28u IPv6 8427 0t0 TCP *:netbios-ssn (LISTEN) avahi-dae 632 avahi 13u IPv6 8628 0t0 UDP *:mdns avahi-dae 632 avahi 15u IPv6 8630 0t0 UDP *:55798 cupsd 652 root 9u IPv6 17212 0t0 TCP ip6-localhost:ipp (LISTEN) sshd 5272 root 4u IPv6 28475 0t0 TCP *:ssh (LISTEN)
If you want to know about TCP connections, you can use -i TCP, and if you want just port 22, then the flag would be -i TCP:22
CORP\mcole@markab ~ $ sudo lsof -i TCP:22 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME sshd 5272 root 3r IPv4 28471 0t0 TCP *:ssh (LISTEN) sshd 5272 root 4u IPv6 28475 0t0 TCP *:ssh (LISTEN) sshd 5556 root 3r IPv4 29166 0t0 TCP markab.corp.class.tu:ssh ->drazi.corp.class.tu:34104 (ESTABLISHED) sshd 5575 valen 3u IPv4 29166 0t0 TCP markab.corp.class.tu:ssh ->drazi.corp.class.tu:34104 (ESTABLISHED)
Suppose you are concerned that a connection from 10.0.6.104 might be hostile. Can you find out what other connections come from that host? Sure!
CORP\mcole@markab ~ $ sudo lsof -i @10.0.6.104 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME sshd 6278 root 3r IPv4 35409 0t0 TCP markab.corp.class.tu:ssh->hyach.corp.class.tu:50795 (ESTABLISHED) sshd 6280 CORP\zack 3u IPv4 35409 0t0 TCP markab.corp.class.tu:ssh->hyach.corp.class.tu:50795 (ESTABLISHED)
If you prefer numeric port numbers, add -P
CORP\mcole@markab ~ $ sudo lsof -i @10.0.6.104 -P COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME sshd 6278 root 3r IPv4 35409 0t0 TCP markab.corp.class.tu:22->hyach.corp.class.tu:50795 (ESTABLISHED) sshd 6280 CORP\zack 3u IPv4 35409 0t0 TCP markab.corp.class.tu:22->hyach.corp.class.tu:50795 (ESTABLISHED)
We can do more than just examine network connections with lsof. Suppose that you want to see what resources have been opened by all commands that contain the text "ssh"; then we use the -c flag; just be prepared for a lot of data.
CORP\mcole@markab ~ $ sudo lsof -c sshd | more
lsof: WARNING: can't stat() fuse.gvfs-fuse-daemon file system
/home/local/CORP/mcole/.gvfs
Output information may be incomplete.
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sshd 5272 root cwd DIR 8,1 4096 2 /
sshd 5272 root rtd DIR 8,1 4096 2 /
sshd 5272 root txt REG 8,1 517112 1049446 /usr
/sbin/sshd
sshd 5272 root mem REG 8,1 52120 921880 /lib
/x86_64-linux-gnu/libnss_files-2.15.so
sshd 5272 root mem REG 8,1 47680 921884 /lib
/x86_64-linux-gnu/libnss_nis-2.15.so
sshd 5272 root mem REG 8,1 35680 921876 /lib
/x86_64-linux-gnu/libnss_compat-2.15.so
sshd 5272 root mem REG 8,1 105288 921913 /lib
/x86_64-linux-gnu/libresolv-2.15.so
sshd 5272 root mem REG 8,1 14360 921860 /lib
/x86_64-linux-gnu/libkeyutils.so.1.4
sshd 5272 root mem REG 8,1 31200 1058091 /usr
/lib/x86_64-linux-gnu/libkrb5support.so.0.1
sshd 5272 root mem REG 8,1 158168 1058085 /usr
/lib/x86_64-linux-gnu/libk5crypto.so.3.1
sshd 5272 root mem REG 8,1 14768 921842 /lib
/x86_64-linux-gnu/libdl-2.15.so
sshd 5272 root mem REG 8,1 97248 921874 /lib
/x86_64-linux-gnu/libnsl-2.15.so
--More--
If you want to know all of the resources opened by a particular user, just specify the UID of the user
CORP\mcole@markab ~ $ sudo lsof -u CORP\\zack | more
lsof: WARNING: can't stat() fuse.gvfs-fuse-daemon file system
/home/local/CORP/mcole/.gvfs
Output information may be incomplete.
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sshd 6280 CORP\zack cwd DIR 8,1 4096 2 /
sshd 6280 CORP\zack rtd DIR 8,1 4096 2 /
sshd 6280 CORP\zack txt REG 8,1 517112 1049446 /usr
/sbin/sshd
sshd 6280 CORP\zack DEL REG 0,4 35464 /dev
/zero
sshd 6280 CORP\zack mem REG 8,1 43480 921623 /lib
/security/pam_gnome_keyring.so
sshd 6280 CORP\zack mem REG 8,1 18704 1049844 /lib
/x86_64-linux-gnu/security/pam_limits.so
sshd 6280 CORP\zack mem REG 8,1 10240 1049848 /lib
/x86_64-linux-gnu/security/pam_mail.so
sshd 6280 CORP\zack mem REG 8,1 10272 1049850 /lib
/x86_64-linux-gnu/security/pam_motd.so
sshd 6280 CORP\zack mem REG 8,1 14472 1053171 /usr
/lib/libck-connector.so.0.0.0
sshd 6280 CORP\zack mem REG 8,1 276392 921841 /lib
/x86_64-linux-gnu/libdbus-1.so.3.5.8
sshd 6280 CORP\zack mem REG 8,1 10296 921621 /lib
/security/pam_ck_connector.so
sshd 6280 CORP\zack mem REG 8,1 10304 1049867 /lib
/x86_64-linux-gnu/security/pam_umask.so
--More--
You can do the same by PID, just with the -p flag.
What do we Know about a Process?
Suppose you see a suspicious process. What can you quickly determine about it? A great deal of information about that process is stored in the /proc directory, in the subdirectory named after that PID.
For example, suppose we are interested in the process with PID 6460. We first change to the proper directory
CORP\mcole@markab ~ $ cd /proc/6460 CORP\mcole@markab /proc/6460 $ ls attr cpuset limits ns sched syscall autogroup cwd loginuid numa_maps schedstat task auxv environ maps oom_adj sessionid wchan cgroup exe mem oom_score smaps clear_refs fd mountinfo oom_score_adj stack cmdline fdinfo mounts pagemap stat comm io mountstats personality statm coredump_filter latency net root status
We can find the command line used to start the process:
CORP\mcole@markab /proc/6460 $ cat cmdline vi./TextFile
The current working directory is stored in cwd and the environment variables in environ
CORP\mcole@markab /proc/6460 $ sudo ls -l cwd lrwxrwxrwx 1 CORP\zack CORP\domain^users 0 Feb 15 10:52 cwd -> /home/local/CORP/zack CORP\mcole@markab /proc/6460 $ sudo cat environ TERM=xtermSHELL=/bin/bashSSH_CLIENT=10.0.6.104 50795 22SSH_TTY=/dev/pts/1 USER=CORP\zackMAIL=/var/mail/CORP\zackPATH=/usr/local/sbin:/usr/local/bin :/usr/sbin:/usr/bin:/sbin:/bin:/usr/gamesPWD=/home/local/CORP/zackLANG=en _US.UTF-8SHLVL=1HOME=/home/local/CORP/zackLOGNAME=CORP\zackSSH_CONNECTION =10.0.6.104 50795 10.0.6.103 22_=/usr/bin/vi
To find the precise executable, we check exe, and if we want to see what files are opened, we can see which file descriptor points where by checking fd
CORP\mcole@markab /proc/6460 $ sudo ls -l exe lrwxrwxrwx 1 CORP\zack CORP\domain^users 0 Feb 15 10:52 exe -> /usr/bin/vim.tiny CORP\mcole@markab /proc/6460 $ sudo ls -l fd total 0 lrwx------ 1 CORP\zack CORP\domain^users 64 Feb 15 10:56 0 -> /dev/pts/1 lrwx------ 1 CORP\zack CORP\domain^users 64 Feb 15 10:56 1 -> /dev/pts/1 lrwx------ 1 CORP\zack CORP\domain^users 64 Feb 15 10:56 2 -> /dev/pts/1 lrwx------ 1 CORP\zack CORP\domain^users 64 Feb 15 10:56 3 -> socket:[36731] lrwx------ 1 CORP\zack CORP\domain^users 64 Feb 15 10:56 5 -> /home/local/CO RP/zack/.TextFile.swp
Information about the memory map for the process is held in maps, while
/proc/PID/net/ is a directory of network information
With all of this information, can you determine what PID 15895 was doing?
Operational Awareness in Windows
As in the case of our Linux system, it is important to know what is occurring on our system.
Netstat
Like Linux systems, Windows systems also come with a tool called netstat. The tools are similar, though the options are quite different. To have netstat show all of the listenting ports, use the -a switch; to have the ports displayed in numeric form use -n and to include the PID of the process that opened the port, use -o. On our Windows File server, this results in a set of output like the following
C:\Users\abester>netstat -ano
Active Connections
Proto Local Address Foreign Address State PID
TCP 0.0.0.0:135 0.0.0.0:0 LISTENING 884
TCP 0.0.0.0:445 0.0.0.0:0 LISTENING 4
TCP 0.0.0.0:8000 0.0.0.0:0 LISTENING 1960
TCP 0.0.0.0:8089 0.0.0.0:0 LISTENING 1752
TCP 0.0.0.0:49152 0.0.0.0:0 LISTENING 676
TCP 0.0.0.0:49153 0.0.0.0:0 LISTENING 956
TCP 0.0.0.0:49154 0.0.0.0:0 LISTENING 988
TCP 0.0.0.0:49155 0.0.0.0:0 LISTENING 768
TCP 0.0.0.0:49179 0.0.0.0:0 LISTENING 768
TCP 0.0.0.0:49181 0.0.0.0:0 LISTENING 760
TCP 10.0.6.100:139 0.0.0.0:0 LISTENING 4
TCP 10.0.6.100:49187 10.0.6.101:9997 ESTABLISHED 1752
TCP 10.0.6.100:49199 10.0.6.50:445 ESTABLISHED 4
TCP 10.0.6.100:49224 184.29.105.33:80 CLOSE_WAIT 3320
TCP 10.0.6.100:49274 72.247.8.72:80 CLOSE_WAIT 2920
TCP 10.0.6.100:49336 23.3.109.48:80 ESTABLISHED 3984
TCP 10.0.6.100:49339 23.3.109.18:80 ESTABLISHED 3984
TCP 10.0.6.100:49340 23.3.109.48:80 ESTABLISHED 3984
TCP 10.0.6.100:49341 23.3.109.18:80 ESTABLISHED 3984
TCP 10.0.6.100:49342 23.3.109.48:80 TIME_WAIT 0
TCP 10.0.6.100:49343 23.3.109.18:80 TIME_WAIT 0
TCP 10.0.6.100:49348 168.63.124.173:80 ESTABLISHED 3984
TCP 10.0.6.100:49357 10.0.6.101:9997 TIME_WAIT 0
TCP 10.0.6.100:49359 10.0.6.101:9997 TIME_WAIT 0
TCP 10.0.6.100:49361 10.0.6.101:9997 TIME_WAIT 0
TCP 10.0.6.100:49363 10.0.6.101:9997 TIME_WAIT 0
TCP [::]:135 [::]:0 LISTENING 884
TCP [::]:445 [::]:0 LISTENING 4
TCP [::]:49152 [::]:0 LISTENING 676
TCP [::]:49153 [::]:0 LISTENING 956
TCP [::]:49154 [::]:0 LISTENING 988
TCP [::]:49155 [::]:0 LISTENING 768
TCP [::]:49179 [::]:0 LISTENING 768
TCP [::]:49181 [::]:0 LISTENING 760
UDP 0.0.0.0:123 *:* 1092
UDP 0.0.0.0:500 *:* 988
UDP 0.0.0.0:4500 *:* 988
UDP 0.0.0.0:5355 *:* 1276
UDP 10.0.6.100:137 *:* 4
UDP 10.0.6.100:138 *:* 4
UDP 10.0.6.100:1900 *:* 2832
UDP 10.0.6.100:52840 *:* 2832
UDP 127.0.0.1:1900 *:* 2832
UDP 127.0.0.1:52841 *:* 2832
UDP 127.0.0.1:55319 *:* 3984
UDP 127.0.0.1:60426 *:* 768
UDP 127.0.0.1:61607 *:* 1276
UDP 127.0.0.1:63587 *:* 988
UDP 127.0.0.1:64122 *:* 3320
UDP [::]:123 *:* 1092
UDP [::]:500 *:* 988
UDP [::]:4500 *:* 988
UDP [::]:5355 *:* 1276
UDP [::1]:1900 *:* 2832
UDP [::1]:52839 *:* 2832
UDP [fe80::7045:880e:7eca:360b%12]:1900 *:*
2832
UDP [fe80::7045:880e:7eca:360b%12]:52838 *:*
2832
We see here that there are both IPv4 and IPv6 connections, with both TCP and UDP represented. If you just wanted, say TCP on IPv4, you could add the additional switch
C:\Users\abester>netstat -anop TCP Active Connections Proto Local Address Foreign Address State PID TCP 0.0.0.0:135 0.0.0.0:0 LISTENING 884 TCP 0.0.0.0:445 0.0.0.0:0 LISTENING 4 TCP 0.0.0.0:8000 0.0.0.0:0 LISTENING 1960 TCP 0.0.0.0:8089 0.0.0.0:0 LISTENING 1752 TCP 0.0.0.0:49152 0.0.0.0:0 LISTENING 676 TCP 0.0.0.0:49153 0.0.0.0:0 LISTENING 956 TCP 0.0.0.0:49154 0.0.0.0:0 LISTENING 988 TCP 0.0.0.0:49155 0.0.0.0:0 LISTENING 768 TCP 0.0.0.0:49179 0.0.0.0:0 LISTENING 768 TCP 0.0.0.0:49181 0.0.0.0:0 LISTENING 760 TCP 10.0.6.100:139 0.0.0.0:0 LISTENING 4 TCP 10.0.6.100:49187 10.0.6.101:9997 ESTABLISHED 1752 TCP 10.0.6.100:49199 10.0.6.50:445 ESTABLISHED 4 TCP 10.0.6.100:49224 184.29.105.33:80 CLOSE_WAIT 3320 TCP 10.0.6.100:49274 72.247.8.72:80 CLOSE_WAIT 2920 TCP 10.0.6.100:49336 23.3.109.48:80 ESTABLISHED 3984 TCP 10.0.6.100:49339 23.3.109.18:80 ESTABLISHED 3984 TCP 10.0.6.100:49340 23.3.109.48:80 ESTABLISHED 3984 TCP 10.0.6.100:49341 23.3.109.18:80 ESTABLISHED 3984 TCP 10.0.6.100:49348 168.63.124.173:80 ESTABLISHED 3984 TCP 10.0.6.100:49359 10.0.6.101:9997 TIME_WAIT 0 TCP 10.0.6.100:49361 10.0.6.101:9997 TIME_WAIT 0 TCP 10.0.6.100:49363 10.0.6.101:9997 TIME_WAIT 0 TCP 10.0.6.100:49365 10.0.6.101:9997 TIME_WAIT 0
If you want to know the name of the process that opened the connection, you can add the -bswitch, however this requires administrator privileges.
C:\Windows\system32>netstat -anobp TCP Active Connections Proto Local Address Foreign Address State PID TCP 0.0.0.0:135 0.0.0.0:0 LISTENING 884 RpcSs [svchost.exe] TCP 0.0.0.0:445 0.0.0.0:0 LISTENING 4 Can not obtain ownership information TCP 0.0.0.0:8000 0.0.0.0:0 LISTENING 1960 [splunkweb.exe] TCP 0.0.0.0:8089 0.0.0.0:0 LISTENING 1752 [splunkd.exe] TCP 0.0.0.0:49152 0.0.0.0:0 LISTENING 676 [wininit.exe] TCP 0.0.0.0:49153 0.0.0.0:0 LISTENING 956 EventLog [svchost.exe] TCP 0.0.0.0:49154 0.0.0.0:0 LISTENING 988 Schedule [svchost.exe] TCP 0.0.0.0:49155 0.0.0.0:0 LISTENING 768 [lsass.exe] TCP 0.0.0.0:49179 0.0.0.0:0 LISTENING 768 [lsass.exe] TCP 0.0.0.0:49181 0.0.0.0:0 LISTENING 760 [services.exe] TCP 10.0.6.100:139 0.0.0.0:0 LISTENING 4 Can not obtain ownership information TCP 10.0.6.100:49187 10.0.6.101:9997 ESTABLISHED 1752 [splunkd.exe] TCP 10.0.6.100:49199 10.0.6.50:445 ESTABLISHED 4 Can not obtain ownership information TCP 10.0.6.100:49224 184.29.105.33:80 CLOSE_WAIT 3320 [wwahost.exe] TCP 10.0.6.100:49274 72.247.8.72:80 CLOSE_WAIT 2920 [jusched.exe] TCP 10.0.6.100:49336 23.3.109.48:80 CLOSE_WAIT 3984 [Explorer.EXE] TCP 10.0.6.100:49339 23.3.109.18:80 CLOSE_WAIT 3984 [Explorer.EXE] TCP 10.0.6.100:49340 23.3.109.48:80 CLOSE_WAIT 3984 [Explorer.EXE] TCP 10.0.6.100:49341 23.3.109.18:80 CLOSE_WAIT 3984 [Explorer.EXE] TCP 10.0.6.100:49348 168.63.124.173:80 ESTABLISHED 3984 [Explorer.EXE] TCP 10.0.6.100:49375 10.0.6.101:9997 TIME_WAIT 0 TCP 10.0.6.100:49377 10.0.6.101:9997 TIME_WAIT 0 TCP 10.0.6.100:49379 10.0.6.101:9997 TIME_WAIT 0 TCP 10.0.6.100:49381 10.0.6.200:135 TIME_WAIT 0 TCP 10.0.6.100:49382 10.0.6.200:49155 TIME_WAIT 0 TCP 10.0.6.100:49383 23.3.109.48:80 ESTABLISHED 988 ProfSvc [svchost.exe] TCP 10.0.6.100:49384 23.3.109.18:80 ESTABLISHED 988 [svchost.exe] TCP 10.0.6.100:49385 10.0.6.101:9997 TIME_WAIT 0
Full details about some of the other netstat options can be found with the /? switch:
C:\Users\abester>netstat /?
Displays protocol statistics and current TCP/IP network connections.
NETSTAT [-a] [-b] [-e] [-f] [-n] [-o] [-p proto] [-r] [-s] [-t] [interval]
-a Displays all connections and listening ports.
-b Displays the executable involved in creating each connection or
listening port. In some cases well-known executables host
multiple independent components, and in these cases the
sequence of components involved in creating the connection
or listening port is displayed. In this case the executable
name is in [] at the bottom, on top is the component it called,
and so forth until TCP/IP was reached. Note that this option
can be time-consuming and will fail unless you have sufficient
permissions.
-e Displays Ethernet statistics. This may be combined with the -s
option.
-f Displays Fully Qualified Domain Names (FQDN) for foreign
addresses.
-n Displays addresses and port numbers in numerical form.
-o Displays the owning process ID associated with each connection.
-p proto Shows connections for the protocol specified by proto; proto
may be any of: TCP, UDP, TCPv6, or UDPv6. If used with the -s
option to display per-protocol statistics, proto may be any of:
IP, IPv6, ICMP, ICMPv6, TCP, TCPv6, UDP, or UDPv6.
-r Displays the routing table.
-s Displays per-protocol statistics. By default, statistics are
shown for IP, IPv6, ICMP, ICMPv6, TCP, TCPv6, UDP, and UDPv6;
the -p option may be used to specify a subset of the default.
-t Displays the current connection offload state.
interval Redisplays selected statistics, pausing interval seconds
between each display. Press CTRL+C to stop redisplaying
statistics. If omitted, netstat will print the current
configuration information once.
Process Explorer
Another useful tool is Process Explorer. This is not part of a standard Windows installation, but instead is available as a separate download as part of the Sysinternals suite. This tool provides a graphical view of the processes running on the system, with the ability to simply click to find more information. Here is what the tool looks like on our Windows Server with the lower pane enabled (CTRL+L)
If you double click on a process, you can find out even more detail; if you double click on the jucheck process for example, you will be given a dialog box that lets you determine, for example, what TCP/IP connections that process has made. Here we see the process has connected to an Akamai server, almost certainly to tell me that my version of Java is old and out-of-date.
TCPView
If your interests are solely on the network connections being made to and from your system, you may want to try TCPView. This is another Sysinternals tool that provides just this information; it even highlights new connections for your attention (in green). Here is what the tool looks like while running on our File Server. You can see the NetBIOS session and directory services connections.
Process Monitor
Process Monitor is another Sysinternals tool; it captures the different actions of the processes running on your system.
ProcDot
If you want to analyze a process in detail, a great tool is ProcDot. This tool does not gather information like Process Monitor does, but rather it displays the results in a useful format.
Installation of the tool is trivial, as it simply comes in the form of a .zipped executable; just be sure to run the 32-bit or 64-bit version as appropriate.
The tool does rely on a couple of other tools though:
- GraphViz, which is a graphical visualization tool, and
- WinDump, which is a Windows version of the venerable tcpdump.
When the tool is run for the first time, it will ask for paths to these tools; simply provide them.
To record process data, start Process Monitor, but be sure to modify the configuration as required for ProcDot; in particular be sure you
- Disable "Show Resolved Network Addresses" from the Options menu
- From the Options menu, select Column selection, and
- Show Thread ID
- Do’t show the Sequence number
You also need to record the packets; I prefer doing so with Wireshark.
Once the activity you wish to monitor has finished, save the packet caputure as a .pcap file, and the Process Monitor data as a .csv file.
Load both files into ProcDot, and select the Launcher button. It will take some time to ingest the data, but when it is complete, you will be able to select a process. Here is an example of the result; you’ll need to zoom in to get the details.
The key to note here, is that this is an Internet Explorer process; looking at the graph you can see that it visited a web site at 10.0.2.250 on port 8080, then again came back out to the same host but now on port 4444. Some work was done in the registry, then a new notepad.exe process was spawned and execution moved to that thread.
This is a classic example of a Metasploit attack on a browser; in this case the attack was MS12-063 Microsoft Internet Explorer execCommand Use-After-Free Vulnerability.
Though ProcDot is fantastic for evaluating a process, remember that Process Monitor collects a lot of data, far more that could reasonably be stored over a long period of time. It is well suited for analyzing a particular piece of malware, but it does not double as an intrusion detection system.
Logon Sessions
If you want to know the details of all of the users logged in to your system, you can use the Sysinternals tool LogonSessions. This is a command line tool; when run on the file server it gives information like the following:
c:\Users\mcole\Desktop\logonSessions-1.21>logonsessions.exe
Logonsesions v1.21
Copyright (C) 2004-2010 Bryce Cogswell and Mark Russinovich
Sysinternals - wwww.sysinternals.com
[0] Logon session 00000000:000003e7:
User name: CORP\MINBARI$
Auth package: Negotiate
Logon type: (none)
Session: 0
Sid: S-1-5-18
Logon time: 2/15/2014 8:59:39 AM
Logon server:
DNS Domain: corp.class.tu
UPN: MINBARI$@corp.class.tu
[1] Logon session 00000000:000003e5:
User name: NT AUTHORITY\LOCAL SERVICE
Auth package: Negotiate
Logon type: Service
Session: 0
Sid: S-1-5-19
Logon time: 2/15/2014 8:59:55 AM
Logon server:
DNS Domain:
UPN:
[2] Logon session 00000000:00010baf:
User name: Window Manager\DWM-1
Auth package: Negotiate
Logon type: Interactive
Session: 1
Sid: S-1-5-90-1
Logon time: 2/15/2014 8:59:55 AM
Logon server:
DNS Domain:
UPN:
[3] Logon session 00000000:00010bbf:
User name: Window Manager\DWM-1
Auth package: Negotiate
Logon type: Interactive
Session: 1
Sid: S-1-5-90-1
Logon time: 2/15/2014 8:59:55 AM
Logon server:
DNS Domain:
UPN:
...
PS Utilities
The next set of tools are the PS utilities. These are all command line tools that let you perform various tasks on different systems, provided you have the proper permissions. Here is a brief summary of the tools, taken from the Microsoft web page that describes them.
- PsExec – execute processes remotely
- PsFile – shows files opened remotely
- PsGetSid – display the SID of a computer or a user
- PsInfo – list information about a system
- PsPing – measure network performance
- PsKill – kill processes by name or process ID
- PsList – list detailed information about processes
- PsLoggedOn – see who’s logged on locally and via resource sharing
- PsLogList – dump event log records
- PsPasswd – changes account passwords
- PsService – view and control services
- PsShutdown – shuts down and optionally reboots a computer
- PsSuspend – suspends processes
- PsUptime – shows you how long a system has been running since its last reboot
The PSLoggedOn tool when run on the file server is able to clearly distinguish between who was logged in locally and who was accessing shares; this is an improvement over what we say with logonsessions which showed too much information.
c:\Users\mcole\Desktop\PSTools>PsLoggedon.exe
PsLoggedon v1.34 - See who's logged on
Copyright (C) 2000-2010 Mark Russinovich
Sysinternals - www.sysinternals.com
Users logged on locally:
2/15/2014 11:41:48 AM CORP\mcole
Users logged on via resource shares:
2/15/2014 2:51:15 PM CORP\mcole
2/15/2014 2:45:36 PM (null)\mcole
2/15/2014 2:51:21 PM CORP\abester
2/15/2014 2:51:35 PM CORP\zack
BGInfo
This is another Sysinternals tool. Unlike these others, it does not monitor your system. It merely sets the desktop background for your system on startup based on the values of various variables. If you have many systems, it is convenient when switching systems to see right on the desktop the hosts’s IP address, name, and other critical system information. BGInfo can provide that for you.
Responding to a Suspicious Process
Students first learning how to defend a network tend to have a consistent response towards suspicious processes- nuke’em from orbit as quickly as possible.
This is often bad- even if the process actually is malicious. Killing a process deprives you of the chance to find out more about what is going on. How did the process get started? What processes are related? Has it opened a network connection? To where? Is that address connected to other processes?
Take some time and look carefully at the process using these tools, and find out all you can about it, if it is malicious, and how they may have gotten in. Then you can decide what to do, and if appropriate, then nuke it from orbit.
Darknet
- Best EDR Of The Market (BEOTM) – Endpoint Detection and Response Testing Tool
- AgentSmith HIDS – Host Based Intrusion Detection
- padre – Padding Oracle Attack Exploiter Tool
- Privacy Implications of Web 3.0 and Darknets
- DataSurgeon – Extract Sensitive Information (PII) From Logs
- Pwnagotchi – Maximize Crackable WPA Key Material For Bettercap
- HardCIDR – Network CIDR and Range Discovery Tool
- Socialscan – Command-Line Tool To Check For Email And Social Media Username Usage
- CFRipper – CloudFormation Security Scanning & Audit Tool
- CredNinja – Test Credential Validity of Dumped Credentials or Hashes
Windows Security
- An error has occurred; the feed is probably down. Try again later.
Security Focus Vulnerabilities
- An error has occurred; the feed is probably down. Try again later.
Security Bloggers Network
- An error has occurred; the feed is probably down. Try again later.
Krebs on Security
- Patch Tuesday, May 2024 Edition
- How Did Authorities Identify the Alleged Lockbit Boss?
- U.S. Charges Russian Man as Boss of LockBit Ransomware Group
- Why Your VPN May Not Be As Secure As It Claims
- Man Who Mass-Extorted Psychotherapy Patients Gets Six Years
- FCC Fines Major U.S. Wireless Carriers for Selling Customer Location Data
- Russian FSB Counterintelligence Chief Gets 9 Years in Cybercrime Bribery Scheme
- Who Stole 3.6M Tax Records from South Carolina?
- Crickets from Chirp Systems in Smart Lock Key Leak
- Why CISA is Warning CISOs About a Breach at Sisense
CyberOperations 