Checking existing verison shows 4.3p2:

$ ssh -v
OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008

simply tried upgrading by running yum using default repositories but it didn’t find any latest version. To grab the latest one, I have installed the CentALT repository, which usually have latest packages. I’ve documented the steps to install it in earlier post here. After having installed CentALT, I tried again but found some dependency issue:

$ yum upgrade openssh
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * addons: mirrors.usc.edu
 * base: linux.mirrors.es.net
 * centosplus: mirror.stanford.edu
 * extras: linux.mirrors.es.net
 * updates: linux.mirrors.es.net
CentALT                                                                                                                     |  951 B     00:00     
CentALT/primary                                                                                                             |  85 kB     00:01     
CentALT                                                                                                                                    256/256
Excluding Packages in global exclude list
Finished
Setting up Upgrade Process
Resolving Dependencies
--> Running transaction check
--> Processing Dependency: openssh = 4.3p2-72.el5_6.3 for package: openssh-clients
--> Processing Dependency: openssh = 4.3p2-72.el5_6.3 for package: openssh-server
---> Package openssh.x86_64 0:5.5p1-1.el5 set to be updated
--> Running transaction check
---> Package openssh-clients.x86_64 0:5.5p1-1.el5 set to be updated
--> Processing Dependency: libedit.so.0()(64bit) for package: openssh-clients
---> Package openssh-server.x86_64 0:5.5p1-1.el5 set to be updated
--> Finished Dependency Resolution
openssh-clients-5.5p1-1.el5.x86_64 from CentALT has depsolving problems
  --> Missing Dependency: libedit.so.0()(64bit) is needed by package openssh-clients-5.5p1-1.el5.x86_64 (CentALT)
Error: Missing Dependency: libedit.so.0()(64bit) is needed by package openssh-clients-5.5p1-1.el5.x86_64 (CentALT)

little searching revealed its due to libedit library which we needs to bump up, got its rpm from phone.net and installed:

$ cd /usr/src 
$ wget ftp://ftp.pbone.net/mirror/atrpms.net/el5-x86_64/atrpms/stable/libedit0-3.0-1.20090722cvs.el5.x86_64.rpm
$ rpm -ivh libedit0-3.0-1.20090722cvs.el5.x86_64.rpm

Then again tried upgrading openssh and its went fine:

$ yum upgrade openssh
...
...
===================================================================================================================================================
 Package                                 Arch                           Version                              Repository                       Size
===================================================================================================================================================
Updating:
 openssh                                 x86_64                         5.5p1-1.el5                          CentALT                         314 k
Updating for dependencies:
 openssh-clients                         x86_64                         5.5p1-1.el5                          CentALT                         598 k
 openssh-server                          x86_64                         5.5p1-1.el5                          CentALT                         328 k
 
Transaction Summary
===================================================================================================================================================
Install       0 Package(s)
Upgrade       3 Package(s)
 
Total download size: 1.2 M

Restart the sshd service and confirm new version:

$ /etc/init.d/sshd restart
Stopping sshd:                                             [  OK  ]
Starting sshd:                                             [  OK  ]
 
$ ssh -v
OpenSSH_5.5p1, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008

We are done.

You may also like other related articles on this blog:
* Quickly change your ssh port from defualt 22 to something higher
* Top 5 most useful tools for Linux Admin
* Top 5 steps to secure your production Linux host
* Ensuring secure access to production Linux hosts
* Bash script to backup essential log files in Linux Server
* Install and Configure DenyHost to prevent brute-force attacks

SSH v1 is not very safe and if you are looking to pass your site/server for PCI compliance then you must disable it. Don’t worry it is too easy to do.
Open /etc/ssh/sshd_config file and disable version 1:

$ vi /etc/ssh/sshd_config

find line: #Protocol 2,1 and remove 1 from it and then un-comment it, the final line should look like this:

$ cat /etc/ssh/sshd_config | grep Protocol
Protocol 2

Restart SSH service to apply changes.

$ service sshd restart

Step 1. Disable direct root access.
Login should be allowed using simple/wheel user and then from there you can switch to root or use sudo to execute commands under root privileges. Let’s disable direct root login:

$ vi /etc/ssh/sshd_config
Look for "PermitRootLogin yes" line in it and change it to "PermitRootLogin no".

Save the file, check for syntex and reload sshd service to make it effective:

$ sshd -D -f /etc/ssh/sshd_config
$ service sshd reload

You can create a new user now which will be used for login, leave it if there’s already a user for this purpose. Don’t forget to have a strong password for root and for this user as well.

$ useradd usertogetin
$ passwd usertogetin

Our step 1 is completed, but why it’s a necessary? A direct access to root user with a weak password is sure-shot invite for compromising security and get inside your server easily. Another powerful way to make it more difficult to hack in server is to change default ssh port 22 to something else, you can find the steps here to do that.

You should also keep looking into security related logs like /var/log/secure for clue about possible attempt of logins from remote places.

Let me show you something:

$ tail /var/log/secure
Feb  6 07:41:56 ip-184-x-xx sshd[20421]: Failed password for root from xx.xx.xx.xx port 48976 ssh2
Feb  6 07:41:57 ip-184-x-xx-x sshd[20422]: Received disconnect from xx.xx.xx.xx: 11: Bye Bye
Feb  6 07:41:57 ip-184-x-x-x sshd[20424]: reverse mapping checking getaddrinfo for 184-xx-xx-xx.abcd.net failed - POSSIBLE BREAK-IN ATTEMPT!

It shows somebody is trying to get in your server by trying random password for root user. You can block such IPs but most important step is to block root access and use strong passwords.

Example of another instance:

$ tail /var/log/secure
Feb  6 07:46:58 ip-184-168-71-156 sshd[20769]: input_userauth_request: invalid user deathrun
Feb  6 07:46:58 ip-184-168-71-156 sshd[20768]: pam_unix(sshd:auth): check pass; user unknown
Feb  6 07:46:58 ip-184-168-71-156 sshd[20768]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=184.xx.xx.xx

Here login attempts are done using random username and password. It implies that do not name your entry/wheel username as obvious one, choose somewhat difficult but what you can remember text for your username. Of course, Password must be strong which should includes chars,numbers and special symbols.

There should be mechanism to block an IP from where hacking/cracking/invalid/failed login attempts come from. If you see that there are 10s of failed login attempt from a particular IP address, then you should block it immediately. To facilitate this thing automatically, there a very good tool available, called as DenyHosts. I’ve already written an article explaining how to download/install/configure DenyHosts in Server. Its a must have package in any prod server.

Step 2. Remove login shell from all other users except wheel user.
You have disabled the remote access to root but do you know through how many users, a person can get inside to your server? many application while installation creates their user but with a major security flaw, they set login shell to ‘bash’ and this loophole can be exploited.

[root@ip-184-168-71-156 log]# grep 'bash' /etc/passwd
root:x:0:0:root:/root:/bin/bash
usertogetin:x:500:500::/home/usertogetin:/bin/bash

Here you can see only two entries, which are fine but if you see more lines here like with users mysql, ftp, postgre etc. then please change shell of these users to nologin so by using these users, login can’t be done.

# usermod -s /sbin/nologin mysql

Here we have modified shell of user MySQL from /bin/bash to /sbin/nologin. You should do this for other users as well except root and wheel user you created earlier.

Step 3. Check open ports and stop unused services.

$ netstat -ant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 184.xx.xx.xx:53             0.0.0.0:*                   LISTEN      
tcp        0      0 184.xx.xx.xx:53             0.0.0.0:*                   LISTEN      
tcp        0      0 127.0.0.1:53                0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:21                  0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:25                  0.0.0.0:*                   LISTEN      
tcp        0      0 :::80                       :::*                        LISTEN      
tcp        0      0 :::22                       :::*                        LISTEN      
tcp        0      0 :::443                      :::*                        LISTEN      
tcp        0      0 ::ffff:xx.xx.xx.xx:80       ::ffff:xx.xx.xx.xx:52502    TIME_WAIT   
tcp        0   3152 ::ffff:xx.xx.xx.xx:22       ::ffff:xx.xx.Xx.xx:5222     ESTABLISHED
..
..

Apart from whatever showing above, your list may have many such lines with different ports with Listen/Established state. Here we used netstat command to list all active/listen/established ports with their IP addresses. You can see here some well known port numbers like 53 (DNS), 21 (FTP), 25 (SMTP), 80 (HTTP), 22 (SSH), 443 (HTTPS). These seems fine but if you doesn’t need any of these services, stop that program. In case you see some unusual listening ports, then you should verify their applications and stop them if they are not necessary.

Here by look at port numbers, you can’t get the information about which application is associated with any particular port number. Just for example, we want to know, which application is listening or using the port number 443. We can get this information by lsof command:

$ lsof -i :443
# lsof -i :443
COMMAND   PID   USER   FD   TYPE DEVICE SIZE NODE NAME
httpd    4335 apache    6u  IPv6 369447       TCP *:https (LISTEN)
httpd    6888   root    6u  IPv6 369447       TCP *:https (LISTEN)
httpd    7343 apache    6u  IPv6 369447       TCP *:https (LISTEN)
...
...

So port 443 is used by httpd (Apache) application/process to provide secure http services.

Step 4. Solid Backup Plan
Ensure that your server have a solid backup plan. All important files/databases/scripts etc. should be included in backup and it should be frequent. I have written a script which can help you to take frequent backup of essential log files/database. You can find it here with its usage.

Step 5. Regular upgrades
Ensure that regular upgrades/security patches/fixes of packages must be applied on time to so that all known vulnerabilities are fixed. You can check sites like http://www.securityfocus.com/ for such vulnerabilities.

Installation of more advanced packages like SELinux and/or AppArmour and/or rootkit detection etc. will provide you higher level of security.

Please note that nothing can ensure a 100% secure server but the steps listed above can ensure that your server have adequate security mechanism in place. One more thing, if unfortunately your security in your Server get compromised then never trust that server as a cracker can have access to 100% of the system. In such cases where we don’t have 100% confidence that Server is ok, its better to rebuild the Server from scratch and restore backup to make it up again.

I’d appreciate your comments sharing your experience or approaches which can help us securing our Servers more effectively.

I’ve suggested them some points as per described below for ensuring secure access to servers. They have 5-6 Linux servers. This is obviously may not be the best way and I’m as always appreciate if you can give your suggestion in comments. My approach is that from 6 servers, we will be able to login only in 2 servers from remote through key based access and from these 2 server, we can access remaining. Here’s what we did:

1. Disable root access
Completely disable root login access from remote. Period. Open /etc/ssh/sshd_config and add/remove comment from this line:

# vi /etc/ssh/sshd_config

PermitRootLogin no

2. Login only through non-root user
Create non-root user and create public/private key pair for it:

$ adduser loginu

login to ‘loginu’ user created above, or if you are in root, just su:

# su loginu
$ ssh-keygen -t dsa

Enter details while generating keys, enter good passphares and always remember it. Now you can go ahead and disable password based access completely so user can only login by using keys but this may be too restrictive or problematic for them if they forget passphares etc. if you want to go ahead, make sure these statements are there in /etc/ssh/sshd_config file:

$ exit
# vi /etc/ssh/sshd_config

PasswordAuthentication no
PubkeyAuthentication yes
AuthorizedKeysFile      .ssh/authorized_keys

copy the key you created earlier (there should be two files in ~loginu/.ssh/ directory: id_dsa, id_dsa.pub. so copy id_dsa) to your pc so from next time you can use this key to login into the server.

Just make sure you are able to login through ‘loginu’ user before applying these ssh settings. Jump to terminal in your pc and try to login with key:

$ ssh -i id_dsa loginu@your.server.ip

It will ask for passpahres and after supplying it you should be able to login into the server. Please make note that this is very confidential key and store it in good place/directory. Alternatively you can also generate keys in your own pc and store them at server to facilitate login. But if you want flexibility to have only one key (like carry it in your usb stick) and be able to login with it, I found this approach good to use server keys instead of pc keys.

3. (Optional) Restrict login by IPs
Now come back in server. You can further strengthen security by allowing only select IPs to log in:

# vi /etc/ssh/sshd_config

AllowUsers loginu@aa.aa.aa.aa loginu@bb.bb.bb.bb loginu@cc.cc.cc.cc

Here replace aa/bb/cc with actual IP addresses from where you want to allow access.

Going ahead, optionally, You can also change port for ssh from default 22 to other by using this guide but as I think we are only allowing access through keys and from select remote places only, this you may skip.

Reload sshd daemon to apply settings which you have set in /etc/ssh/sshd_config by:

# service sshd reload

Without closing current login session, try to login again from other terminal to check you are able to login into the server.

4. Secure other servers
As mentioned earlier, I preferred to treat first 2 server as ‘login’ server in which we can login from anywhere using user ‘loginu’ with key and then can login to other servers. So effectively other servers would not allow direct access from remote. Jump to server 3-6 and set following:

# vi /etc/ssh/sshd_config

AllowUsers loginu@aa.aa.aa.aa root@aa.aa.aa.aa loginu@bb.bb.bb. root@bb.bb.bb.bb

here aa.aa/bb.bb indicates IP address of server #1 and #2 (login servers). So in this (#3) server we can login from those server(s) only. After making changes, reload ssh daemon to apply settings in all of these servers:

# service sshd reload

5. Other services
I suggested to disable every service that we don’t need in servers. That’s the best approach to secure them . These servers has role of web servers and rsync process is there to sync files. In that case, created another non-root user:

# adduser rsyncuser

generate keys (you can generate passphares less keys) for this user as well. Create same user in all other servers and put first server’s (from where rsync initiate) keys in them. Dont allow this user to login from remote but only from server where rsync initiate. I’ve documented rsync process here, if you want to go ahead and configure it. Similarly, if you need services like FTP then allow this only from selected IP address (by configuring /etc/hosts.allow) or firewall etc.

6. Configure DenyHosts
To further prevent attacks and block any IP address from which several failed login attempt originated, you should configure DenyHosts script ( I have documented howto on DenyHosts here) or equivalent.

Other Most Read Articles:
* Top 5 Linux commands for Administrators.
* Install and configure Munin/Monitor for monitoring.
* Change time zone in your Linux machine quickly.
* Detect directory changes in Linux.
* Script to backup essential log files.

Suddenly after few days, same problem popped up. My friend complained that rss feed page of blog is not working. Now my eyebrow raised as I didn’t touched the site from my side since last cleanup (few days before) and entire blog files were again infected by that malware. Submitted a ticket with GoDaddy explaining everything but got lame response having points like We were unable to find any issue in your site, Make sure you are not running outdated version of blogging software blah blah.

When I searched over net about that issue, there were thousands of users having sites/blogs on GoDaddy who were facing same thing. I got many references/conversations like here and here indicating massive no. of users facing this issue. Whatever, I didn’t got any promising/supportive response from GoDaddy even after its very clear that hacking/malware spread was happened in their servers and instead of accepting and investigating, all they replying is crap.

This was a good indication for me to hunt another hosting provider and after lots of hunting, I zeroed down on nearlyfreespeech.net. These guys are simply awesome. Pay per usage plan, only SSH access (no FTP by default), very simple but intuitive interface and dirt cheap rates. I promptly shifted my personal/financial blog www.investorshine.com from GoDaddy to them and its now more than 3 months and honestly, I’m more than happy with them. Now going to shift this blog also very soon.

Just an update as on July, 2011, I’m running this blog on HostGator and very satisfied with their services.

At the end, GoDaddy lost one more customer having several domains and deluxe Linux hosting package. I will put all efforts to educate my friends as well to keep a distance from GoDaddy, at least as far as Hosting is concerned. Do you also have kind of same experience with GoDaddy? Please share here in comments for benefit of all community.