How to keep your user's secure, even when one has been compromised
Your database has been compromised... now what??
The name of the game here is to slow down attackers. Everyone has heard of salts, and if you have half a brain cell, your company is using a strong encryption algorithm, such as bcrypt, scrypt, mcrypt, or pbkd2f. But if your hackers have access to your db, what should you do?
My password hashing always consists of 2 salts for every password. One permanent salt, stored as an environment variable, and 1 random salt stored per user. These two salts are combined when your password is hashed to help slow down an attack on your DB. If a hacker gets a hold of your database, they only have half the salt used to hash your passwords. Granted, if an attacker has gotten a hold on your app server, your pooch is already screwed since your attackers have access to your DB by proxy. But, if attackers have only gotten a hold of your DB, change your DB credentials and your server-side salt immediately (keep a backup of the old salt offline, just in case, but make sure it is somewhere not web accessible). This will not stop your database from being cracked once it's been downloaded, but it will slow attackers down. Hopefully it will retard those with malicious intent down enough that your users will have a chance to change their passwords before anything can be done with what is uncovered.
Next, you should notify every user you have of this breach immediately. You might think to save face by trying to hide news of your security breach, but that is WRONG (and should be considered criminal). Your users are not smart. They reuse passwords, and use insecure passwords ALL. THE. TIME. They need to be given notice the second you think you might have had a data leak so they can change passwords on all their other accounts with the same credentials before the attack spreads.
Now that you've changed your credentials, and notified your users, you can get to work on uncovering the reason for the breach. Chances are the breach is because you or one of your employees is using the same password they used somewhere else that has been breached previously. Don't believe me? Just look at the recent release of Dropbox passwords. Their breach was caused by an employee that was using the same password for their LinkedIn account as they were for their work credentials. After LinkedIn was compromised, it was then short work to steal data from Dropbox as well.
Secure your data before it's a problem
Knowing what to do in case of emergency is important. Even more important, however, is preventing that emergency in the first place. Here are some important guidelines that should be followed to the letter:
1) Force employees to change their passwords on a rotating schedule. No employee (or non-physical user such as a db user) should have the same password for more than 90 days, preferably shorter. This is extremely easy to implement and there's no reason you should be allowing anything otherwise.
2) Employees should be explicitly forbidden from reusing a password ANYWHERE. I cannot stress this enough. You cannot control the passwords that your users decide to enter, but you absolutely have power over the people you pay to work on your system. No employee should use any password that they also use outside of work. They also should never use the same password in more than one place within your company.
3) Use STRONG passwords. Any password that can be potentially used to cause a data breach should be no less than 32 random characters. I personally use 64 character or greater passwords for any critical data. If you don't want to type it out every time, use a GOOD password manager, such as last pass.
As long as you follow the three rules above to the letter of the law, you should stay relatively safe. For more information on the Dropbox data breach and the insecure SHA1 encryption they were using check out this blog post and this for SHA1.