Server hacked (again)

Written by Clem on Wednesday, August 27th, 2008 @ 11:44 am | Main Topics

We’ve been hit again by this: http://www.linuxmint.com/blog/?p=235

The good news this time is that we’ll be faster to get rid of it (we’ve got really up to date backups), the bad news is that we’re still obviously vulnerable despite the measures we took the last time. I’ll ask Michael (our sysadmin) to look into this and to find out how this could have happened.

I’ll keep you posted.. I just found out about it a few minutes ago.

Update #1: A backdoor virus was found so it’s possible we got re-infected from the inside. I’m currently re-applying updates to clean the website first.

Update #2: The Wiki, forums, blog, software portal and main website are now clean.

Update #3: I’ll be upgrading the forums to the latest version of phpBb today so they might be offline or disabled for a while.

Update #4: The forums were upgraded to the latest version of phpBB. We’re missing the global announcements and there’s a little problem with the theme but overall they’re back online and they should be working fine.

Update #5: The blog was upgraded to the latest version of WordPress.

Update #6: The wiki was upgraded to the latest version of MediaWiki. We also know more about the problem now.. the first attack left a virus called PHP.RSTBackdoor.

Update #7: The planet was upgraded to the latest version of Gregarius.

Update #8: All the cleanup is done. All our tools were upgraded to their latest versions and we made new backups. Michael identified malware uploaded via mintUpload. We’re discussing the possibility to restrict, secure or even discontinue the free part of this service.

37 Responses to “Server hacked (again)”

  1. Fred Says:

    Just a shot in the dark, as I have no details, but check for a stub in the boot sector. This has been around for a while but this type of dual infection is difficult to get rid of once it is in place.

    Fred

  2. belovedmonster Says:

    Should anyone who visited on Windows scan their computer again?

  3. Clem Says:

    Yes.

  4. Carl Says:

    What about if we visited using Mint?

  5. Clem Says:

    Same as before. It shouldn’t harm your Linux box, but you may as well run an antivirus just to remove all potential traces of it.

  6. Tom Rinker Says:

    The people who create these pieces of malware code deserve a spot in the lowest circle of hell, right between the pedeophiles and the 419ers.

  7. Spas Says:

    Did I get that right it is a virus for Windows OS ?

  8. Eddie's Guitar Says:

    If you haven’t gotten off Servage,get off ASAP. I see many horror stories about them all the time, especially on Webhostingtalk.

  9. Alex Says:

    Yes this a virus for the Windows OS. People on Linux should be fine but those who visited the site on a Windows OS should run a virus scan.

    And man am I glad I fully switched to mint right before all of this :P .

  10. Sari Says:

    i think you should concentrate on the files or folders with 777 permission. cauz most of the time the trouble is from them :-)

    i hope this could help some how.

  11. Whomp Says:

    Why is the Linux Mint web site run on a Windows computer?

  12. Clem Says:

    ?? It’s run on Gentoo and it’s on a dedicated server. Servage doesn’t do our hosting anymore, they only act as our domain registrar.

  13. infamous Says:

    U guys r cool for actually telling us, I don’t know of any other web sites that do that for their users.

  14. exploder Says:

    The forum currently has no topics for threads displaying…

  15. Ahbrahm Says:

    My Wine had a virus last time this happened…It was in Hotkey.exe…I used avast workstation to get ride of it…love my Minty and I will Protect Her.!! :)

  16. kezdeth Says:

    @clem

    Once again, i appreciate your candor and honesty about something like this. I have confidence that you and the team will solve the problems and continue the great work you do on Mint. Thanks for a great distro and for your honest & hard work. You guys rock!

  17. merlwiz79 Says:

    The forum is missing the last post subject name.
    Other than that it seems fine.
    I actually have 0 new messages.
    It use to say 4 new messages when I had 0.

  18. Chris Brainard Says:

    You need to re-partition and reinstall. I had this happen and it is almost impossible to find where they broke in and what is infected.

    Install only what you need. Close all ports that aren’t used. Switch to unknown ports such as 29 for ftp instead of 21. Shut down anything anon to get into the server. Change all passwords of course. Make sure you limit your database access for web based programs and don’t put other system passwords into your database. They have found an sql injection in your web software.

  19. jungar Says:

    I ran the antivirus on windows and calmtk on linux… didnt get anything.. should i be worried?

    i have norton antivirus (2007 or 2006 edition) AND i got zone alarm security suit. (the antivirus in that is disabled cuz i got norton)

  20. matthews Says:

    Maybe you could get some white and grey hats to test the server set up.

  21. Jim Goulter Says:

    It just shows what decent people run the Mint site, that they immediately inform us of the problem. I occasionally access it from work so it is good to have this timely knowledge.

  22. alessandro Says:

    I dunno too much about servers. But somehow these frequent attacks represent that, both, Mint distro and Mint community are growing fast as hell, so people in the other side feel envy and jealousy about us :D It must be.

  23. Alex Says:

    Hmm, I hadn’t thought about the possibility of a Wine virus. Do you have any specific advice for Linux Mint users who browse this site? (That’s gotta be everyone, considering the default home page in Firefox is this page.) “Scan for viruses” is fine advice, but any specific steps to take?

  24. lucian_ge Says:

    i use GNU/Linux Mint for more than a year and never had problems with viruses,malware,etc,etc. the system is stable and very good!i totally forget about windows! :)

  25. dw5437 Says:

    I for one installed SELINUX on my Mint just in case. I also installed and use rkhunter. I highly recommend bot. Both are in synaptic.

  26. Justin Says:

    I realize one probably has nothing to do with the other, but it’s difficult to feel confident in Mint’s security when the website is being hacked repeatedly.

  27. poloman Says:

    Thanks for he virus warning. I just navigated here and saw it on the page…

    @alessandro
    Your probably right. But why don’t they target Ubuntu.
    Well i guess that Mint is Ubuntu derived.

    Kinda new to Linux but Mint finally got me well and truly in.
    Try a hardware firewall, if thats not already installed.

    j

  28. Acid_1 Says:

    hehe. windows…

  29. Mathieu Says:

    Your honesty in the matter is greatly refreshing. Still, it is troubling that they were able to do this two times in a row. Hopefully up to date servers will solve the problem and it wasn’t an underlying vulnerability in apache, PHP or mySQL.

  30. Matt Anderson Says:

    I don’t understand, if the site is hosted on a gentoo box and not a windows server, then why would it be a problem?

    Also, why not run on Mint?

    I’m new to Mint but I sure do love it so far!

  31. Husse Says:

    It’s not a quite simple matter. The safety for a system running Mint is in no way compromised by this. Unless you run Wine nothing has happened, Wine as well as Windows can get hit.
    And the server was not hit, no security hole in it was used.
    It is an SQL injection, meaning that commands where sent that tricked the server to execute “dangerous” commands, and the server “believed” they were issued by the right people. Somehow we had not secured ourselves completely against that. Unfortunately it is possible to do just about anything in this situation and a trojan was left behind that was not detected.
    The origin is a bot that “trawls” the net looking for vulnerabilities – any server can get hit by this (Linux, Mac, Windows)
    But of course our credibility is hit by this, marginally I’d say as we share what happened with the community.

  32. Will Says:

    Can’t you prevent SQL injection?? It’s a relatively simple matter once you identify query string concatenations and entry points. In fact you could skip that and just use the PHP function for filtering SQL escape characters on all input.

  33. Clem Says:

    Yes. We’ve audited all our code after the first attack. And we’ve upgraded all the PHP software we use to their latest versions after the second one :)

    I don’t think the second attack came from outside though.. I think the first one left a backdoor. Also, we’ve identified vulnerabilities with mintUpload.

  34. Chris Says:

    Clem, thanks for the update. I for one will continue to use and endorse Mint for my windows customers making the switch. I’m no expert on SQL, but what SQl database are you using? Would switching to something like Oracle 11g prevent this? I’m sure it’s expensive as all get out. But, might be a worth while investment.

  35. Husse Says:

    It’s not the database, but the code used to “manipulate” it that is the cause here – and Oracle – well they are definitely not in a hurry to patch….

  36. Please Says:

    I like mintupload. don’t take it away please.

  37. steogede Says:

    Clem, do you use any sort of application level firewall – mod_security can be very useful for protecting against this sort of thing. Obviously in addition to a regular firewall. Obviously if you are sure your PHP is now secure, then you can do with out ModSec, but it is a very useful extra layer of protection.


Trackbacks & Pingbacks