My old personal site got hacked and trashed. When I built it, I didn’t know much about the web world, and stumbled into a rough custom solution. I had stood up a MediaWiki server, which uses MySQL, as a place to record some project ideas. I was doing this funky thing where I would write a blog/project entry in MediaWiki’s markdown forms, and then convert them to HTML to display on my site. It worked well enough for what I was using it for, but it was pretty ugly by most standards.
I use the same machine for my personal site as a bunch of other projects. In the past few months, I noticed that some of my other projects were periodically failing. They’d go down for no reason. I initially blamed these failures on a recent project. This project has an infinite loop awaiting websocket connections, and I didn’t spend much time verifying that this loop was behaving correctly. I assumed for weeks that this was the root cause. Every time I noticed an issue, I would hop on the server, restart the related services, and the issues would clear up. Didn’t seem to warrant further inspection.
After the third or fourth round of this service restart game, I started getting more suspicious and tailed the Apache logs, and BOOM - smoking gun immediately. Tons of requests that took the form:
220.127.116.11 - - [18/Jan/2016:06:33:26 +0000] "GET /index.php?title=One_Piece_Necklaces_-_On_Line_Shopping_Security_Tips_To_Get_A_Safe_And_Fun_Digital_Shopping HTTP/1.0" 302 695 "http://wiki.jacobra. com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36"
Weird IP address, weird user agents, weird requests to index.php with crazy query string args. Terrible. Then I looked at the MySQL database behind the MediaWiki site.
mysql> select * from site_stats\G *************************** 1. row *************************** ss_row_id: 1 ss_total_views: 428036 ss_total_edits: 103020 ss_good_articles: 3 ss_total_pages: 87945 ss_users: 16975 ss_active_users: 7390 ss_images: 1
Ugh, no good. 428000 views and 16000 users on a server I’ve never shared with anyone… Gotta shut this down.
I clicked on a few of the articles that had been created on my MediaWiki. They were very strange. Each one had some title, like “One Piece Necklaces - On Line Buying Security Tips For A Safe And Enjoyable Virtual Shopping”, or something something dieting advice, or some medical supplement idea. And the actual articles were somewhat lengthy and poorly written. Written well enough to vaguely pass as a blog post, but had tons of typos and grammar mistakes. Notably, each of them had links to other websites, which they themselves were similarly poorly written, but vaguely passable blog posts on the same topic.
Originally I blacklisted the IP addresses I saw. I figured it was just some person with a few IP addresses and a script, but my blacklist grew to about 15, and the unique IPs kept coming. Maybe this is a botnet? The blacklist wasn’t going to work, plan B. I ended up creating an Apache rule that redirects all requests with index.php in them, to a download speed test site. Specifically, to a 1GB zip file. muhaha! Hopefully this will cause the requests to stop.
I poked around a few other spots on my server to see if anything else was compromised, and it didn’t seem likely. Most assuring was that no other user had ever gotten SSH access, so I’m fairly confident that this attack was just abusing the MediaWiki instance.
I’m not actually sure how the attacker gained access to the MediaWiki server. Could be that I didn’t lock it down correctly, could be that it’s out of date. I’m running 1.19, and looks like there’s a fairly big list of known vulnerabilities on this version. Latest stable is 1.26.
My theory is that the person behind these requests is running some sort of shady SEO operation. Another datapoint is that my MediaWiki server had 185514 external links. My MediaWiki server, by linking to other blogs, is building credibility for other sites, since part of Google’s Page Rank algorithm measures the number of incoming links a page has.
Sad to see it go, but it’s time to upgrade. Hugo is where I’ve gone. It’s got an active developement community, lots of theming options, and supports markdown entries.
RIP Old Site