After weeks and weeks of deadlines at work, and sickness and minor injuries at home, I finally had some time to get back to work on UserLand this weekend.
There are still too many servers running UserLand’s sites, making the hosting fees higher than they need to be, which of course limits the amount of time available to get things under control, so the first priority is still to consolidate servers. Not only does this reduce costs, but it also makes it easier to migrate later if necessary.
A couple of weeks ago, I moved radio.userland.com to the main UserLand server that hosts the UserLand.com site, and the Manila and Frontier sites. I started work tonight on another server, with an eye toward moving the sites to the same machine.
I haven’t finished this one yet, because as I was exploring what data and sites were there, I noticed a huge amount of traffic on one sub-domain for /RPC2. What on earth is this RPC handler? Well, it turns out it’s spam. The requests were all coming in to rcs.salon.com, which was the backend host for the Salon Blogs, and the RPC method was weblogUpdates.ping. Whoops!
So for years, there’s been an open ping server sitting there, waiting for spammers to send URLs to it, and posting links to their spam sites up on the web, under a prominent domain name. This had to stop right away.
Fortunately it was pretty easy to do. UserLand already had code in place to drop requests from spammers. We called it the blocker responder. Its job is to examine all incoming web requests (as efficiently as possible), and look for reasons to reject them. If the request is a “bad” one, it responded with empty content. But it didn’t know anything about RPC requests.
So I made it smarter. Now it looks for any requests that are RPC requests for the weblogUpdates.ping method, and it drops them. I also updated the responder to simply drop the request, instead of returning empty content. Hopefully this makes it more likely that the spammers will notice that it’s not working, and turn their bots off.
I should note that this responder runs only on UserLand’s servers. If this is something that you want to do on your own Manila server, let me know, and I’ll be happy to share the code with you. (It’s super simple.)
Tomorrow I’ll move the sites on this server over to the main UserLand.com server, and shut off the old one. After that there’s more old stuff to turn off, and the problem of how to generate viable static HTML from Manila sites.