Tag: wordpress

Misadventures in blocking referrer spam: How a rogue WordPress plugin wreaked complete havoc on one of my blogs

There’s no real original news article or Wikipedia article to link to here. This is the original story, and I figure there’s no better place to put it than here.

As many of you may know, I blog about my arcade and pinball adventures (mostly pinball these days) over on SKQ Record Quest. Over the past few weeks, it’s been painful to do much with the site as it had become increasingly unreliable. First it was an occasional “500 Internal Server Error” and then they came increasingly frequent to the point where, at times, the site was basically unusable.

To make matters worse, the host I am on bills by resources used. Taking a look back, the resources used started skyrocketing at the start of the year. I did a few things that appeared to fix the problem at first, but it would eventually come back with a vengeance.

That is, until I finally took a look at what was going on and noticed the CPU for SKQ Record Quest was much higher than the other WordPress sites I have running. I finally began going through a guide that recommended a plugin called WP-Optimize. So I started going down through the list of options. At some point it told me about some cron tasks which had not been completed. Something like over 100,000 (I didn’t notice the exact number, and honestly I thought this was a glitch). So I followed the instructions and did a wp cron event run --all (after some blundering around to find this exact command). I was then greeted with:

Executed the cron event 'wsrs_update_blacklist_twicedaily' in 6.925s.

and many more lines like it, with different times on the end. Turns out that this was left over from a plugin I used to run called Stop Referrer Spam. Not only did this plugin do little, if anything, to get rid of the referrer spam problem I was having, apparently it left behind thousands of cron task turds behind.

I’m not waiting for nor paying for all of that junk to run, so I had to play Go Fish in the database server. Turns out deleting the “cron” key under wp_options (well, what my wp_options table is renamed to) is enough to fix this. Or so I thought, as after the first time it came right back. A second nuke of this key appears to have permanently fixed the issue.

I hate referrer spam, but right now I’m afraid to try any other plugins to fix the issue given what has just happened, and what it took to find and fix the issue. The truly horrifying thing is that I was only a couple more weeks away from taking an extended break from blogging to finally abandon WordPress and migrate almost a decade’s worth of content to a static site. Now, this blog and the other WordPress sites I currently have would be much easier. SKQ Record Quest, however, makes extensive use of Jetpack’s image galleries feature, meaning a large number of posts and images would need to be migrated to whatever is used by the static site generator I am migrating to. On top of this, I would have needed to find an alternative solution for analytics, and it would become much more difficult to post from my mobile phone should the need ever arise again. Also, just for good measure, I would likely have to edit many other links to WordPress-specific things like wp-content/uploads/2024/04/{image filename}. We’re talking easily a month or two of spare time involving a considerable amount of caffeinated beverage consumption and a nontrivial uptick in stress and uttered profanities. Oh, and this is spare time that would come out of bar/arcade visits (i.e. less time playing pinball) and possibly other social activities.

Now there would be advantages to such a move, the least of which would be that backup copies of static sites can be mirrored much more easily and on censorship-resistant platforms. Hopefully the potential censorship won’t ever be an issue; it’s a nontrivial amount of work to go from WordPress to a static site and there’s no guarantee I would be able to easily reverse the process should the original reason no longer make sense from whatever standpoint, be it technical, social, etc.

I’d like to try to make this a learning experience for as many people as possible. So here’s what I learned:

  1. Be mindful of what plugins and themes you install. Most of the time, especially if you stick to plugins available from WordPress’s own directories, you shouldn’t have issues.
  2. Take it seriously when weird things start happening on your website. You should never get a “500 Internal Server Error” and even the occasional one means something is going really wrong. I assumed a later plugin update was going to fix the issue eventually. It did not; manual intervention was needed to get things back to a sane state.
  3. When troubleshooting, always thoroughly investigate anything and everything that is out of the ordinary. As I remember it, there were errors in a previous round of troubleshooting that should have pointed me to the problem had I investigated them more thoroughly. What I actually did was slap a quick bandage fix on it, which of course didn’t solve the real problem.
  4. If you still have the Stop Referrer Spam plugin installed, get rid of it! Like this, if you have WP-CLI installed:
    wp plugin delete stop-referrer-spam
    and then, just to be sure, delete the entire cron row from your wp_optionstable. (Generally, open phpMyAdmin, go to your wp_options table, sort on option_name and look for cron, then delete that row. I’m not sure how to do this by typing in SQL commands if that’s your only option, but that’s the basic idea.)

Hopefully you’ll never need this advice, but it’s there if you do.

The WordPress “new Coke” moment is right around the corner, for real

Around this time last year (specifically, 2017 November 24),  I published a post comparing the Gutenberg editor due to be a part of WordPress to “new Coke”. It’s about time for a retrospective on this, since in a couple of weeks, WordPress 5.0 is due out and will have the Gutenberg editor be the new default, and so the true “taste test” is about to become a reality.

First, the serious stuff to my fellow WordPress users out there: If you have tried the Gutenberg editor already and you know it won’t be a good fit, or if you don’t want to be screaming in horror when you upgrade your site to WordPress 5.0 and wonder what the hell happened to your editor,  then you will want to install the Classic Editor plugin. If you have access to the WordPress CLI, you can simply run wp plugin install classic-editorand call it a day. Alternatively, you can install from the plugins menu as normal.

Alternatively, if you’re daring, you can install the Gutenberg editor as a plugin today: wp install gutenberg from the CLI, or you can install from the plugins menu. As I mentioned in my previous post, Gutenberg is a radical departure from the classic editor, and for some, it has already failed the “taste test”. Take, for example, this recent post on WP Tavern (quoted in part):

Testing Scenario: A user has written three paragraphs and decides to add an image to the second paragraph. This user wants the image to be aligned to the right.

[…] [describing the task in the classic editor] Adding media to a paragraph is as quick as placing the mouse cursor at the beginning of a paragraph, clicking the add new media button, selecting or uploading an image, and choosing its alignment.

[…] [describing the task in Gutenberg] In Gutenberg, each paragraph is a block and each block has its own toolbar. This is important because after writing three paragraphs, you can’t click on an add media button. Instead, you need to create an image block.

[…] select[] an image [and] move the image block above the paragraph block where you want to insert it. […] [T]ry[ing] to drag and drop the image into the paragraph […] doesn’t work. […] [U]se the up and down arrows or drag the block into position.

Once the image block is in the correct location, click the align right icon. […]

[For now] the Classic editor wins this use case.

Translation: for some simple tasks like this one, the “new Coke” tastes terrible. I can see new users getting frustrated at this, until and unless they figure out how to go back to the classic editor. Or, they may well give up WordPress completely and move on to something like Drupal (hopefully not), Joomla, etc.

Perhaps even worse is the WordPress Accessibility Team’s assessment of Gutenberg (as detailed in another WP Tavern post):

The [WordPress accessibility] team, largely a group of unpaid volunteers, collaborated on a detailed assessment that publicly challenges Gutenberg’s readiness for core in a way that no other WordPress team has done through official channels to date. After a week of testing the most recent version of the plugin, the team concluded that they cannot recommend Gutenberg to be used by anyone who relies on assistive technology.

[…]

The mistake of not having consulted accessibility experts in the design phase cannot be easily rectified at this point, but the Classic Editor is still available for those who need to preserve their same workflow. […]

Either the accessibility and usability issues the team identified are not as bad as they purport or this document is a last-minute clarion call that could prevent WordPress from shipping an editor that excludes users who rely on assistive technology. Due to the gravity of their claims, the accessibility team’s statement on Gutenberg demands an official response.

It is my hope that the accessibility issues in Gutenberg can be fixed sooner rather than later. I find it quite difficult to believe, as a website platform intended for the masses, that the developers of WordPress would not have thought out accessibility issues in Gutenberg earlier in the development process.

As for me, I do plan to use the Gutenberg on one of my sites (a business site, not one set up as a blog, though I may use the blog feature as a “news wire” at some point down the road). For straight-up blogs like this one and SKQ Record Quest (my pinball/videogaming blog) I see myself sticking with the classic editor for the foreseeable future. My biggest concern is what the move towards Gutenberg will do to the time-honored method of editing blog posts using an external editor (such as I did for many months using GVim and Vimrepress, though I should note as of last time I tried, it was too broken to be usable.)

The Evernote two-step: a tale of caution regarding privacy and data ownership

This recent story from PCWorld about Evernote changing its privacy policy followed by this story covering a very abrupt about-face from Evernote say pretty much all there is to say about why it’s a bad idea to blindly trust companies like Evernote with the privacy of your data.

Basically, Evernote changed their privacy policy on a whim to allow employees to snoop on user notes ostensibly to assist efforts to train its algorithms. Originally, individual users could opt out of the algorithm training, but not out of the part of the privacy policy still allowing Evernote employees to snoop on their data; businesses could opt out but would not get the benefits of the new features if they did.

Evernote was quick to backpedal and change to an opt-in model following the fully justified outrage of its users. It’s quite possible some users no longer trust Evernote with their data after this two-step, and it would be hard to blame them. It is possible to store data locally with Evernote (by creating a local notebook instead of a “synchronized” notebook) and they intentionally make it easy to get all your data out of Evernote if you want to leave for what you believe are greener pastures. This gaffe might be the impetus for quite a few users to do just that.

The lesson here is a very powerful one: there’s very little stopping other companies from doing this tomorrow, and there’s no guarantee the CEO of that company will even give the appearance of giving a tinker’s damn about its users. This is a rather direct reminder to take a look at the companies you trust with your data, and how easy it would be to get your data out of those services/products should you decide you want to leave. Of course, it would be wise to remember the best time to find out how easy it is to get your data back out of a product/service is before you put your data in it.

I’ll add a personal angle to this. At various times over the past year and change, I have considered moving this blog to a static site updated with Pelican, among other possibilities. The hard part is not getting the data out of WordPress–that’s actually about as easy as they get. Even the free-of-charge wordpress.com platform makes this relatively easy, even if one is not moving to self-hosted WordPress (a.k.a. “wordpress.org” to differentiate it from shared-hosting wordpress.com).

No, the problem comes if I find out Pelican (in this case) doesn’t work out and I have to move entries back into the last backup of the site as a WordPress site. That might be easy if I make no more than about five posts before finding out it’s not going to work out. But what if that’s ten? Twenty? Fifty? One hundred? Two hundred? It’s a potentially painful process because I don’t see an easy way to automatically make even a WXR file with the new posts in it. Sure, I could ease any future transition by keeping a local install of WordPress and add the new posts manually as I make them, but that is an implied admission I have no confidence in the new platform–and that would be the case even if it was something besides Pelican.

Of course, ideally I want to change platforms only once. There’s always the chance the first change doesn’t work out. There’s also the chance I would want to later switch back to WordPress after making this change, or switch to a future WordPress fork or even to something like b2evolution (which forked from the same blogging software that WordPress was forked from).

Another great example of what not to do, unfortunately, is what the WordPress Foundation (at the time) did when they struck a deal with Meetup.com regarding the online presence of local WordPress groups. The thing to remember about Meetup.com is that they intentionally make it difficult to impossible to change platforms. Vendor lock-in, combined with the (justified) fear that organizers might lose some members (in fact, almost certainly will lose some members) when transitioning to another platform, is the business model of Meetup. It’s a huge departure from the Meetup that I used as an early adopter and that’s sad.

Anyway, it was and still is really disappointing to me that the WordPress Foundation (at the time) deciding to just pay a bunch of money to Meetup. The first reason is it’s been sort of an informal goal of the WordPress community to do everything with WordPress that can be done with WordPress. As an example of this, WordCamps aren’t ticketed with Eventbrite or other such sites; they use a WordPress plugin called CampTix written for the purpose. Thankfully the deal with Meetup.com is about the only time WordPress was intentionally eschewed for a function central to the local WordPress communities around the world.

Adding to it, of course, was the unfortunate experience we had with the Houston WordPress Meetup in most of 2012 going into the early months of 2013 and the response from Meetup (at the top of the post). Basically, Meetup’s standpoint was that the people (community) in the group didn’t matter as much compared to the organizer paying their dues on time. The only silver lining to this cloud is that the WordPress Foundation (the entity paying Meetup which is the nominal organizer of the actual group; maybe this, too, has changed to WordPress Community Support, the new PBC) has a bit more skin in the game and can replace inactive or unresponsive organizers. Then again, they would still be able to do that using a home-brewed WordPress-based solution, without paying Meetup one bloody cent.

Regarding the use of Meetup, it’s hard to see which is the chicken and which is the egg. A lot of groups were using Meetup.com prior to the WordPress Foundation deal. Still, the better move for the community would have been to start a WordPress multiuser site similar to wordcamp.org with a plugin to replicate the Meetup-like functionality in much the same way CampTix fills in for Eventbrite. Better yet would be a fully free software alternative (GPL, if need be Affero GPL) to Meetup.com; even if it is not built on top of WordPress, that would be a step in the right direction.

Examples are plentiful, but the lesson remains the same:

  1. Know how to get your data out of a platform before you need to. If unsure, ask questions. If you don’t like the answers, use a different product or service instead, after getting satisfactory answers to the same questions.
  2. Trust your gut. If a privacy policy change rubs you the wrong way, raise hell about it, and minimize the damage by taking your business and your data elsewhere (see #1).

The tale of the WordPress XML-RPC attacks

I can’t believe I’m writing this post. But, I am.

This concerns something in WordPress that a lot of people don’t even know is there called XML-RPC. The XML-RPC feature has been turned on by default since version 3.5 (released 2012 December 11) and which I think has been in WordPress itself since the beginning even if not enabled by default.

Without getting too technical, XML-RPC is responsible for two main features of note: pingbacks and external blog post editor support. Unfortunately, it has recently become the vector of choice for what I can only assume is some kind of worm/malware which sends thousands of requests in an attempt to either break into a site or just plain cause havoc and increased resource usage. That’s where the problem begins.

This wouldn’t be that vexing of a problem except for one thing: I actually use XML-RPC legitimately. This post, for example, was made via the XML-RPC function, not WordPress’s internal editor (though I have made posts via the internal editor before). Most of the posts you’ve seen me make in the last three years have had the XML-RPC interface come into play during editing at some point. So, just turning this thing off isn’t an option.

I tried a couple of plugins which promised to “secure” XML-RPC and they did so by completely breaking it. (I can do a better job of this with “rm xmlrpc.php” or “chmod go-rx xmlrpc.php” myself, I don’t need a plugin to effectively do something like this for me.) It is difficult to know for sure, but given the relatively low traffic I have, it does not make sense that my resource usage is what it is. (Resource usage which I pay for, I might add.)

So, what the h-e-double-hockey-sticks am I, or other WordPress site admins, supposed to do?

Well, right now (actually, by the time I post this, as of a couple of weeks ago), my solution is akin to a duct-tape-quality fix: copy the existing xmlrpc.php to something else, and use that for my editor’s XML-RPC endpoint. It’s the only thing I could think of to do, and thankfully almost everything still appears to work as designed.

I say “almost” because this doesn’t yet solve the problem of legitimate pingbacks potentially getting dropped. My host has XML-RPC protection, but unfortunately that protection is to change permissions on xmlrpc.php at the first sight of an attack (breaking the functionality until I manually restore it). Unfortunately I don’t have a way to tell sites sending me pingbacks to send them somewhere else automagically just yet. It also means I have to repeat this step on each upgrade (well, at least if xmlrpc.php changes). Like most actual duct tape repairs, it’s probably not the best solution. But, for now, it works, and to me that’s what matters.