The History of The Beat, WordPress, and Media Temple

abstraction.png

Be forewarned: Unless you are interested in nitty gritty web crap stuff, this post is NOT for you. In fact, if you don’t like it, I suggest skipping down to the section called “So what have we learned here?”

Although this is going to be as nerdy as shit, in the hopes that others who find themselves faced with the same challenges as I have met may google this and find some guidance. For those who don’t want to wade through it, there are two lessons to be learned:

1) OWN AS MUCH OF YOUR CONTENT AS YOU CAN.

2) DON’T BE AFRAID TO LEARN

So…

A Seven Year Journey

The Beat started, if I remember correctly, in July of 2004. (Aside: Jesus Christ — six and a half years of daily blogging.) It was hosted at Comicon.com and, after a very brief struggle with a CMS called Greymatter, mainly used Movable Type as the blogging platform. Seven years ago, MT and WordPress were competitors for the blogging crown; MT at the time was a bit wonky and unstable. Several crashes slowed things down and I even lost about two months of blogging at one point. I was a web neophyte at that time and Comicon’s mastermind Steve Conley had wisely given me extremely limited access to the backend — The Beat was hosted at a massive ginormous site that housed many other subsites and tinkering was out of the question.

Eventually I got a fantastic offer from Publishers Weekly to move The Beat and actually get PAID to blog, so I moved over, and switched to WordPress which I was very happy with — I was already using the blogging client Ecto, which I found super useful, and WP had even more functionality. That was Spring, 2006. Reed Business International, PW’s parent company, was going headlong into this “web” thing, and they were adding blogs, and it all made a lot of sense for everyone involved.

As time went on, of course, problems arose — because no popular website is ever static, and amazingly, it turned out that the “Web operations” department at Reed Business “did not support WordPress.” As in they wouldn’t do ANYTHING to fix or upgrade the site. That’s because most of their site and some of their blogs were run on a ghastly Web 0.9 software called eLogic, which RBI owned, along with Variety, Library Journal, School Library Journal, and many other B2B magazines. In some ways, I could understand sticking with the home product. But it made no sense for contemporary content management, especially something as dynamic as a daily blog. Even with no support, this was the Golden Age of blogging and The Beat’s traffic soared — to the point where it was soon nearly 50 percent of the PW website’s traffic.

Despite this, the web department still wouldn’t do a thing to fix problems or upgrade it. Despite the fact that at some time in the future they were moving to WordPress for their blogs

.

This had a lot to do with inter-department politics, of course. But it was even more so a part of corporate philosophy. Of everything that has happened in my career, RBI’s disinterest in growing their web traffic remains one of the most puzzling. Here was a major web initiative. Here was a proven successful site which doubled traffic and cost NEXT TO NOTHING — but support this cheap strategy? No way.

Even when hackers broke into The Beat and started replacing posts with ads for Viagra, the Web people would do nothing. Every time I complained, they said I should move to eLogic. Fortunately, I had the support of the entire editorial and advertising teams at PW who didn’t want me to move to a CMS that would effectively kill my traffic. Luckily, the web people didn’t actively try to muck things up either, which I’ve seen happen elsewhere. So I would definitely choose neglect over micro-management every time.

I did, however, make one crucial mistake. I had access to the FTP and control panel for the site, so if I had been a little bolder, I could have gone in and fixed everything myself. But I didn’t, because a) I was chicken and b) I thought it was the responsibility of Web Operations to maintain their websites, and not editorial. In retrospect, I should have just gone in and gotten my hands dirty and done it myself, or hired someone to do it. I was pretty much given carte blanche by the Director of Web Operations, who, whenever I had a problem, suggested I call the webhost myself. (She also told me they never called her back when she did contact them — while in my experience, they responded very quickly.)

Anyway…years went by. Civil War, Final Crisis, the death of Cap. The hacking problem was always a pain in my butt — it was like a cancer eating away my content. All I could do is try to go in and clean things up by hand in my spare time, which there was little of. Meanwhile, RBI put PW and all their magazines on the block, so I was getting more worried about what might happen to The Beat. RBI itself was in a massive state of upheaval with huge layoffs every six months that left the remaining staff increasingly dazed and demoralized. In other words, my little problems didn’t amount to a hill of beans in the larger picture of media meltdown.

But there were still attempts at progress. About 15 months ago, RBI announced they were moving to a new CMS – and taking all the blogs with it. The Beat would be hosted on WordPress — which was great, it would finally be supported! — but WordPress 1.7. This…at the time when WordPress 2.8 was being used. Yes, they were using an out of date version of a free, open source software. God knows why. But needless to say, I was…alarmed.

I made plans to move The Beat to my OWN site. And this time, after all the problems I’d had with webmasters who couldn’t or wouldn’t respond, I decided to take matters into my OWN hands. After talking to the proper people at PW, I exercised my right to cancel our contract. Editorial and advertising were sad to see The Beat and its traffic leave, but with the magazine’s sale up in the air, everyone understood.

When the NEW Beat launched on February 1, 2010, I had set it up entirely by myself. I had downloaded and tweaked a free theme from WooThemes, and had the superb logo from Comicraft which you see. Other than that…I did it all myself! Woot! I had the comicsbeat domain from a while ago when I stockpiled a bunch of useful domains, and set up the entire site on Media Temple all by myself. I even did things like fix the .httaccess file ALL BY MYSELF.

Now why Media Temple you ask? I had been in discussions with a few site developers who were interested in hosting The Beat. I had a lot of meetings and discussions about other people taking it on, but when it came to business and money, it was always a shared revenue model. After a while, I decided that perhaps sharing with myself was the best idea and that’s what I did. I had asked around to knowledgeable web hosts about potential hosting sites and most people said that Media Temple was the choice. When I saw that they had a GridServer option for $20 a month with 1tb of traffic a month, it sounded like a good deal. I thought I could always tweak it if I needed more juice.

2011-01-02-BOINGTHUMP! Chapter 4 Page 12.png

The Media Temple Era

So I signed up for a $20 a month website. So thrifty was I that I even fund a coupon which brought it down to $16 a month. Good deal!

After setting up The Beat on MT, I did hire some people who really knew what they were doing to port over the old entries and set it up in a completely professional manner, namely Ron Croudy at Smartbomb and his web development team. Ron is an old friend, a comics reader, and he knows his onions as they say. Unfortunately, probably because of the same vulnerability that allowed the hacking, the old Beat database was corrupted, and all the entries could never be imported smoothly. (That’s why most of 2009 isn’t even archived at the bottom of the page.)

Needless to say, this was very frustrating to me. I still retain a vague hope of bringing everything together someday. All the old posts are still available at whenmonkeysattack.com, and we’re still trying to find a way to bring them over here.

The New New Beat’s traffic picked up steadily and was soon at its old levels. And that’s when I started to get these GPU overage emails from Media Temple.

I don’t know the whole technical end of it, but a GridServer is basically some kind of moving array of servers so that traffic is spread out over a whole bunch of machines. This allows traffic loads to be spread out. Even a “Boing Boing” link or “Warren Ellis” or whatever won’t bring it down. And indeed, The Beat’s servers held up incredibly — the server was very very peppy, the interface was very clean and I really didn’t have any problems.

Until I got the bills.

My $16 shared server had ballooned to over $200 a month because of GPU overages. That’s a “grid processing unit”, a measure used only on MT’s GridService hosting. Traffic was well within reason. I could look at these logs, and to me it seemed like it was just the normal running of the blog. After MANY conversations with tech support, they said it was because I had some missing files (from missing images) and I needed a custom 404 page.

So I called up Ron and Ryan and we got a custom page.

The GPU overage barely shrank a bit.

Now, for $200, you can get your own giant dedicated server and a million emails and whatever else you want. And, to be honest, this charge was not in the budget and it was a pain in the ass.

Every month I would look at the overage report and every month I would call MT’s tech support and ask what was going on. To be fair, they did waive the charge one month because I had made so many efforts to bring down the grid processing. However, NOTHING seemed to help. Even when we cleaned up all the 404s, the traffic was growing, and backlinks were keeping older posts getting traffic and so on. All of which was creating more GPUs.

After a while I came to the conclusion that you just shouldn’t run a busy WordPress blog on a Media Temple GridServer. Even though we were using Supercache to prevent dynamic pages from loading every time you hit the site, it was the normal site itself which was one of the biggest process hogs. The Beat is not a ginormous Gawker site, but its traffic is healthy and steady. My advice to anyone looking to host a professional level blog would be not to use MT’s GS service….BUT on the other hand if you are running a professional site, you can probably afford to pay more then $16 a month — YMMV. Just be aware that there are going to be more charges than advertised.

Despite the fact that these charges were killing me, over the summer with with travel and all I couldn’t really do anything about it. Finally after New York Comic Con, I called up the team and we readied for a NEW move, to a DV server — a dedicated server, hosted by Media Temple. For this they charge $50, which is pretty reasonable, and by moving it I would save some money. I did consider other web hosts, but I will say MT’s accessible and friendly tech support made me deicide to stay with them. For all my suspicion about what a “GPU overage” is, everyone I talked to at MT was VERY helpful — when The Beat started getting hacked a few months ago, they went in and cleaned it up immediately. It was definitely worth a few bucks to have good service.

However, as you may have noticed around Thanksgiving, when we pulled the trigger on the new site, it immediately ground to a halt. It took forever to load, images weren’t coming up, ads weren’t coming up, and I couldn’t post new images. After a few frantic calls to MT, I was advised that the site was a memory hog and needed to be immediately upgraded to more memory and a $100 a month site.

I also encountered a problem with access to the uploads folder, which said I didn’t have permission to post to it. I had a lot of technical notes on this, so that anyone googling the problem would be able to find this solution, but the solution is that I renamed the upload folder via FTP, and WP created a new one automatically with the right privileges. I had to move all the most recent images into the new folder and bingo, all the images were back in place.

The new DV server is also more vulnerable to spam messages for some reason — the GridServer filtered them out 99.9 percent of the time. There are are some other security issues which my team had to shore up, but by and large, it seems to work fine.

As for the slow loading time and massive memory drain — this was all due to a script in theme, Daily Edition, specifically the one that automatically resizes the front page image to create a thumbnail. Whenever I turn this script on, the entire site begins to resemble a spinning beachball factory. The reason is that WP suddenly starts creating thee different sized thumbnails of all the images in the posts. Since the Beat has 1000s of images in the archives, you can see why this is a drain. My man Ryan tried to download a newer version of the resizing script but Woo doesn’t seem to have one available (plus they were hacked about a month ago and have had ongoing problems.)

However, one of the goals for the new year is to get this script fixed so that the front page will look normal again. We have a bunch of tinkers and improvements in the works which should roll out before the one year anniversary of the comicsbeat site.

batcave60s.jpg

So what have we learned here?

1) In retrospect, it was probably this image resizing script which was causing most of the GPU overages. But this is pretty much blog standard these days and isn’t something I would consider above and beyond performance wise.

So — WooThemes + WordPress + grid server = LOT OF OVERAGES AND EXTRA CHARGES. If you are looking to run a site cheaply, I would not recommend Media Temple for a WP blog.

2) MT is still a fast and reliable webhost, and their customer service department is very knowledgeable. If you have the scratch it’s worth it. It just isn’t a budget option.

3) WordPress is still one of the best CMSs available. If only their iPhone app weren’t such a shocking piece of shit.

4) The personal satisfaction I have gotten at learning more about webhosting and setup from doing this myself is a source of huge pride for me. Of course there are still things that leave me in a fugue state and I tend to forget the most complicated bits as soon as I fix them, but in general I haven’t fucked up anything so badly that it can’t be fixed. Knowledge is power. After all the grief I’ve had because of busy webmasters, knowing how to accomplish the web equivalent of changing a toilet seat or putting up a shelf provides a very valuable peace of mind.

5) Most important of all — ALWAYS OWN WHAT YOU CREATE. A blog post isn’t on a level with a comic a story or video, but add up 6 years of them and you have something. For Rick and Steve at Comicon, my owning all The Beat content wasn’t even a question — Rick is a veteran of the original Creators Bill of Rights, and anything less isn’t even in his nature. I’m extremely, extremely fortunate in that RBI, for all its faults, wasn’t all that interested in owning my archives or content moving forward. This may have been part of the same negligence on their part, but no matter the reason, it worked in my favor. Finally, by believing in my own IP means, I bring equity into any potential deal. It’s not like I own Facebook, but I do own something. An in today’s attention economy, good Google rankings and reader engagement hold significant value.

(BTW just as an epilogue, RBI sold all but a handful of magazines, including Variety, whose main web strategy right now is a paywall. While still one of the most valuable B2B brands left, it’s struggling in a world where Deadline and a refinanced Hollywood Reporter are making the web a priority.)

Since running The Beat on my own, I’ve enjoyed tremendous advertiser support. Indeed, my biggest problem at times has been having enough space to slot everyone in. I know a lot of sites would envy this problem, so I’m incredibly grateful for the advertisers who continue to show their support. And of course, I’m grateful for every reader who stops here, whether it’s every day or surfing in from the archives.

mr.jinlges mile.jpg

There’s a lot more I could say but I think those of you who aren’t web wonks fell asleep long ago. Mr. Jingles, are you still alive? In 2011, I hope to turn my attention to making this site more useful and informative for readers, and to fix the remaining tech problems. If you have any suggestions, consider this open house.

And now back to the news.

Art credits:
xkcd: Abstraction, Randall Munroe
Wizzywig Comics, Ed Piskor
The Batcave
The Green Mile