11 Responses

1. chriskalani

   January 22nd, 2008 at 5:45 pm

   1

   I agree. That is THEIR problem… not the users. All the users care about is the site they are trying to access. I saw the tumblr one today and thought it was a bit lame. But I didn’t say anything. They should only say something about their host if something insane happens, like all the servers explode or get eaten or something.

2. Matt Hellige

   January 22nd, 2008 at 6:48 pm

   2

   I totally agree. I think it’s an in-group/out-group thing. From the user’s point of view, you and your hosting company are one entity. The right thing to do is to represent the whole group and take responsibility. If you want to sue them behind the scenes or get them to pay up according to some SLAs or whatever, go for it. But don’t try to put yourself on the customer’s side against “them.” It’s condescending and kind of shitty and nobody buys it.

   Obviously this is what you mean by ‘black box’. So yeah. I agree. Take responsibility for yourself.

3. holcomb

   January 22nd, 2008 at 7:55 pm

   3

   Blaming things on your vendor just makes you look like a middleman. Middlemen tend to get cut out (or should be at least).

4. Fitz

   January 22nd, 2008 at 11:30 pm

   4

   Blaming is a waste of time. If you have a service, you have chosen to use certain providers for your service, be they datacenters, networks, or what have you. If a provider fails and you don’t have a backup, no one is to blame but you.

5. slicematt

   January 23rd, 2008 at 4:40 pm

   5

   As a partner at Slicehost, we take the responsibility of keeping our customers’ servers running very seriously. Everything outside of customer control (power, cooling, server hardware, network) falls on our shoulders, not yours.

   I’ve been on both sides of this equation - as a service provider trying to get customers back up and as a customer at the mercy of a service provider. There isn’t a worse feeling in the world than having to tell your customers you know what the problem is, but someone else has to fix it.

   37signals accepted full responsibility and explained what happened. At some point we all have to trust someone else to do *their* job, sadly they appear to have been let down.

6. Jengates Blog » Blog Archive » links for 2008-01-23

   January 23rd, 2008 at 6:25 pm

   6

   [...] Putting the blame on your vendors for the win. » Harper Reed: Tech, Phones, Yo-yoing and Death Meta… [...]

7. Harper

   January 23rd, 2008 at 11:29 pm

   7

   Hi Matt. Thanks for the comment.

   I agree - at some point we do have to trust someone to do their job - and i agree that it is a harrowing experience to have to tell your clients that something is broken. however i don’t agree that you need to get into specifics of what is broken and who is going to fix it. because - in the long run - it is the company who is choosing vendors and if the vendor fails - i would go as far as saying that it is the companies fault for choosing a faulty vendor. Or at least not making sure that the vendor is doing it right/having backups and redundancy.

   Whenever i have had large failures. I have tried to apoligize in the following form:

   Dear users. I am sorry for the downtime. We are working hard to fix it. I am sorry for any problems this has causes.

   However - tumblr and 37signals seem to say stuff like:

   Dear users. Somebody caused us some problems and they are doing a poor job of fixing it fast. I am sorry for any problems this has causes.

   Those two statements are VERY different. Which would you rather hear as a consumer? I would rather hear the first one.

8. Gil

   January 24th, 2008 at 11:39 am

   8

   The 37signals thing bummed me out. They delicately phrased their communication so that it sounded like they were taking the blame, but when you read it closely, they are actually blaming Rackspace for taking 2 hours to replace a failed piece of hardware. The fact is that hardware does fail, and you can’t reasonably expect them to replace it much faster than that.

   The transformer thing was a totally separate issue, and it sounds like Rackspace hadn’t been keeping up with their backup systems to make sure they were working properly. Hopefully they learned their lesson, and I won’t discredit them unless it happens again.

   I’ve had my own frustrations with Rackspace, but after the fact they’ve always gone out of their way to fix the problem. I still refer people to them on a regular basis, and will continue to do so in spite of the 37s/tumblr nonsense.

9. Harper

   January 24th, 2008 at 11:56 am

   9

   Hi Gil!

   I think that having frustrations with vendors is totally part of the game. It happens to everyone. It is just a bummer that people are so into passing the blame onto someone else - instead of taking responsibility.

   i think that blaming is easy and a bit thoughtless - whereas taking responsibility is usually hard and possibly shows weakness (because it is YOUR fault instead of someone else).

   but yea. rackspace has had a couple rough spots with us too. but they are always quickly resolved and remedied within a day at the most. and usually they do not cause downtime. so regardless of tumblr and 37signals bitching about how bad it is. i will also continue to represent them in anyway i can.

10. slicematt

    January 25th, 2008 at 6:44 pm

    10

    Harper, in our experience, customers WANT (perhaps demand?) to know exactly what happened. Granted we serve a very technical community, but the explanation you outlined in your response would not fly. They would rather hear exactly what broke (insert technical jargon here), how we’ll prevent it going forward, no matter if it’s our fault or not. I think 37signals has a similar user base and perhaps reached the same conclusion.

11. Ryan Kanno

    January 25th, 2008 at 8:08 pm

    11

    Harper - I blame you for writing this damn post. As much as I agree with you, I also agree with sliceMatt. As a consumer, I want to know what went wrong and why, as long as everyone is honest - I can make my own decision about who to blame.

    As for 37signals, I think that if they’re blaming their service provider, as Fitz points out, they’re to blame for putting all their eggs in one basket - albeit a generally big, protective basket in Rackspace, but a single point of failure nonetheless. Even in DHH’s comments on TechCrunch he admits that Rackspace’s hardware agreement has long since been broken. Umm, I’m not a smart man, but if that’s the case, um, why are you still with them (and since you are, why aren’t you mirroring crap?)Without either a backup service, or a Google Gears type of app, they’re the ones to blame. Hopefully, the customer can see this. ;)