Earlier this week Dennis Howlett posted on ZDNet a fairly scathing attack on poor analysis and its creators – be  they traditional firms or people with a different approach. In his post (in which I came in for a lambasting myself) Howlett makes a call for a change in the trenches of business IT analysis, he asks those in the know to;

step forward and push aside the fashion driven marketers masquerading as analysts. It is time to squish the idea that media is analysis and ask the question: Did you really think that or were you gaming Techmeme? Do you truly believe what you are saying or chasing the latest buzz phrase? Tell me, where is the evidence of which you speak so boldly and with such paper thin conviction?

The truth of the matter is that everyone is finding some balance between self-promotion, the creation of quality content and identifying (or as he put it, chasing) the latest buzz phrase. One only needs look at the traditional analysts who make extensive use of new media to see that personal brand (albeit backed up with substance) is a non-negotiable requirement these days. Howlett himself is a voracious user of Twitter and has created for himself a compelling (if somewhat abrasive) online presence. Is this not a tacit admission that content needs exposure in order to surface, and that in return, exposure create opportunities to create content?

In critiquing the new breed of commentators/analysts, Howlett takes the line that:

pretty much anyone with a half baked opinion and a skill to write catchy headlines gets to call themselves an analyst. It has come to the point where anyone with a following of more than 10 people gets to spout whatever they wish with a good chance they get some attention

He drives home his point by giving the example of Quora, the rapidly growing question and answer site which he dismisses in the post as little more than a hangout for “the latest crop of attention seeking anal-ysts”. In taking this line however, Howlett seems to dismiss the very notion that collaborative platforms can build a meritocratic system that can allow the quality content and quality practitioners to rise to the surface. I’ve been using Quora for a short period of time and already it has become obvious that it’s a remarkably effective way to rapidly get a crowd sourced, crowd qualified and crowd moderated take on a particular issue. Many would argue that this beats the conflicted view that traditional analysis creates. Interestingly in a subsequent post all about Quora itself, Howlett said that

I like that because it either forces you to be smarter or it drives you away until you become smarter. In theory that should mean we all get a bit smarter at what we do. Or, as someone said: there are no stupid questions, only stupid answers and on Quora I get to decide.

All of this would be an interesting, but not overly relevant topic, were it not for a recent Cloud Magic Quadrant produced by Gartner. The Magic Quadrant is shown below for reference.

I don’t want to get into an extensive critique of the MQ results, but there are some glaring issues that one can only put down to commercial imperatives (more course commentators would call it “rampant pay for play”). To get some completely neutral feedback I spoke with a global systems manager from a major enterprise (who asked to remain anonymous) to get some end-user critique of the report. At an initial glance he criticizes the following;

  • They’ve created some kind of hybrid IaaS with contract-based hosting – very much apples with oranges
  • Despite a partial focus on colocation, many colo providers are missing from the report
  • Amazon web services is included as a visionary which in Gartner’s own words means they are “new and unproven” – Huh?
  • Gartner have entirely ignored security and accreditation issues in their advice to customers section

Contrast this if you will with this two part series from James Urquhart (Part 1 and Part 2). Without wanting to sound too much like the Urquhart fanboy that I am, the difference in quality, insight and, most importantly, neutrality between these two works is stark. Urquhart may not have Gartner’s Analyst’s combined centuries of experience and he may not have private briefings with execs from the particular companies he writes about but if I was a betting man, I’d put money on his predictions long before I do those of Gartner.

And what modus operandi does Urquhart utilize in his day to day work with the cloud? Those same meritocratic, noise-filled and low-brow channels that many would dismiss. And those same channels allow Urquhart’s content to rise to the surface – surely analysis or commentary – call it what you will, is in good heart, despite the noise. The great thing about meritocracy is that it self corrects, as a friend pointed out to me, if this new breed of self-serving “spouters” are nothing but attention seekers then they will soon be cast into obscurity by the audience (those who are the real consumers of the commentary).

And lastly, and in a nice closing-of-the-circle. So long as the analysis is fair and without prejudice, the personal brand that gets built from meritocratic kudos can give companies what has recently been coined the “Scoble effect” – a benefit of being given approval by those whose opinions count.

Enhanced by Zemanta
Ben Kepes

Ben Kepes is a technology evangelist, an investor, a commentator and a business adviser. Ben covers the convergence of technology, mobile, ubiquity and agility, all enabled by the Cloud. His areas of interest extend to enterprise software, software integration, financial/accounting software, platforms and infrastructure as well as articulating technology simply for everyday users.

20 Comments
  • Ben, you / the guy you talked to seem to be pretty confused about the role of colo in the MQ. I’ve answered you in a blog post of my own:

    http://cloudpundit.com/2011/01/07/why-cloud-iaas-customers-care-about-a-colo-option/

    There is ZERO pay-for-play. Gartner never does it, period.

    You might find some of my other blog posts interesting, as well, as they touch on various other aspects related to the MQ and the cloud market in general.

    • This is a frequent proposal, especially by current or past analysts of Gartner. However, a long list of research demonstrates that conflicts of interest of the type Gartner routinely involves itself with leads to biased research outcomes. Gartner could remove this conflict of interest by ceasing to take money from the entities it ranks. Secondly, Gartner habitually overestimates large vendors (that pay more) than smaller vendors. Gartner’s research results show a clear relationship between those that pay the most and their position in the magic quadrant.

  • You say: The truth of the matter is that everyone is finding some balance between self-promotion, the creation of quality content and identifying (or as he put it, chasing) the latest buzz phrase. – I challenge that. Fact is people like you are little better than pimps for whichever vendor is paying you to spout their words.

    • I guess I dispute that, if you look at my disclosures page you’ll see that the list of people I work for is pretty short when compared to the extensive list of subjects I write about (many of which are not vendor specific at all)

      • I see your disclosure….you’re a vendor biased pimp with no original analysis to your name.

        • Without wanting to engage in a circular discussion, I guess the challenge to throw out there is to ask to give an example of commentators/analysts who don’t derive some revenue, directly or indirectly, from vendors. If, as I contend, we all earn revenue from vendors to some extent, exactly how do we differentiate “vendor based pimps” from those who aren’t?

  • If it was otherwise then you might have something original to say. Sadly I don’t see that.

  • I like this post. I think the difference between people like yourself and maybe the likes of Gartner is that you are not bandwagonning. You have well thought out and articulated reasons why you think SaaS / Cloud computing is a paradigm shift (reasoning with which I sometimes disagree).

    I have similar thoughts about free & open source software (which some folks, occasionally, disagree with).

    So we write about this and people know our positions and agree/disagree in with that understanding and context.

    It does not seem to be “pimping” to me unless Google/MS or someone is paying you to promote their message, which certain “highly respected” analysts and companies definitely do.

    I don’t understand the comment about Gartner not doing “pay to play”. I have certainly seen studies that Gartner has been paid to produce which, strangely enough, match exactly the outcomes the payer desired.

    • Thanks Don – and to reciprocate I’ve always appreciated your passionate position on FOSS (even if I sometimes disagree). Very few companies I work with would be silly enough to even ask me to shill, those that do get short shrift…

    • Where have you seen such reports? Unlike other analyst firms, Gartner does not do any commissioned research whatsoever (no white papers or anything). We stopped doing so years ago, for exactly the reason you cite — you can try not to be biased but if you do commissioned research your clients will naturally expect that the outcome isn’t going to be negative for them. We also sold our primary research business a number of years ago, so we don’t do any research-for-hire. You can ask Gartner’s Ombudsman (ombudsman@gartner.com) about these policies if you’d like.

  • @benkepes That was quick! Been thinking about it over the w/e…Loved the “holier than thou” response from Gartner, who are they kidding 🙂

  • Lydia – when did Gartner stop doing commissioned research? They certainly used to offer such a service.

    • Years ago. It was within the last eleven years — we were still doing some commissioned research when I was first hired, although we had some strict policies that tried to ensure that anything written would still be neutral, and I remember rejecting requests (with full management support) for things where I expected the outcome would not be the client’s liking. But the policy was changed many years ago — to my recollection, somewhere around 2001.

      We have a consulting division, which means we will do surveys and such on behalf of clients, but they are only allowed to use those results internally, and cannot publish them or use the Gartner name (i.e., it’s for the purpose of the client gathering data about a market in order to do internal strategic planning and so forth).

  • I am a little late to make a contribution to this discussion but I have a few points to make about the traditional analyst business, having worked for some of them over the years.

    Most analysts and analyst firms do indeed seek to be as objective as possible and put processes in place to try to ensure objectivity. But, ensuring complete objectivity i,s of course, impossible.

    The main reason that analysts often produce material that seems biased in favour of major IT vendors is because they spend more time with these companies than with anyone else. This impacts their thinking, in particular their overall view of the market, their vocabulary and their segmentation of markets.

    The companies with whom analysts spend most time are also often their clients so they seek to produce work that is most relevant/useful to these clients.

    As long as Gartner, IDC etc have IT vendors as customers, there will be some bias in how they look at the world. They need to produce a view that will sell to IT vendors in order to survive.

    • While I’m sure that familiarity breeds understanding. It is a relative clear history that objectivity is not influenced by relationships but by hard analytic evidence. When this isn’t the case bias become the order of the day. While it might seem that the popular vote makes the best case, it deprives a complete picture for those buying the services. You ever wonder why it’s only Tier 1 players (with the war chests) who are continually in the spotlight?

      • I think it’s more than just large vendors being in front of analysts more. (In my experience that’s actually not necessarily the case, either. I’ve met plenty of smaller organizations who actually use more time, and/or just have far more effective analyst relations despite being small.)

        For firms like Gartner and Forrester, where end-user inquiry makes up a significant percentage of an analyst’s day, a lot of the information that one gets is naturally going to be focused on who those end-users are asking about and therefore, also telling you about. That generally reflects not just large vendors but also buzz in the market, and any up-and-coming interesting vendors that are doing a reasonable job of marketing (especially SEO).

  • Bill Fathers, senior vice president of global sales and marketing at Savvis commented on the report: “We believe our position in Gartner’s Magic Quadrant as a leader is a testament to our overarching commitment to delivering visionary, enterprise-class solutions to our growing list of clients on a global basis. We are honored that 33 of Fortune’s top 100 companies depend on Savvis for their mission-critical applications in this fast-paced, dynamic environment. We look forward to continued growth and momentum on the heels of such a respected industry acknowledgment.”

Leave a Reply