IntApp recently put out a press releases commenting on the findings of an ILTA survey (see text 1 below) however Alp Hug of The Frayman Group challenges this interpretation of the statistics (see text 2 below). Feel free to draw your own conclusions…

(1) IntApp Inc announced that an independent survey by the International Legal Technology Association (ILTA) of its membership shows Wall Builder continues its market momentum, with significant product adoption reported by large and mid-sized law firms.
ILTA’s 2010 Technology Survey presents responses collected from over 460 law firms, comprising over 80,000 attorneys and nearly 180,000 total users. Report data show that 74% of large law firms (700 or more lawyers) using commercial software to enforce ethical screens to secure access to confidential information have standardized on IntApp Wall Builder. Of firms with 350 to 699 lawyers using commercial confidentiality management software, 60% have chosen Wall Builder. The survey report presents market share data on several software products available to law firms.
Survey Data: Confidentiality Products Used by Firms with 700 or More Lawyers
Wall Builder (IntApp)                                  74%
iMPrivate (DocAuto)                                    17%
SecurityGuard (Olson Consulting)                 9%
WincWall (Wertheim Global Solutions)          0%
MasterEthics (RBRO Solutions)                    0%
The Wall (Younts Consulting)                       0%
CompliGuard Protect (The Frayman Group)   0%

Survey Data: Confidentiality Products Used by Firms with 350-699 or More Lawyers
Wall Builder (IntApp)                                      60%
iMPrivate (DocAuto)                                       18%
SecurityGuard (Olson Consulting)                    4%
WincWall (Wertheim Global Solutions)              9%
MasterEthics (RBRO Solutions)                        4%
The Wall (Younts Consulting)                           4%
CompliGuard Protect (The Frayman Group)       4%


(2) Alp Hug of Frayman writes… As an enterprise software veteran, and as head of marketing for Frayman Group, I am always interested in market numbers and statistics. Ultimately, marketing is no longer an art, but a science based on available data. This is why I exchanged emails with Randi Mayes, Executive Director of ILTA, shortly after the ILTA2010 Technology Survey got released. I wanted to make sure I understood what the numbers meant and that I was interpreting them accurately. Here is what I found out:

The technology survey results are based on 464 law firms’ responses.
Of these 464 firms surveyed, 323 (69%) have only 149 lawyers or less on staff.
Of the 464 firms surveyed, only 67 of them (15%) have 350+ lawyers.
Of the firms surveyed, only 26 (a mere 6%) have 700 or more lawyers on staff.
Questions were formulated and answers provided by members with no vendor participation, ensuring self-policing and self-governance (which is very cool !)

We, marketers, like to leverage industry-issued independent data to demonstrate a point, and to leverage the numbers to position ourselves as the best, the most capable, the most advanced, the fastest growing, <insert exciting statement here>. And this is fine. We obviously want to present the findings in the best possible light to our audience. But there is a catch. It is important for the marketers and for the readers to ensure the numbers are not taken out of context.

Example: A 50% dominance in the ILTA 2010 Technology Survey in a given category among the law firms with over 700 attorneys means the vendor has 13 large law firms as clients. A 70% dominance equates to 18 firms. This is great news. But it does NOT translate to the vendor owning 50% or 70% of the market.

This is where the sample size comes in. Let’s take a look at the AmLaw 100. According to AmLaw 100 2010, 95 of the 100 firms have over 350 lawyers on staff. Which means 30% of the largest AmLaw 100 firms are not part of the ILTA survey results, since the ILTA survey only had 67 participating. How many firms is that not included in the ILTA survey results? 27. That’s almost twice the perceived market share the vendor portrayed they had with 13 firms. And this is not even counting the global international firms that are not headquartered in the US. You can see the potential for margin of error in the interpretation of these numbers if they are taken out of context.

I urge the readers of this survey, vendors, and law firms alike, to pay attention to the context within which the statistics were provided. The ILTA Technology Survey is a great effort for the law firm industry and the resulting survey, when the numbers are used in context, are very useful to predict patterns for law firms and vendors. Let’s just make sure we view those numbers for what they are.

One more data point from the ILTA2010 Technology Survey. This statistic is from page 21 of the same report where IntApp derived their conclusions. According to this data, only 7% of the 464 firms are IntApp Wall Builder customers, with 16% having chosen other vendors, and the remainder not using a solution at all or using a custom built solution. So, there is an opportunity for 78% of these firms surveyed to choose whatever technology they deem is appropriate.

Comment: Please don't say there are lies, damned lies and statistics because nobody can even agree who coined that phrase. You have a choice of Mark Twain, Benjamin Disraeli, Charles Dilke, A J Balfour and Walter Bagehot to name just a few.