Latest News

IntApp and Frayman go to war (again) over ILTA statistics

IntApp recently put out a press releases commenting on the findings of an ILTA survey (see text 1 below) however Alp Hug of The Frayman Group challenges this interpretation of the statistics (see text 2 below). Feel free to draw your own conclusions…

(1) IntApp Inc announced that an independent survey by the International Legal Technology Association (ILTA) of its membership shows Wall Builder continues its market momentum, with significant product adoption reported by large and mid-sized law firms.
ILTA’s 2010 Technology Survey presents responses collected from over 460 law firms, comprising over 80,000 attorneys and nearly 180,000 total users. Report data show that 74% of large law firms (700 or more lawyers) using commercial software to enforce ethical screens to secure access to confidential information have standardized on IntApp Wall Builder. Of firms with 350 to 699 lawyers using commercial confidentiality management software, 60% have chosen Wall Builder. The survey report presents market share data on several software products available to law firms.
Survey Data: Confidentiality Products Used by Firms with 700 or More Lawyers
Wall Builder (IntApp)                                  74%
iMPrivate (DocAuto)                                    17%
SecurityGuard (Olson Consulting)                 9%
WincWall (Wertheim Global Solutions)          0%
MasterEthics (RBRO Solutions)                    0%
The Wall (Younts Consulting)                       0%
CompliGuard Protect (The Frayman Group)   0%

Survey Data: Confidentiality Products Used by Firms with 350-699 or More Lawyers
Wall Builder (IntApp)                                      60%
iMPrivate (DocAuto)                                       18%
SecurityGuard (Olson Consulting)                    4%
WincWall (Wertheim Global Solutions)              9%
MasterEthics (RBRO Solutions)                        4%
The Wall (Younts Consulting)                           4%
CompliGuard Protect (The Frayman Group)       4%


(2) Alp Hug of Frayman writes… As an enterprise software veteran, and as head of marketing for Frayman Group, I am always interested in market numbers and statistics. Ultimately, marketing is no longer an art, but a science based on available data. This is why I exchanged emails with Randi Mayes, Executive Director of ILTA, shortly after the ILTA2010 Technology Survey got released. I wanted to make sure I understood what the numbers meant and that I was interpreting them accurately. Here is what I found out:

The technology survey results are based on 464 law firms’ responses.
Of these 464 firms surveyed, 323 (69%) have only 149 lawyers or less on staff.
Of the 464 firms surveyed, only 67 of them (15%) have 350+ lawyers.
Of the firms surveyed, only 26 (a mere 6%) have 700 or more lawyers on staff.
Questions were formulated and answers provided by members with no vendor participation, ensuring self-policing and self-governance (which is very cool !)

We, marketers, like to leverage industry-issued independent data to demonstrate a point, and to leverage the numbers to position ourselves as the best, the most capable, the most advanced, the fastest growing, <insert exciting statement here>. And this is fine. We obviously want to present the findings in the best possible light to our audience. But there is a catch. It is important for the marketers and for the readers to ensure the numbers are not taken out of context.

Example: A 50% dominance in the ILTA 2010 Technology Survey in a given category among the law firms with over 700 attorneys means the vendor has 13 large law firms as clients. A 70% dominance equates to 18 firms. This is great news. But it does NOT translate to the vendor owning 50% or 70% of the market.

This is where the sample size comes in. Let’s take a look at the AmLaw 100. According to AmLaw 100 2010, 95 of the 100 firms have over 350 lawyers on staff. Which means 30% of the largest AmLaw 100 firms are not part of the ILTA survey results, since the ILTA survey only had 67 participating. How many firms is that not included in the ILTA survey results? 27. That’s almost twice the perceived market share the vendor portrayed they had with 13 firms. And this is not even counting the global international firms that are not headquartered in the US. You can see the potential for margin of error in the interpretation of these numbers if they are taken out of context.

I urge the readers of this survey, vendors, and law firms alike, to pay attention to the context within which the statistics were provided. The ILTA Technology Survey is a great effort for the law firm industry and the resulting survey, when the numbers are used in context, are very useful to predict patterns for law firms and vendors. Let’s just make sure we view those numbers for what they are.

One more data point from the ILTA2010 Technology Survey. This statistic is from page 21 of the same report where IntApp derived their conclusions. According to this data, only 7% of the 464 firms are IntApp Wall Builder customers, with 16% having chosen other vendors, and the remainder not using a solution at all or using a custom built solution. So, there is an opportunity for 78% of these firms surveyed to choose whatever technology they deem is appropriate.

Comment: Please don't say there are lies, damned lies and statistics because nobody can even agree who coined that phrase. You have a choice of Mark Twain, Benjamin Disraeli, Charles Dilke, A J Balfour and Walter Bagehot to name just a few.

9 replies on “IntApp and Frayman go to war (again) over ILTA statistics”

Alp has a valid point here about how folks in the business use stats to their advantage, More importantly Law Firms need to look carefully at the numbers in these surveys to understand what they really mean. This same survey indicated that close to %74 of large firms use Autonomy Records Manager. The KEY here once again is that this data is a valid point that reflects only the choices of those who RESPONDED to the survey.
A closely look at the facts on the ground would show that 47 of the AMLAW100 firms use LegalKEY RM and ~29 of them use CA/MDY Records Manager. Beyond that there are still plenty of folks using records packages from Elite and Aderant. IOW's even if they gave it away for free there is just no way that the Autonomy Records solution gained 74% of the large firm market share in the past 3 years….. Caveat Emptor

A recent survey suggests the 80% of statistics are wrong…..

Not my area of expertise, but there's still hardly anyone using the Frayman software…..right? – so fair play to Alp for responding, despite the lack of numbers for his product.

Yes, interesting point and follow on comment about your other product, LegalKEY, Alp.
But can't help but notice that however the data are presented, complicated math notwithstanding, some is more than none.
0% is 0%.

I agree completely with both comments above, and would further comment that many aspects of this (and other surveys) is that they are looking for specific answers for statistical purposes, whereas real answers many times do not fit cleanly into one pigeonhole or another. For example, even comparing these different products as functionally or architecturally apples-to-apples is extremely misleading. The DocAuto iMPrivate product, for example, does ethical wall management in iManage WorkSite, but only on documents. DocAuto's WorkSpace Manager is used more commonly in matter-centric organisations for managing ethical wall security on WorkSpaces, Tabs, Folders, etc., as well as documents. The IntApp and Frayman products cover a much broader range of platforms, but they are not 3-tier applications in WorkSite, so they have some limitations there. Some of the other applications listed are also DM-centric, working only with WorkSite or DOCS Open/DM5/Open Text, but they are also based on direct manipulation of the SQL database, which is limited at best, dangerous at worst (like another company's product that is not listed here because they scrambled a few too many WorkSite databases…).
Maybe someday someone who is really independent will do a “shoot-out” comparison on these types of products, as well as other products that perform similar/related functions…
Any takers? (No vendors please.)

We all rely on CIO circles and there have been a clear indication that Frayman is coming up in the ranks. We all make our decisions based on demos and due diligence – with that said, Vendors should stop exploiting percentages that does not make any sense.

I thought the whole point of Alp's post was that in fact 0% is NOT 0%.

Comments are closed.