DiscoveryMining

Let’s talk numbers. Last night I attended the sf geekSessions event about scaling web apps, specifically ruby on rails web apps. We don’t develop in ruby around here, but the general takeaway applicable to our business/industry was measurement. We’re dealing with ALOT of data, taking hundreds of gigabytes, that are un-structured, putting structure around them and building inventory is hard. Not only is that hard, keeping them in one single repository, with sub-second search, a dynamic set of attributes/tags/folders(you name it); is VERY hard.

In an industry that throws around terabytes like they are floppy disks, I find it amazing that there is no real sense of measurement. Where are the metrics? Why don’t we, the vendors, put our stats out there for all the world to see. My guess, it’s not as rosy a picture as we’d like and we’d have to stick by it. I see it all the time, cases that start small and get big, take systems to the edge of usability.

What we are dealing with is not a one-solution-fits-all problem. Each case is different and sometimes it makes no sense to get a vendor involved, however when you do it’s important that you know where the strengths and weaknesses are. A weakness in one area should not be a deal breaker if it’s not relevant to the case, but you should know what you’re getting before it’s too late.

Let’s start with a simple one: Document Account Ceiling

I’m going to define it this way:
The maximum number of documents in a single customer account (matter) that can be searched with a single term, from a single interface, and return results in under one minute.

There, that wasn’t so hard was it?  In computer science circles around the world they would say that any search that returns in a minute is broken, but it’s a good starting point. (I’ll be posting on this subject later)

I’d be interested to hear what you’ve got to say so add comments: the good, bad, and not so good.