Unstructured Data: Yet Another "First Mile" Challenge to Effective BPM
OK, this is likely my last Lotusphere-inspired rant for this year, for those of you who have been waiting for me to stop harping on that event. However, I did see a fair amount of thought-provoking, BPM-related stuff there, especially related to the "first mile" problem of information capture...
One such item was a prototype "smart personal assistant" that "lives inside" a user's computer. In the demo and discussion presented to me by an enthusiastic chap from IBM Corp. in Ireland, I saw software "skim through" incoming e-mails and rapidly and accurately triage them in order of importance and urgency. Criteria included key words in subject lines and message bodies, as well as sender information. These were manipulated based on simple rules given to the "assistant" by the hypothetical user.
So what's this got to do with BPM? Well, software that can perform such analysis rapidly and accurately can do so because of its ability to navigate through what the buzzword-heads call "unstructured data." For the purposes of this discussion, "structured data" is data that is all but "computation-ready" without further manipulation – typically data already stored in or produced by a computer. "Unstructured data" is how humans typically communicate – e-mails may be on computers, but they do NOT all contain orderly rows and columns of text and digits.
Structured data is of course useful, in part because it is almost always easier to assimilate into and manipulate with automated, IT-based solutions for analysis, consolidation, sharing, visualization, and similar tasks. However, there's lots of unstructured data that could add immeasurably to such efforts, if it were easy to collect and integrate into the enterprise "data stream."
For example, the unstructured data contained in any notes that support providers make while solving specific user problems could be very useful in the analysis and refinement of support processes. Similarly, notes and e-mails captured and exchanged during almost any user or customer interactions could help process managers deliver processes that are more human-centric and business-driven.
Business processes are, almost by definition, made up of data that is difficult or impossible to "structure" in the ways traditionally employed by computers and software. So, until and unless you can figure out how to translate all process-related information into accurate and complete structured representations, you're going to have to figure out ways to capture and use unstructured data effectively. Technologies are evolving that can help (and the people running database management or analysis efforts at your enterprise may already have some). However, for the time being, there is no substitute for old-fashioned conversations, interviews, and surveys of representatives of all key BPM constituencies, from external customers and internal users to support providers and process managers themselves. You may have some mostly manual "heavy lifting" to do to get such information and to integrate it into your BPM efforts. But those efforts will have little to no business value without that information.
So, how do you and your colleagues deal with unstructured BPM-related information now, and how are you planning to do so, if at all? Do let me know, and we'll revisit this issue as your interest levels dictate.
Michael Dortch has been an analyst, 'information entrepreneur,' speaker, and writer about IT and 'the real world" for the past 30 years. After almost 10 years at the Robert Frances Group (RFG), Michael begins a new position with Aberdeen Group in September.
Dennis Byron is an analyst with ebizQ, focusing on Open Source Software as well as Business Process Management technologies.
