The Future of {Re}search

As many of my colleagues and followers know, I have been given the brilliant opportunity through my book research to work with a company called Quid and their extraordinary semantic search tool.  This post is a summary of my observations and my best guess at the implications of such a tool for both general search and focused research after a couple of months of hands-on usage.  As a full disclaimer, I have no direct economic investment or interest in the company. I like their tool.  They believe I have something to add to their discussion with influencers and companies. We are both happy!

This is a long post, here is the TL;DR version:

Quid is a tool that is changing the research and search game. It has four implications:

  • The Future of search is Semantic:   Understanding deep inter-connections of people, places and things is essential to navigating an increasingly complex world. This is done – in part – by semantic ontologies and Quid has done a fabulous job at this.
  • Visualization Matters:  New methods of information visualization and presentation are required to show and interact with Semantic search  results and what has been started with Quid is very compelling.
  • Humans and Machines need to do what they do best:  The true value of the Quid tool is the head start it gives those with some domain knowledge. You can immediately see what the “division of labour” should be when you use a tool like Quid.
  • Bots are the Future of Search:  Looking ahead and speculating, I think that automation and “search for me” tools will become essential next steps as the speed of change increase in every domain.  This would be a natural add-on to Quid’s technology.

In summary, on the surface, Quid is changing the very nature research and search.  But more profoundly it is changing how we interpret the future and how professionals can provide value in this future.  It is also changing the opportunities in public policy research and in understanding complex ecosystems such as start-ups and health care.

Quid here…

Observation 1:  The Future is Semantic

At the heart of a Quid search is their engine that provides a comprehensive, curated and closed set of data that, when searched, returns  a first guess at the “semantic ontology” of the results (that is, a ‘clustering’ of concepts that the returned results can be broken down into). Without getting into a dissertation on semantic ontologies (very cool field by the way and a component of a product I built several years ago – so I know just enough to be dangerous!), imagine that the you knew something about Geology.  Your ‘domain’ knowledge has a set of vocabulary that people with similar knowledge – reservoir engineers, production accounting and drilling people – can use to understand each other quickly and efficiently.  In addition to the domain vocabulary, a semantic ontology provides the rules of how this vocabulary relates and connects. It short, it provides human meaning and interpretation – independent of spoken or written language – of a specific field of knowledge.

If that makes sense to you,  compare that to what a typical Google/Bing search returns:  A list driven by hyperlinked connection weightings and ads.  When you see the result differences with Quid, you will understand IMMEDIATELY why they are not the same in any shape or form.  Quid returns a dynamic, real-time ontology of the search terms that you provided it.  Or more accurately, Quid provides a first order guess at what the ontology is. And here is the thing:  If you are even remotely subject matter familiar, this first cut is a mind blower.

Implications:  If you are doing a search for the best restaurant or car or where something is, Google is perfect, efficient and frankly pretty extraordinary.  On the other hand if you are doing deep, broad PhD level research, there is an entire industry of tools for managing complex, long running deep research.  But if you are looking for something that requires an understanding of the known AND the unknown relationships among a connected set of concepts, things or people, then Quid is your tool and the future.

My bet is that the entire business of strategy, trend analysis and deep new thinking that is found in the management consultant, venture capital and public policy areas (to name just a few) has been “searching” for a tool like this.  They will eat it up because 1) it makes them smarter – faster and 2) It uncovers “adjacencies”  among the data that are impossible with any other tool.  And it is in these adjancies and intersections of concepts that new insight and innovation lie.

Observation 2:  Visualization really matters

Overview

Sample Quid Network Cluster diagram. (c) Quid Inc

It would be cool enough if the search results from Quid were returned in a list with a set of identified clusters and ontological hints.  But it is the visualization of the “network” of clusters and inter-connected nodes that is the real differentiator.  By visualizing the returned clusters it provides a clue into the inter-connections among the concepts – including information densities, adjancencies and outliers.  See the visual above.

Nodes intersect with other nodes by degree and weight and nodes connect to others.  Nodes can be companies, cities, countries, people or whatever the underlying data allows.

While the network view is very cool  – and you have to see it to believe it –  it is when you combine this Quid generated view with a bit of smart, human clean-up and filtering and then spin the views by analysizing the metadata that come along for the ride, well it then becomes a killer analytic tool.

It short, I can now peer into a field of business, very quickly find out who’s who in the zoo and then drill in using a whole host of data analytics and then – when satisfied – export the visual outcome in a JPEG or the data in a CSV that can obviously the feed anything your inner data nerd is used to.

Implications: One of the coolest examples of the research is to go from visualizing clusters of quantitative investment  data in a particular space by company with some sub clustering by city, then using the same query against the qualitative text information in curated periodicals and then look at the same query in a visual patent search.  After all of that is understood, spin the dial to look at a timeline of the same field and look at sentiment analysis by a certain meta data variable (City, gender etc) and then finish with a summary of the “social reach” by published article scatter plot that tells you what really has been talked about in the journals as well as socially shared.  That will tell you really who’s who in that zoo.

Once you get a handle on the somewhat challenging interface, all of the above happens in literally clicks of the mouse.  Export the results into PowerPoint and blow people’s mind. Ask for a demo and you will see what I mean.  I haven’t seen anything quite like it.

The new visualizations are game changers.  They dramatically up the intensity, depth and value of a domain visualization to a point where I would be hard pressed to show any other type in a presentation of an industry.  However, like all powerful data analytics tools, Quid can suffer from both the “garbage in / garbage out” and the “little bit of knowledge is dangerous” syndromes.  If I simply take the default return from the search results and visualize it, it will be an oversimplification at best and flat out data lies at worst.

So that leads me to my next observation…

Observation 3:  Do what Humans & Machines do best

Semantic clustering combined with smart people changes everything.  Humans do rapid pattern recognition – especially when seeded with even a basic level knowledge of a field.  Our brains are acutely sharp at quickly seeing and filtering key patterns once we are given hints of the connections.  We can also filter out noise and extraneous stuff really quickly when given this head start.  Trust me I have seen it over and over with this tool.

On the other hand, what machines do really well is a combination of brute force – i.e.  massive ingestion and analysis of patterns of words and phrases to create suggested connections and human training – i.e. the ability to re-do semantic connections based on what a human has suggested is the better data organization and then repeat.

Of course this in addition to the ability of the connected, modern browser to deliver graphically intensive visualization. Machines do that really well!

But as I said to a colleague earlier, the future of data analytics will be to pair a data scientist with a domain expert and sick them loose with Quid.  Watch what happens the next time a so called consulting “subject matter expert” walks into a room that has been seeded with this kind of knowledge.  They better have something to offer, because one hour with a combination of business / data folks and this tool will blow away most consulting resources.

Welcome to the future research and management consulting, folks!  As one of my clients (Big four management consulting firm) asked themselves, “What happens when we are no longer the smartest in the room…what’s our value proposition and what happens to our business model?”

Really good questions!

Implications:

The ability to let powerful semantic engines “lead” our research will profoudly change how experts are viewed by their clients and how they will need to re-imagine / re-think their value propositions.

It is early days for tools such as Quid but they provide a tantalizing hint into the human / machine / AI interactions.  I for one am guardedly optimistic.  The strong caution is that tools such as Quid can create “false positives” – attractive visual representations of incorrect connections and intersections that are only guesses and without the proper finesse from human experts, they become “data visualization eye candy”.

Observation 4 {future}:  Search ‘bots’ will drive knowledge acquisition

To sum up so far: with Quid we have a tool where one can quickly and easily understand relationships among concepts, companies and things that heretofore have been impossible;  one can view these relationships in ways that are truly innovative, insightful and unique and I can slice and analyze data in ways that are both simple and powerful – especially if I apply some of my experience of the domain.   Pretty cool.

So what’s next?  I think the folks at Slack – while in a completely different software universe – have given a hint of where this might go.  Slack’s greatest contribution to team collaboration is more than its simple channel based chats.  It is actually its open API and ability to create and run “slackbots” against the running “stream of consciousness” inside a channel that give us a hint at where analytics software is headed: Let’s let software get to know us, interact in the background and learn as we go.  Domain knowledge changes SO fast that I cannot possibly keep up on a near real-time basis.  And in the case of many of the fields of disruption and technology this cadence is very real.  I know because I am trying pin some of this down as I write my book.

What I really want now is the power of semantic search, next generation visualizations and analytics all within a background AI that learns about the semantics as I go, works while I sleep and shows me what’s new when I am ready.

Simple, right?

Welcome to a new way to look into the future.

Conclusions

I like this tool a lot.  Like any powerful “machine” (think cars, weapons, other AI capabilities etc) it is only as powerful as the intent and capabilities of the human brain driving it.

Or is it?

I think companies like Quid will be able to show us the way forward from advanced analytics and ecosystem mapping to a world driven by AI and smart bots.  Our role in that new space is something I am sure they are thinking about.  And I know the smart folks in Government, consulting, health care and a host of other domains where big problems remained to be solved are thinking about it too

Exciting times.

Game on!

2 responses to “The Future of {Re}search

  1. Jim – cool stuff. How would you compare/contrast Quid with what one can do with IBM’s Watson or IPsoft’s Amelia? More general purpose, less ramp up time, more for consumers vs. corporations, similar but different?

    Like

  2. Pingback: Quid Cheerleading: The Future of Search : Stephen E. Arnold @ Beyond Search·

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s