Thursday, October 1, 2020

 I've recently enjoyed a number of "Sherlock Holmes pastiche" stories by Donald Thomas.  He's written nonfiction about Victorian crime, and his depth of knowledge shows - the stories evoke a wonderful atmosphere.  I always liked the Conan Doyle stories, but you can only read them so many times, so finding his work was a real pleasure.

As my trade involves Artificial Intelligence applications, a few weeks after finishing most of his Sherlockian works, a thought began knocking around.  Could one write a "Sherlock Holmes AI system"?

It might work like this: taking an image (or consecutive stills from a video) it would, for instance, examine a person and categorize what it could: is the clothing a good fit, is it in style, is it worn in unusual ways; are there environmental indicators such as being wet from rain or snow.  Concerning the face, are there any telltale signs of illness, stress, makeup, etc.  Gait might be analyzed for its medical or employment implications.  I think we all know the typical Sherlock Holmes story passage in which he deduces amazing stuff from such information.

I realize that the author of such stories has the freedom to put in what he wishes Holmes to get out of his observations, so it's somewhat contrived, but some of that works in reality.  Years ago when commuting in Boston I used to play a kind of game (not really inspired by Sherlock) of trying to guess what I could about someone from the appearance of their vehicle.  Bumper stickers give away a lot.  I recall riding with a friend and I just commented on the driver ahead of us - "A former marine, with a son in the Army, and a daughter at <I've forgotten the private school> who's into equestrian activities".  This did surprise my companion, but it wasn't my intent.  The whole game was driven by the boredom of being stuck in slow-moving traffic.  One could also tell a lot from the car itself - had it been in accidents, the nature of the repair history (and possibly the economic status of the owner), the residential or academic parking stickers, the cleanliness of the windows.  So I do know that some thinking along these lines really does work.

An AI system might similarly also look at other aspects of scenes - lighting, mirror reflections, furniture type and condition, indicators of taste or travel, family photos, pet presence, education level, political orientation, level of social activity - to be the basis for further analysis/deduction.  At this point it's just a matter of basic information collection and interpretation.  In fact that may be enough for such a system to be useful; making that information available to a human in some way that allows exploration and correlation with other data might be as far as it needs to go.

The thing that seems "hard" about what Sherlock Holmes would do is it's comprehensiveness - normal people only pay attention to the key elements in a social interaction, not all of the ancillary detail.  This of course is a key trope in the Sherlockian world, but it's very hard to do in reality, certainly for any length of time.  But a computer system could do it, being "always fully on".  Toss in facial recognition and the like and it might be very powerful.

The other element in the stories is that Sherlock had a sort of encyclopedic knowledge of arcane topics, with a library at 221B Baker St. to back it up.  Now we have the web, which might even be better.

Maybe such systems are out there for specialized use; not sure anyone would fund one for practical use, but maybe there's grant money out there... if not, it might be a great basis for a story.