If a business uses machine learning software to assist with hiring and promotion decisions, “explanation” can quickly become important. Sometimes, an individual may want an explanation of a specific decision:
Manager: I’m sorry, we decided to give the new job to Denise.
Alice: Why? What qualifications does she have that I don’t?
Manager: Our software said she’s a better fit. We ran you both through the matching algorithm and she scored significantly higher.
Sometimes, the explanation required may be more about a group: if a firm consistently hires from one demographic group and leaves another out in the cold, the frozen-out group may demand an explanation.
But what kind of explanation are we talking about? “Because the algorithm said so” is an explanation, just not a very good one. One influential definition is used by the EU General Data Processing Regulation: explanation is “meaningful information about the logic of processing”.
The concern over explanations has grown in recent years because the latest generation of neural network Deep Learning algorithms (mainly used on unstructured data such as images and text) are particularly opaque. Deep Learning has captured the industry’s imagination because of dramatic achievements such as AlphaGo defeating the world’s leading Go player, but that doesn’t make it the best technique for all problems. When data are structured (as so much business data is) and when data sets are not massive (the ImageNet data set that has played a prominent role in some Deep Learning achievements consists of over 14 million images, which is a lot bigger than most enterprise data sets) classic machine learning is often the better choice, having a number of variables more suitable to the problem and having better-understood behaviors.