I've come across this article at the NYTimes about deep neural networks and the amazing things they do that we just cannot. The problem is that if a human has to make an important decision, like a medical diagnosis, based on an AI recommendation, will he/she do so with no understanding at all of how the AI came up with it, even though it is likely correct?
It reminds me of the way computer chess tends to be baffling to us humans, but is nonetheless "superhuman" at the same time.
Am curious what talkchess thinks of the piece. Sorry if it's been posted before.
Neural Nets can't explain what they do and this is a problem
Moderators: hgm, Rebel, chrisw
-
- Posts: 5
- Joined: Sun Aug 20, 2017 4:09 am
-
- Posts: 4366
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: Neural Nets can't explain what they do and this is a pro
It's a well known issue and also applies to other kinds of complex learned models, including random forests, SVM with non-linear kernels, etc.
--Jon
--Jon
-
- Posts: 2272
- Joined: Mon Sep 29, 2008 1:50 am
Re: Neural Nets can't explain what they do and this is a pro
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.
-
- Posts: 1334
- Joined: Sun Jul 17, 2011 11:14 am
Re: Neural Nets can't explain what they do and this is a pro
I'd like to point out that also applies to humans. We don't know how the brain works, and possibly won't for a long time.althus wrote:I've come across this article at the NYTimes about deep neural networks and the amazing things they do that we just cannot. The problem is that if a human has to make an important decision, like a medical diagnosis, based on an AI recommendation, will he/she do so with no understanding at all of how the AI came up with it, even though it is likely correct?
It reminds me of the way computer chess tends to be baffling to us humans, but is nonetheless "superhuman" at the same time.
Am curious what talkchess thinks of the piece. Sorry if it's been posted before.
If the brain was so simple we could understand it, we would be so simple we couldn't understand it. The same goes for neural nets.
Some believe in the almighty dollar.
I believe in the almighty printf statement.
I believe in the almighty printf statement.
-
- Posts: 7218
- Joined: Mon May 27, 2013 10:31 am
Re: Neural Nets can't explain what they do and this is a pro
Artist can't create good art if he follows rules only.