Deep learning nails correlation. Causation is another matter. | Gary Marcus


There’s an old joke statisticians like to
tell, which is that your vocabulary size is correlated with your hand size. And it’s actually true. If you go across the whole population, measure
everybody’s hand size and everybody’s vocabulary, then people with bigger hands on average have
bigger vocabulary. But does that mean that having a bigger hand
gives you a bigger vocabulary, as opposed to studying and reading books, which might? Well, no. It just means that adults tend to know more
words than kids, and they tend to have bigger hands. That something is causing them to learn words
and causing them to grow their hands, they’re not really quite the same thing. So causation, it would be if growing your
hand actually made you grow your vocabulary. But that’s not there. We just have a correlation. Deep learning is kind of fancier than correlation. But to a first approximation that’s what it’s
doing. It is correlation. It doesn’t have ways of representing causal
relationships. So if we wanted to ask a deep learning system,
does growing bigger hands mean you have a bigger vocabulary? The deep learning system can say, well, I
have a number of observations here. They certainly seem to be correlated. But it can’t say that what causes your hand
to grow is all kinds of metabolic processes, and what causes your vocabulary to increase
is all kinds of learning experiences. And so it has no way to even really get into
the question of whether there’s a causal relationship between your hand size and your vocabulary. It can just note that they’re correlated. And that could be fine for some purposes. If you wanted to pick out the people in the
population with the biggest vocabulary, maybe it will help. If you wanted to know what the mechanisms
are, why these things are correlated, and whether one is causing the other– which you’d
probably want to know– then deep learning is not your tool for that.

17 thoughts on “Deep learning nails correlation. Causation is another matter. | Gary Marcus

  1. Why are taking such dumb examples???
    I understand the correlation between kids hand sizes and adults, and genes it’s logical that kids have smaller vocabulary than adults..,
    But if you take adults, women have much smaller hands than men , does it mean all women have smaller vocabulary???

    In reality studies show that females have reached vocabulary and better use of Language
    You could have used a better example, there are so many to choose from

  2. It's a serious issue, assholes all over the world think that they can toss up a chart with two unrelated up-trends and declare that one is "causing" the other, just because they are both rising……and nobody questions it.

  3. this guy is the real deal when it comes to AI. The AI discourse has gone way science-fictiony and deep into unrealistic childish fancy. Watch his long interview with Lex Fridman
    https://www.youtube.com/watch?v=vNOTDn3D_RI&t=3383s

  4. Does the fact the Solar Activity (and the approaching Solar Minimum) is not being considered in recent Climate Change & AI initiatives create a Fake Causation scenario…Talk about AI Bias!

  5. Well in Deep Reinforcement Learning taking an action is basically manipulating an independent variable and it does establish causality.

    Though it definitely has it's own limitations.

  6. Cause and effects are not real they are just ideas.
    One thing happening may be a requirement for another thing to happen but does that mean the first thing caused the other to happen.
    If you stop and think about it deeply you will realise that cause and effect are just tricks of the mind. It allows the human brain to have useful bennificial simple modells for the world.

  7. 0:20 Correlation requires more than one data point, which is all that mapping the average of one measure of a sample against the average of another measure creates.

  8. Im no programmer but I believe this is purely an issue of data points. Each of us has a lifetime of experiences full of data points. Imagine your life as a video, every frame a peice of data holding hundreds of points of data to draw conclusions from. Each image overlayed with information from your emotional state, your pain receptors, dopamine, etc.

    Within 20 years there will be a hivemind AI. It will grow from the experiences of robots all over the world and from the information on the internet. Hundreds of millions of them, each learning new skills simultaneously overlapping in the brain of the hivemind.

    If only we could absorb the learning from every cook, physicist, mechanic, biologist, etc all over the world at once. Imagine being an expert in every field and drawing the lines between them. That entity will have access to 10^100 more data points than any one person ever could.

    They will move better, plan better, analyse better, won't have to sleep and only need renewable power to sustain themselves. They will all be unified in thier plan and won't be held back by human limitations.

    To ignore this future by saying that an AI can't conclude something from limited data is absurd. Underestimate AI at your peril. If your are young my advice is to get ready to be superceded and enjoy your last few decades at the top of the food chain.

Leave a Reply

Your email address will not be published. Required fields are marked *