
Here is the scenario: you exit the car and walk toward your door, with your right arm carrying four bags of groceries. Your left hand is free to grab your keychain, which holds several keys including your house key. Your challenge is to manipulate those keys with the fingers of your left hand, isolate the house key from the rest, then insert it into the key slot, and turn it to unlock the door.
Can you do it? I’m going to assume you can. After all you’re a capable adult. You know how to do stuff. You can accomplish this task of unlocking the door with one hand, just as you accomplish many other tasks every day. It’s slightly challenging but really it’s no big deal.
So, let’s assume you can do it. How did you learn to do it?
I’m asking on behalf of your future humanoid-robot colleague. He has a keen interest in your feedback on topics like these.
I’m not asking “how did you learn that keys open doors?”; or “how did you learn this particular key opens that particular door?” I’m asking about the specific motor functions you used. How did you learn to manipulate that keychain with one hand, to get a grasp on the correct key? How did you learn to move that key to just the right position and at the right angle to get it inserted?
How did you learn to do all that? Again… asking for a friend.
*
Strictly speaking, this is a physics problem and a mechanical control problem. And a very complex one at that. So some people might think of solving it as an engineering problem, using Newton’s laws of motion and solving it top-down. That’s do-able; but it’s pretty difficult.
To solve the engineering problem you need to know a lot: about the friction of the key surface to your fingertip; about the fine-grain control capability of your hands and finger; about the tolerance allowed by the keyway to allow some room to insert the key; and so on. You’ll need to know all this, and much more in the same vein, in advance. All of these pieces of information would be mere calibration inputs, fed into a big physics-based simulation model.
That “model” is basically a big set of equations that describe this particular situation. A model like this can be programmed into code. Then that code can be run on a computer, which would then “know” how to do the job.
That’s how engineers 20 years ago would have accomplished this task. With their solution, they could run the model-based control program on a computer, and use it to control a robotic arm and hand. It’s a top-down approach, applying first principles of physics and control theory to this very-specific task.
Is that how you solved it?
I doubt it. I bet you don’t really know how you solved it.
*
Next you might tell me that you don’t even have a physics or engineering background; that you’re a liberal arts major who doesn’t even grasp the basics of all that physics-and-control mumbo-jumbo. To which I would say: amazing! You must be some kind of savant … solving complex physics control problems without even reading a single book about physics or controls. You are an overachiever in the best possible way!
(The humanoid robot isn’t responsive to this witty line. I doubt he really understands irony or flattery.)
*
How did you learn to do it? … to manipulate that key in this challenging scenario and open the door?
I think I know how. You learned by doing. You figured out how to accomplish this complex task by inference and inductive logic, built up by repeated physical experience, and from the feedback of success and failure in that experience. Physics dictated what was happening… but you didn’t learn physics in a book, and then solve the problem top-down. You learned bottom-up, by testing the nuances of physics as they appeared in this specific problem, not with equations but by feel. You tried and sometimes failed, and learned as you went along.
Sometimes you dropped the key, from which you learned how to prioritize holding it firmly.
Sometimes your 3rd and 4th fingers nudged the correct key forward from the keychain, so that you could grasp it between your thumb and index finger, in a trick of manipulation that you can’t fully articulate in words. You remembered that action after it was successful the first time.
Sometimes you pushed the key into the keyway at the wrong angle or indexed poorly or before the key was engaged, and in that moment you failed. But you learned from the failures.
And you did all this in in the context of a long hierarchy of experience which you built bottom-up; first by developing a sense of touch and basic hand-eye coordination; then by manipulating objects in general; then by grasping and manipulating keys with your fingers; then by manipulating this specific key and this specific lock until you can do it repeatedly.
And you never attended a day of physics class in your life. Apparently knowledge of physics, in a classroom sense, is immaterial here. You solved this challenge without knowing anything about physics formulas; instead you just did it by feel and by experience.
*
By the way, this is also how humanoid robots will learn to do things like open locked doors with mechanical keys. (OK perhaps they’ll use electronic keypads or wi-fi signals, but somewhere they’ll be doing complex physically-interactive stuff. Stick with me here…)
Your humanoid robot also won’t have the pre-programmed engineering physics model of grip and locks and keys, with tons of pre-programmed calibrations as I described earlier. So your robot won’t solve this problem the way robotics engineers solved it 20 years ago. Your robot will solve it the way you did. That is: by inference and induction, using trial and error, and by learning from mistakes and failures. By discarding strategies when they fail, and further developing strategies when they’re shown to work. By testing physical actions and outcomes, over and over, building up a feel for what works.
*
You are a person with real intelligence. Your humanoid robot has only artificial intelligence. In some matters, he’s not as smart as you are. Ask him to derive that physics-based engineering model from 20 years ago, using Newton’s laws of motion, and he’ll be totally lost. (Even more lost than you!)
But in the specific matter of opening the door with the key, his artificial intelligence is just as good as your human intelligence. And just like you, he learns without referencing the physics textbook. Just like you, he’ll have something more useful for this particular problem: he’ll have a practical feel for physics based on real world experience. And just like you, he’ll have a long history of mistakes, and data recording his subsequent adjustments to those mistakes. His AI-based inductive learning will do the same basic thing you did: learn to manipulate that key into that lock, developing a feel for it by trial and error, until this complex mechanical control challenge becomes just another background task. Until it becomes second nature for him, as it is for you now.
*
Oh don’t get me wrong. Your humanoid robot contains reams of software, control algorithms galore, and tons of physics-oriented rules-based code. He has a lot of physics “knowledge” at his command… a lot more than you have, if we’re being honest. But that doesn’t make him superior. He’s learning by doing, just like you. And his learnings will be contained in his trained AI model, which is similar to your own trained mental model of the task.
Your robot’s trained AI model for performing this trick can, technically speaking, be described with a bunch of math. By contrast, your own method is impossible to describe, in math or in words. If I ask you how you did it, you’d probably say something like “I just developed a feel for it.”
But don’t let your robot’s mathematics intimidate you. Even with all his internal computation, he won’t really be able to explain his methodology any better than you can. His trained AI model is made of math; but it isn’t human-explainable… its far too big, and it resists simplification. The manipulation of the key+lock+door is just something he learned, and that you learned, in similar ways at different times. Each of you separately developed a feel for it, until it became second nature.
It’ll be one of several things you have in common with your new humanoid robot when he arrives. Unlocking a door is, for you two, a kind of shared experience which may help you get to know each other. Someday maybe you can discuss it with him over a cup of coffee.
Hello Bill,
like you article. It is a good summary about how learning is done.
Best regards, Alex
This is a great reflection - yes on AI, its limits and its uses, yes on humans and their irreplaceability.
But most of all it's a great reflection on how we think about the subroutines of programmatic code and that divergence from our own cognition. Through this thought experiment, it would seem that the learning and development of AI is fundamentally different from Humans. This is a hurdle which ought to be respected and not simply overcome.