okay so just um a brief
brief bio okay i was born in los angeles
uh
california and i grew up in compton
california
which nowadays is a kind of a heavy
place but
it was good when i was a kid though we
had music lessons in school and all that
kind of stuff
and i went to graduated from high school
compton senior high school
uh in 1960 and then i went
i worked for a while then i went to
school at a little school in ohio called
graceland college
because i had a lot of friends that went
there it was a religious school
and then i
decided that i wasn't going to go to
school anymore but i within a semester
i was back in school i went to long
beach state got a degree in mathematics
after that i went to ucla and got a
degree in physics i was going to be a
particle physicist but concluded that
the employment
opportunities were very grim so i
thought well i'll take a year off
and this is back in 1972 and
so i started you know resigned from
school and started looking for a job and
jobs are pretty hard to find in 1972.
i had fortunately had a summer job that
this
summer of 1971 at information
international
where i learned to do machine language
programming
and so then after when i was looking for
a job there in early
early 72 i saw an ad in the la times it
said
come to dayton ohio and do machine
language programming
and having lived in iowa this little
college uh
for uh you know a year and a half i
thought well i kind of like the midwest
uh
you know it's rolling hills as pretty
green trees not as much
cactus and that kind of thing so i went
down and interviewed with this fellow
and he said he's i said what are you
going to do he says we're going to build
a smart robot
and i didn't have any experience with
robots
after that point other than reading
science fiction and stuff like that but
at any rate he said
we're going to build a smart robot and
uh yeah i thought that's kind of
interesting and i was planning to take a
year off of school
and kind of figure out what i wanted to
do do a little programming or something
and then uh
go back to school but uh so then he said
she's gonna build a robot i thought well
that sounds kind of interesting so i
i went back and interviewed with them
and uh
concluded yeah i wanted to do that it
also was true that it was
pretty hard to get jobs then i had one
other job offer here in l.a but it was
programming missile systems or something
and i
just thought it'd be much more
interesting to build a robot
so i you know matter of fact i got
married
the day we got married we got in the
volkswagen and drove to ohio
and um the interesting thing was that uh
you know i started asking around like
okay we're gonna build a robot how does
it work
and nobody knew i mean honestly they
there was this high
neat idea they were going to build a a
smart robot and the idea
was that by the 1974
machine tool show they were going to
have two of these bendix robots and they
were called
pax which stood for programmable
automation cybernation system
which is kind of a pompous title but uh
anyway the idea was they were going to
have
two bendix robots on a conveyor belt
they're gonna have one of them putting
together chrysler air temp
air conditioning compressors and they
would put it on the
on the conveyor the other one to take it
apart and put the parts back in the bin
and they've just continued to do that
and the idea was it would have computer
vision
uh quote adaptive reactive intelligence
and i remember asking what the heck is
that
but the thing is nobody knew how it
worked and i go around asking everybody
i said well
how does this work well i don't know i'm
working on interfaces or no i'm
designing the arm
but since i i was the only professional
programmer
on this project so i thought well okay
this actually can be kind of fun
because i'd always wanted to write an
operating system and always wanted to
you know do that kind of thing so i just
very gently started going around saying
well
would you mind if the angles increased
counterclockwise
no sounds good to me would you mind if
if the radius increases from zero and
so we do we define the coordinates of
the robot in a very gentle way
and i just ask questions that could we
do this can we do that
finally within a couple weeks i was the
project manager of the control system
because nobody knew and
then i began to wonder i says i didn't
really have any
any hard requirements that were
specified in any kind of a mathematical
way
the idea was that if if there was a
problem the machine would look at it
figure it out
and fix it and i thought to myself you
know i don't have the slightest idea how
to make something that smart
i said but i know that i can make
something or we can design something and
build something that that
you can have a reasonable basic like
language or something
uh that you could program to do things
kind of like machine tools do
so that's what we um that's what we set
out to build
and as a matter of fact we were able to
to build that uh
but it was actually a couple of years
after i started working there
um the uh i saw a write-up
of um what was supposed to happen and
they were going to hire
two they only hired one which is myself
they were going to hire two
so-called intelligence engineers and
within six months
they were going to build this this this
smart robot that could look at stuff and
figure out what was wrong and that kind
of thing and i'll have to say that
still today we can't do that i mean
we're a long way we're a long way from
having a robot that could say
you could say hey go get me a beer out
of the fridge and when you get back
we'll talk about
who's running for governor or something
that is so such a difficult problem that
that we we simply can't do that uh
currently but that doesn't mean that
robots aren't very useful because your
cars are put together with robots all
kinds of
uh machinery and appliances and things
like that are put together
with robots which in fact do very useful
work
and that but there's an interesting
thing about it um i remember
i was pretty fascinated with building
this robot because he basically started
off with an empty mini computer
and a little you know this is 1972 and
and
having uh we probably had uh
it would take several nanoseconds to do
a register to register
ad but it was pretty fast mini computer
but very slow by
you know comparison with what we have
today but the um
uh we filled the robot up with the idea
of having an empty computer
and providing its software so it could
actually respond to the environment
and and do things move around that was
very fascinating to me motor control
and uh you know i learned an enormous
amount from that um
and we uh you know we wrote a lot of
software this machine didn't even have a
push down
stack so we had to write push down stack
software and
linked list software which was all
fortunate that i had that summer job
previously because i learned all about
that kind of thing
in that job and so we were actually able
to build this robot and it had
began to realize that uh you wouldn't
just want a robot that could pick in
place you wanted a robot that could pick
and place and adapt to things like
if you had taught it to pick up that
that tape cassette holder
and then somebody moved it that the
robot could actually
adapt its its motions to do that uh this
is all
you know without path planning so you
had to be kind of smart but they were
industrial robots too so they they're
in a pretty organized environment
but in those days i thought well you
know it's really fascinating if you can
make a robot solve problems
and that's true especially if you're as
we do at jpl if you have an environment
where
it's unstructured and you're not sure
what you're going to accomplish
but in the case of an industrial robot
you don't really want them to be
you'd like them to be smart if it was
affordable but fundamentally you don't
want
a robot that could i always thought this
was a good a good fiducial problem
you say make me an envelope so the robot
can pick up scissors
maybe it traces an envelope a pattern on
a piece of paper picks up scissors
cuts it out and gets a glue bottle glues
it and makes you an envelope
and that would be a real tour to force
from the standpoint of dexterity
and problem solving stuff but nobody
wants to buy envelopes
that were that expensive because it
would cost you a lot you want to have a
roll of paper that unrolls at 60 miles
an hour and goes into a machine that
just
spits these things out and automatically
puts them in boxes that's how you can
get envelopes that cost
you know put a couple of together not a
nickel a dime something like that
so anyway it's really very different um
but then after ben
i could tell that bendix uh was
interested in making profits right away
and back in the early 70s companies
weren't yet convinced that robotics
i mean they were using them to weld cars
together and things like that you know
very
very precise things but they weren't
convinced that adaptive reactive robots
were going to be really very useful
and i could sense that bendix was going
to
get out of the robotics business so i
went to the r d center in schenectady
because general electric the general
electric r d center they were very
interested in
in robotics for their appliances and
that kind of thing
and i worked there for a year um but
while i was uh while i was there i ran
into some people from jpl
and um uh
at a conference a robotics conference
and they thought well why don't you come
out for a visit if you're
ever out here and i happen to go to the
stanford research institute
now sri international uh to
import a computer vision system of just
a laminar computer vision system
uh for our lab at general electric and
so i stopped here just for a
you know professional courtesy find out
what everybody's doing
and the supervisor of the time said oh
when are you coming here to go to work
i said oh i didn't come and i said this
is an interview i just came here to
talk about what you know everybody's
doing and but they they were a little
bit persistent my wife hated schenectady
so we ended up i mentioned her that
these people from jpl kept calling up to
come here and go to work and she said go
back to california
and sure enough but when she said that i
thought you know
i'll bet you in six months i'm living in
pasadena and it wasn't quite true it was
it was uh la crossinta but uh so anyway
we moved to jpl
partly because i really became
fascinated with this whole
neurological idea of how do you you know
what does it mean to be smart what does
it mean to be conscious and those kinds
of issues which we
don't understand really by any means
but i thought if there was an
organization in the world that really
needed
capable robotics it was going to be a
place like jpl because if you're going
to go to mars it's going to be a long
time before we
put humans on mars if we want to get
them back alive
and so i came to jpl and we were just
winding up the uh the mars 1984 rover
project we thought maybe we're going to
send a rover
in 1984 which we never did but uh and
then the
nasa funding for robotics started to
roll off but
i just kept pushing i thought you know
this is going to be good for us and
then the supervisor left and went to
boeing so i became the supervisor and i
that was an interesting problem because
i thought you know if i become a
supervisor that means i'm going to do a
lot of administrative stuff
and i won't be doing you know real a lot
of research
but if nobody who has a passion for it
is a supervisor
it will just die you need you need
entrepreneurs you need people who are
willing to push things so i became the
supervisor
and i just you know i pushed this and i
pushed that and i
was able to uh it was kind of
interesting but
one day because caltech at the time did
not do robotics
now although mark rayburt who later went
to you've probably interviewed him he
later went to mit he worked here
when i first arrived he had been at
caltech but they didn't really start a
robotics program
and so i one day i just called up the
head of the
engineering and applied science
department cold call i remember i was
kind of nervous but i
told him that well you know we we uh do
robotics at jpl and caltech doesn't but
you know most universities now starting
a robotics program
and isn't there some way we could work
together to uh
you know do some robotic stuff and um
so he said well you know it's
interesting we've been thinking we ought
to do that but the way it works at
caltech is that you get
you have to get a professor who's really
interested in in
championing this he says you need to
talk to fred kuhling
okay so i called up whom i didn't know i
called him up although i had seen him
before but i didn't know who he was uh
feynman and carver mead and john
hopfield were
uh conducting a course this was in 1982
uh called the physics of computation
which i went down with a few other
people we audit we took the course
and i saw this this this looked like a
you know must have been a professor or
an older student but he was in there all
the time turned out that was fred
hewlett
anybody talked to him and he says yeah
why don't you come down and eat lunch
with us and we'll see
you know we'll talk about it so i went
down to the atheneum and uh
met with the well there were four
professors there and myself when we
talked about how we might collaborate
um then but that the great thing was
that getting the campus involved since
since we're all caltech employees and
and uh
jpl pays a lot of attention to what
campus people think
so getting the campus involved the next
thing you know they said well can you
teach a robotics course
which i did we set up a little robotics
lab and
then we fred and i uh wrote a proposal
in 1984.
uh we wrote a proposal simultaneously to
jpl and caltech we said well look
we need to do robotics uh if jpl's going
to prosper in robotics we need to have
some
good investments so why don't you give
us 500k a year to do robotics and we'll
do part on campus and part here
and so they didn't give us the 500k but
what they did do is they started we have
internal
our research grants they started uh what
they called
called a large ddf that was the
director's discretionary fund
and uh so that what they said they would
give us 300k a year
so i made the executive decision that we
would not
um that we would not do manipulation
since a lot of people are
doing you know because of industrial
robotics a lot of people are doing
manipulation
and we ought to just do rovers because
the idea of a rover was it had to be
able to rover
rove autonomously so we took that money
and
uh and actually built a matter of fact
brian had a big role in that a
very very very big role uh we took that
money and did the first
ever that we were aware of autonomous
mobility demonstration out here in the
arroyo next to jpl natural terrain
uh that was in 1989 nobody there had
been a lot of mobility
demonstrations but people were going
down the hall and avoiding waste but
baskets and stuff like that but we
actually went out
started the rover and it took pictures
and it
navigated over behind some bushes all by
itself and that was really really a
first
but then the interesting thing was that
uh so i got to know fred
and and uh i had taught this robotics
course for a couple of years or
i thought it for one year and then uh
to be frank we i got really disappointed
at the lab here
for the following reason we had busted
our hump putting together a proposal for
darpa
to do rover type work i mean you know
this late tonight you write the proposal
this and that and
and you had to write it in in a fashion
where you could you wrote
a kind of a proposal to be able to
propose and they were being very
selective and there were only six
organizations in the country that were
chosen to be able to propose later
and so i happened to be in washington dc
at a at a meeting
and we had submitted this proposal and
uh i was
just itching to know how did it go so i
called up our program office here and i
said well how did we do
and they said oh we were accepted we can
propose but we withdrew
i said we withdrew why he says well we
can't compete and
you know a bunch of stuff and i thought
and i happened to be standing next to a
fellow
from david nitzan from sri
at the time and whom i had met he and
charlie rose and had
run multi-client studies on robotics
which i participated in when i was at
bendix and general electric so i knew
them but anyway i was standing next to
david and they had written the proposal
and they were also selected
so he said uh you know the old days of
no cell phones were on the
on the on the landlines the pay phone
and uh he says how'd you do i said
because i knew that they had made it
i said you know we made it but but they
withdrew they withdrew and i says you
know i feel so disappointed i feel like
quitting
and he said well why don't you come up
and work at sri
lanka i said yeah i want to finish my
phd because i never did
you know i got involved in robotics i
never did finish my phd at that time
uh at ucla in physics i said but you
know i
i want to finish my phd he says oh you
can go to stanford
so i i applied to stanford
and i knew bernie roth who was a
professor there
and he agreed to he'd be my thesis
advisor and so i
i actually sold my house
and went in and resigned told my boss i
was leaving
here at jpl and then he uh
then i called fred kulick up and i said
well i won't be able to teach a robotics
course this summer because i'm i mean
this
this spring term because i'm moving up
to stanford and are
you know going to go to school there and
so then he um
and i knew people at stanford knew tom
binford and a lot of people at stanford
have been very active
um lou paul um and uh
so so but then a couple days later fred
kellet calls up he says well
if you go to if you could go to caltech
would you would you stay
i said boy that i says you know i
already sold my house and everything and
i said well yeah i'd really be tempted
because i always had a soft spot in my
heart for caltech and i had
you know helped the lab together and
this and that and uh
so the net result was that i stayed here
uh and went to fred was my thesis
advisor then so i got a phd in my bodies
from caltech
that just had kind of just started the
robotics program joel burdick is a
you know professor there now he was a
had graduated from uh
stanford i guess and um so anyway it was
uh it was kind of an interesting uh
it's kind of an interesting uh
interesting way for things to happen but
then after the uh
after i finally finished my thesis i was
over on campus for five years and then i
i had to come back here and and work and
so i finished my thesis at night
uh you know working at home and um
but then you know jpl by getting campus
involved then jpl got much more involved
and we uh you know they started thinking
well you know because of this grant we
had in the in the
autonomy and robotics that both the
campus and jpl
decided that was really something they
wanted to do and so jpl
because of that grant we had doing the
autonomous mobility
and because we had gotten some money
david miller whom you may interview he's
at
oklahoma he was working on
small robots uh with a debt with what
they called
behavioral control the idea is if you
run into something is really simple you
run into something back up turn left a
little bit and then go forward until you
finally
get out and you kind of get stuck that
way but it
it was a a useful paradigm and
but because of that experience we were
able to get the
the pathfinder project which was uh it
was a mars the sojourner rover it was a
mars
it was actually a flight experiment it
wasn't a real hardcore
absolutely rigorous science because it
was a flight experiment to show that
robotics would be
useful and uh or the mobility would be
useful on mars and the funny thing was
that before we
actually flew that
there are a lot of people here at jpel
says oh we don't want these robotics we
don't want those damn robots they're
going to take up mass and
take up instruments we would rather have
instruments there
and or let's put an arm on the robot on
the lander and they can reach out and
grab some stuff
but so journal was so successful that
now everybody pretty much believes that
mobility is a good thing to do with
uh with robots for planetary planetary
exploration
so anyway i did finish my phd at caltech
and
but then i became the section manager of
the robotics section while we were doing
sojourner donna shirley
was the was the project manager for the
rover
uh the uh for the sojourner rover the
people who were running
pathfinder didn't necessarily want again
didn't necessarily want that rover
taking up mass on there but but you know
higher pay grades than they
uh intervenes intervened so we um and we
actually flew that
but it was kind of interesting some of
the stuff like just a
story that really makes me kind of still
chuckle is it that uh
i mentioned that bendix that we had we
had this empty computer
all it had was a it was an assembler a
fortran
interpreter and a link loader so you
certainly couldn't run a robot very fast
with a fortran interpreter so we
decided to do all in the same language
but so we had to write all our own math
routines and everything
and i was really an empty computer but
so we figured out a way to
to do fast sines and cosines using
interpolation and
some stuff like that but so we we wrote
this routine up to do signs and cosigns
and i asked the fellow there was a
fellow helping me do programming he
didn't have a background in programming
and he didn't have a
much college education in terms of
mathematics and stuff but he's a good
hard worker
and so i said all right i want you to do
a test i want to say take all of the uh
and instead of using degrees we
organized organized it by
binary fractions of turns so full turn
was
you know all ones and and
that made it very easy to index into a
table and and we could get a
we could get a uh a sine or cosine in
seven
microseconds okay so that was pretty
fast but
so anyway i told him i said we want to
test this i want you to
just go all the way through of the first
quadrant
uh get the sign square it get the
cosine square it and add them and then
print the results out
so he came to me says you know there's
something wrong i said what he said it
always adds that to one
i said oh well i said that's exactly
right you know it just uh
but anyway it was very interesting uh
when when people started in robotics
there was uh
i remember hearing the story that back
around 1965 at stanford at the ai lab
they thought that a couple of grad
students could
program a robotic vehicle to drive to
san francisco from palo alto
autonomously and do it in six months
and i will say that we still you know
there's a lot of work going on
mercedes and various uh companies are
trying to develop that but yeah that's
pretty dangerous and it's not really
the kind of thing where you're gonna
easily solve the problem
if you know if you took all the cars off
and you painted a yellow line down the
street
and you had some some or some
fluorescent line or something and you
just had a sensor yeah you could do that
but that's very simple that's not
recognizing features in the environment
and deciding trying to decide what their
intent is and that kind of thing
so anyway now jpl you know we we did
sojourn that was successful we did mur
and you know those the rover one of them
is kind of crapped out but the
rover is still going it's gone i just
read about 30 kilometers
just today i think it was and so now i
think people believe that
robotics has a place in lots of
activities and certainly the military
is interested in all kinds of robotics i
mean cruise missiles are really
robust they aren't trying to do
dogfights and evade you know made people
shooting them necessarily but
they still track the environment and
listen to gps or
look at uh topographical signals and um
and get to their very precisely to their
to their targets so
so i guess uh robots will be with us for
a while but i think it's going to be a
long time
before there are really dexterous
smart robots i mean for example you know
you take a person you say
all right you know a good example from i
remember going to
automotive and assembly plants and you
know a guy
working on the line can grab a handful
of fasteners just out of a bend
go like this and stuff them in and
magically they kind of come out of your
hand
with their right end pointing forward
install them and then grab a screwdriver
and an air
powered screwdriver and screw them in
and
the the dexterity to be able to do that
and recognize
the the shape and the orientation of
something or even recognize you know
yourself if
if i gave you that pen and
uh let's say it was really cold you
picked it up you'd go
what's wrong with that pen you have all
this built-in knowledge and it'll be
very difficult to
to convey that you know in a way that
you could reason about and anticipate
but again that doesn't mean that robots
can't be very very
capable and and really enhance
productivity
but i mean the basic problem of how over
four billion years we have these
these neurons that can think about
things like how are we here
and how did the universe start it it
makes us
recognize what a difficult difficult and
subtle problem that is
but that's so that's really interesting
just to cover some of the historical
elements sure
what year was it that you went to ge
okay i came to jpl in 1978 in october
and i went to ge in 1977 in september
and i went to bendix uh in in april of
1972.
and then he went back to caltech oh okay
i went
i started in 1985 i went back as a
as the oldest graduate student there i
was 43 at the time
when i finally graduated i was one of
the two graduate students with
pretty graying hair
well as a matter of fact okay fred was
interested in
robotics and i say he was the champion
to get the robotics program going
but what i did was i was very much
interested in
neural networks and learning and so i
took a lot of
biology classes my minor advisor
i had my major was mechanical
engineering with fred
my minor was computer science carver
mead was my
was my minor uh advisor
and so i was very much interested well i
had kids by this time
a matter of fact i had four kids when i
went to went back to cal tech to grad
school but
but you know i observed that kids i
watched my daughter
when she was just an infant and she was
interested she had this red bunny
and you know that's just she was
intrinsically for genetic reasons
interested in the red bunny
and she wanted to grab the red bunny oh
yeah she's watching like she
couldn't even sit up yet and she's
watching this thing and she reached she
i could tell she was trying to grab it
and she put her hand over here
but i watched her for maybe 40 minutes
and then she then she
put her hand here and then she realized
that she could
go like this and get her hand on the
bunny
and so then after that she began to get
closer and closer
and finally after 40 minutes half hour
or so whatever it was she got so she'd
see that bunny and she would reach for
it
she couldn't she couldn't yet manipulate
it but she could get her hand on it
which made her
again intrinsically solve solve problems
and so i thought you know that's pretty
interesting because how do how do kids
learn to control themselves you look at
babies
i just have i have a new grandson just
two months old
and you watch them and you know they're
going like this
and that's how you learn to control your
arms because you issue commands
and you see what happens and there's
something intrinsically that says that
wasn't quite right
and uh or i mean i i've been played the
violin for years still do occasionally
but
it was always interesting to me that how
you could learn to play the violin it's
true that
your teacher beats you up if you don't
do it right
but you know you listen to it you're
doing this you say oh that sounds awful
but you learn somehow to change the
stiffness of your muscles or the amount
of force
um and you make it sound better
and you make it sound on too and there's
all these kinds of loop closure things
but you have have capabilities in your
nervous system that recognize
if something isn't right and they
if you make corrections they recognize
how to correct
but the interesting thing is that you
could let's say you're a concert
violinist
or a concert pianist and you're going to
be in a skit so you want to play
badly so you can actually learn to play
badly
because your your knee-jerk reaction is
to play very very well but
you know if you're supposed to play
badly and say okay what am i going to do
okay this and this and then you
but you actually have to learn that so
the point is you've got internal models
uh that you can adapt to uh whether it
be good or bad
but you're you're figuring some or
optimizing some figure of merit
uh that means that it's it's acceptable
for what you're trying to do
but uh anyway so my my thesis project
was uh
two things uh learning hand-eye
coordination
which basically means you reach for
something and and i actually had to do
this in simulation because we didn't
have any
money to to build hardware but learning
hand-eye coordination was just basic and
that was a that's a
common robotics problem when i worked at
bendix we had the first time we did a
robotics demo
people thought we thought well you could
just take a tape measure and measure
from the middle of the robot to the
vision stage and this and that and you
know as long as you put the part
in exactly the right place it would pick
it up but if the part rotated
the arm would go to the wrong place i
think everybody was like oh we got to
have the ability to do these coordinate
transformations that i mentioned
but then you still have the problem of
how do you how do you get
when the computer vision system sees
something it's going to see it in its
coordinates
and you need to translate that to the
coordinates of the robot so it can grab
it
and you know how do you do that and we
ended up with little things of spinners
and we
figured out that you could if you always
poked a piece of clay
uh you could look the the deviations if
you rotated the
piece of clay it had pointers on it so
you could tell where it was oriented
but it would form a little circle of
mistakes but there was a well-defined
relationship between the last point you
made and where it ought to be and so you
could go in there
and and and kind of tweak your your
tables to make the uh
to make the uh vision calibrated but i
wanted to learn how to
again how does it how does my daughter
do this well she
placed fiddle with it and so i you know
built a system that could do that
and then i was also interested how do
you learn to catch
because you know yourself if if you have
a well-defined ball somebody throws it
to you
yeah you can learn to catch that but
then if somebody throws you a beach ball
instead instead of
you know they go like this they just
they fall descending fire so to speak
and so i the simulation there was a
learning neural network that you could
teach that it would learn just by
watching it would learn it would watch
the ball
decide where it's going to land based on
experience
decide is it a light bulb or a heavy
ball and then
it would it learned to move the arm by
just moving the arm back and forth and
watching it a lot
so watching it's what knew what
commanded issued and knew what it did
so then it learned to associate the
appropriate command
with what it saw the ball doing and
predicting where it was going to land so
they
and then but then the interesting
problem is it's not always possible to
catch a ball this is a
this is a two-dimensional problem if
sometimes it's just not possible to get
there
or you're going to get there at the
wrong angle and it's going to bounce out
there's no way you can get it into the
cup
and so that ends up being kind of a
subtle it's not just the number of
successes versus the number of tries
but it's the number of successes versus
the number of possible catches
which is a actually a much more
difficult problem that uh
i learned a lot about trying to simulate
physical systems doing that because you
don't you know you start trying to it
bounces and it's got
elasticity and damping and all that kind
of stuff and so you end up
you know very interesting problems but
that that gave me an appreciation for
the kind of things people do here
when they're trying to do a very
sophisticated control of a spacecraft
with sloshing fuel and stuff like that
because you know this spacecraft
aren't rigid they they wobble all over
the place and if you're trying to
you're trying to stay pointed on a star
or a galaxy or something like that
uh there are a lot of very subtle
problems involved in in doing that
so better the caltech you know i i took
uh
to control classes and that kind of
thing gave me a a much broader
appreciation of the kind of thing
what it means to try to control a
physical system
and um so but then you know the you know
you become a section manager and then
you become a
division technologist and you but as you
get older
you realize it more and more that you're
you kind of become a tribal elder and it
you provide a very useful service by
helping people write good proposals
and helping them focus their scientific
ideas and technological ideas well
so that's the kind of stuff i do now i
read lots of proposals
and i and it's funny when i when i was
in high school
i told my uh my high school english
teacher in 10th grade she gave us an
assignment
every day we were supposed to to
encounter a new word
and we're supposed to make a dictionary
entry for that and we're supposed to do
it every day
well of course i didn't do it till the
end of the semester and i
i told her i finally i was so frustrated
i said you know i'm going to be a
scientist or an engineer i don't need to
do this
and the hilarious thing is mostly what i
do
is right text and it's very important to
write it
in a lucid fashion and i tell my
students that i teach at ucla and i tell
my students that you know
you think you're going to be an engineer
but you're going to be selling your
ideas and writing a lot
and as a matter of fact we joke with
that the scientists and engineers here i
said hey
when you're in graduate school do you
think you're going to spend most of your
time writing
you know people write papers they write
books this and that so it's you know a
lot of writing and so
the i think a matter of fact i think the
the uh
the best indicator for success whatever
that really means
uh but is you know if you're talking
about success in an organization or
something it's
it's a an ability to given that you're
reasonably smart
which you are at a place like jpl but
it's uh
that you're able to express yourself in
a lucid manner and uh
and sell your ideas because if you don't
sell it you know the job one if you're
gonna do science is to be funded
and if you aren't funded you aren't
gonna do it did it fall off i don't know
i i ran
over it yeah okay so
is that uh what else would you like to
know
can you know a little bit about the the
group here
um so when you came here how many people
were there how did it grow
how did you what were some of the
difficulties
well okay when i first came here the uh
this is again it was 1978
we had been working on the 1984 mars
rover
and that brian may have showed you some
pictures of that but it was a uh
it was basically a chassis a metal
chassis with plywood on it had a big uh
pdp-40 computer which is a huge thing in
those days
sitting on the on the chassis it had
volkswagen wheels
it had a laser uh scanner
and stereo vision cameras
and uh it had a shineman especially
built
shineman arm vic shineman had built it
it was a it had to be longer so it could
reach the ground
and so the idea there was that we had
actually built some automatic path
planning
uh algorithms and that's done before i
before i arrived
um but the funding started to dry up
they but they actually were able to uh
kind of like what we do with the rovers
today if you look at the video
image of let's say a rock you're
interested in picking up
that you could a robot would actually
plant a path and go over and pick up the
rock
and it could actually execute
three-point turns to get itself in an
attitude where it could pick up the rock
as a matter of fact mike griffin was
one of the people that worked on he
actually published a paper at a
conference about three-point turns with
the jpl
jpl rover uh but the funding kind of
dried up
for that uh but at that time nasa was
very much
interested in in
the space shuttle manipulator because i
hadn't phoned the shuttle yet when i
first
when i first arrived and uh so we uh i
became a supervisor of the group and uh
you know mark rayburn was interested in
hopping machines he was doing some of
that on campus and
and uh and mostly not here mostly for on
campus but
campus is very close as you know and
oh john craig was here uh there are
several people we had uh
the people in an ai group we had they're
doing
path planning and that type of thing we
had people in the computer vision group
which was actually in division 38 which
is a division i'm now in
and we had people in the uh in the
robotics group
but they all worked along very well as a
matter of fact the uh
the rover lab was a helicopter hangar
which
was in the place where that building
that big greek building now is they
they tore it down a few years ago and
built the uh built the
uh flight project center but we uh you
know we fiddled we had a little bit of
funding here and there we get some uh
in those days nasa was they had
art what are called our tops which was
research and technology
objectives and plans so every year we
had pretty close working relationships
with our headquarters sponsors that time
and every year we'd negotiate with a
sponsor about what we would work on
and so we had funding to keep going um
but then you know that started to dry up
um but then the space shuttle you know
people's well people it wasn't really a
space shuttle but the
back in the early 80s people started
thinking very seriously about the space
station
and uh some congressman had said their
congress had said that nasa should
consider
something like 10 percent of the space
station study budget to be
devoted to developing automation
technology
and so we thought well maybe we're going
to be windfall 300 and some million
dollars a year to do robotics which
didn't quite happen
but nasa started funding research
that was oriented toward doing space
servicing
and uh we had a this is actually kind of
funny we had a
meeting we had what was called the
tele-robotic inner center working group
and then all the nasa centers were
involved in it and
so we had a meeting at headquarters and
this is right matter of fact this is
when i thought i was
still leaving to go up to stanford i
didn't realize i was but i had
i told him i'm going to resign but my
boss still made me go to the meeting
and uh so the this headquarters sponsor
a fellow named ron larson
uh he ended up leaving nasa after he i
guess he put in his 20 years he went to
university of maryland to run their
computer
center but always like ron liked working
with him
but he was he was starting trying to put
together a program
to deal with space servicing and this
wasn't really
robotic mobility so much as it was you
know could you replace
modules that failed and that type of
thing and so we went to um
we had a meeting at headquarters and uh
just before lunch ron said well you know
when we come back i'd like to hear some
specific ideas for what we could do
and i thought my god i've got to talk
first or we lose
i mean this is the kind of thing where
you know if you if you even if you have
the same ideas if you say at second your
second fiddle so i i spent lunch time
drawing drawing the uh basically the
outline of a
of a pragmatic proposal a lot of times
people who propose robotics and i i get
irritated of this today
i still see this people say oh we're
going to build a bio-inspired robot
that's going to be fully autonomous
and fully integrated and so i always say
okay
question could you tell me how it works
at all what does that mean
how are you know you know you're not
going to just build a neuron here you've
got a bunch of things interacting
how is it going to be bio-inspired and
how is it going to work
and then i say so don't say that say
that you're going to build something
that's better and that's still useful
but don't say you're going to go mimic
some kind of a dog or something
but anyway so i so the point is that i
put together a proposal where
the idea is for for well-defined local
tasks
that once you've defined it well enough
a robot can do it like if you told you
say robot okay
pick that up okay once you see it okay
and the robot could actually you could
actually
take an image enhance the edges and take
a light pin or something or a mouse and
draw a picture on it say i picked that
up
and if the robot knows you're supposed
to pick up something that looks like
a little parallel pipette you know okay
it knows that you can plant it or you
can have the
the motions pre-coated or something
anyway you can relocate them and do that
but the um but the
if this were a harder problem where you
had a whole bunch of stuff a bunch of
rocks or something you're on the surface
of mars
or you know maybe you had a wreck on the
space station and
you know things no longer look like
themselves so you say okay
now the robot can't figure out what to
pick up but if i can tell it what to
pick up it can do it and the robot you
know you're not going to be out there
anyway the robot's got to manage the
forces and this and what so
those are doable control issues so we
propose what we call local autonomy
which means that for high-level stuff
the people supervise
and low-level stuff the robot does it
and that in fact really enhances your
product productivity
so the funny thing was so i put this
down during much time
as soon as i got back in the room i said
hey i've been thinking about this
lunchtime i have an idea
so ron says ah okay carl go ahead and so
i put this and i could tell all the
other nasa
people were so pissed at me boy they
were irritated but you know the thing is
that you gotta hit
the oh that's an old saw a hit while the
iron is hot and so that what that
happened was with jpl we got
the telerobot uh killer robotics project
um that's starting in 1985 and we ended
up with
you know something like 30 people here
working on manipulation
uh human you know we call it you know
man machine interchanges or man machine
interface
or human guided tele robotics so it's
got a robotic aspect
where the robot's capable and it's got a
human aspect where it requires
who's going to pick out the guy who's
going to run for mayor
and go over and give him a beer okay
that's the kind of thing have to be a
high level
human level uh uh decision in human
level planning
um but it was very useful i mean we we
worked on that for several years and
finally the uh
you know i just the space station still
doesn't have a great deal of automation
on it
the space shuttle cross space shuttle's
stopping now but that doesn't have a
great deal of automation they do a lot
of you know human
interactive astronauts uh helping
but that's exactly the the way that we
control the rovers on mars
we uh you know we get back images stereo
images we can
you know you can reason about them a
little bit but the human operators help
say okay we like that rock because you
need some scientists who have some
smarts we don't have
have robotic grade geologists yet
and so they're the uh using
multi-spectral you know
rama spectrometers and and
multi-spectral images and things like
you can tell a lot about what might
interesting rock might be so the
scientists say you know that's really an
interesting rock let's go
let's go braid the surface and see
what's inside and uh
you know that a moderator we went uh we
went to ford
a few years ago since we built rovers
and they're computer controlled we
thought yeah let's go talk to the ford
research lab
people because they your car today has
lots of computing
and so we found out that you know
typical 1909 2005
ford was going to have something like 12
and a half megabytes
of power control computing and this
controls the engine
timing the fuel injectors the
transmission
and it senses the air temperature the
air pressure all that kind of stuff
and it was absolutely necessary to do
that to be able to meet the pollution
pollution requirements biggest
breakthrough in cars in the last
you know 50 years anyway has been
computerized
controls of various sorts even controls
to keep you from tipping over in your
suv it's just
just remarkable and a lot of different
processors and different
local area networks all this kind of
stuff is just remarkable
if you look at an old car i have a model
t at home
and you know basically all of the
computing happens
uh you know you decide how to advance
the spark you decide what the throttle
is you step on the pedals
yeah yeah yeah yeah it just it's just
fascinating they're a lot easier to work
on
because it's hard you can see the
computer chip looks just fine but you
know
somehow there's a short in it or
something and and it's easier to debug
an old car but um
anyway the uh what we were talking about
the uh
telerobotics yeah human human machines
and there was
something specifically we were dealing
with um
well so uh who are some of the people
you brought in for the telerobotics
project
okay well we uh well let's see we have
been a paul schanker who
actually is not here anymore um brian
wilcox we uh
you know i hired brian in probably about
1982
uh and it's uh you know i thought he was
very
very capable which he is of course but
he i he's he happened to read an article
i had written about science and cosines
in an old robotics journal that no
longer exists
one of the guys originally at jpl alan
thompson
uh started when i first came started
this magazine i think called
robotics today or something anyway um
so we wrote an article about the fast
way to calculate signs and coal signs
and indicate into a table or index into
a table
and brian happened to read that and he
sent me a letter saying you know i read
your article on such and such he says i
thought you might
be interested in my resume and he you
know i could he had
started a little robotics business which
failed uh
when the first uh apple twos came out
because suddenly there was a real thing
that you could buy
but so he had experience you know
dealing with the business aspects of
staffordshire thousands but he
had built he built a little rover a
little not a rover but a little
uh robot that could crawl along the
bottom of a river
and and sift for gold so the idea is
while you
you know powered by the by the by the
water and
it would try to get good than he's after
a few days he'd go
the idea was he would build it and then
after a few days he'd go back and see if
there was any gold there
i thought now this is a this is a
creative guy that we gotta you know we
have to
have here and he when i went to caltech
to go to um
the grad school he became the supervisor
i said you know you really ought to
he said i want to do research and i said
yeah but you know that if there's not a
good supervisor
a good entrepreneur a person with ideas
and it doesn't go anyplace so he became
a supervisor for years he's
no longer supervisor of the group but um
he's continued to how many people i told
you about
athlete and a lot of things and he had a
central role in building the sojourner
rover
rover as well um
okay what else the i still haven't
remember what we were talking about
before your tape right now but maybe
you're talking about the local
you mentioned kind of local autonomy and
that
um a lot of stuff that was going on
was um not so much
autonomous robots per se but um for
example the mers were also
yeah well
machine control and oh i know okay now
okay thank you
um so when we first the first mers
at first you know you'd say okay i see
this rock we want to go to the rock
and so every day you uplink some command
you you
interrogate the robot and then you
uplink some commands well
at first to get over to the rock
and they had to kind of drive it and
figure okay now go over here and then go
here and so it would take
like three days to get the rover to a
point
where it could actually use the the rock
abrasion tool the rat
to abrade the surface so they could look
at the
beneath the skin of the rock but we knew
you know we knew that okay once you've
defined where the rock is and you know
clearly you're not going to drive over
there unless you can get there i mean
the human operators are kind of saying
yeah that's okay
but toward the end of the mission then
we we uplink software that would allow
us to say okay go take a sample of that
rock
and then the rover itself can go over
estimate the local surface normals
do the correct positioning and do that
operation itself and it saves a lot of
time
you can really improve mission
throughput just by automating little
stuff as you know like
you know you could put gas in your car
or something and you'd go up there it
puts gas in your car and it takes you a
few minutes rather than a
long time to uh to gas your crop so that
that kind of thing we actually have
have implemented and we're trying to
implement things like
if a rock looks interesting
maybe pick that rock up or decide
based on local information that you have
using your spectrometers you know we
have all kinds of instruments we put on
on the on the rovers as well and we're
we're continually
miniaturizing them so they get less and
less massive and
more and more capable but so you can
give the robot
given high level commands you can give a
robot an ability to do a lot of stuff
autonomously
it doesn't have to be brilliant it just
you know it's like my boss said to go do
so and so
it's a little bit like the um in 1803
when the uh who was that the people went
up to
explore the louisiana territory uh and
they
nobody could talk to them they said go
up and find out you know if you can you
find a northwest passage is what's there
what did we buy
uh you know document the flora and fauna
and um and come back and uh
so we're we're moving in that direction
and over a long period of time it will
happen partly because you know the
deeper we go into space
uh the longer it takes to close a loop
it's one thing if you were on any matter
of fact this is an interesting story
about brian
when he was 12 years old his father was
working at
at the general motors uh facility up in
santa barbara goleta i think it is and
they had built a rover
and which was supposed to go to the moon
and the idea there was that's only like
a second or two delay so you know you
could actually
almost control a rover from the ground
uh because you don't have to wait for 20
minutes or 40 minutes depending where
you are
um so he uh but he actually drove one
he they had an open house he said he but
he would it would have an image and then
it would go
dark and have an image and go dark he
said he actually drove it into a ditch
because he wasn't getting enough
feedback but
the funny thing was that uh when i was
the supervisor and
brian was in the group we had a trailer
parked out behind the
building 198 which is where our robotics
lab was
and the had a bunch of old junk in it
and
so uh the the division where division 34
was as a controls division but the uh
the
section administer the division
administrator desperately wanted to get
that trailer out of there and all that
junk gone
because he wanted to uh
okay that's not um that was my daughter
i needed to take her home sometime
um but uh he wanted a parking space
and he told me says but if you get rid
of that trader i'll let you have the
parking place in those days
at jpl you could have a space with your
name on it which is you know cool
but anyway so uh i said i wonder what's
in that trailer so i i said hey brian
let's go over and look in the trailer
and see what's in there and the first
thing we see is it's a rover
he says god he says that he's just
because gm had built it for jpl he said
you know i think that's a rover that we
that i drove when i was a kid and so i
said wait you know
we can't get rid of that we've got to
save that so we
we we took it out we put it in the lab
we talked to the program office they uh
they bought some new wheels actually off
of a honda atv
uh we put some new wheels on it and
that's what brian
did a lot of work with he painted it
blue and he did some stuff out in the
royal
and this wasn't the autonomous demo one
but he did a lot of
of kind of human machine interface type
of type of control work with that
but i thought it was really really
hilarious that his uh
his old rover returned to haunt him but
it was very good because we
at that time nasa wasn't funding rover
work
and i remember the one of the nasa
sponsors that
uh mel montemerlo accused us at one of
the triwick meetings he said
are you jp or you just want to build a
rover i thought
of course you want to build a rover
what's wrong with that you know rovers
ruffles will really benefit science and
and benefit nasa and so
but anyway that's a kind of funny little
story about brian and
and things that kind of return to haunt
you later
why wouldn't weren't they building
rovers at the time well because
they they thought they thought they were
going to do the 84 rover
but it was much too expensive you know
these flagship missions
you know i mean in recent years of
course nasa has started uh
you know we end up competing for
everything and they put these cost caps
on missions and it's just
you know the idea of you know hey let's
have five billion dollar mission those
are just
they aren't you know the united states
were having a lot of economic difficulty
and so you know you start you know what
can you really afford to buy
and uh so they but at any rate it was
going to be a very expensive mission
and we didn't really have the technology
in place to do it at that time
now the sojourner was a much cheaper
mission again it was a flight experiment
which meant that
the the flight certification wasn't
nearly as rigorous
and because the idea okay they thought
it could you know if it lasted for 30
days or
a few days it was going to be fine and
it did last a lot longer than that it
really captured the public's imagination
but yeah it ends up being a matter of
funding and a matter
you know how does nasa decide what it's
going to do now we have
every one of the main nasa directors in
earth science
planetary science and astrophysics they
have decadal studies
uh and they've the
powerful wheelbase scientific community
weighs in on what is it relevant to do
and uh you know if if there's a if we
discovered
something that looked like life on mars
there'd probably be a lot of funding to
do mars stuff
um if uh if there were if we discovered
some new thing about dark energy i mean
we of course
want to fly some missions that relate to
dark energy and that kind of thing but
if we found some really interesting work
really interesting uh scientific tidbits
and probably be more funny
well for example the you know the allen
hills meteorite was found in antarctica
which i do believe was a martian
meteorite uh
they remember dan goldins got on you
know
a press conference said you know we
think that these might be fossils of
life
well that really influenced the later
funding because then people were curious
and say okay that's you know that's
interesting to do
uh some of the astrophysics things uh
people think well the universe has been
here
if we understand it correctly in at
least 13 million years
and maybe the universe will be here in a
couple hundred and maybe
some astrophysics missions aren't the
most important things to do
but if you have a strong enough
constituency that can talk to their
congressman and the
nsf and this and that then you you it
ends up being very political and i don't
mean that in a negative way that's what
humans do
you know how do we decide what we're
going to do we all get together we talk
about it and
whoever's got clout has the loudest
voice and
and uh somebody senator has uh is
a chair of a certain committee then
maybe they win
but that's all that's uh you know the
kind of kind of way that politics works
but
but nowadays it's uh you know the the
country can't afford quite as much as it
did
and so they're very uh try to make sure
that we're doing things that are
relevant and now
at least you know some people
notwithstanding who think that climate
warming is a big
humbug uh there are enough people saying
you know there's some
things going on that maybe we got to
know about and uh so
earth science seems to be uh on the
ascendancy now but that's
you know if you gotta talk about not
being able to raise food versus
did you have black energy or dark energy
maybe uh
maybe being informed about your decision
so you
you can be reasonable about climate
change and that kind of thing is a
useful thing to do how has the
relationship of the robotics group with
the scientists
that are also part of the missions well
let me say i i think it's actually very
good the i know that at first
before we flew sojourner a lot of it
i won't mention names but some of the
people who
sit there on those panels now and talk
about the stuff the rover rover picked
up
you know we're just kind of opposed to
it because well a was unproven
technology and they knew that they could
put instruments in that mass that might
make
makes for some good science and of
course be very frustrating if there was
this really interesting rock that you
couldn't quite reach
um but i my feeling has been that the
relationship is very good there are
people and i also work in division 32
i'm the division technologist in
both this division the instruments
division and the science division
yeah but you know of course you do
science with instruments
but there are people like geologists uh
bob anderson for example is a geologist
uh
in 32 but he works very very closely
with the myrrh people he worked very
closely with the phoenix people
and he said he was one of the people
that discovered
that you know you can pick up sand and
you think you're going to put it in the
instrument but it doesn't always go in
the hole
so you know he learned a lot about how
do you how do you actually get
samples and get them into the
instruments so you can
you can make the measurements that you
want to make and uh
you know as an institution we've learned
a lot about that too but
my feeling is that planetary scientists
have worked quite well with the uh
technologists and as a matter of fact
people in general in division 38 we have
people who are really expert at building
superconducting detectors
and they work so closely with the people
in division 32 who will do the
astrophysics
that sometimes you can't tell what
division or groups are in because they
you have to have the really
sophisticated super conducting detectors
to be able to make measure things like a
cosmic microwave background where you're
talking about
differences in temperature of 10 to the
minus 5 degrees
and so they they watch out for each
other and they
they work very very profitably and
productively together
so what were some of the biggest
technical challenges
or technical accomplishments along the
way of developing these
these various rover systems well let me
let me make you just remind me of
something that i
think was it dawned on me when i was
still at bendix
you know i was really pleased that we
were able to build a robot that you
could
if you could throw something out one at
a time on a table
it could look at it decided it was right
side up or upside down and
what orientation it was and then we
could the robot could go pick it up and
then put it down where it was supposed
to
and we could even you know we had some
force sensors and you could make little
tests um
uh i was very proud of we built a whole
bicycle break
and uh but yeah i was using the you know
there's a part really there because
humans when you're doing an operation
say like if you say pick up the pin
okay there's no pin there
or that's not a pin that's a chopstick
or you know people would be able to do
that
but rovers say you know if you say pick
up a pen and it doesn't
just pick up nothing and i've watched
rovers self are not rovers but robots
self-destruct because
in one case doing a demo the role was
supposed to pick up a washer with a hole
in it
and then it's supposed to put the hole
put the washer over over the axle of
this bike brake
okay but the thing is the the but the
washer falls out of the hand so then if
the hand is is
force controlled it closes so then when
the rover goes and when the robot tries
to stuff it down on the axle
it destroys the hand because there's no
hole there okay
so the thing is you start you start
really appreciating the kind of stuff
that human beings can do
but at any rate the um we so we built
this robot and it is
provided everything was in reasonable
shape or you could program it so you can
test but that of course really slows
things down
and human beings just look and say ah
it's not there they reach for it
but a robot you know you end up going
over and nudging and sometimes like
like having thick gloves on and trying
to decide what you're gonna do
so that slows it down and then people
don't want to buy it because it's too
slow
um but the uh so we had a guy we're
admitting we're trying to we wouldn't be
able to make this system that would
do assembly and so you know i could do
all these nice clean coordinate
transforms yeah i could do that
robot was really very good at doing that
but we hired a guy who was supposed to
the big problem is let's suppose you
have a bunch of springs in a
in a bin you're going to pick a spring
out
you know a human being you know metaphor
springs are pretty hard because they get
wound up but
you imagine you had to pick up nuts and
bolts or something you know they reach
in there you grab some and you
and you put them on but uh having
a non-human try to get nuts and bolts
out one at a time
especially if they tangle up is a very
very hard problem and so this
this fellow his name is uh bob molique
and i used to call him rube because a
lot of the stuff he tried to come up
with was kind of
rube goldberg-ish and he never liked it
i was just teasing him but
but he you know he's a very smart
creative guy too but i finally realized
one day that the hard problem
isn't dealing with an object that's out
there
pristine located on a plane the real
problem is you got a bunch of junk
and you gotta figure out what to do with
it and deal with it one at a time
and so you know okay we've been solving
the easy problem the hard problem is
getting those doggone parts out
one at a time and uh but then you know
if you were
in space for example let's say when the
russians did this they ran their
their their their vehicle into the space
station into mer
right and it messed it up well things no
longer look like they're
they're they're they're graphics models
and so they say well if this is all
smashed you know human beings say oh
yeah because it looks clear and it's
going to see these little corners and
stuff yeah you smash that that's what it
is though it's a smashed
it's a smashed cassette case but you
know
robot isn't necessarily going to know
that because it doesn't have all the
capabilities
so one of the big problems is dealing
with uncertainty
and uh i think that's still
matter of fact the problem of like
trying to drive to palo alto
uh on the freeway with under computer
control that's a
problem where there's a lot of stuff
going on a lot of things that
you know human being can't do that until
you're like what's 16 years old maybe if
you're really good you're 12 you can
drive the tractor or something
but handling those kinds of problems is
is in the dexterity those are just
incredibly difficult uh one thing i did
work on
it was ken salisbury who you may have
really he and i
built a multi-fingered hand he actually
designed it and built it as his
as his phd dissertation but um
i remember we were we thought hey we
should build a multi-fingered hand and i
went up to talk to tom benford and
at stanford and i said did somebody can
work on this and they said oh yeah
so ken salisbury and so that's became
his career but uh the idea there was you
know could you actually program
something to do dexterous
things and it turns out it's very
difficult it's you know you can
kind of pre-program it but the kind of
really
effortless dexterity or even a squirrel
you know i feed scrolls in my yard at
home they pick them pick up peanuts and
then they knob and they throw them down
and but they even them and they don't
have a great deal of cognitive
capability but they know enough
that it's a peanut and they can eat it
and they can peel it and they can get
rid of the shells and
those kinds of problems dealing with the
uncertainty and dealing with situations
that are not
really well defined those are the big
challenges
uh you know trying to you know like i i
wonder myself how is it that i know
things
like i was something i hadn't thought
about for years i was
i occurred to me and i thought about i
right away pick up the names and this
and that
and i'm saying how is it that that that
you even in your brain you
format you formulate a query
and you aren't even you don't even know
the name but maybe no yeah that was the
thing that was that weird looking thing
that we saw down at the museum in san
diego
and what was oh yeah that was the sun
you know how does that work
and and we really don't understand that
i mean just
a matter of fact if you if you talk to a
neuroscientist
they will admit that we mostly don't
understand the brain at all we know a
lot about
the the chemistry and the physiology of
neurons
and how you get signals but how you you
go from a neuron
to maybe your spinal cord neurons talk
to maybe 200 000 other neurons
and how how information gets gated
and represented and how you learn you
learn to walk
okay and after you learn to walk you
kind of you know you don't think about
it if the floor is smooth it's kind of
flat floor
yeah you talk to your buddies and you
walk down the hall if somebody has left
a banana peel there you might slip
and then suddenly you have an interrupt
and you recover with reflexes but then
after you've decided the world is
organized again
you start walking if you're walking down
even stairs it's straightforward if
you're
trying to go across a creek and not get
your feet wet and there are rocks there
you don't sit there and solve physics
problems while you're doing that you
might solve physics problems while
you're walking down the hall but
if you're going to get across a wet
creek and not get your feet wet
one thing you got to test is this rock
stable there's all kinds of stuff that
you do
and it typically interrupts
conversations
because it requires a great deal of
thinking and an organizational skill to
make that happen but
yeah i think uh you know i used to think
that well yeah we'll be able to
make these smart machines and
they get smarter and smarter but they
aren't very smart yet
and i think again that doesn't mean
they're useless because they do lots of
really
really interesting and useful stuff but
uh the whole kind of you know like how
do you know
you know if somebody touches you with
this you say oh yeah that's the
yes that's a i know i expect this to be
a pin or a
point of a pencil but you could also
take a scorpion and you know because
okay then yeah that's it oh god it's a
scorpion
and um you get more and more reverence
about
especially you see see images of of
neural tracts and all that you get more
and more
reference about how it and how does that
even work
and how is it that you that you end up
being conscious now i can imagine
programming a robot
a very sophisticated robot that we can't
build yet but so
it could look like it was conscious
because it says oh yeah that's me
i call that me but does it does it have
the feeling of being conscious
you know you taste coffee you could have
you could put a spectrum you know mass
spec here i ask coffee
and that's arabica beans or something
like that but you know it doesn't taste
it it doesn't have the
the sensation of of taste or that's red
or that's green or whatever those are
those are really really very fundamental
problems that i
but i don't even know if humans are
smart enough to answer them yet
but we may experiment with genetic
engineering and
build some people with really a more
more facile brains that could do it
but uh anyway what about the challenges
of actually
so you mentioned the man-machine
interfaces and so really it's the
interaction between humans who have
all these um capabilities and then
robots who have different kinds of
capabilities but not the same ones as
humans what are
the challenges in that well the
challenge you know the
big a challenge okay here's a good
example
people talk about uh remember years ago
people were trying
to work on implantable gait assist for
people who were
were paraplegic okay so then i thought
well okay
that's suppose you're gonna build a
really smart robot that could you know
say the floor is flat and this and that
but how do you or you know that'd be one
thing maybe you could have an
exoskeleton you control a person so they
can walk
but if you wanted to have something that
actually helped the person
to interact how do you convey the
information and in what form from the
person's nervous system
to the control system of the robot
because if people have you know severe
you know als or some kind of a severe
injury
they're cognitively very capable we just
had a colleague here john klein just
died
last week he you know a year and a half
ago he yeah a year and a half ago he
he said woke up one morning and he
couldn't walk and then they kind of came
and went but it turns he had als
okay but his mind was crisp like you
know stephen hawking mine is crystal
clear
they can hear they can understand but
their motor control capability is just
gone
those diseases uh really degenerate
motor neurons
how how would you get the information
and you know now we can
people put electrodes in people's visual
cortex and you can kind of see
you know the people and they can you can
put these neural implants and allow
people who have cochlear damage
uh to be able to hear to a certain
extent uh you know stimulate the right
kind of nerves but then
how would you know if you wanted to give
a person a little pack that could
stimulate their motor neurons or
you know even directly maybe you can
innervate the muscles with the
artificial neurons but how do you get
the information out in the form that
would make sense
to a machine those are really because we
don't know
you know and matter of fact i don't even
know when i taste coffee does it taste
the same
to you i mean the quality of it you know
i don't like cheese
you know a lot of people like cheese but
so there's a different quality i think
these
i hate it but the but people say oh this
is delicious cheese so you know what
what is that and uh
you know ultimately we can all agree
that this is red but you might be
perceiving blue
or green or or purple or something like
that so i think that those are
those are very fundamental issues uh and
also trying to
you know people want to build robots
that are self-healing actually look
we're at the genetic
level we heal ourselves you cut yourself
and
genetically it knows what it's supposed
to be providing it's not a terrible
injury but
it was a little cut you know fixes
itself but if you if you lose a finger
you don't grow a finger back unless
you're a starfish or something but
but uh you know how do you how would you
make a robot
that could heal itself unless you
started providing
very biological things like biochemicals
and amino acids in the bloodstream and
this and that
and enough information that could grab
them and reformulate
those are those those are very
fundamental issues if you let's say you
wanted to send a
a robotic colony to mars are you going
to send a whole warehouse of parts
or are you going to be able to maybe
dissociate
and fabricate new parts with native
materials
with you know more advancing like 3d
lithography kind of stuff that people
now do
but those are you know i i think that um
or even wiring up a brain
okay i mean that's not i don't mean
there's a trivial task but remember
several years ago we were doing a task
here uh
for darpa and the idea what we were
trying to do is
active vibration suppression so the idea
was we had a we had a
stock we had a focal plane mounted on
the stock and we were exciting it
but we wanted to to look at the
vibrations but we wanted to be able to
actively damp it
to stabilize the focal plane so you can
get a clear image well the guy who
it was a fellow group supervisor in my
section at the time
he was responsible for building this and
so i noticed they were way behind
schedule and every time
and they do some testing and they would
find a problem so they would open up
this
this box to fix the problem and they'd
fix the problem but they'd make a new
problem because they'd break a wire or
something
so i began to think i said you know
didn't you design it so you could open
it up yes no we figured we're going to
build it and then it's you're going to
fly it i said but you have to test it
and
uh so it occurred to me you know being
able to wire something up as subtle
as a nervous system with enough
information content that they can do
something subtle
is a remarkably difficult problem and if
you've ever talked to a brain surgeon
we had a brain surgeon come by here a
couple years ago was interested could
they use
robots to do brain surgery but it turns
out that you know
your brain is not designed to be worked
on
and if you have to breathe and this is
actually kind of chilling in a way
but he said you know if you have to
operate on somebody's
inner brain you basically have to kick
their head all apart
you know unless you could you know if it
was cancer or something you might be
able to zap it with very directional
radiation but if you've got to get in
there and try to fix something like an
aneurysm or something like that
you've actually you take people's faces
apart and their skull apart and
they're all unfolded and you get in
there and you do the operation they put
it all back together again
and so the idea of being able to wire up
something
with the kind of tools that we have
today for doing micro devices even
even really sophisticated computer chips
uh it's just really a challenge
and so i think that you know being able
to make really capable robots that have
enough
nervous uh enough sensory capab
excuse me capability to make very
informed and subtle decisions is going
to be a long-term challenge
but now the kind of thing but you know
one thing that robots can do let's say
you want to reach into a
into a furnace and you want to pick up a
piece of hot iron
and because people do that you're going
to put it in a forge and you're going to
stamp it well you know robots don't hurt
of course people don't either if you
wear iron gloves with enough cooling you
don't hurt either but uh
so those are issues you know robots do
very useful stuff welders
you know your car if you buy a car today
is welded together and they have big
welding machines that
humans had a hard time coping with but
they're very consistent you know your
cars are pretty
pretty pretty consistently welded
together unless there's a maybe a glitch
of some sort
but much more consistently than in the
old days when people had welding guns on
these spring
uh spring supports and had to move them
manually to uh
to weld your car together so i think
we'll see
uh and a matter of fact you look at in
the military now
[Music]
a fighter pilot you know if you're going
to see offensive
uh offensive uh planes of miles away
it's radar and all kinds of displays
that try to
assimilate and distill what the operator
is
going to see and make it meaningful for
the operator so then they can
they can make a a high-level decision
and they can do you know you see it in
movies all the time lock on okay you're
locked on okay once you're locked on you
can track
and then you fire but trying to find it
in the first place
can really be really be a challenge and
so i think that's so we'll see more and
more of that where people
do more and more high level things and
robots do more and more
but then you know you look at the way
the world economy is that people in
spain are rioting because there's no
jobs
we historically said well we'll use
robots to free people to do more
productive stuff
and that's great if the people are
really capable
but you're not going to take every
person and teach them general relativity
i mean you just there are very few
people in the world that can really
comprehend those very esoteric things
and so basically what you do is you end
up with people that don't have
much to do that's within their
capability of doing it and i think
that's going to end up being a
significant problem we we aren't
organized socially yet to
let people just live without working
because you say i'll go get a job
you know so maybe you need robots to
make lots of stuff for people but then
you got the problem
of the uh the carrying capacity of the
earth
and uh yeah so i just sometimes think my
goodness
you always think you're going to solve
these problems but there's always going
to be new ones
and sometimes it's going to be a billion
years from now and there will probably
be some kind of
sentient life and you wonder what kind
of issues they will deal with
what would you say is the future of
robotics in space exploration
oh that we'll do more and more of it
for one thing let's say you wanted to uh
you know people often talk about doing
an interstellar mission
well one thing you're going to have to
fly really really fast to even make it
uh
even make it possible to get there in a
reasonable amount of time and you're
going to have to
use use
ion propulsion that kind of thing and go
faster and faster but
you aren't going to send people there i
believe now maybe sometime
you know maybe we will discover some new
physics and we you know in 1899 the
guy named buell was head of the patent
office and he said we should just close
the patent office because everything's
been invented that can be invented
and you know that certainly hasn't been
true
as a matter of fact we invent more and
more stuff and uh you think about your
your disk drive okay the
the disk drive on your computer now you
have 80 gigs 100 gigs 200 gigs
in a tiny little package and that was
because
some people fiddled around in the lab
and they discovered giganto magneto
resistance
which suddenly really shrank i remember
when i when i
had my first computer with a disk drive
but you had 250k
and so now but we we discover new things
all the time maybe we'll discover some
some interesting physics that will allow
us to the wormholes or
you know this stuff all sounds like
science fiction now and knows what a
wormhole really is
it doesn't really go other places but
you know we we've discovered lots of
interesting physics that people didn't
know in the middle ages
and if you had a cell phone with or you
know you take your cell phone with
images on it
and if you were by yourself they'd
probably burn you as a witch in uh
you know in 1600 or 1500 because
it would be stuff that somehow wasn't
mentioned um
but we will do more and more robotics
it'll they'll be more and more
capable um but we'll for a long time we
will need
human capabilities to uh
to govern the overall uh task
uh you know maybe you what you really
want you don't want something that's
totally autonomous
because totally autonomous means it says
heck with you i'm going to do what i
want
you want a very loyal agent that says
yeah boss
uh this is what we'll do and and we'll
do the best we can to
uh to meet your uh meet your needs
so um if we could just go back to uh
some of the other people that you've
that you've worked with over the years
okay well i've touched on already
well let's see well there's okay at mit
uh there was dan whitney and jim nevins
they had a
multi-client study uh as a matter of
fact we gave them
our first the the engineering model
robot we built at bendix that we did a
lot of our exploratory work
uh we ended up giving that to them uh
when bendix when we just we built some
other some more
uh more product oriented robots and so
we gave
them the uh our first one the uh
then when i actually i actually when i
left bendix i went to as i went
i mentioned i went to ge but one reason
i went to ge's because i wanted to go to
rpi
because i they had a professor um i had
a professor there who had been
uh i wanted to see hillary nor herbert
but he uh he had been working on puzzles
and stuff like that he was in so i
talked him into being my thesis advisor
um so then i bendix was getting out of
the robotics business so i
called them and said hey why don't you
donate your robot
rpi so they they got a robot
but then i and i enrolled as in the
computer science curriculum at rpi but
then i
you know left to come here and uh
i always thought i would go back to ucla
and finish my phd in physics but uh
that just never happened but i get to do
a lot of physics and kind of stuff in
the science division now so it's uh it's
interesting
um other people it's okay though nevins
and whitney um
then at stanford there was you know tom
benford and and bernie roth and that
at sri there were
charlie rosen and david nissan they had
a multi-client study going
um and those are you know principal
people and they were always uh well okay
there were other people
matt mason was a grad student at mit
when when i was
uh when matter of fact i think he was a
grad student when we first went
uh we made a presentation of this pax
system
because my my boss at the time wanted to
get out and find out what people were
doing in robotics because we were pretty
isolated there in dayton
and i remember that we showed a movie of
the programmable automation cybernation
system
and all the grad students laughed when
we said that
that's what made me think that's kind of
a highfalutin sounding sounding thing to
say but
i went to mass lozano perez and
i think he's still at mit i haven't seen
him for a while regina baitsy
she was she was running the grasp lab at
uh
university of pennsylvania and then she
left and she's been nsf
she's in berkeley at berkeley uh but
then but she got a she got a phd
she already had a phd but she got a phd
from stanford in computer vision
uh about the time that you know back in
the early
early mid 70s yeah because when i was
being active in
in those multi-client studies and things
so then there were some people that
people that i didn't work that closely
with within nasa each
each nasa center it seems started a
robotics group
of some sort because it all suddenly it
got to be
when the when the space station started
to be real
and they started to fund this um
this uh teller robot uh project that we
had here
uh the nasa centers all started and
matter of fact jsc
changed their name from the man space
flight center because they and they
in one year started several ai and
robotics groups
and you've probably talked to them they
have their uh
what are they called robonaut and uh and
things like that but
um it's interesting i think that a lot
of times you know one fellow here at jpl
built a multi-fingered robot
bruno yao and the funny thing is that
even though it had a lot of fingers and
stuff
it's still when you wanted to pick
something and they couldn't do this kind
of stuff
you know it hold something they would
slip and then it didn't know how it was
slipped and they couldn't tell
that i was actually holding holding the
object so uh but yeah that's been
you know it's been a live course i've
been doing robotics for almost 40 years
now that's kind of stunning
and now i have grandchildren and all
that stuff
it just kind of time passes and
you realize that you don't have forever
to do things
what's your recommendation for young
people who are interested in robotics
well it's interesting you have to decide
what you like to do because robotics
robotics is one of those system level
activities
uh you know it's in it if you do control
theory
people can talk really precisely
formulated problems in control theory
if you're doing detectors you can do
very precisely formulated problems in
solid state physics and
that kind of thing as a matter of fact
it's really easy to say hey look
everybody's impressed here's that this
has got a trillion transistors on it
looks like a piece of silicon to me um
but robotics has got all that stuff it's
got you know it's got
the whole computer organization uh how
do you
how do you integrate sensory information
with the overall control of your task
a matter of fact how do you even
remember what you did robots don't
remember what they did
you know it's almost like they have a
hippocampal lesion and so they can't
remember what they just did
because you know a human a human
explorer would
you walk through and say oh yeah i've
been here before yeah i remember that
little red rock
uh when i remember that we weren't
successful when we tried to
try to use that last that last
drill drill bit because it had a little
crack in it you know humans say remember
all those details
and they put them together and they
still managed to get the job done a
human being you know if you tied
somebody up
and and they needed to get out of
something they might even like if they
could grab something with their jaw they
might even grab and
pull themselves up by you know just the
force of their jaw and using their neck
so the human beings are
very versatile at solving those kinds of
things so you need to um
and if you try to solve an animal
robotic problem in all of its
complexity there is too big so if you're
interested
in in looking at overall robotic
projects
then i think that you should you should
learn take some control and you need to
take some signal processing and you
certainly need to take software
you need to know about detectors and
sensors and things like that
if you want to get down to the details
of
controlling an arm then you need to have
a
good a good basis in control theory
itself
uh because you know sometimes you you
know you have the idea that
that you can uh control a mechanism
but if you don't uh oh you know just let
me call my daughter
faster okay just because i'm just i
didn't realize it was so late yeah
yeah well no if that's gonna this is fun
uh but anyway i think that
that uh you know if you're really
interested in the subtlety of
of neural reasoning and stuff like that
then you need to
to look at uh you know neurobiology
but at you know there's so much stuff it
just just really really gets complicated
but
there's a lot of things to do and it's
it's a it's a feel that we'll be growing
being able to make a smarter household
assistance or hospital assistance things
like that
and it'll be interesting to see what
things are like in
in 50 years compared to now and maybe
we'll have
you know robotic biological hybrids
or maybe we'll use biological ideas to
be able to build the circuits i mean
you know your brain gets built from
amino acids and things like that over a
long period of time but
you know maybe maybe we'll do things
like that but then of course
your you know if you don't feed your
robot it'll die you can't just turn it
off and put it in a drawer
well they have the robots that eat
various organic materials now
so yeah no it's just it's it's it's a
fascinating problem
so okay well it's been a bit interesting
talking to you that was our final
so so you got a fair amount of
robotic philosophy in this net thank you
people always
kind of have thought of me as a robotic
philosopher i remember
vic shineman once uh it was a funny
story he he was here
we had another group it was actually in
the same group but uh
tony batesy used to be here and he was
doing he was very much interested in
tele operators
and uh so he he but he had vic doing
some consulting work with him because
tony built a big
uh wrist sensor for the shuttle manager
he actually went down to texas and was
using it to
show that he could minimize overloading
uh you know orbital modules and things
like that
but uh i remember that one day i was
walking i was i was
taking somebody on a tour of the
teleoperator lab and i
mentioned something about vic shineman
and vic walked in
and i said what are you doing here and
he was consulting with tony but
i remember once he introduced me to
somebody said he said well yeah this is
he's really a robotic philosopher
but these are fascinating these are
really fascinating issues the kinds of
things that
being able to start with inanimate
matter
and program it's so that it can actually
do things
and respond in a reasonably effective
way when you look at
mark rayburt's uh you know big dog his
walking machine
and you know you go kick it it it
invokes reflexes and tries to
tries to stay upright and it uh so it's
got it's almost like a headless horseman
it doesn't doing a lot of
planning and high level reasoning but it
does a good job of climbing
and walking on on things that are uh
that are difficult as a matter of fact i
you know i should have mentioned that it
was mark rayburt who actually kept
calling me up
when i met him at a conference and he
kept calling me up and saying why didn't
i come to work at jpl
and uh so i i neglected to mention that
but uh
mark was here for uh when i first
arrived and then he
he decided to leave and go to cmu and
then after that he went to mit and then
i guess he's still in boston dynamics
because he
maybe got tired of academia i don't know
but he
he was always a wanted in a successful
mover and shaker
and stuff so well good talking to you i
probably should go pick up my daughter