so if you could tell us when and where
you were born
i was born in rahway new jersey on may
22nd
1959. and where did you go to school
um up until the time i went to college i
was in seven different schools my family
moved a lot
so mainly in the north the eastern
seaboard
and then after that i went to duke
university as undergraduate
where i majored in chemistry and
mechanical engineering and then i went
to stanford for grad school where i had
a master's in phd in mechanical
engineering and then i came straight to
caltech so i've never had a real job
in my life so how did you get interested
in mechanical engineering
um that's a kind of interesting story
because when i started as an
undergraduate i actually started as a
chemistry major i thought i was going to
cure cancer
and that's as i went along i decided to
take
for fun one engineering course and
really liked it so i ended up then
finishing my undergraduate work
primarily in mechanical engineering
um and uh
why mechanical versus electrical
something else i'm not sure
but i you know always like to build
things and so uh the mechanical part and
treat me more than not to go apart
what are some of the early things you
built oh
um so in my graduate study days
especially the first half the graduate
study days i worked with a guy named
usama khatib
when he was a postdoc at stanford and
now he's a professor there
and so i worked on robotic force control
with him so there we didn't build so
much as we sort of used existing robot
technology
and worked on algorithms to improve the
ability of robots to do what's called
force control to be able to
monitor and control the forces the robot
applies to the environment
and the latter part of my graduate
career was primary theoretical
but once i got to caltech that's when i
really started building things
and so one of the earliest things i did
here with my first graduate student
uh great trichia news now at johns
hopkins university
greg and i were interested in these
hyper-redundant or highly redundant
robots or snake-like robots
and so in addition to working on the
algorithms or theory
and i'm probably more of an algorithm
person than i am a mechanical engineer
per se
we we built a variety of snake-like
robots so the first
kind of full-scale construction project
i did in robotics was actually uh
how many degrees of freedom 30 degree of
freedom snake-like robot
and how did you become interested in
those particular types
well in the latter half of my um phd
career i was working on a class of
robots that were hot then called these
redundant robots
so these are robots with more than six
joints in their arms
and quite frankly they didn't pan out as
well as we'd hoped at the time
but it was a fairly natural extension to
think about these hyper redundant as we
call them which
means lots of redundancy and so i
already had some exposure to that
and uh fortunately my first graduate
student greg when he arrived here for
graduate school he was interested in the
topic also he thought some about it and
so it was a
kind of a you know natural marriage i
was interested he was interested so he
said let's dive in
uh and just do it so what were the hopes
for redundant robots
well in those early days the thought was
by adding these extra degrees of freedom
that you can make the robots much more
agile and able to do more things and
while that's true
it also means that the robots are very
expensive and more difficult to design
and build because they have that much
more complexity
to them and what were some of the other
robots that you worked with when you
were in stanford
um like i said stanford primarily was
just with robot
arms and these robot manipulators and
doing this force control work
occasionally you know because the lab i
was in uh was one of the first labs in
the field of robotics had a long noble
history
certainly we were exposed to lots of
different robots that were there at that
time in the field
including some mobile robots and
actually now that you mentioned it i
forgot the first
jpl stanford hand
built by ken salisbury i actually helped
build some of the electronic interfaces
for that
and so that could actually be used uh
you know in the early days
so you worked with ken he was just
finishing his phd when i was a first
year graduate student i can't say i
worked with him closely
because he was you know frantically
trying to write his thesis and get out
but you know the arm that hand was there
for a while and so i helped
uh people in particular ron fearing
who's now
at berkeley um was a junior graduate
student like i was and he sort of
inherited that mechanical system
and so uh while it was ron's main work
certainly you know we had discussions
and i said occasionally help with
interfacing and doing a few other things
but he did most of the work and who are
some of the other people you worked with
there
oh um so people who uh
went on and still work in the field of
robotics uh or or related things so that
group was both a robotics group and a
computer vision group
so there are lots of computer vision
people also so
uh not in necessary order of importance
uh dave kriegman
who went on to yale and then illinois
and now uc san diego
was a co-graduate student at the same
time ron fearing as i mentioned
brian armstrong was my office mate and
actually i'm still in contact with brian
uh these days uh he's up in wisconsin
now a professor of electrical
engineering
um jean ponce was a postdoc back then
he went on to be a professor at illinois
for many years in the computer vision
area
he's now he's french by birth and so he
went back to france
uh to one of the top schools there as a
professor
um you know some other notables in the
field of robotics um
he was certainly earlier than me but
after he went and postdoc
at mit rod brooks came back to stanford
for a while
and so i knew rod a little bit then um
root this is a name not in the robotics
field but it's kind of funny bruno
ulsthausen was a master student then
he went on to become a very prominent
sort of biological vision researcher
he's now at uc berkeley um so that's one
of the things which was uh
exciting at that time is you were in one
of the kind of few labs that had a long
history of robotics
and you knew that the other students
around you you know were very talented
and were likely to go on and have
impacts in the field
so oh i completely forgot so i said i
didn't know you're going to ask this
question ahead of time so i'm thinking
here
john craig who was important in the
early field i knew him quite well
at the time and and for
many years after a grad school maintain
you know casual contact with him
i've lost contact in the last or ten
years or so
as someone who didn't necessarily play a
big role in academia but was quite
active in the robotics field for many
years was jeff
kerr so we were sort contemporaries and
we had the same phd advisor
um another student then uh was sunil
agrawal
who is now professor at university of
delaware um he wasn't it was kind of a
funny story he wasn't working in the lab
with me day to day but he had the same
phd advisor so we ran across
each other and i just saw him last week
as a matter of fact so we're still
friends and colleagues
and probably during the interview more
names will pop up i hope you don't mind
if i just free associate
uh when things come to mind at all
that's actually because i'm sure i'm
missing some you know really important
oh just some other ones because
because of the dual nature of a lab
there is a whole vision side that i
wasn't much involved in but uh should
tender malek
who went on to be very prominent
professor at berkeley who was even
chair of the berkeley cs department for
a while came out of the group uh
just a few years before i left as well
who was your advisor
that was a complicated story so when i
went to stanford i went to do fluid
mechanics
because i was a mechanical engineer and
a chemistry major and then i decided
that was just going to be very boring
so i just happened to take a control
course in my first year liked it
and then tom minford's lab who was the
at that time directing the kind of
lab which was had the long history of
stanford robotics
um and so he was my first advisor for a
while
but then fairly late in the game i
eventually switched to a guy named
bernie roth
um who was my final phd advisor so i
really had two advisors
in that period tom a more cs
vision person and then bernie roth a
real mechanical engineer
to work with mentors can i speak off the
record
yeah of course tom binford was not fun
to work with
um he was a unusual guy
and in fact during the time i was there
because of some
unfortunate things that happened he was
basically demoted by the university
um and uh um
he was a very bright and interesting man
there's no doubt about it and he had
sort of good intuition but he wasn't
much fun to work for
but then now back on the record um
bernie
roth was my second advisor and um in a
large part i'm in academia today because
of bernie
um he sort of showed you know that you
could be
uh you know both a researcher and uh you
know a good human being
while being an academic so he's kind of
this terrific sort of
guru master uh type uh and i can't say
enough
positive wonderful things about brony i
think uh certainly
if you haven't interviewed him you will
but certainly he's
he's mentored a whole generation of
people that have a big impact on the
field and so
um i think it's due to his uh
mentorship and his personality and his
just sort of force of nature
that he had the impact that he had and
what were some of the
you know big discussions or challenges
that people in the lab were very
interested in at this time
oh um yeah
i have to think back um well certainly
it's kind of surprising how many of the
issues you know
uh we were dealing with back then are
still prominent issues today
um i think what happened is in those
early days we
uh could foresee a lot of the important
problems but we didn't necessarily have
all the technology then to really
tackle them so we spent a lot more time
then
worrying about the computational
complexity of algorithms because
computers were so slow if you couldn't
you know get them implemented on the
minimal hardware we had
then they just didn't work those issues
have almost
completely gone away these days
but all the you know kinds of things
there were just different flavors so the
you know computer vision people for
example
studied all the same problems are being
studied in vision today but they just
didn't have the
you know tools that we have today not
only in terms of hardware but in terms
of the principles and theories and
algorithms
as i mentioned i was only tangentially
involved in the sort of you know
grasping
a little bit in fact actually a couple
years after i came here i got involved
in grasping i still work in grasping
today
and so we were thinking about it back
then but it was much more
you know trying to discover what are the
basic principles
uh that should be used formulating some
of the first
mathematical or algorithmic problems and
so
those problems are now well understood
but not only solved today so i think
we're starting to formulate more of the
basic problems back then
and so certainly in the force control
area i wasn't one of the first people
with usama to work in force control but
we were pretty
early in the game and so you know we
were able to at least i hope
um you know elucidate some of the key
problems and a couple of solutions which
people of course built upon or advanced
or come up with new ideas since then
and how was osama to work with assam was
a lot of fun it was a kind of a funny
story then
he was a post-doc he's a syria syrian i
believe by birth
and so then he went and did his graduate
work in france and then came to the u.s
and so
uh by the time he come to the u.s it had
been a long time since he spoke arabic
his french was pretty good but wasn't a
native language english wasn't his
native language and so
we used to actually in the early days
converse by pascal we would send each
other pascal programs
and that's actually how we we did a lot
of our early conversations but then his
english advanced
very rapidly uh and he's a incredibly
sweet uh
and gentle guy and so i learned a couple
of big things it from him so the one
thing i learned from him that i still
pass on to my graduate students today
is if you ask the right question then
that's the most important problem
research is asking that good question
because once you can formulate the
question properly then you can do the
hard work and roll up your sleeves
and find the solution to it so i said he
was a very very gentle and sweet guy and
so that
made working with him you know very
pleasant
uh what weir's were you actually at
stanford
82 to 88 or to early 88.
what's that that's what it says online
right
and uh what did you do after you
graduated so i came directly here
uh and so um my graduate career was a
little bit longer than average and
longer than i hoped because i switched
advisors fairly late in the game and so
the downside of that is i spent more
time at graduate school the upside was
that i had a lot more experience than
the average
graduate student and that's in some
sense helped me land my position here
um i had more you know breadth of skills
which you know made me attractive and
there's actually a kind of a funny story
how i got my job here
so uh caltech was quite interested in
hiring john craig that we'd mentioned
before
and uh john at that point had also had
an offer from berkeley
and uh um was also um working with
one of the companies silma that he
helped found so he was trying to decide
between academia and soma but he knew he
wasn't going to take the caltech job
but he said i have this friend who's
graduating soon you should talk to him
and so that's actually what led to me
getting my position here
because they weren't really advertising
that position so i can think
thank and curse john for my job here
what was the engineering department like
and were there any other
robotics at stanford or here right here
no um the mechanical engineering group
here when i first came here
um up until maybe two years before i
arrived here it was
all thermodynamics and fluid mechanics
there was
very little in the way of mechanical
design mechanical systems
control theory the kinds of things which
go into robotics
and so um they hired another faculty
just two years before me eric anderson
who was in the mechanical design area
and then a few years after me richard
murray who just you talked to earlier
today was hired
and so for a while we formed this
mechanical systems group
and it was a lot of fun um richards kind
of drifted off
first into the controlled dynamical
systems and actually i'm now part of
that program also
and then into bioengineering and
actually i'm part of bioengineering
should we sure
yeah and so um we sort of you know shift
the department away from a total focus
on fluid mechanics and heat transfer
although uh kind of in some ways sadly
we're kind of back in that
mode again i'm the only person left in
me that doesn't do some sort of
continuum mechanics
which is now why i'm sort of more
closely affiliated with the control and
dynamical systems program
so i don't know if it's a topic we'll
talk about too much here but just
six miles away is the jet propulsion
laboratory and they have you know 40
some researchers in the field of
robotics and so
i have done off and on with them
collaboration
and in fact right now pretty much all of
my robotics research at the moment is
done in collaboration with jpl uh and so
there's a bigger activity here than you
might imagine um
who are some of the people you
collaborate with on jpl oh a whole bunch
of people over the years
and so i'm going to actually work
backwards so my two active
collaborations right now
um there's a whole bunch of names but
i'll give sort of the senior people
and so the senior people would be larry
matthews m-a-t-t-h-i-e-s he's a computer
vision person
a guy named paul bacchus b-a-c-k-e-s
he's more from the mechanical side and
then a very fun collaboration i have
right now with uh issa issa
nestness n-e-s-n-a-s so we're working on
some cliff climbing robots
in fact we were just out in the arizona
desert last weekend testing these robots
and other people in the earlier days um
a person who's risen up in the ranks
there is um
karl ruff r-u-o-f-f he's been involved
in robotics from the
early days and so if you haven't
interviewed him you should
um and um samad hayati is another person
that i worked with
in my earlier days and he's risen in the
management ranks also
hayati is h-a-y-a-t-i
another name that i'd like to mention um
is homie and saraji
s-e-r-a-j-i i worked quite a bit
with him in the my first couple of years
here and the reason i mentioned it
because he unfortunately died of cancer
a few years back you know kind of
prematurely
which was a big loss but i enjoyed
working with him
and so when you came here since there
wasn't really a focus on robotics but
they obviously wanted to hire a robotics
person i don't like that it was um i
think in large part you can trace that
to two people
one was a car ruff carl was at his
master's degree and i apologize i don't
remember where he got his master's
degree from
but then wanted to come back and do a
phd and so there's a program so that jpl
researchers can do their phds at caltech
in a not too painful way and fred kulick
was a professor here in the
thermosciences area
and propulsion area and so it really was
fred and in some sense working with carl
who
first taught some robotics courses here
just to sort of see what it was all
about
and then really push the department to
kind of hire you know
in the non-flu mechanics area and in
particular in robotics i really
uh owe not only john craig for making
the contact but also
really fred kulick for pushing the
department to think about that
and so what was some of the early work
that you did here in robotics
well as i said um in the the first thing
i got involved in because
my first student was just a terrific
student i i in some sense owe him my
career
it was great tariq and so we worked in
these snake-like robots
and the reason i mention it is because i
was
very motivated to do this work by the
terrific work of
shigeo hirose i don't know if you're
going to interview him he's a huge name
in the field of robotics and justifiably
so
he's just an incredibly inventive man
and
he had done some beautiful mechanical
design work on these hyper-redundant
robots
and so my interest was to try to
contribute to the algorithmic side of
these robots so
to develop a theory of how to plan the
motions and do sensing and do lots of
operations with these redundant robots
and the reason i mentioned that is even
though i dropped that work you know
six or seven years after i got here you
know research goes in
phases the pendulum swings back and
forth and so
it was not an area where i could just
sustain long-term funding
on the other hand the problems that i
worked in in those early days actually
spawned pretty much
most of my research that i'm doing today
in many ways
so with these redundant robots you want
to send them into very unstructured
environments
so these snake-like things we were
thinking about using them to
crawl into collapsed buildings or into
difficult-to-reach places and therefore
you would want to do a lot of
sensing and be able to plan your motions
using sensing
and so uh for many years and still even
to this day
i worked on sensor-based motion planning
for robots
we with greg we kind of as a fun thing
we looked at actually how these
uh snake-like arms could grab stuff and
manipulate them
and that really kind of got me started
on uh grasping which i still do to this
day
uh and then because these snake-like
things were
you know one of the modes of operations
to locomotor move
for a decade or so i had several
students working on various aspects of
robotic locomotion
and kind of indirectly then also one of
the early
applications we had envisioned and we
weren't the only ones uh
for example professor ekuta in japan was
prominent in this area
of thinking of using these stake-like
robots for medical operations
and so i didn't really become a
prominent person in the medical robotics
field but
um i did a little work in the area which
opened me up to starting to work with
neurobiologists about 12 years ago and
so
basically for the last 12 years almost
half my group and now half my group
has been working in different
technologies related to paralysis
and many of the technologies are
motivated by robotic technology
a lot of the algorithms and technology
that we've been using robotics we've
been finding ways to
transform them and bring them to bear
upon some of these problems of paralysis
and so that's a very active area in my
research group today and actually i
found also that the paralysis work
the algorithms we developed there have
actually had a positive impact on the
robotic side we've actually
first developed things in the signal
processing for
reading neural signals that we've
actually found great applications in
robotics as well
what kinds of applications so i have a
crazy project right now which is winding
up
it's funded actually by the korean
military so we're building
little sensor pods with radars on them
to sprinkle all over the demilitarized
zone
uh between north and south korea to
actually find and track people as they
enter the demilitarized zone
and so it turns out that the mathematics
of finding and tracking these people
is absolutely identical to the
mathematics that i had to solve for an
earlier project where we had to
put electrodes in the brain and i sort
of introduced a new concept of robotic
electrodes which would drive around the
brain and optimize the
signals which are recorded it's
mathematically the same problem
so we have to find and track these
neurons inside the brain
and they give off signals that are
exactly the same as the signals given
off by
ultra wide band radar reflecting off of
humans
and within the neural stuff have you
been able to get robotic control from
in a different way our goal isn't so
much that and so in the early days when
i was working on
these paralysis problems we're working
on neural prosthetics so the idea is you
would put electrodes in the brain
uh and again i develop some robotic
electrodes get the signals out and then
use that to move a cursor on a screen or
a robotic arm
and other groups have gone on and done
that i've been
putting that work aside because about
eight years ago i really started working
very actively on spinal cord injuries
and so there the goal isn't to
move the legs mechanically the goal is
to actually
tap into the nervous system and find a
way to rehabilitate the nervous system
so they can actually
move the person's legs in a fairly
natural way and so that works actually
gone very well so we actually had our
first human studies this year
and so you may have noticed two weeks
ago there was a big press release about
advances in paralysis that was us and my
colleagues at ucla and also university
of louisville
and how did you get into that line of
research well it was uh going back this
new prosthetics work
because i'm kind of the robot guy on
campus as well as richard murray
a guy in richard anderson who's a very
famous cortical neurophysiologist here
at caltech
called me up must be 12 years ago now
and say hey i want to hook a monkey's
arm up to a robot arm
excuse me a monkey's brain up to a robot
arm and i'm like yeah count me in sounds
great
uh and so that's how it started um at
that time the leading um
sort of primate neurophysiology research
labs were
reviving an idea which has been around
for a long time this neural prosthetics
implanting electrodes into the brain
trying to bypass the broken circuitry to
get information out of the brain and use
it in some competent way
and so he was thinking about hooking the
brain up to a robot arm
and he has live monkeys in his lab and
so we worked together for nearly a
decade
doing that work and there's still a lot
of exciting work to be done in that area
i'm just focusing more on spinal cords
because it's an area where
there aren't so many other techniques
that are working right now and so
uh we're hoping to make you know better
progress
what are the biggest challenges both for
the the interface with the monkeys and
for the
sort of prosthetics you're working on oh
there's an infinite number of challenges
and so
certainly um moving from the mechanical
to the algorithm theoretical
um making a long-lasting interface with
the nervous system has proven to be a a
challenge
uh and many people are working on that
and i've had my uh working contributions
to that area
certainly another one which i focus on
is these algorithmic aspects
so the nervous system is incredibly
complex and we don't have anything like
newton's laws of mechanics
to tell us you know what is the model
and so it's fairly complex to come up
with a good model
so that a engineer like me can either
build a mechanism or build an
algorithm to interface with the nervous
system and so
just developing you know the models and
developing the theory of how you
interface is still an emerging in
art and science we're not done yet we
have a lot more to do
and then a part which i didn't
appreciate until more recently until our
human work started is
just dealing with human patients and all
the issues that they have
are complex so just to give you a brief
example so in the paralysis work
you would think that the number one
thing that most people want to recover
after a major spinal cord injury is the
ability to walk
it's not um so for example most people
with a
major and high spinal cord injury have
difficulty regulating their blood
pressure
difficulty breathing they can't control
their bladder and bowel
they have all these problems in fact
that's actually what they want solved
first
and so part of our work has actually
been aimed not only at restoring the
locomotion but restoring these
additional functions as well
as i said yeah going from a laboratory
animal to human is a big jump
and you mentioned that this was feeding
into your work back into your working
robotics as well absolutely some of the
the feedback is just serendipitous
just lucky um some of it's more
deliberate uh
and that is um and i'm not the pioneer
of this i'm sure you're talking with
sebastian throwing up at stanford
and so certainly he's pioneered the
idea of probabilistic robotics and
what we know is all of our work on
interfacing with the nervous system
the nervous system is very uncertain
it's a plastic or adaptable as we call
it
and so treating it as a kind of a
probabilistic system has been
the right way of doing it and i'm not
the only person to do that other other
people treat it exactly that way also
but uh it's really opened up my eyes to
bringing probability in from my
neural side to play or to bear upon all
the work i do in the robotics side as
well
and in terms of the the multi-degree of
freedom or hyper
degrees of freedom what are the biggest
challenges in controlling this
well i said i don't i haven't worked
that area for quite some time
and certainly in the early days the big
issue was
how do you deal with a large number of
degrees of freedom because
the kinds of algorithms that i and other
people looked at during my phd days of
these redundant arms
they didn't scale very well once you
started had a large number of degrees of
freedom
and so that's why with the work with
greg trichi and we were trying to come
up with a different way of doing that
and we had some successes there and had
some impact on the field
um and then i think still the the one of
the key issues is that now
you have this incredibly complicated
robot if you can build it and we're
pretty good at now
designing algorithms to control its
shape and its motion but
still uh trying to put sensors all over
this robot and make the response
much more adaptive uh based on the kinds
of sensory information that comes in is
is still an open challenge in that area
and that's a problem in robotics in
general so
that's in some sense that early work got
me involved in thinking about
sensor processing and algorithms for
planning that rely upon sensory data
and assume that you don't know about the
world ahead of time and that what you
can find out about the world is
uncertain
and noisy and dirty what kind of
different sensors have you
used overall uh you know i'm not a
vision researcher but we use vision
um i don't necessarily per se do
research on tactile sensing but we use
tactile sensors um
we've used lasers like everyone else in
the field of robotics does
the unusual thing as i mentioned we've
been using ultra wide band radar in fact
we're looking now at using ultra
wideband radar for
mapping buildings and things as an
alternative or as in in conjunction with
vision and radar
you know forced torque sensors even from
my early days even now so
one of my big projects right now i'm
working with the team at jpl as part of
the
national kind of darpa grasping
challenge we're one of the six teams uh
that we've got hardware
given to us by darpa and we're every
month having to solve a different
manipulation challenge so
pick up a flashlight and turn on the
light pick up a key and put it in a
key hole and turn it and unlock a door
and so there
we integrate all kinds of sensors lasers
vision
tactile joint sensors force torque and
so
a lot of my current interest is how to
fuse all that information together
in an interesting way what were some of
the other
robots you've worked on throughout the
jdp well so
at jpl in here you know i built several
generations of these snake-like robots
as i mentioned it worked on locomotion
for a long time so we built
um this little fun robot called a snake
board that actually it's one of the most
highly referenced papers i ever wrote
even though it's just a cheesy little
conference paper
um you know built fish like robots built
hopping robots so i had a lot of fun for
a couple years with
paolo fiorini at jpl building little
hopping robots
building with some visitors from japan
little walking robots
so so you know pretty much most
different morphologies except for
humanoid robots i've built one of them
sometime in my career and um
where they prefer particular
applications of jpl
um so some of the uh this is you know
not unique to me
um but typically either you would uh
build the robot for an application so
uh one of my students tom lee was a jpl
employee
and so motivated by the earlier work we
did in stake like robots he and the team
up at jpl
with assistance from us built a snake
like arm
which was meant to be picked up by a
large arm to use for inspection inside
the space shuttle when it was down on
the ground
so that was one was motivated by an
application other ones like
the early snake like robots i built we
didn't necessarily have an application
in mind
we built them more as a demonstration
device like look if you want to build a
robot
that's crazy like this you know we can
build it and we have the algorithms to
sort of you know
control it so most of the time for me
the mechanisms are more
vehicle for demonstration of the ideas
rather than
particularly for a specific application
and have you with jpl have you worked on
any of the rover missions
uh yeah indirectly and so um one of my
first students uh who did her phd up at
jpl sharon laubach
who's now actually risen up to quite
some prominence in the current mars
missions
um she actually took some time out and
actually was uh
was a rover ops person so um in the
sojourner mission
um the sojourner vehicle on the
pathfinder mission um
in the latter day she actually was one
of the person that actually all the
control commands went through her and
she
double checked all the commands and i
think she actually even got to drive the
rover one day
um and she's gone on to i said to play
an important role in the the mars
activities on the flight side at jpl
sam feaster another one of my grad
students
took off about a year during his
graduate time and sort of
you know uh with some input from me
worked on the current odyssey and spirit
rovers that are on
jpl and so particularly all the camera
pointing and sun
angle estimation and orientation and
body estimation of the vehicle sam wrote
while he was a graduate student here
what was like working with paulo
palo and i are all friends and so we
still collaborate uh in an indirect way
in fact i just saw him in china you know
two weeks ago
at the big conference and so um i met
paulo when i was a
graduate student um so because at
stanford
we were one of the labs that had figured
out how to break into these uh
unimate 560 puma robots and so they were
a prominent
device used by many research labs so i
you know i had sort of knowledge of
actually how to break into the system
and hook up computers
so i came to jpl to just do a little
two-day consulting job to help them
make an interface to their research puma
robot
and that's when i first met paulo and
then after i came to caltech
at that time i didn't know i was coming
to caltech we just sort of kept casual
contact
and then we found a way to collaborate
together with these hopping robots
and paolo is just a terrific gentleman
and a scholar and uh and a dear friend
of mine
and so then after he left jpl i went and
spent sabbatical time with him in verona
and i still we send email back and forth
and so we're we're you know personal
friends
have you gone to other labs
i spent some time also um in um
a lab called inria inria which is in the
south of france
in sofia antipolis so inria is a big
national french organization of research
and um at that time i wanted to learn
more about computational geometry
and so a guy named jean danielle
who actually uh played in robotics for a
while
was a host i didn't know him that well i
knew mainly through my french mafia
contacts
with sama khatib and also jean ponce so
they really helped set it up and
just coincidentally happened to be you
know in the cote d'azur
and so it was a great place to spend
sabbatical time so i spent a summer
there
with them and learned a lot about
computational geometry i actually made a
contact there with a nice guy named
subir gauche we actually ended up
writing a few papers
together and i had a great time living
in the south of france it was a real
life-changing experience
and have you also worked with the
japanese
more closely no i can't say i've had a
detailed collaboration with them no but
um obviously japan is a very important
country in robotics and i've
you know kind of gone out of my way to
to make visits there
and so i've you know met many people
there i actually had a small
collaboration with a
guy named omata who was another grasping
researcher so we wrote a paper together
and so elon ramon and i have worked
together for 20 years now
on grasping as kind of a hobby we had
one student together now we actually
have several students working on
grasping now
now the grasping's hot again and
so elon and i were motivated by some of
omad's work and so we asked him to be a
co-author on a paper that was motivated
by his earlier work
he made some nice contributions it was a
great pleasure to work with him
and no it's mainly just i i you know met
people in the years over there
and there's also a national science
foundation uh joint japan
activity that graduate students can go
spend the summer there so i've had
at least three and maybe four of my
graduate students go spend summers there
and then the main activity i've had is
also the japanese government has a kind
of a sabbatical program for young
researchers
they don't necessarily have to be in
academic labs they can be in government
labs
and so i had two researchers uh come
over uh one was
nobleki takanashi who actually i just
saw last week as a matter of fact
come over and spend a year in my lab and
he that was during the days which i was
doing snake-like stuff
and so he made some terrific
contributions in the technology in my
lab and which is
a lot of fun to have around and then
another researcher that you may run
across
shuji kajita k-a-j-i-t-a
he was working in the lab which at that
time was called mel the mechanical
engineering laboratory
which later got folded into the now aist
and so
he he got one of these sabbatical grants
uh and then uh
spent a year and a half here and we
worked on some walking robots together
and he built a terrific little walking
robot
and so with him i co-authored my only uh
publication in the japanese language
so he had to write it of course i can't
speak or write japanese he's on the hrp
he's on the hrp project yeah yeah yeah
so he kind of heads up the locomotion
part of the hrp project
and so he's a funny guy we had it was
really a great pleasure to have him here
um i'm curious could you tell us a
little bit about
um your funding sources over the years
oh yeah i think i have you know most the
typical funding sources that most
researchers do
they've changed over time in the early
days of my career more of my work is
funded by the national science
foundation
and to some extent the office of naval
research and that was you know really
valuable funding uh
because when i finished up in robotics
in the you know late 80s
it was right at the peak of the sort of
first wave of you know
explosion robotics and so i was lucky
that's what got me this job here
was that was a good time and then
robotics crashed uh
and went through a hard time boy that
was not fun and so i was very lucky to
to have support from nsf and onr
in those early days and then off and on
you know i've had support either from
nasa or joint work with jpl either for
nasa or for other sponsors
i've had you know department of energy
funding these days because of all the
biomedical work i do the majority of my
lab is funded by nih
um yeah and then i've had little odds
and ends like this korean funding right
now
i've certainly had some very pleasant uh
collaborations with a number of
companies
uh toyota denso another japanese car
company
yamaha another japanese so i actually
have had more japanese collaboration
than i guess
than i've i told um and so
um let's say department of energy uh
onr ns nih nsf uh darpa of course
i've had several darpa grants that have
one right now with jpl
so those are the main funding agencies
and then also some of the medical work
um
there are non-profit foundations and so
my lab per se hasn't received funding
from christopher reeve foundation but
the people i collaborate with do and so
i benefited
from the christopher reeve foundation
and also some uh the beckman
there's a beckman medical foundation as
well
and how did you get in contact with the
koreans
so i was working with um denso
and they were interested in um being
able to detect
pedestrians at a distance so um car
manufacturers have reached the point
where there's lots of great safety
technology for the drivers and
passengers
but um pedestrians particularly in the
newly
urbanizing asia are getting killed at a
horrific rate
and so they first came to us because of
all my sensor-based planning work it's
like okay
you know here's someone who knows about
sensing in a broad range of sensing it
can also integrate it into planning
uh and we looked at a number of things
and eventually decided hey you know
vision later we have researchers in our
labs that can do that
why don't you look at this ultra
wideband radar and i didn't know
anything about ultra wideband
radar uh and so fortunately my japanese
collaborator
so denso actually sent someone to sit in
my lab
for a whole year dr mitsumoto who's
another great guy who spent three years
in my lab
and he actually helped me find a postdoc
named sang yoon chang
from china china excuse me of korea and
so
we were working with denso and then
senun because he was korean
you know had contact with korea and they
were finding out about the work we were
doing
and then when the financial crisis had
denso i had to drop out
but it was perfect timing that was when
actually the korean
military and government wanted to
advance this ultra-wideband radar
technology
so it was really because of the postdoc
and he's
what not only is he a terrific guy but
um he's made this all possible because
he can speak to the sponsor in korean
uh i you know so it's made things much
easier
um and for the first time sorry to be
jumping around no no no no no no no no
no that's okay
i know you'll edit this later yeah yeah
we will um so you mentioned that there
was kind of a downturn
in robotic spending around the 90s yeah
um early 90s
well i'm sure other people have told you
about this well um here's the um the
stock phrase that everyone says
and there's this is primarily the truth
there's some nuances though
so i think uh what happened in the early
80s everyone got excited about robotics
and there was a big push and advances
were made
but i think um again the cliche is there
was sort of
people pushed robotics too much and
there were sort of more expectations
that could be really delivered with the
technology and science at the time and
so
many of the funding agencies and
companies that got frustrated uh that
you know these robotics people
told us we could do all these great
things and none of it came true or very
little that came true so i think
um the field had to lay fallow for a
while because the funding agencies and
the
corporate sponsors got you know
frustrated that all this money wasn't
seeing too many results
there were results they just weren't
necessary results leading right to
factory or to other economic
implications
and so funding in large part dried up
and certainly on the corporate side and
even within the government the amount of
money put into robotics didn't disappear
but it was
certainly lower than that first blush um
and but in some ways it was good for the
field to have that hard time it sort of
shook out a lot of the people
and only the kind of diehards remained
and then you know we kind of started
rebuilding and so
certainly i would say now we're in an
area of much more active robotics
funding
it's not the you know giddy rush like
the early days but
that way is good it's much more
sustainable now you know robots are
working big computer vision works now
um you know just wasn't working outside
of the lab before
and could you compare it kind of some of
the funding agencies since you've
you know had contact with so many so i
um
so there's a joke i tell um and so um
funny agents could be mad at me about
this but it's true to some extent um
so when my junior students are finishing
phd students go to academia and they
want to know how to write a proposal
so they say well you got to know your
funding agency and so
nih loves innovation as long as you can
guarantee no failure
and nsf you just got to make sure you
refer to everyone in the field because
if someone's on the panel you don't
refer to their paper you're dead
and darpa i used to tell them tell the
biggest lie that you can think you can
get away with
and now i tell them tell the biggest lie
you can imagine and so you know
that gives you some character of the
different funding agencies
in terms of uh students you mentioned
several that went on to work at jpl were
there
other students in academia yeah i
um actually is uh kind of funny i did a
count
last year so between the number of ph
students i finished and also the number
of postdocs my lab i've actually had 50
people come through my lab
so far uh and um i have probably
i have to count this but you can send me
an email i'll count it up but
especially in the early days a large
fraction of my students went out to
academia so i have
at least 15 and i can think of you know
post-docs and
phd students who've gone on to academia
you know
all across the country if you want i can
try to name them all
here or i can send you a list later
but so particularly in the early days i
said it was popular and i was lucky to
have students go on to academia
and so hopefully when my kids go to
college i'll have lots of ins and
different places
to help my kids get to do into yeah yeah
yeah okay so would you
um if you were giving advice to somebody
who's in the future
interested in robotics or what would you
tell them
um yeah you're kind of catching me
flat-footed on that one
uh yeah i don't i'm sure you've heard
these cliches from everything but
certainly um uh robots are
you know computer-controlled
electro-mechanical devices
uh and so one thing which is both an
advantage and a disadvantage for the
field of robotics is a very
multi-disciplinary field
uh and so it's good to sort of pick one
area that's of your interest so you can
go deep in but you need to actually have
kind of broad exposure to a variety of
things to be effective in the field of
robotics so
i would encourage people yeah that if
people like to do only narrow-minded
research robotics isn't the right field
for you uh
it's something where you have to have a
fairly kind of broad interest
and any other you know advice i have is
you know the same for any research
field which is find something you're
passionate about and do it
um you know as well as you can what do
you see is the
the big outstanding problems for robots
in the near future
well here's my take on uh why why don't
we see robots
you know everywhere today uh and um
other people have different opinions on
this but in my mind there's sort of five
things you can tick off
so as if you go back and think about
robots or computer controlled
mechanical devices in the early days
computing was awful
now that's kind of getting solved and so
sure for some of the really difficult
things we want to do computing still is
limitation but it's just getting much
much less to be limitation
so that one's falling by the wayside the
other one is we want to put robots in
the real world they have to use sensing
and again that's been one of my
interests and sensing technology
has come a long way both in terms of
quality and also the cost
particularly with kind of the again as
computing gets better and also as
micro machining and other kinds of
fabrication techniques get better
sensing's getting close to being solved
there's still a lot to do but it's not
the the real bottleneck that it used to
be
there's still two real practical
bottlenecks uh why we don't see robots
everywhere so one is power we don't
really have a great way to store a lot
of power on board a robot
so they're kind of like tethered uh to
you know a power source they can't get
too far away from power source and power
engineering's always a hard one
another one is if you want robots to be
cheap the kind of dc
servo motors that we have right now to
drive robots are just still too
expensive
um they're coming down in price and they
get you know incrementally better every
year but that's still a bottleneck
and then the last one of course is what
all of us researchers worry about which
is the intelligence part or the
algorithmic part
that's going to be with this for decades
and so
you know but we're hoping soon my own
opinion is if those
other two you know sensing needs to move
about another decade further
and we need to move actuation and power
another you know quite substantial leap
and then what's left is just trying to
make these robots smarter and that's
going to be an outstanding problem for a
long time
and what are the big challenges for
integrating sensing into planning and
decision making
um uh i'm speaking as an academic not
necessarily
you know as a battle heart and industry
veteran here and so um
anyone can come up with an algorithm um
what's hard is can you come up with an
algorithm where you can prove some
property about it so
what you'd like to know is if i send a
robot to mars and you know it's been
in this it goes in this environment
where it's never seen things before
you'd like your algorithm to have
certain properties so that you know
um that will either be able to solve the
problem if a solution exists
or come back and tell you the problem
can't be solved so this is a classic
idea we call completeness in in motion
planning
and so with sensor-based motion planning
because you're dealing with things where
you don't have much prior information
and there's a lot of noise you assume
there's noise in your sensors and
uncertainty
you know how can you make these very
robust reliable algorithms
not only being able to prove things as
an academic but also making them
practically reliable and robust in the
real world
um you know it's still a real challenge
is there anything else you'd like to add
i think you guys are doing a great
service and so in fact i was almost
thinking
you know that someone should go around
and try to capture all this stuff so
i'm not uh i don't know how you guys may
see it differently than i do but
um you can think of their sort of
different generations that sort of come
out i'm like the third or fourth
generation i'm not the really early guys
so there's a funny uh interesting story
um so stanford was one of the first
places
that did robotics research starting in
the early 60s and so they used to have a
place called the stanford dc
power lab and it wasn't to study dc
powers it was named after a guy named dc
powers
and so the um
i may have that wrong uh you still check
your sources on that uh but there's no
doubt that some of the really early
uh you know phd students that had you
know important impacts in the field used
to work in this lab up in the hill
behind stanford and
it attracted a pretty uh by today's
standards curious
uh kind of eccentric crowd and
so when i was a graduate student right
at the tail end of my graduate days they
were actually tearing that lab down and
so they actually had a reunion of all
the people who used to work
there and it was a just a great
adventure to meet kind of the you know
really early
pioneers robotics because they did this
you know not knowing if there'd be a
future or not
they just thought it was exciting uh and
they those people even though that
by our standards today the tools they
had to work with and the technology was
incredibly primitive
they really kind of figured out a lot of
at least the problems to tackle
um and they were surprisingly good at
sort of you know figuring out what
should be solved but
you know it took a long time to figure
out how to solve it
yeah this is now we're going to have to
uh
um you know the canonical person that
i'm sure is on your list actually i saw
it on your list
vic shimon um you know has been in that
robotics and was a grad student those
early days with bernie roth
um uh russ taylor
who's now at johns hopkins i'm sure
you're going to talk with him
here's a name um that probably has not
been on your list at all and he's
actually here in l.a
at uh cal poly pomona barry soroka
s-i-r-o-k-a and i see barry occasionally
here
he's a funny guy and he would actually
have great insights into those really
early days
and bernie roth would also give you you
know great insights into those early
days and
a list of students um
well you know uh
dan piper one of bernie's first students
um
so there was actually uh at bernie ross
70th birthday they had a reunion at
stanford i went to that it was a lot of
fun i'm just trying to remember that you
know some of the early
people that used to show up there ken
waldron uh
of course um that you know has had a big
impact on the field and also was uh you
know in the early days and
is a great storyteller too and she'll
probably give you some great stories
um and what i'm trying to think about is
some of the more esoteric ones that sort
of left the field of robotics soon after
that that you might not run across like
i said barry's one of them
um
you know that's all that comes to mind
right now uh yeah
that's a shame yeah i feel like i should
have written down all those names there
uh for posterity's sake
well if another one pops up okay let us
know okay
did you ever run into hounds more no i
never met
uh hans while i was there um
he left long before my time um
but certainly yeah that's a you know a
character you know a known character in
the field of robotics from those early
days
all right that was all we had
because there's something that you can
think of that would be important to add
that we haven't
no i think you cover a lot
great thank you so much