um so usually just start by asking you
to introduce yourself
and tell us where you were born where
you grew up and
went to school so
i'm usa makatip i was born
in a country that is
very much in the news these days
unfortunately
for the wrong reasons i was born in
aleppo
aleppo is a very beautiful city
the city of aleppo goes back to
[Music]
the sixth century bc
and it is sort of
the terminus of the silk road so
it is one of uh the oldest
continuously inhabited city in the world
and i grew up there i
came to france to do my
undergrad and graduate study after
high school i lived in
montpelier in the south of france
where i did my undergraduate study and
then
i went to toulouse to lose the capital
of the aeronautic
and space in france where i
[Music]
did my phd and what did you study as an
undergraduate
so i studied automatic control
what is called automatic control
electrical engineering
and i was doing a project
uh at uh what we call in france grande
cole
at a county called superior which is
a school focusing on aeronautic and
space and i was doing a project on ai
and i was
working completely away from the
physical robot
and one day my advisor
told me about a project of industrial
robotics
and was asking me whether i would be
interested to do
uh further studies and that's how i came
to know about robotics became involved
in
in that project and did my phd
in robotics and that was
probably one of the first
phds at superior in robotics
but certainly it was uh among the
the very few uh thesis in
in france uh we were
few hundreds at the time and
today we can say we have
a tenth of thousands of people
involved in research in robotics
so what year did you did you finish your
phd
so i finished my phd in 1980
and by
i don't know destiny or chance
in 1978 i was presenting my
first paper on the artificial potential
field
the concept that allow us to
move around obstacles reactively
i was presenting this paper in italy and
i met a professor
from stanford university this is
professor roth
i'm sure you had an interview with him
professor roth uh invited me to
visit stanford as a postdoc after my phd
which i did but what happened
is he invited me to stay one year
and i never left and he keeps reminding
me of that
that's great so um what was your thesis
project on
and who was your advisor so
the advisor the professor who was
supposed to be my advisor left so
it's in uh it was a sort of um
self-made project that
that really focused at the time on
this idea that
we had a lot of work in
in analyzing the kinematics of a robot
in analyzing the dynamics
and even in controlling the robot so
i was wondering what can i do i mean
they solved all the problems
and i thought maybe i should do
something
more than just tracking a trajectory and
i was thinking perhaps we could do
something more
in the control if the control
of the robot trajectory can accommodate
obstacles and the inspiration i had
came from
actually human the way human
move and
at the time i i was observing
my son
who was learning so fast
compared to any robot that can learn
and i was realizing that there must be a
different way of controlling robots
among obstacles and i propose this idea
of
creating an attractive potential a
repulsive potential
and just sort of move
with the with a goal position
attraction for attractive force
and when we come in the proximity of an
obstacle just to create a repulsive
force to move
around those obstacles so this idea was
was very simple and
in fact i was able to implement it few
years
later because there we didn't have
robots at the time
so i implemented it in montpellier at
the university where i did my undergrad
went back there
and implemented it on
a robot one of the first robot
that was developed for nuclear energy
plants and we managed to
to to get it to work after a week of
adjustment
and then i realized that it's not enough
i mean
the simple idea is great but we really
need to
understand uh more
about the system to understand the
dynamics of the system with respect to
its motion
in in the environment in the task space
and that led to a lot of research that i
was doing
later on how to model
the dynamics of a robot in terms of the
task
behavior and how to model
also that dynamics when we are
interacting with the environment
and that is we are making contact
physical contact with the environment
and that became the theme in my research
the theme is
not only about moving in free space
without collision but really moving
around obstacles while in contact
or multiple contact and then still
managing to do that in a way that
uh that is reactive so that was
the work in my thesis what kind of
robots were you working on
so this is this is uh the interesting
part is
the robots at the time were
mostly hydraulic robots
very large robot heavy robots there was
a robot
from renault
that i think it was called the r80
huge 80 kilograms it can carries
and and i
i really didn't implement
my approach on that robot but i used
this cable driven robot
called the ma23
manipulator that was designed and
developed
by john jean vertu jean vertu
is probably the father of
uh robotics
robot design in in france and um
there is a number of very very
influential people
in my uh early career in robotics
and jean-verti was one of them jean
vartu
developed most of the the concept that
led to
cable driven actuation
the three-fingered hand can salisbury
develop was based on a lot of those
concepts
many of the robots that were later
developed
were based on uh ideas and concepts that
jean vertu developed jean vertu passed
away
very quickly uh years back
and the whole robotics field missed him
but i had this privilege to to know him
and also the privilege of having my
paper selected by him
uh my first paper uh to to be
then the introduction to me to
know professor roth who
eventually invited me to to come to
stanford
so who else was working in robotics in
the early 80s in france
so i became involved
during these years in one of
the early projects in robotics called
spartacus spartacus
is a project to develop
a robotic system that is aimed to
assist human and in this project uh
a number of research groups around uh
france
from the south to the north india
centers
were involved in that that
pioneering project
i i think this was probably one of the
earliest projects in europe
and it brought a number of
people who are still today involved in
robotics
so if we go over the the group of people
who
is today involved in robotics we will
find
many of them who participated in that
project
however my research
itself was very closely
connected with another laboratory in
france
because of my uh my uh
uh work was
pursued at the school of superior
uh next to separate there is a very
important center of robotics
and this is called las l-a-a-s
and at last
there was a very important
project initiated
and led by george geralt
and george geralt was in fact
one of my other mentors in fact he was
the director of my uh post-doctoral
study he was the
the chair of the committee of my defense
and
with george we had a long
long
years of interaction uh sharing a lot of
the ideas
uh that uh in fact uh
became part of my own research
or the research that uh has been pursued
uh
uh over the years at last uh georgia alt
uh was the leading uh the
robotics research effort at last
and one of the researchers there is mark
renault
mark renault was one of
my uh very close colleague because both
of us we were interested
in dynamics and uh
mark renault together with uh uh
wisama khalil from montpellier
were also the three of us we were
working in different
aspects of dynamics
uh in fact we sama and renault
uh developed a
software to generate automatically
symbolically the dynamic equation of
robots
that in fact i used myself in developing
some of the
dynamic models but my approach to
dynamics
as i said was based on the idea that
we need to understand dynamics at the
level of the task
not just at the level of the joints
so i was doing this transformation and i
had a great partner
with mark renaud for discussion and uh
mark renault
was was very close uh
to me in toulouse and uh
we shared a lot of moments of discussion
on dynamics
great and so who was working at stanford
when you arrived here as a postdoc
so arriving to stanford
ah it was wonderful uh uh we
we had the ai lab in margaret jacks
in the basement of margaret jacks next
to
the deck 20 and um
and we had a small room uh with one
table
and four robots and uh uh
the ai lab just moved from the hills
down to stamford to the basement of
margaret jacks
so at that time the
there were a number of uh
uh i mean extraordinary people
that that were involved in the ai lab
the ai lab attracted
a lot of uh talent
in robotics over the years
just behind me you can see the
uh vic simon uh uh
robot the stanford chairman armed robot
and uh uh
the there there were a number of people
who who were
involved in the 70s
in the ai lab but in the 80s when i
arrived
we had john craig
was there
tom binford was leading
the ai lab and
the my friend the very famous the very
famous researcher in both
a.i and robotics
rodney brooks was there
in fact
many other people came
through uh as visitors
uh we had uh also a very famous
uh designer uh ken salisbury
who was there there were
a number of people from the field of
vision uh there was harlan baker
[Music]
i i don't want to go over all the names
i have the list and i have their photos
if you would like i will we will we will
look at that later but
but it was an amazing dynamic
environment that
i've never experienced in my life
it was it was just the discovery of that
exciting place that made me
decide to stay one more year and one
more year
and then stanford just kept me forever
because it is
very very difficult to leave this
environment the research environment the
excitement
the the opportunities to to to take on
the new project
is really unique and and what is
interesting also is
the fact that uh while i was at stanford
i was in france as well collaborating
with my colleagues
i was uh in fact uh
involved with other collaborations in
japan
in the year 84 83
and in fact 83 december 83
i i visited japan for the first time and
i had the great great
opportunity of meeting professor inoue
professor omatani hirose
uh professor
um nakamura who was a student at the
time
and uh his advisor uh
professor hanafusa and professor
yoshikawa
and uh and also uh
a number of i mean just pioneers in
robotics
all all of all of that was done during
one trip i had
to japan in 1983 and since then i kept
in touch with all these different
laboratories
over the years and that was really an
amazing experience
so what i discovered about stanford and
about the ai lab is the fact that what
we are doing is so much connected to the
world
and that became one of the other
dimension
for for my research and so how did that
influence your work and what sorts of
projects did you start working on
well this is really interesting uh uh an
interesting question
uh looking back
at my phd where i started
looking at the vision i had in that
that that work
i feel like i'm still working on my phd
that is that is uh
from the beginning i took on a challenge
which is
to try to understand
robot system with respect to
the it's with respect to descriptions of
the task
that is to try to not just worry about
modeling the physical robot in a
convenient
space which is essentially the joint
space
i was interested in understanding
uh the characteristics of the robot when
it's interacting at a given point with
the environment so what would be
the effective masses that i will feel
here when i'm going to touch the
environment
what what are the the
what is the normal space on which
i can exert forces and what would be the
corresponding
tangent space where i can move a lot of
questions
related to the dynamics and the control
of robots in task space uh
have not been really fully addressed
there are few people
uh around the world uh who really looked
at those
uh a problem novel hogan
thought about it in the context
of analyzing human motion and
proposed very interesting
models using impedance control
there are a number of other
researchers who looked at those problems
but the
sentences in task space
uh has not been
something that uh a lot of
work uh was uh done
uh in and that was uh really
one of the things that i i was committed
to pursue
and and to to work on so
there was that work that was uh taking
me from
thinking about uh how i
can we control just one effector
then how can we go from one a factor
that is not only moving in free space
but
obviously interacting with with with the
with the environment
but then how can we go from one robot
uh and a factor to to a in the factory
so
how can we do cooperative manipulation
when we start to have internal
forces then how can we have
general cooperation between multiple
robots
then how can we make this robot mobile
manipulators
so just behind me you can see romeo and
on this side you have juliet and these
are uh the two platform that we
developed to explore the
um the workspace of a human
that is uh we always thought about
uh these manipulators attached to
a table and bringing
the the component to be assembled by
those robots
now we if we are really to think about
robots that would interact with human or
move
in the human environment we would need
to have those robots
not fixed but mobile so we started to
look at
mobile manipulation and that brings a
lot of interesting
issues in the dynamic of macro mini
structures
the properties of dynamics in relation
to
reduce effective inertias redundancy
and all kind of things that were
amazingly interesting
to to analyze and work on as we
pursued this work we we discovered that
well there is more we
when we go to humanoid robotics we have
uh the branching complex branching
structure with
uh multiple tasks that need to be
simultaneously
achieved while maintaining constraints
while maintaining balance giving the
fact that
these robots are under actuated it's
amazing you have one problem
after the other it's so rich so exciting
so that was one line of the work that uh
i was really interested in dynamic
control
uh uh and control structures
but uh there was as uh
you know uh from the beginning i was
interested
also in in the uh
aspect of interaction with the
environment through
uh uh understanding of obstacles
and the need to avoid collision and they
need to
create motions around those obstacles
so having resolved the problem
in in terms of uh reactivity
uh doesn't mean that we resolve the
global problem
because motion planning requires us to
think about
a global path now
if we think about the computational
complexity of this
we realize that this is exponential in
the number of degrees of freedom
and it is costly to to find a path
to plan a path especially for a robot
with many degrees of freedom
so i was always interested in
finding a way to connect
those planning levels that is the level
where we are planning
uh the motion with
the levels of execution where those
levels
are running at very high frequencies
so if we think about the control
we need to to have a millisecond of
control
uh servo tick whereas
if we think about a motion planner
well between the time we
find the obstacles and perceive the
environment to the time we
come up with the new plan it's going to
take a long time
especially in the 80s so
what what we really tried to do was to
create
a an intermediate level
that would allow us to think about
uh this connection between
motion planning and control uh
in a way again similar to that of human
so the idea that
became known as elastic planning
is to start with a plan
and continuously adjust
this plan reactively locally
by mechanisms very simple
as repulsive potential forces
but the the trajectory
itself is treated as a physical entity
uh with property of elasticity
that will make it shorten and adjust
to the environment and the changes in
the environment
and that became a very very interesting
technique that also is being pursued by
a number of
my current student and
previous students like oliver brock
who worked on this sean quinlan
developed the early version of it and
now a number of new students are working
on
on taking this further
not only to move in the free space but
also to think about the contact space
that's really this kind of interface
between logic planning
and control theory right uh
then i mean the whole idea uh
that that that really is pursued here
is human i mean if we think about human
human never uh
go through this exercise of
precisely planning every aspect of their
motion
we do not compute a full trajectory or a
full plan
what we do is like we are sitting here
and uh after the interview you're going
to to go back to the parking
you're not going to plan all the details
of that motion
you what what you what you know is you
need uh
to go uh to a door uh
and you you know that it's going to be
feasible to go to the door
and probably you have some uh
intermediate landmark
along the way so we plan with
this concept of feasible reachable
locations and along the way
we know that we we are going to
accommodate our motion using
our reactive behavior so we are
interacting with the world as we are
moving
in in your car you you are you are
driving around
uh other cars and people and and
bicycles
it wasn't completely in the plan but you
know that skill is there
so we make use of skills
and what we do when we are interacting
with the physical world when we are
making contact when we when we are
assembling uh
objects when we are putting things
together
we are also using skills when we are
playing
tennis when we are doing uh any
uh challenging task we are using skills
so the idea is
let's then plan first of all
let's develop those skills for the robot
that is skills are
a more abstract a more advanced
capability for the robot then the simple
control of
following a trajectory or reaching a
point or
just touching a surface
it is a skill to say i'm able to
accomplish this task taking into account
different things
so by by think
building those skills the planner can
plan
using those skills so the the task of
planning becomes
much less difficult and complex
and in real time the skills can perform
at much faster server rate than
any uh dependency full dependency on
on the planner that would
be required if we didn't have the skills
so all the challenges is
how can we uh teach
these robots those skills
and um this is what we are doing
we we we
we are making use
of a lot of uh
fundamental characteristics of those
robots in terms of
their kinematics dynamic models
so that we build the first layer and
then
we abstract that layer to create
a behavior that can can
accomplish a different
uh task in the environment
with obstacles with object so we we
we talk about compliant motion behaviors
that allow the robot for instance to
to move to a multi-contact with the
environment
uh by uh just detecting and feeling
uh those contacts you
you uh you can imagine uh
a task of just putting a lead
on a box and this is something very hard
if we want to do it just by
controlling the motion and the position
but actually
this is something that becomes very easy
if we understand compliant motion
so this becomes a skill that that then
could be used and reused in different
situation
it's really the sort of direct
engagement with the world that
you're sort of utilizing to simplify the
problem
yeah the the the
it is it is really
it is really uh i mean to say that
we get a lot of inspiration from the
human but
we are not trying to
record he just record the human motion
and replay the motion
so what we are trying to do is to
is to really to understand what is
behind this motion
what is the strategy and how can we
uh encode that strategy in something
that the robot can use
as a strategy so we can generalize
but this is this is this is uh again
part of the whole approach we
are seeking solution that can
address a versatile number of problems
we're not just looking for
finding or engineering the solution to a
specific
problem because robotics is about
ultimately uh all kind of tasks in
all kind of environment and we we really
need to
to to look at this versatility
when it comes to compliance it's this
you're actually taking information from
the things that you're touching and
instead of
trying to figure out all of the what
you'd have to do to sort of manipulate
it
you you just start acting in resistance
to it and
solving the problem yeah so
so actually
i mean if we look at the history of of
robotics in the last 50 years
52 years now
well what we
definitely find is that
robotics has been dominated by
the idea that we need to generate
we need to program a trajectory for the
robot
to move in order to perform a task
so this was in fact
most of the effort in in
the 70s early 80s we were
building robot programming systems
actually one of the first robot
programming system was developed at
stanford al arm language
that became val by when it was
developed and adapted to the unimate
robot now
the question i mean obviously
in many uh many industrial applications
we want to move from one point to point
uh we we want to really follow
uh specific trajectories in free space
we
we are doing a lot of things
tracking different motions in painting
in in
in many of the industrial applications
when it
comes to making contact with the
environment
when it comes to saying uh i want my
robot to do something
we have a problem because using
a position control controlling the robot
to follow a trajectory to reach a point
and if this point is a surface
we're going to touch then
[Music]
because of imprecision and errors we are
going to make
a contact with the environment at a
location either before
the surface or little after the surface
and as we control position
the this this uh error could cause
large force to break the robot
now what we had to do was to try to
accommodate this
with trying to slow down or control the
velocity
and little by little we found ourselves
really just unable to handle the
interaction with the world
so there were a lot of work in
laboratories
trying to create compliance uh through
all kind of
techniques that brought velocity control
uh or uh admittance control
or but there was something
that was missing in the design of the
robot
that really
caused a lot of a lot of
difficulties to implementing many of the
control techniques
that a lot of researchers were
developing and building
and that is the fact that
there was no ability for those robots to
control
the forces or the torques
so the joint torques were
were absent we couldn't control these
torques which meant that we can only
control the position
[Music]
and because we are controlling the
position
we are we are making use of high gears
reducers that cause
uh the robot to
not be always back drivable
which caused these robots to be
well ill-suited
for interacting with the world so
when we look at the robots around
especially in
the context of industrial robotics in
manufacturing
we have position controlled robots
so as early as the day i arrived to
stanford
i tried to convert the puma robot which
was
controlled in position by the unimate
controller
into a torque controlled rover by
cutting the cables of the controller
and going directly to the amplifiers
to control the current in the motors
well obviously you cannot
compensate for the
friction that you have in the gear
transmission system
but that was sufficient to be able to
control
a good part of the talk
and that allowed
a lot of interesting development in
terms of force control
as early as 83 we were able to do
uh compliant motion moving over a
surface
that we didn't know its location
moving over a wavy surface
moving
[Music]
micro switch electrical micro switch
and making the assembly of a double
peg and hole that was quite surprising
in terms of the ability of force control
to perform very complex tasks that were
really really difficult
at the time and i think i think
that idea that force control is
important
uh
[Music]
completely transformed the way we
program the robot we think about
so we were talking about compliance the
idea of compliance
changes completely the way we approach a
problem
by creating as needed
center of compliance in relation to the
object location in relation to the
object geometry
and the property of the environment we
were
going to be
[Music]
yeah talking about the
center of compliance in assembly
uh so i don't know where did you stop do
you know where
where it stopped the opportunity it was
the very end of that sentence
okay all right
so you tell me when we start okay we're
going
so so the idea of
force control is that
now the motion is guided by
uh the relationship of
the object carried by the robot and the
environment
so the interaction forces are going to
guide the motion
and in a way we can from the geometry of
the object create
a strategy based on the center of
compliance
to take the object from one contact
state to another context
well despite the fact we studied those
problems
despite the fact that there has been a
lot of research around those issues
we could not really implement it in many
of the robots because of
the lack of the robot ability to
to control torque which limited its
ability to control forces
so
another aspect was the fact that
if we talk about dynamics of robots and
if we
were interested in the ability of the
robot
to pour to move a longer trajectory but
little faster
this is very important in many
manufacturing
uh applications you want uh
performance and you want speed
and acceleration well
the study of dynamics shows the
different interactive forces that are
going to be
produced by the motion so we can compute
those forces
but in order to compensate for those
forces we need torque control
so both for performance
that is dynamic control dynamic
decoupling
contact with the environment
advances in in assembly
capability of the robot to do assembly
and and
part mating we needed
torque control which was not there
so by the end of the 80s mid 80s
into how to create torque control
abilities
and one of the things we did here
was to take the puma
and say let's place a torque sensor at
the output of the puma
now we retrofitted the puma with a
torque sensor
on joint three and we demonstrated
amazing abilities once you you have
you have the output torque measurement
you can compare it to what you
your input desired input is and if there
is an error you can
create a controller that compensate and
drive the motor
to achieve that desired talk
so that removed the friction
that uh almost uh rigidify
the that joint and made it high
performance
right that lead us to to think we should
build a robot
that is completely based on torque
control
in the late 80s so we launched a project
to build artisan
so you have a piece of artisan also
behind me
this is a robot that was
designed to
have a high performance store control
and we introduced a lot of ideas
using inductive torque sensing
lvdt's all kind of
configurations to build a torx
control ability and
we built uh two degrees of freedom of
this robot
basically we built the wrist
structure of this robot and we were able
to reach a level where you could just
blow on the
arm and the arm would move it would
respond to just the wing
and that was a remarkable achievement
and demonstration of the abilities of
the robot to be so sensitive
to external forces and uh
and the need for control well obviously
in a laboratory like
at the university you do not build the
robot it has to go and be developed
so we collaborated in fact with dlr
and uh my friend gerd hersinger
uh was very very interested in
those technologies in fact one of my
student
dieter fisher who was working on this
project
worked also at dlr on on some of
the aspect of torque control and
there is a very good connection between
stanford uh robotics lab and
dlr in uh the collaboration
that uh was very fruitful in fact
dlr were the first
to produce an amazing machine the
lightweight robot
that brought
a torque control not only to the lab
but somehow moved it
together with ralph coppe to
kuka and this is
this is a very successful uh
transfer of technology that that moved
from
the university in collaboration with a
research institute
and eventually uh resulted into a very
very
amazing machine that brought
tor control as a basic capability so i
believe
uh torque control is essential to
render the robot compliant
and to allow the robot to interact with
the world and also
to to to think about robots in a
different way
it's not anymore programming the robot
it's really thinking about the
interaction of the robot
with dynamic forces or
contact forces so what were some other
projects you worked on through the 90s
so
we moved we moved forward with the
with the development of store control
and
we as i said we we moved
to to take manipulation uh
to the field to uh the environment by
combining mobility and manipulation and
then
and then we started uh
looking at interaction between platforms
to do collaborative work so romeo and
juliet were
working together well sometimes romeo
would be
doing a lot of homework like
domestic work like ironing vacuuming and
all of that but sometimes uh the the two
will be
interacting to carry a uh a heavy object
and the concept there was was was sort
of different from
just the concept of saying well we built
a robot
and they will do and do the work the
idea was
we wanted to create a machine that can
assist the human
and work with the human and the idea was
essentially you can
represent the concept as follows
the human provides the brain
and the robot provides the muscle and
you put the two together and you have
amazing
synergy so
first of all you provide the muscle with
the robot it means you need to do
a lot of coordination of many different
motions
you're creating sufficient amount amount
of autonomy for the two robots to
to carry a task to control internal
forces to do all of that
and to move and interact with the human
to follow the guidance of the human
and now the human is just like touching
slightly the robot to position it
and all the load is carried by the robot
so that was the concept
the concept of a robot that is assisting
the human
but at very very high level
and that started our uh our idea of
human guided motion and human robot
interaction
now human robot interaction we were
pursuing that
directly on the robot but from the
early 90s we started uh another
important research project
of human robot interaction through
haptics
so we
probably were among the few first
groups to look into the problem of
haptic rendering
there was the group of ken salisbury at
mit
working on that problem and
here at stanford we developed
a comprehensive approach to dealing with
the the problem of collision detection
so in haptics what you need to do is
to represent your objects virtually in
the machine
graphically and you you want to
to find where
your finger or hand is with respect to
those objects
so you need you need to do collision
detection
and objects are represented with
polyhedral so you have a lot of
things to check and that means
you are going to spend a lot of time
doing
uh collusion detection at the time
robotics and haptics came together
through what i uh just was describing
earlier
about the elastic planning so
the elastic planning started with the
idea of an uh
with what we called a bubble band
or elastic band so the elastic band was
this idea that
we would like to model
the free space
around the path so we have a trajectory
that we planned
but in instead of just planning for one
path we are going to plan for a tunnel
of the free space
and this tunnel is essentially you can
imagine
spheres that are intersecting so from
one sphere of free space
to the next sphere if they are
intersecting then you can have a passage
now uh the the
the obstacles that we have are
represented by sort of
a very fast way of computing
distances to those obstacles from where
you are
would be to represent those obstacles
with
a combination of spheres
so you can put the whole world in one
sphere
and then you break it into two and break
it into two and
this sphere hierarchy let you go from
where you are doing the check down
to the leaves that are in contact
the same concept that we were using in
motion planning
elastic planning we applied to haptics
and we developed collision detection
very easily
one of the i mean at the time was really
the fastest
uh technique to find um to
to resolve the collision uh uh with
a large number of uh objects uh
uh and and at the time that
that was amazing because uh we were
doing
uh i mean as you know haptics requires
very fast interaction
now that that uh technique became a part
of
other algorithms
to do collision detection that are
actually making use of combined sphere
hierarchy and balding boxes and other
things
the other problem in in
haptics is the problem of collision
resolution
and collision collision resolution is
something that
we know it's difficult how you resolve
collision how you simulate the collision
now simulating collision between three
objects is something that we studied we
know how to do
but working with
the problem of collision
or multi-collision between multiple
links
that is at multiple point you're making
the collision
is a problem that we don't know very
much how to solve
because you have constraints
so in fact barav one of the researchers
who looked at this problem in the
context of graphics and simulation
resolved in a very interesting way where
just before the collision you have the
velocities before
you remove the joints now you have free
body colliding you collide after the
collision you put back the joints
that is you eliminate the constraints
you remove the direction that are not
possible
which takes time eliminating concerns
now in our approach
again this is the interaction of
robotics with haptics
in robotics we know how to compute the
effective mass
at a given point on the robot
because we we are interested in
controlling this point
in contact we need to be able to
stabilize this point so this is
an information that comes from
projecting the dynamics along
well now if i'm going to resolve
collision
this is very useful i can take this
effective mass
i take this effective mass and make them
collide i replace the robot with two
muscles
now if i have multiple collision i will
have
two muscles here two muscles there two
muscles there and
in addition i have the contact i mean
the interaction between those masses
so that led to a very very effective
solution in resolving
multi multi uh collision
between multiple links
so in the 90s we saw haptics
progressing we saw the
elastic planning progressing we saw
a mobile manipulation progressing
and by the end of the 90s
by the end of the 90s something very
interesting happened which was
suddenly the robotics world discovered
there is
a robot that is going upstairs and
downstairs without
falling the honda robot
a lot of people who were working in
biped locomotion were depressed
i remember and a lot of people
actually became really energized and
excited robotics is moving
i think that was a really big challenge
to the community
coming from a company that that built
a machine with those capabilities
now honda really
produced one of the most remarkable
mechanical system and developed
one of the most interesting machine to
to perform by bit walking stable
by bed walking but obviously they needed
much more
to to really do useful things with the
robot
and one of the
things that we explored with
honda just in the
following year i i remember in 1999 was
to
to try to take the capabilities we
developed for romeo and juliet
to ashima and
in fact i i recall at the time i said i
don't
want hmo i need to study the problem so
in 1999 we started the project
and we've been working with honda
on pursuing implementing developing
the capabilities of a honda robot
or a humanoid robot to
go beyond just walking
to enable a humanoid robot to interact
with the world
to be useful to do things and that was
that was uh the direction that
our work has taken in the last almost 10
years
i mean 12 years and
that was one of the major projects
that we have and it is really uh
remarkable that
a company uh lyconda
kept a long-term relationship with
a laboratory to explore
this direction so
from the basically 99
to now we've been working on on how
to deal with this problem of getting a
human
robot to interact with the world so what
were the challenges
first of all if you
have been to a
an ashima show you would hear
the following they would tell you
with ashima there is one rule
and that is do not touch
the robot that is
this robot is controlled to balance
but it is not controlled to
be interrupted by any
external force unless
the controller is going to accommodate
that
and that requires specific way of doing
it
so ashimo is essentially position
control draw what we had to do
we had to first of all
take our concept and models
from the structure that we had with
mobile manipulation of romeo and juliet
which was essentially a seri like
structure
to a branching structure and
deal with the dynamic interaction
between the right hand the left hand all
all those different parts of the robot
model the robot in term of effective
masses
and dynamic responses of the different
part we would like to control
but then we have all kind of redundancy
that
remains so if the arm is fixed the body
can still move
so we developed a technique that make
use of
the idea of similar to the human
task you're moving your hand you're
reaching with your hand
and your body is following the posture
you're controlling the posture
but the task has priority we started
building
uh those priorities two students work
with me
on on on this uh uh
first of all uh jihong park
and uh lewis sentence both of them
uh contributed uh to to to this work
in exploring uh the contact
and the strategies for dealing with
the
priority control of of the different uh
tasks and sub tasks that we are
controlling
i mean when you think about a humanoid
robot you think you need to
control the posture but you have to be
consistent with the task
do not interfere with the task at the
same time
you have constraints you have to be
consistent with those concerns
you have joint limits you have obstacles
you have to balance
you have so many things to deal with and
people
very often go and say all right okay i
built a controller to deal with this
problem i built a controller to deal
with that problem
and i put them together and they will
start fighting each other
so what we end up doing is reprogramming
everything and
choreographing every motion to produce
what we
think is a robotic motion so what what
we've been doing was to create
a framework that
explicitly from the beginning integrates
contact integrates constraints
integrate integrates the the
the posture control task control the
task at multiple points you're
controlling center of mass you're
controlling pressure your control you
you you have all the degrees of freedom
all the constraints all together so
the task oriented
control approach that i developed in the
80s
and in the 90s
called operational space control this
comes from the title of my thesis
operational space control operational
space control
became constrained
consistent
task space control that is the task
is multiple thing but
you need to control it under a lot of
constraints
and this task control
is done with constraints that are
obstacles but can be joint limits that
self-collision all these things are
coming together
so sir your robot is
is inside a field of forces trying
to move away from all these different
constraints to perform others
now how do we control this rule
you said torque torque control
do you know a humanoid robot with torque
control
all right we're back to square zero so
what do we do
well so you developed the simulator
you developed the controller we spent
many many years building psi our
dynamic simulation system
with collision detection
resolution similarly so we are
simulating the environment we're
simulating the interaction with the
environment
we're reproducing all the contact forces
and we are controlling
the robot using that framework and it
works beautifully
now taking it to ashimo again
is a big challenge because you have to
deal with the fact that ashima is
controlled in position control
but if you identify a problem
i mean in research the most difficult
part is
finding the problem to solve once you
decided
when when you set your mind to the
problem
the solution will come and you solve it
so we said all right we need to find a
way to go around
the server loop of position control
of oshima we came up with an idea
which is called the position
to torque transformer
and the position to torque transformer
is essentially
an idea to fool the
servo controller to make it think
that it's following a trajectory but
actually it's producing the right talks
so it's like inverting
the the controller and driving it
in a way to make it compliant
and we managed to make it compliant and
we managed to
render ashimo really really
uh like almost like
uh a robot with a compliant joint
so
what we see through this from the 80s to
90s to to it's the same progression in
the same direction
toward task always task control
with dealing with contact
interaction and dealing with
constraints obstacles self-collision and
but all of it is done in term of
thinking about the robot as a system
submitted to
all kind of dynamic forces external
forces
so it is moving in for space
and through that mechanism we
never talk about inverse kinematics
whereas most industrial robotics
most robotics is actually
developed with the idea that all right
i need to move like this in space
well let me compute what
joint motion i'm going to have and let
me find the
inverse kinematics and then let me
control those joints
that works very well for free space
motion
but once you start making interaction
and contact with the environment
you are going to have a problem
controlling
joint motions and forces
at those contact in a way the task
oriented control
unifies motion control and force control
or contact control in the same space in
the workspace
and this is also really interesting from
uh the the fact that this is almost like
how we human do it we think in the
workspace
and we operate and control uh
relationship in that space
and it also translates very quickly to
the multiple robots
yeah exactly and and
by the way i mean understanding models
of
what the hand dynamics is makes you
say all right now my skill
is related to the hands i mean what the
hands are doing
so i can develop the skill independently
of the robot
so when i'm going to do learning when
i'm going to learn human skills
i'm not talking about a robot that is
going to
establish connection between motor
actions and resulting emotions
so my learning will be dependent on that
specific robot
rather i'm going to just think about
forces and movements applied to the
hands and
resulting behavior on the on those
objects
and then i can take that strategy and
apply it to any robot
and the relationship is very simple
forces and movement here are related to
torques
using what we call the jacobian
transpose
jacobian transpose capture the
movement arms of the different joint
axes so
so this is basically the the the the
major
the major uh direction
is think how
you can produce those tasks
in their own representations
and then you can transform it through
the jacobian to produce those forces and
and controls uh as needed for different
robots
just by modeling
the dynamics the kinematics of that
robot
well i know you're going to ask me what
did you do
in the new millennium right you were
going to ask me that
so
one thing beside ashimo that happened
was very very interesting
so when you have a humanoid robot and
you're interested in understanding
how
how can we better be inspired by the
human how we can
uh go further
in exploring human skills and
building controllers that are uh
proven with the human to work so
when when we really consider
that we realize that well
either we we need a way to
to look at every task human uh
is doing and they try to copy that task
and reproduce it by the robot
or uh really
to go and and identify
the way human perform those tasks i mean
identify in the sense that
we need to model have a dynamic model of
the human kinematic model
perform these tasks and look at the
the musculoskeletal
control of the human and how human
are handling these different things
so we started a study
that took us to the human
and what we discovered there was that
a lot of the techniques a lot of the
algorithms
developed for robotics for articulated
body systems
are amazingly powerful when applied to
musculoskeletal systems because we
developed a recursive algorithm very
efficient algorithms
that that make use of generalized
coordinates
in a very efficient way we were able to
reproduce reconstruct human motion very
quickly
in real time whereas a lot of the
algorithms and software that the
bio mechanic community use
are still far
from that optimal characteristics
and efficiency that is needed to create
real time
interactions a lot of the measures
a lot of the characteristics and and the
metrics
we developed in robotics can can be
applied to the human
so we started applying robotics
to human models and we started doing
a very interesting analysis
that led us to a lot of a lot of
addressing conclusions
not anymore about robotics that is
there is there is always this this idea
that robotics is
about robots actually robotics is
sort of this body of science
that we are developing with
sort of like algorithms
models that efficiently
allow us to explore articulated body
systems
high dimensional spaces and take us
through that analysis with
insight coming from the physical world
but
but really taking us to those high
dimensional systems
and their non-linear interactive nature
well humans are more complicated than
most robots that we know
but but but we
have been able to apply those techniques
to
to robots i mean those robots
this those techniques that we developed
for robots to the human
and the result
is really really interesting so here is
the first result
the first result that we found was
just related to
something very basic which is to say
if i was going to push
an object so
you you want to push the table
what posture the body will take to push
a table if i'm going to
um to lift
or pull a weight what would be the
most appropriate posture for that
specific direction
it turned out that if you are doing this
repeatably and if you really discover
the the proper posture to push
or to pull or to apply a force
you are using a posture of your body
and this posture
would cause the force
and the effort of your muscle to produce
that force
to vary depending on what posture you
are taking
so if you're drinking a cup of coffee
drinking a cup of coffee
this is a very common posture
we never do it this way we do we do not
do it that way
we do it here and the reason for that
there is something special about this
configuration
some 40 some 40 to
43 degrees
and this is the fact that at this
configuration
the effort associated with
the muscles that you have
is minimized so
it's sort of like when we are building a
machine what do we do
we build the machine and we use the
machine
in a way to make use of its mechanical
advantage
somehow a human discover the mechanical
advantage and its use
and we use it we use it because
we are then able to
uh to minimize the effort
associated with our muscles in
performing producing
the forces needed for the toxin
and this property is
uh resulting in
sort of an energy minimization
associated with
with your muscles so if we take one
muscle
of tension m
and the energy will be m square
but you have big muscle and small
muscles so the
you need to to consider the capacity and
weigh your energy by the capacity
so the energy we we discover that we are
we human minimizing is
e the energy is equal to c
m squared
[Laughter]
so
how do you follow that um
so how did you get interested in
robotics at the very beginning i went
through to the field of engineering and
robotics
there is an interesting story about
engineering um
after
my baccalaureate i went to
to enroll in aleppo
at the university it was just before i
get my fellowship to come to france
so i was went to the university and uh
i enrolled in engineering
and the clerk
said what you cannot enroll in
engineering
you you are the top student you have the
highest score
and you are accepted in
the medical school so you can be doctor
so you should enroll in medicine
i said sir i really
i cannot see blood i really really want
to i love
i love science and i love the the the
but i really like to to work with
machines and uh
i i would like to
enroll in engineering he refused he said
come tomorrow with your father
and uh i had to bring my father to
to get to engineering
yeah i always liked physics math and in
in
i mean and and i always uh found it like
uh
limitless i mean the the the the thing
was
uh the the the textbook we had
were not enough for me so i was creating
my own problems
to solve in math or physics and
exploring like
uh beyond the boundaries of the textbook
but uh
but the idea of robotics uh
was really not there i mean i i was
interested in
electrical engineering in computers in
circuits
in electronics in all of these things
was fascinating by the way
we we can
create those amazing machines and and
then
and then it was sort of
by accident
i was at the right time
with the right group who started
robotics
uh at uh superior
and uh pointed me to robotics
and i felt this is it
because what is wonderful about robotics
is you're not studying electrical
engineering
you're not studying mechanical
engineering
you're not just looking at biomechanics
you're not looking only at the
psychology
it is this cross-disciplinary
field that brings all different aspects
of computer science
with electrical engineering with
mechanical engineering
in in in a way this is
why we have
had a lot of difficulties in the
beginning of robotics because
we have computer scientists approaching
the problem as a programming problem
we have mechanical engineers designing
robots
that are difficult to control for the
purpose of controlling the robot to
create interaction
so it took time to build
skills for people to understand
not only their discipline but across the
circle
so to realize that we need to design
mechanisms not only to
to be fast or light
or heavy or strong or precise or
accurate but also we need to
design them for the characteristics
that we need in the control and
we can optimize those machines
when we we look at what is needed in the
control but
if we do not look and analyze what is
needed in the control
in the design so there is the cycle
uh
i think the industry of
building aircrafts went aviation
went through that as well for for many
years
but i think robotics also
benefited over the years from
well many of the previous technologies
but
with so much focus on manufacturing
and industrial robotics that it was
optimized
for precision accuracy speed
and and
the intelligence the abilities of the
robot
of creating
what robotics is about was not really
needed
because the manufacturing environment is
structured so if you have a structured
environment
you you you just program the robot for
that structural environment you know
that what
what your what your environment is
in the 90s my thought was
robotics will never will never
find all its
[Music]
depth if it is imprisoned in
manufacturing
because it is adapting and
automation is integrating the robot
in their processes
robotics
started seriously to develop when it
escaped
and moved out to the human
environment as robots moves outside
they are facing the unstructured world
and now you have to
do perception now you have to
adjust to the changes in the environment
and now you have to deal with dynamic
environment
that is always changing and you're
interacting
with human in that environment you have
different requirement for the robot
their safety you have different
requirements
for the navigation you have all kind of
new things that brought a lot of
new things and new concepts and
amazing development since
the turn of the century so
i was working before to
deal with a joint torque control
for the robots building artisan
after
or since robot escaped from
the manufacturing plant and went out
we realized we need to make safer robots
so that was one of also the
other projects we we have been working
on which is to
to see how can we build a robot that
is light has low impact
but guaranteed to be intrinsically safe
and uh maxine and
another student don john shin
worked with me on this
question together with professor roth
and also ken salisbury
in fact the willow garage robot
sort of came from that initial project
that started with an
nsf support that led to
one of the solution of the problem of
making robot lighter and safer
uh and that is to do gravity
compensation
so behind me you have two robots that
were built
to address those problems through uh
distributed macro mini actuation what we
call dm2
and uh which later
was implemented with pneumatic actuation
and electrical actuation
and this is the hybrid actuation
solution
that brings strength
with softness through for the from the
pneumatic actuation
and fast but light
and small uh impact
with the electrical motors so
that was another another interesting
project that that
that was quite uh
successful in fact gm uh was very
interested
in this project and was sponsoring it
for the short time before
gm had some financial difficulties
and we had to suspend it but
i was glad to see gm involved with
nasa on building the robonaut 2
that also brought a lot of safety
to the system compared to robonaut one
now
one other project
that we also
launched in the
probably the beginning of the
2010 probably 2009 i don't remember
exactly when we started
is a project dealing with um
the fact that okay now we are studying
the
musculoskeletal model we understand how
what is the closed loop in term of
the behavior transmitted from muscles to
to the task but what
about the control what is
what is the motor control that is taking
place
and how can we
connect the representation of the task
in terms of the what the eyes
are seeing in the world to the muscles
and the way the muscles are operating
this is something that
is taking place in the brain and
understanding what is going on in the
brain is very complex
now what
people usually do is they stimulate
some neuron in the brain and look at the
response
muscular response and you can connect
this region with that
muscle control
our understanding in robotics of
articulated body system
shows that there is enormous effort
that goes into the coordination of
motion
so you're coordinating a lot of joint
motions
to be to produce some coordinated motion
at the
hand and what we've been interested in
was to
think about how can we
how can we get data
that will capture this coordination
between different joints so one solution
is to go to
fmri inside an fmri you can
you can capture
an image of the activities in the brain
and by moving part of the body you will
be able to
say connect that activity with
the physical activity the problem
i don't know if you try to go inside
fmri but it's really confined
it's very compact and you can maybe move
your hands but
not more so
what we try to do through our experience
with haptics
is to say well i really need to
stimulate the brain to do coordination
and i can
imagine all the tasks
i can do inside fmri if i had a haptic
device
but forget about
taking any haptic device with you inside
of fmri because
all the magnetic field that is there so
what you need to do is to think about
a haptic device that is
fmri compatible and
this is what we built so
in fact over there you can see a haptic
device that
is compatible with fmri
and it is placed outside and you can
reach to it
and now you are able to stimulate the
brain
and perform complex tasks you spend a
variety of tasks
that involve applying forces
using your muscles to produce
these different specific forces and you
can use tools
so you remember earlier i said we do the
synthesis
of controlling robots by thinking about
forces and moment
and apply them to the arms using
jacobian transpose
well in the case of the muscles there is
more transposes because there are other
mapping
but what i believe is there is a
jacobian transpose in the brain
and we are trying to identify it
great um so i also want to talk a bit
about
some of your work as an administrator of
the robotics motivation society and just
how
the field as a whole has been developing
and coordinating over the last few
decades and
what the role of various conferences
and organizations
good question
how about if we if we do that after a
small break i need water
sure
so i was wondering if you could tell us
a bit about how different kinds of
organizations
and associations and conferences have
evolved and emerged and shaped robotics
over the last few decades
so the first conference i attended was
romanci
this is uh in houdini and it's part of
iftan iftam is the international
federation of machine
and mechanisms and in fact
professor roth was president of the
organization and uh that's where i met
him in romanci romanci is
a robot manipulator systems
uh symposium and uh romanci is still
running and i was
co-chairing the last meeting of romanci
in paris
uh last year so
that was in fact one of the very first
meetings in robotics it was dedicated to
mechanisms and
and machines and mostly focusing on
design
um later on
in the probably the early 80s the first
one of the first the very first meeting
uh
in robotics was uh called isr
inter national symposium of robotic
research
that that was
that was a sort of a small meeting that
brought all the experts in robotics
at the time and uh
to meet and discuss the different aspect
of robotics
from vision to planning to control to
design
all these different aspects in one
meeting and
this meeting is still running and it is
organized now by the international
foundation of robotic research
ifr ifr
organization is
an organization that organizes
different meetings and sponsor different
meetings
including isr the international
symposium of robotic research
and iser the international symposium of
experimental robotics
now experimental robotics is
a symposium i i
myself with vincent hayward from mcgill
initiated in 1989
because we felt it is very very
important
to not only develop the theory for
robotics
but also to validate
or invalidate those theories
and the first meeting took place in um
in montreal and last year we organized
the meeting
back in quebec city not far from
montreal
so this meeting went around the world uh
from montreal to toulouse to sydney to
rio de janeiro to so it goes
continuously around the world
and we started to try to go to the south
so we're going to have it in morocco
next year
there are a number of fantastic very
important
thematic topical conferences
symposia workshops that brings
small communities
to focus on that theme
and i think these meetings have been
playing a very important role in
robotics as robotics has many different
sub-areas however
the community as a whole come together
in one meeting as we know
and this is ecrap or iris
so at icra and iris all
the
researchers uh in robotics
and related field are interested in in
coming
to meet uh with other researchers to
to to find out about the state of the
research but also to interact and
especially for phd students
newcomers to the build they really want
to to find out
what i mean this is the place where
everyone is coming
so acra
started in 1984 and uh
in atlanta and it was not ikra it was
just the robotic conference that was
organized by
the what was called it was not yet
the society of robotics and automation
the society of robotic and automation
came
a little later and we had the first
official icra in saint louis in 1985
and uh that was
uh one of the
probably i mean it was the first and the
smallest
uh gathering i remember
i presented uh my paper
on
the artificial potential field at that
meeting
and and
had a a lot of discussion
discussions not only with one discussion
but many discussions
uh with a lot of colleagues
and it was i think the first time i met
with gert herzinger
it was uh in
through those discussion that we had
coffee together and we started talking
and then we started he started to
visit stanford and uh
connect with with me the
the the interaction idea
the contact between among people is very
important
and this remained for me one of the most
important aspects of a conference
it's a conference we're not just going
to to listen
to a presentation and leave we really
need to talk and interact
and discuss so
in 2000 when uh i organized ecra in san
francisco
i thought we really need to
to bring more interaction among the
participant
and there is always a challenge because
the
the problem is when you have
a large number of participants you're
going to have
more and more parallel sessions and
a direction becomes harder i
participated to meetings where we go to
the
session you find the authors
the co-authors the chair the co-chair
and among themselves and
because we have 15 17 18
parallel sessions so one of the thing
we tried to do in ecra 2000 was really
to
reduce the number of parallel sessions
by reducing the time of the presentation
by squeezing the uh the time between
uh sessions by immediate all kind of
ways
to to bring the conference to
to be more to closer
in term of uh uh even
i mean the geography of of the meeting
was
was such that everyone has to stay
within
uh uh a small space so that they
they really have to meet and interact i
remember some meetings where
you go to the third floor you come to
the two second floor
and you end up losing everyone in every
direction
we wanted also to attract people not
only by
putting them in a space but also by
creating
uh interesting attractive presentations
and we we developed the idea of mini
symposia
with a high
quality of presentations and
interaction and i remember in 2000 we
were monitoring how many how many
participants would be in a given session
and the numbers were amazing
very high numbers we had like 70 in
those mini symposia
and this model came to to to be used
later in
other conferences uh one idea we
introduced
uh i think ikra 2000 was the meeting the
last meeting where
uh we were we've been using uh
proceedings i mean still we we use
printed proceeding
after but we were using paper
submissions
i remember in that room over there
we had so
many thousands of piece of papers to
handle
because every author would submit
eight copies and then you add to them
all the papers for reviews and you end
up with
a huge amount of uh pages
to handle uh for that submission
and when you have uh
i i recall we we we had about
over 2000 submissions so it was a huge
number of papers to handle
and that was the last time uh before
going to
electronic submissions
electronic submissions and the the lack
of
uh and the large number of papers and
heavy proceedings
led to a situation where
a participant will go to the meeting
with a
a program and the program would list the
title of the paper and the name and the
room
so this was really difficult to to
conceive
as sufficient information to to to plan
your
participation so we introduced the idea
of the
um instead of having just a simple
program
uh is to have something called the
digest
so the the the conference digest
became now uh a very useful tool
in many conferences but it was
introduced in
icra 2000 anyway uh
many of those ideas including
also interaction not only
in the conference space but in the
on the boat on
providing a social uh uh
space where people can really discuss
and and meet
uh in ios 2011
uh i don't know if you attended but
we had the banquet in a museum
and we had
a cruise that
that brought so many uh participant
that that was quite memorable
uh to many who who were there
but i think i think the quality
of every aspect of
the experience of a conference is very
important
and it's important because
there is i believe a huge effort
by everyone who has who is attending
those conferences from japan
from europe to the us or from the us
going to europe or to japan
and we are
taking time for
a conference so the quality of the
papers we are
we're we're we're going to have the
quality of the
social interaction the quality of all
the aspects of the discussions
and uh in that meeting are going to be
very very important
and i think uh iris
icra moved in that direction and
in the coming years uh i i think i'm
going to have um even
a more challenging role uh in
uh dealing with conferences in the
future
i think conferences are a very important
window and
vehicle for our community
and i don't think about it
in terms of just one conference
that is better than other every
conference is serving
a very important role icra is
is the flagship conference for
our society iris is supporting
a large number of participants of our
society and others
all the other meetings
humanoids rss and
wafer iser isr
or ark all these other meetings are
contributing to themes
and topics within the
the areas of robotics
and having smaller
gathering
in one track presentation
where people from vision from motion
planning
from mechanical design are sitting
together
through the whole conference listening
to all these papers
goes back to what we said in the
beginning robotics
robotics has all these different aspects
and unless we really bring that
understanding
between people
in computer science in electrical
engineering mechanical engineering
to really understand what are the issues
and the problems
it's going to be really difficult to to
build the field so
we need the small meetings the symposia
the workshop
the one track high quality what we can
call
uh slow conferences like
slow science it's slow food
in other settings we do not have that
amount of time
we still have the three four days with
thousands and thousands of participants
the last meeting
in san francisco was nearly 2 000
participant
iris and those numbers are
are growing and i think we really need
to think
uh um hard about how we're going to do
the
future conferences but what is clear is
that
we are going to need
to have more
and more of the specialized conferences
to create that interaction
and those discussions between the
communities that are being formed one
thing i wanted to add is
also the fact that robotics is making
impact in other fields
in biocomputing
protein folding in in biomechanics
in a lot of other fields
makes it that there are new communities
that
are coming to uh
interact with robotics and
they have their own conferences and we
are
seeing a lot of of those interactions
so um i think i think
this is why we are looking at
robotics with all its different uh
dimensions and it's becoming
harder and harder for
newcomers or even i mean
people who are working on the field to
find their way
and this is this is not new this is
started
back in in the late 80s i remember i
mean we started to see
so many things happening so
back in the 80s i in the late
80s 89 i proposed
in addition to starting iser
i propose to to have
a way to think about
uh a sort of a place where we can
find out about what's going on in the
field
and we created uh
what was called the robotics review
the robotics review was published
by mit and i
worked with john craig
and thomas lozano perez on
its
editing uh in fact we we edited
two volumes in 89 and 91
and with the contribution of
about 50 60 people each time
and with a lot of editorial work
by us a huge amount of work because
uh the idea was to review the field and
come up with what is going on in the
field
well this is maybe uh was the
the first step towards what
bruno and i worked on
later in the handbook of robotics and
it wasn't 40 people that were involved
it was
much much more 160 people
with 1600 pages and
i'm really excited about the
the new edition that is coming because
it will bring
yet a new dimension that is very
important and this is
the multimedia the technology is
developing and it is
helping our robots to compute faster we
have
a lot of new technologies in
material in sensing etc and i think we
really need to see it we need
ways to communicate
algorithms we need to share
software to to
put videos and
to even introduce material to
help understand
concepts uh through uh simulation and
different
ways of uh presentation so i think
this is something that that is going to
be really interesting and
we will see how
the first result of this would be but i
i think the multimedia is going to be
quite interesting uh thinking about how
the coherence of the
group and the kind of history of the
robotics
automation society to ask you how you
came up with the idea to do
an oral history of robotics and how that
idea
started and proceeded
well
i think in every domain
we have so much
that is not written that is not
published
that is uh really um
maybe maybe only the the the
uh historian of science uh
can can discover
later that
[Music]
we can really not
find
i think i'm mixing up can you can you
start
let's start again yeah this one was
i was thinking about others or something
else and then
right i'll just ask the question so can
you tell me about the origins of the
oral history of robotics project
well i mean robotics went
through a long history of
many many years and i think so much is
not documented there is a lot of things
that
are unpublished and uh really having
a direct uh
statement by uh those people who have
been involved in the field
uh presenting their uh
point of view and answering your
questions uh
is is very helpful i i always really
thought that we we really need to look
back at
the history in order to to know where
we're moving
and uh where we uh
made mistakes and uh where we can
uh try to to uh
um to address issues that that uh
have slowed down uh our
our uh progress so so
the history is really important and i
think uh
historian of science will go probably
further with
with what robotics brought and how
robotics developed
and there is a much more needed in terms
of sentences
uh technical uh understanding of
the the different uh aspect of
how the field moves so if we think about
the field
we we spent so much time optimizing
uh our algorithms because of the
limitation of
computers uh in the 80s and
in the 70s i remember implementing
uh my potential field
in pdp 1145 that have
80k memory and the program was
in and out overlays and we were
running slow server we could
have a vision system
uh monitoring obstacles moving obstacles
with one millis one second
one second rate
so the motion was so
jerky and uh despite all of that
um the
there was a sense of need of generality
we knew
computer power will increase but it was
only when computer power
really increased that we
we discovered many different ways of
looking at the problem and focusing on
different issues
and i think technology
technology uh has a huge
role in the way uh
systems like robots are developed
and material science
in new material fabrication techniques
are affecting the the field of robotics
in many different ways
and today
many of the components of
computation fabrication and material
are changing sensors communication
and that is changing the way we we think
about robots
so i think that's why i think it's very
very important to look at the history
and
and see how this evolution took place
and uh we are uh
also recording that history uh
especially with the pioneers who uh
really contributed to the early ideas
and and explored the
that that
environment that that was uh completely
unknown to
to to many in fact uh we are still
somewhere in on the way uh to building
robotics and i think there is
so much to be to be explored and
every every time we think we
we made some progress we discovered oh
there is much more
so i tell my student this is a filled
with
infinite resources of problems and a lot
of fun
and excitement to explore which leads
right into my next question which
what do you see as the biggest
challenges facing robotics and the most
exciting
future directions
so robotics
as we said earlier
grew from the industrial application
that was
confined to manufacturing that is the
structured environment
and now it went outside and
it is changing robotics in
in many different ways where really
the whole space of challenges
is brought together to the robot in
terms of
the perception we need
very very
fast processing of a lot of data
in real time to
to provide information about the tactile
contact about the dynamic motion about
the visual environment and and also
uh uh we we have more degrees of freedom
to deal with
we have a human uh to interact with
uh we have all these uh aspects that
are uh sort of exciting
all the dimension of the problem which
is great
so this is one of the way to justify why
we want to humanoid robotics well
at least we are able to look at all the
complexity of the problem
we we probably don't need uh
the the the full system in every
application later
but we need we need to understand uh
many aspects of many the dimension of
those problems
now when we think about
robots developing in
human environment we are going to build
those skills we are going to improve
their capabilities
and i think what's going to happen
is that robots
will go back to manufacturing
now much more intelligent much more
capable
and they will change the structure
environment
that we have
used to develop in
manufacturing so in manufacturing we
have
much we spend a lot of effort building
the assembly line
very costly maybe the most costly part
is how to program it how to adjust it
how to tune it up
well now and in the future i think
manufacturing will
if we have those intelligent machines
we should be able to uh to change the
whole picture
to get away from the assembly line
and get these robots to move and do the
assembly
so there will be huge implication in
manufacturing
there are uh also implications in
in our lives uh
robots that that can really ultimately
finally
help us in homes uh
a challenging environment
i didn't have the chance to discuss that
but
two three new projects we launched
recently
one is a project to build a humanoid
humanoid-like
robot that is a diver
that is it is
a project for exploring
the red sea reef
i mean the uh the natural environment
uh the in the red sea
that is quite fragile so
that if you if you imagine
uh taking a robot in that environment
the marine biologist would be
really really upset because robots
that typical robot that are quite heavy
and
would be very dangerous for those
environments so in those fragile
environments we need
something as safe as the robot that we
built for the human
and that that's what we we are
developing we are developing a robot
that
uh that in fact would be sort of
uh like a robonaut
a robonaut in the sense that uh this
would be aqua nut
working with human divers
to reach down to place sensors to bring
samples to interact with human
and to cooperate and do everything we
talked about
on on the ground underwater
this project started last year and uh
this year we have a we have
we have a prototype design and we are
collaborating with the mac robotics in
fact on
on on this project
there is another project and a very
important
aspect of robotics that
that we really need to address and this
is how to deal with
disastrous areas so we saw fukushima
and we saw how little robotics
contributing to helping with focusing
and one of the reason uh for that is
we get used to the idea that robots will
be working
in our uh
comfortable buildings in
those flat
surfaces flat floors
so they are moving on two-dimensional
environment
and and evolving uh in those
environments
through biped walk now
if we have an earthquake or a disaster
the environment becomes completely
difficult you you will have to go over
three dimensional
objects and you you
you need a a different way of thinking
about
uh controlling those uh robots
especially in their locomotion
then then um
the typical zmp zero moment point
control
inverted pendulum control for viper
so we we are launching a concept that
that is called a
contact supported locomotion and that
goes back to the idea of
of interacting with the world and using
the environment to support
the motion and this
this concept is taking us further
with new tools that the robot can carry
to bring extra support
something we call suprapad that
brings a sort of poles that the robot is
using
to move in the environment and this
concept is
uh really
taking a biped system
to become quadruped to become whatever
pad needed for the stability as
we evolve in those environment and that
will make
at least the system capable in moving in
many
different three-dimensional environment
then
there are a lot of other challenges to
for the perception and for the planning
but
these are issues that can be addressed i
really think
the hardware is a big challenge unless
we
we build the hardwares that address the
problem of
torque control unless we deal with
the issues of the design of the robot
with integrated sensors with
realistic and effective
hands and grippers with
the autonomous uh power to
operate uh uh
beyond the the 20 minutes and 25 minutes
that
we we usually usually have with with the
humanoid robots
i think uh we we are going to
to to face a lot of difficulties unless
we
we achieve those those those uh
challenges with the technology of
building the machine i think the
challenges
in in in terms of software and and
algorithms i think there are there
but they
not as critical as the hardware because
we had made a lot of progress in
robotics
the field has matured to a point where
we have
much more capabilities in in the
software than
uh in the in the platforms so
uh the the the the
development that we are seeing today uh
all around the world in terms of
new platforms new hardware
is is very helpful but sometimes
some of those platforms are are missing
the key capabilities
that are needed and and this is what we
talked about
in terms of the communication and but we
are seeing much
more and more uh torque control force
control
silly elastic actuation all that are
leading to more compliance
uh in in the in these systems one
one other thing that we are seeing also
is
a great increase in the use of
robots in the medical field
with with
the integration of haptic devices to
operate those systems
and this goes back to the idea that
robots
are useful even if they are not
completely autonomous
that is robots can have
a lot of skills to be interfaced to
human
and the human are guiding those robots
at the right level at very high level
but are able to interact with them to
make them
very very useful in surgery in
operations in remote areas or in
different other
[Music]
applications underwater for the
underwater we
are operating those robots from the boat
on the boat you have uh you have the
same
uh haptic devices that we use in surgery
and now the operator is doing surgery
underwater
so so the whole concept is that
we should
[Music]
be open to the idea that we have
we have a robot that
are interacting under
some level of control of the human
guidance of the human
the robot is assisting the human but the
human is also assisting the robot
in some cases uh in
smart transportation we are
making use of the haptic in to to alert
the driver uh to augment the driver to
augment
the uh the experience of driving
and all of these things are creating
interaction between the
user and the machine in ways that
not only through switches
and sounds only but through a tactile
feedback
and and tactile interaction so
i see a lot of new applications coming
in all these fields and i think the
robotic technology is
is going in many different areas and
specializing for those applications
but the underlying technology with its
component
in the hardware in the software are
already the same
and as we approach the complexity of
humanoids we approach also
models similar to those of the human
but all of this is still at
a very low level compared to
the cognitive aspect that are needed in
building the full intelligent system so
the way i see it is that
we increasingly will bring autonomy to
those machines
and uh there are studies
at the higher level and somehow
this development will meet
at some point what is dangerous is
to do that work in separation we really
need to understand
the overlapping of those fields and
this is why we need
more interaction between those
communities
the ai community the robotics community
the developer
the designer and the the people who are
working on
the control architectures
so the the question we usually wrap up
with
is what's your advice to young people
who are interested in careers
don't hesitate this is a field
that is a lot of fun a lot of excitement
and it is it is actually
amazing i mean after so many years i
started by saying
uh i feel i'm i'm still working on my
phd
i really feel that i'm i'm i'm exploring
uh a
a road that that is it keeps stretching
and it is a great pleasure to do this
exploration because it is
we are after uh uh
things that relates to us to human to
to behavior to uh to actions and
uh we are discovering things all the
time and uh
i feel uh keep the imagination
and enjoy
a wonderful field robotics