before you started school oh so i'm
pradeep khosla
i'm the dean of engineering at carnegie
mellon
and to take you back into my life
history i was
born in india
and i went to school high school and
undergraduate school in india so my
undergraduation was from
indian institute of technology kharagpur
which is one of the five iit one of the
five iits and the oldest iit
and i graduated in 1980 um
and i decided that i was going to go
work in india for a while
because i was patriotic kind of stupidly
and i was in india working till 1982
in the area of real time control of
power systems
and then i decided it was time to come
to the us because i wanted to be a
faculty member i wanted to do research
so 82 i came to carnegie mellon
what was your degree my undergraduate
degree was in electrical engineering
but my honors thesis was in computer
science
it was designing a disassembler for
tdc316
which was the indian equivalent of pdp
8 i believe and where did you come to
school
i came here in 1982 to get a masters
and hopefully a phd and i was fortunate
enough that i got both at carnegie
mellon
so when i applied to the u.s i had
admissions in
three places and at carnegie mellon
there was a professor who listed
his research interests as computer
controller power systems
which is exactly what i was doing what i
wanted to do so i said okay
i'll come to carnegie mellon plus i had
a classmate of mine who had come here in
computer science
bud mishra who's now a faculty member at
courant
so i came here what was the relation
between
computer control power systems and
robotics
well it turns out that when i came here
that catalog i was reading was four
years old there was no web at that time
so people had moved on and robotics was
becoming very hot in 1980s when raj
ready
at carnegie mellon started the robotics
institute so there was a lot of
excitement a lot of people involved in
it
and so i decided to work in that area
because it was still computer control of
electromechanical systems
who was the initial professor that you
wanted to work with
so i came here to work with chuck newman
but that i did my master's with him
but chuck was a control systems guy so i
had a co-advisor
fritz springs who was in mechanical
engineering
so my master's thesis was seam tracking
for welding robots where the idea was
that if you're going to use these robots
for welding
then how would you track seams and so i
developed an algorithm
and then while i was doing my masters i
really got
interested in very interested in
robotics and for my phd i worked with
takeo canada he was my advisor for phd
what was your thesis so at that point
uh this is about 1983
late 83 takeo canada was working with
harry asada who's now a professor at mit
on developing what is called the cmu
direct drive arm one
so that arm was based on new technology
to a six degree of freedom manipulator
based on a new technology where there
were no gears so the motors were
directly coupled to the links
using uh samarium cobalt
motors high magnetic intensity motors
but that arm hung from the ceiling it
weighed like
i don't know six seven hundred eight
hundred pounds
uh so its payload capacity compared to
its weight was
way too small and we had this idea of
creating uh what is called a scara
configuration manipulator which would be
very
appropriate and useful for high speed
assembly so my phd thesis was cmu direct
drive arm 2
and that design was a scara design
and uh for example the adapt robot
is a scalar design and it's a direct
drive robot
the first as far as i know please
continue
the first car robot was made in japan
right before
yes okay did you have any connections
with no so what we wanted to do was uh
so we knew the advantages of scara
configuration
and we knew the advantage of direct
drive because
uh of lack of gears they were higher a
higher precision
faster so we combined the two to create
a new design for a
scara configuration robot that was
direct drive
what were the advantages of the scara
configuration
uh the scar configuration is
like the two links are horizontal in a
plane
so gravitational forces don't act on
these lengths
which means the motors don't have to uh
pick up the weight of the robot so they
could be sized smaller
and they have if their size smaller that
means the torque is used for speed and
direct drive configuration gives you
better precision
and what kinds of things have you been
using before
you had the cmu1 and cmu2 arm
all kinds of systems i was not using any
system that was not even in the business
right but this was actually a very
challenging uh phd thesis
uh i may not be exactly right but i
think i'm nearly right this was also the
first
experimental thesis where the thesis was
the design of this arm the
electromechanical design of this arm
now because this arm is higher speed
higher precision
it required higher sampling faster
sampling rates
and at that time there were no computers
fast enough to create
what we wanted was thousand hertz
sampling
so example the control system a thousand
hertz so i built
an architecture uh
multiple multi-processor architecture
using a special processor that had just
come out in the market it was called
marinko i remember that
it was a 64-bit word length
cpu and i had to write code
not in assembly but in bytes
i had to write code in a c f g because
that's all they had they did not have an
assembler at that time
so it was actually rather challenging so
we did that and we
for the first time demonstrated the
real-time control of a directory mr arm
at 500 hertz
uh now in the process of doing that it
turns out that
the computation of these
dynamical equations is extremely
intensive
which is why no computer could compute
those dynamical equations and sample at
500 hertz it took
longer than two milliseconds so that's
why we had to build the cpu
but there was also an implicit
assumption these where the dynamics of
uh the system are known so there's a
very famous scheme called the computer
torque scheme
which works extremely well if you know
the dynamics
or the dynamical parameters of the robot
so
the other contribution i had was demons
showing
theoretically that even though it's a
non-linear system
it is linear in the dynamical parameters
under some transformation
and then we implemented a real-time
estimation of dynamical parameters and
then using those parameters we
implemented real-time control
to show how computer torque really
worked very well
and who were some of the other people
that you worked with
during this project uh so the
my advisor was takeo canada so it was uh
him and i uh
taken myself uh we also had some staff
members
don schmitz uh was a staff member and he
was a mechanical designer
and an embedded systems person so he had
great contributions to make i mean
honestly without
his creativity and design i don't think
we would have built a system as good as
we did
uh there was uh another electronic
technician
very creative guy mark de lewis
so after your phd what did you do
so i finished my phd in 1986
and at that point i was looking for
faculty positions i had like several
offers
and cmu was one of the offers and
no better place to do robotics research
than cmu
secondly i had a lab already set up at
an experimental
setup set up and so i decided to just
stay here
what were the other places that you were
considering
uh it was current institute
it was university of maryland college
park it was
university of rochester
i can't remember the others might have
been caltech
what was the next project you worked on
after that
so i did two things
i continued on this direct drive arm
research
and we moved on to demonstrating
force control because that was a big
thing when robots come in contact with
the environment
there are forces of interaction that are
generated so the idea there is not just
to create
to control the position but to control
the force of interaction so that you
know
you can slide around an object you can
pick an object
so you can see a lot this has a big
impact on
grasping research for example designing
hands
it has and the fundamental work in force
control
was done by mark rayburn
at mit uh i believe it was
part of his uh wait i'm trying to is
that mark rayburt or matt mason
one of those two so don't uh it was
called compliance control
it's a very fundamental piece of work
that showed that you could
actually when a robot interacts with
a surface or an environment there are
certain degrees of freedom
where you can control the position so if
you think about a surface and i'm
touching it
out here i'm allowed to control the
position but going
into my palm i can only control the
forces because i can't control the
position
so they had this model of separating the
space into where positions could be
controlled and where
forces could be controlled and that was
called compliance control
and robots to become useful had to be
compliant
with the environment so so we
demonstrated again for the first time i
believe
uh high sample rate force control
strategies
but the most interesting part was my
first phd student was richard volpe
who's now at jet propulsion laboratory
and richard was a phd student in the
department of physics
and i was a professor in electrical and
computer engineering
when richard wanted to do his phd in
robotics
so i went and talked to his advisor in
physics and we agreed that
i would become the co-advisor and this
is carnegie mellon you know we allow
faculty to advise just about across the
whole university
so richard was the one who did work in
force control
for his phd and with a physics
background he brought
a wealth of knowledge and a style of
thinking that
typically would not have existed so
one of his he had several contributions
but one of his contributions was
showing the equivalence of impedance
control and force control
so at that time there were two camps one
was
impedance control which was neville
hogan and the other
people were doing force control and
there was some impression that these are
different strategies and what i wanted
to do was run
the equivalent of a touring test which
is if i implement a force control
strategy
and a impedance control strategy and i
don't tell you which one is which can
you tell the difference
and we demonstrated that there's
actually an ignorance between the two
which was rather interesting
what kind of tasks did you demonstrate
it in
what's the robot doing but this in this
case it was more like following a
surface and there was also a strategy
for impact control like
when a robot came in contact with the
environment at high speed
could you do it in a way that it just
touched and never bounced around so if
you imagine two stiff objects when you
hit
one against the other there's bouncing
so we wanted to
demonstrate what is impact control which
is there was no bouncing
right and that would then lead
automatically to compliance compliant
control
what was the reaction of the community
to your
you know that i think uh when you look
at
research uh
and there are a lot of young people
involved the initial reaction
there's always this notion of trying to
manage and maintain
your space in the community but looking
back 30 years 25 years
the reaction is more reasonable more
muted and more rational i believe so
one cannot judge the reaction to
research at the instant you publish the
paper
so what's some other projects you worked
on so after that
i had this notion that when they're
following way
so if you think about computers
they're called general purpose machines
where the idea is that
any function that can be computed can be
implemented on any general purpose
computer
now people talked about general purpose
six degree of freedom robots
and intuitively it's obvious that not
every task
that can be done by a six degree of
freedom robot a
can be done by a six degree of freedom
robot b a
and b being two different configurations
so that led to this
uh both scientific and philosophical
question in my mind as to why are these
called general purpose robots
because they're both six degree of
freedom robots i call them general
purpose but i know that
this robot a can do a task and this
robot we cannot do that same task
right so clearly there's a problem so i
started thinking about what would a
general purpose robot look like
and to develop that notion so what i
wanted to really do
was build what i think of as a touring
robot
just like a touring machine so
i came up with this notion of a
reconfigurable modular manipulator
system
uh which was a project again i started
initially with the takeover
so the project was called cmu
reconfigurable modular manipulator
system
which went the following way which said
that
if you give me a set of tasks a finite
set of tasks
let's call it set t and if you give me
and if i create a set of modules
set m these are like independent these
are like lego independent modules
then if i can demonstrate that for every
task you pick from this set
i can pick a robot from this module set
i can build a robot from that module's
head
which would do that task then in some
sense you would have defined a general
purpose robot
which would be a set which would be the
box of lego vis-a-vis the set of tasks
that has been predefined
so the only constraint was the set had
to be finite uh
right but it didn't put any constraints
on the number
of tasks and that said it would be
million two million five million just
could not be incorrect
so conceptually this made a lot of sense
to me but it had significant problems so
now we have to think about design
methodologies
if you pick a set of a task from here
how do i figure out what kinematic
configuration works
right so let's assume i have a method to
figure that out because that was part of
our research
on designing uh mapping tasks to robots
and that was chris paredes phd thesis
who's now a professor at
georgia tech and that was some
very interesting work uh that was done
so the next question is so so let's
assume that problem is solved so you
give me a
set here or set there i can pick a task
and i can tell you what kinematic
configuration works
so we put it together just like you
would put
lego together so then the task was how
do i design these modules so that i can
just put it together
and again don schmitz was involved in
helping me design this system
and the idea was that whenever you
connect two modules together
power data communications automatically
get connected
so when the system is configured the
system is totally connected there's
nothing else to do
but i'm still left with a problem of
programming the system i have to figure
out the kinematics the forward
kinematics inverse kinematics the
control strategies
right so we had a project which would
automatically determine kinematic
configurations
so as soon as you build a robot every
module was a smart module it
knew about itself but it didn't know
about
anybody else and then we had algorithms
where as soon as the system was built
they would all transmit what they knew
about themselves and the system would
figure out what this kinematic
configuration looks like what are the
forward and the inverse kinematics
and that was a master's thesis uh laura
kelmar so
i have lost track of herself where she
is
which was fine too but i'm still left
with the problem
of controlling the system now
automatically determining control
strategies
uh and then we
realized that this was a non-trivial
problem because to do this right
you had to build you had to have
capabilities in the operating system
that would allow me to configure
controllers automatically
and there was no operating system at
that time that allowed us to do this
so my phd student david stewart uh
his project was called camera
c-h-i-m-e-r-a
it was a real-time modular operating
system
where this operating system allowed you
to create
or assemble modular code so the idea was
just like i'm assembling this robot
using lego blocks
if i had uh lego block equivalent of
software pieces could i then connect
them together automatically
and create a controller for the system
automatically so
camera was an environment that allowed
us to do this and we demonstrated how
this is going to be how this would be
done
on a physical system so all my life it's
been all about
theoretical work uh mapped and or
supported by experiments
so we demonstrated this it was really
really successful
and if you look today a lot of
work in modular robots
in different shapes sizes forms across
the country across the world
and i believe that the work we did
was probably the first or the second
work in the world in that area and that
has laid the basis
or at least demonstrated you can do it
modular systems of today
look different than what we had
but nonetheless it showed the utility of
these systems and how they could be done
so once we had demonstrated this system
uh the next goal was to say okay can a
high school dropout
program this robot because at that point
and even today to some extent you
literally need a phd in embedded systems
or a phd in something to program robots
that doesn't make sense if they're going
to be
useful so we developed this iconic
programming language called anika
and that was matthew gertz it was his
phd thesis
where the idea was that
somebody with an understanding of the
task but no understanding of the robot
or the technology could program
so these this iconic programming
language was puzzle pieces
so each icon was color coded
and shape colored so it looked like a
puzzle piece like a thousand piece
puzzle
uh so the color coding so the shape
coding was
for colorblind people to be able to
connect them together
so when you brought two pieces together
that didn't make sense where the puzzle
pieces didn't fit in
the system would reject it automatically
so it was an
aid to help you program too now because
you could not just bring any two pieces
together and create arbitrary programs
because
they had to follow a certain sequence so
now
every one of these puzzle pieces in the
back in the background
had sophisticated code behind it so when
i brought two puzzle pieces together
the code got combined automatically
based on this
real-time operating system camera that
allowed me to do this
so now i could program a robot at a very
high level it was all seamless the code
got configured automatically
the controllers got kind of configured
automatically the kinematics got
configured automatically
and you just do the task and we
demonstrated that with
and what level was the code so one
puzzle piece what would it
be for example what would it stand for
uh
one puzzle piece for example might be
move the other then you would have like
move
left right whatever that's right
and then uh how would that compare to
say that the lego mindstorms kind of
programming environment that they
developed
yeah so conceptually it is similar but
remember this piece of work is
nineteen
right uh so then after that we got uh
interested in uh swarm robotics
but it was a little bit
so the okay so the idea then was uh
so 94 95 96 i spent three years at darpa
and there i managed large programs in
robotics
in real time control and in
manufacturing automation and design
engineering and that gave me
a broader insight into two things one
was the needs of dod
and how we would have used or we would
want to use robots in
warfare and second was capabilities of
several universities who were all my
contractors like stanford like berkeley
like u-pad including companies like
lockheed martin boeing northrop grandma
these all
they're all my contractors so that was a
spectacular three years
where using my own background i could
influence the direction of the field
and at the same time using collective
intelligence
of the community uh i could learn a lot
more
and influence the direction of how
robots were going to be used in
uh warfare how would you how did you try
to shape that
direction how would you characterize
what you sought to achieve while you
were at darpa
so the real-time planning and control
program for example was a
more science-based program developing
algorithms and so on and so forth
there was another program which was
called small unit operations
where the idea was how would you empower
a soldier
so at that time uh there was this
view in dod which said that conventional
techniques of battalions and platoons is
not going to work we need to have small
units like six to eight people
special operations who will do something
interesting and useful somewhere now to
empower these people
because they're in a very dangerous zone
you need to be able to deploy
forward-looking sensors you need to be
able to collect information
so the idea then was that could we
create robots small enough
that could be thrown then one robot
would
give you vision information like camera
the other might give you thermal
information
infrared whatever so that was
the program there so when i came back to
carnegie mellon
dick urban who was a program manager
also at darpa at that time
had another thought where he wanted to
combine
uh small form factors
rob we wanted to build small four factor
robots so dick told me perdi can you
build a robot five centimeters on the
side
right now sounds easy like there are
many robots right now which are
that size but this is again 1996
long time ago so at that point we built
these robots about
this size and they were called millibots
right now clearly in a robot this size
they were wheeled
robots so you could not pack all
capability that you needed
so we had this vision which went the
following
which said that no single component of
this
system has to be able to do everything
but
as a system it has to be able to do
everything that means
there might be in this system of
millibars robots that have ir sensors
there might be robots that have sonar
sensors that might be robots that have
uh camera sensors
right so each one so think of this
uh system of robots being several blind
men
feeling an elephant but they have to be
communicating with each other
so that the system operator me
knows that it's an elephant right so the
vision here was
can you have multiple robots behave like
a single logical machine
so i want to interact just with one
machine
so as soon as i give a command to that
machine says tell me what's in this room
the system should automatically figure
out what are the capabilities of each
one of these
subsystems it has who can do what create
code
dump it on that let them do it collect
the information and
interact with me as a single logical
system
so this was now an extension of this
modular robot that i was talking about
but in a slightly different way where i
had several
independent systems who were going to
behave like a single logical machine
so that's the system we built it's
called millibars and there was a
scientific american article that we
wrote on that which
is pretty well cited so what would you
say was the overall role of darpa in
shaping
robotics both while you were there but
sort of in the
whole history of robotics i think if
there's
one agency that can take credit for
okay so no agency can take all the
credit but if there's one agency that
can take credit for pushing the
technology at the cutting edge
uh it is darpa uh darpa had a specta
a very big role in robotics in general
but if you look at for example the
autonomous land vehicle
program right without darpa
there would not be any grand challenge
one or the urban grand challenge or
what was the other grand challenge
called there were two main challenges
recently
in the last two years those grand
challenges
we were able to demonstrate capabilities
because of
20 30 years of darpa investment in
autonomous land vehicles for example
similarly darpa made investments in
manufacturing and automatic assembly
uh and tarpon made investments in the
next generation controller so darpa has
had a history
more than a 30-year history i believe a
30-year history of
investments in robotics that have
brought us to where we are
and from your experience of being in
darpa
how does the policy making about where
robotics is going
um how is it made i mean do you have
meetings
obviously you mentioned having um
a broader view of what your colleagues
are doing in various places
but how does the interaction with for
example the military happen or
what are their meetings or what does it
look like
yeah so so there are meetings but the
way the interaction happens is darpa
brings
program managers who are domain experts
mems robotics computer vision you know
solid state physics whatever doesn't
matter and darpa has
a mission which says that
they want to be doing cutting-edge
research but of
relevance and application to the
military but not necessarily in one year
could be five
ten years right so as a program manager
you know what your
passion is you know what the mission is
of the agency
and you have access to just about
every other agency in the country
including the three ladder agencies
besides the dod so what you're expected
to do is
go talk to various people figure out
what the needs are
understand what the state of the art is
understand how much can it be pushed to
satisfy this need
and build a program plan which then you
pitch to
either your office director and to the
director of darpa so there's layers of
not that many just two layers of
other smart people who look at it and
say yeah that makes sense but have you
thought about this have you thought
about that so there might be a couple of
iterations
and once the program is approved then
you go
uh put out the ba and you run it like a
project
unlike uh the national science
foundation where you are funded you are
funded i'm funded and we are all doing
our own work
darpa we would put together just like my
modular robot
a project with you doing component day
you're doing component b
somebody doing component c and the
program manager would
have the overall oversight to make sure
these components connect and when they
all put together they solve a problem
that is of interest and relevance to the
country and especially the dod
but the people who are doing the
projects aren't necessarily aware of the
larger
no absolutely they are because when you
build
a vision for a program like that you
cannot build it
top down you talk to the customers to
see what their needs are
you talk to the community to see what
the state of the art is and what their
passion
is so it doesn't make sense to define a
program
where nobody in the community wants to
work on it and we want the best minds to
work on it
and the only way you get best minds to
work on it is to have a challenging
problem
and give them freedom and empower them
to do great stuff
so it's a delicate balance between uh
enabling them significantly total
freedom
versus getting your job done so it's a
so i learned
a lot in terms of uh project management
managing people
and these are spec these are the best of
the best so
and so how do you when you're kind of
making the decision of how it's
something that looks much like very
basic
science how do you make a decision of uh
whether it's going to have an actual
impact on application in five to ten
years
so this is why darpa wants to hire
people who are
top-notch domain experts in the area
right because these people have a vision
these people have good taste
they have a sense of quality they know
what can be done and how long so
it's intuition it's experience it is
experience of other people all coming
together
and how did you before you started
working in darpa for the three years
was your funding previously
generally from darpa was it from
industry yeah i had money from both nsf
and darpa but it was
purely accidental that i started working
for darpa where
uh gary denman who was the director
so i joined darpa in 1994 january
so gary denman i think it might have
been 93 june or something
came to visit carnegie mellon he was the
director of darpa
and i remember this clearly i pitched to
him
this project camera the next generation
operating system and a programming
environment and how it was going to
change
everything we do in embedded systems and
rightly or wrongly for some bizarre
reason he liked it
and he said how much money would you
want for it and uh
i forget what the number was i said like
10 million or 5 million dollars
something it was a big number
way beyond a professor or seven years
out of phd would say
and his response back to me was why
don't you just come
work for darpa implement this you might
get 3x the money
and the world will solve the problem for
you and
just at that time in 93 i had just uh
been promoted and i was looking for the
next thing to do
and this came at the right time so i
talked to a couple of my mentors and
they said hey this is a great thing go
do it
so i took three years off and i said
okay i'm here gary
tell me what you need to do
so did you get the did you get the
funding for the camaro
at that point no so you know camera was
just one project
right so the funding was real-time
planning and control that was a program
which involved planning algorithms
control algorithms
the software and the underlying software
basis it was much broader
than just one real-time operating system
and after
darpa when you came back to cmu did you
continue
to implement that program uh no so that
when i came back to cmu that's when i
did this project on milliebots
right which was the next generation of
modular systems where
you would have multiple physical systems
behave like a single logical machine
and so what came after the millibots
uh so after the lilybots what happened
is i started getting interested
as you can see so my career involved
electrical and computer engineering it
involved
software engineering it involved
mechanical design
and embedded systems so towards the end
of melee bots i started realizing that
once you have many of these systems out
there
where the code gets generated
automatically the code gets downloaded
the probability that one of these
systems could be taken over by an
adversary is extremely high
so this is now 98.99
so that led me into this whole notion of
security and embedded software so how do
i make sure
that one of these machines subsystems
does not turn
rogue on me right because that would be
a bad
thing and from there on i just got
interested in security and embedded
systems
and in 99 i became department head of
electrical and computer engineering
and as a new department head i was
looking at the department thinking about
what's the next big thing we should be
doing
and i realized this computer security
for embedded software is going to be big
things so i created a center which now
is scilab
it is the largest cyber security
research center
in the country at a university so i just
got interested more in
software security and uh
and who are some of the people that you
collaborate with
on that research uh adrian perrig uh
who's a professor in uh at carnegie
mellon
uh he's one of my main collaborators and
the other collaborators
rohit nagy but rohit is an information
theory guy
so with him i collaborate or i have with
the ph 2 phd students on
a sensor network so if you think about
these little millibars that i call them
and they start running around you can
think of them as
deployable mobile sensor networks
right so now the fundamental question is
okay so if i have
enough of these i know i can sense
enough of the environment but i don't
know
what the sensing capacity is so just
like a channel has information capacity
like sharon's theorem so we got
interested in what is the sensing
capacity of this system
so rohit was an information theory guy
brought to bear
his knowledge and we have some results
on what would this capacity look like
so we did a lot of work so we've been
doing work in uh
sensor networks and what are
some of the special challenges for
security in embedded systems versus
more traditional computer systems
i don't think so on a we're at a very
low
technical level there are uh the
problems are different the challenges
are pretty much the same
uh but in embedded systems uh so for
take power systems for example
a lot of cpus and that are used for
power system management and control are
embedded cpus
and uh the problem there is that the
control
critical infrastructure like the cpu in
your car
is an embedded cpu it's an embedded
software system it controls your car
if that gets taken over and your car
does to do things it starts to do things
that you don't want to do it could be
bad for everybody same with power
systems right so it's the
implications that are different not the
fundamental challenges are still
uh security of software systems
and when you so it seems like in terms
of the general application of
your work initially you were doing a lot
of things that were more
kind of industrially oriented oriented
towards industrial application
right and then after darpa now you're
looking at some other kinds of
applications so you can see the
transformation so there's a project i
didn't talk about where
it was a similar notion as a modular
system where
uh people it was called the troika bot
project uh where there was a
system of three puma robots like in a
triangular configuration
that westinghouse wanted to get rid of
but i had an interest in getting so they
gave it to me
and where people
were talking about robotic assembly
where all the assembly would happen with
robots
but clearly programming these systems
was painful
right and you can so at that point uh we
had this vision of
here's what should work if i build a cad
model of an assembly right
what should and i put it into some
system what should come out of the other
end
is a program to assemble that from
components
and then i should download this program
to these to the system of troikabot and
demonstrate automatic assembly and we
actually did that project
which had geometric reasoning involved
in it because now you take a cad model
you reason on the cad model to figure
out how it's going to be disassembled
then you more more often than not do a
reverse
sequence which creates the assembly
sequence right
which tells me what motions i should
have in the robots
one once i have that i have this iconic
programming language that i talked about
which at a high level and encapsulate
those motions i bring them together
right the software gets created
automatically i decide which robot this
is the software goes on which one of
these three
and then out comes the assembly so we
did a big project that lasted for like
five years in that area so
yeah so my pre-darpa was
more industrial applications but more
thinking about this touring robot and
even today i mean
very few people at least from what i can
tell have been pushing this notion of
what a touring robot looks like we build
these robots to solve a task or two
but we have never really addressed the
problem of what would this
if there was a touring robot what would
it look like and then post darpa
was mobile robots milibots
making multiple robots act like a single
logical machine
right more uh field slash dod oriented
right and 1990 2000 onwards it's more
uh software security and better than
secure uh
systems i'm sorry security for better
systems
and that's also the time when i became
department head 2004 i became dean
so my interest and expanded to building
organizations
that could accomplish these big goals
and tasks without me
having to do all the work on my own as
the
as the dean how do you how do you see
the position of the robotics institute
relative to other robotics programs
i think
the robotics institute over the last 30
years has made
a significant impact in the whole on the
whole community
there was a time when the robotics
institute
was the dominant dog when it started
then there came a period when mit
stanford also
had great michigan had great robotics
programs
uh and then there came a period after
that
where many of these programs kind of
got diluted not in the quality sense but
more in the sense of looking at other
issues in life or in the research area
and i think we're at a point right now
where there are more robotics
institutes quote-unquote in the country
than we ever had before and many of
these are populated by
students that graduate from here and the
robotics
scenario in the country the research
situation
on one hand looks bad when it comes to
the amount of funding
but on the other hand if you look at the
amount of technology and the impact it
has created
is purely amazing because if you think
about computer vision as part of
robotics
a lot of visualization a lot of
hci type of technology came out of that
as the core
so robotics now is more diffused
as an area and it's made and this has a
very big impact i think
how do you see kind of culturally and
socially the trajectory
changes in the robotics institute you've
been here since almost the very
beginning so uh so
you know every organization when you
start
there is a core group there is a very
strong camaraderie
there's only so many people that you
have to interact with that you have to
remember
and you know just about everything about
nearly everything that somebody else is
doing so you're really well informed
right uh so that was robotics institute
today where robotics institute is about
60 50 million a year operation
it has got components like the nrac the
national robotics
center it has components like social
robotics
it has more faculty than even some
departments at carnegie mellon
so it's bigger it's more diffuse having
said that
i don't remember everybody's name which
is unfortunate i don't know what
everybody else does but i know
approximately what's going on
but at the same time it is higher
momentum higher energy
it is more exciting where i need to work
with somebody in
uh modeling cloth dynamics
there is somebody jessica hodgins for
example i go talk to her and
just more exciting and do you remember
when the phd
program started yeah actually i was
involved in defining the phd
program let's see if i can remember i
might have to look at my
resume to figure out the date
uh but that was but how was it decided
to start a phd program and who were some
of the people who were involved in that
so there was a committee typically at
carnegie valley was very entrepreneurial
place we said up you know giving a phd
in robotics would be good so we put a
committee together and i was
one of the committee members when we
just defined a phd program
end of story yeah
unlike unlike most universities where
you would spend like two years
evaluating it i think it was less than
six months we defined a phd program but
how did you decide that then was the
time
to have a phd program in robotics
because at that point the field had so
if you look at a phd program it has like
two components mainly
one is uh what what one would call the
qualifier or the coursework component
and the other is the thesis component
now at a place like carnegie mellon you
could do the thesis like i did in
electrical and computer engineering and
you cannot tell the difference but we at
that point had the feeling
that there were a set of core
courses that described the domain
and having some confidence in them was
necessary
right so these were courses and
perception cognition and manipulation
was the three areas initially
defined just like uh electrical
engineering would define signal
processing
solid state you know so i think these
were the three areas
and so that's why we created the phd
program it was more to create a
common base of knowledge on which you
could build your research capability on
and insur you've been another besides
darpa you've been another
government and policy making what would
you call them
i'm not agencies or well you were just
talking about
um being with president mcrobbie and
oh so so darpa i spent full time
nearly three years then i am
on the advisory board of for example
csiro australia
uh nist in the u.s
i was on the advisory board of
ucaf program which was the unmanned
combat air vehicle program
where starting from just
autonomous flying machines
to literally fighter aircraft being
unmanned and autonomous
that was a program so i've been on
observing advisory boards of several
universities government programs
government agencies and including
companies
and so what do you see as some of the
future challenges in robotics or
where is the field going i think
i think when the field started uh
manipulation was the basis i mean
creating a robot that could do
human-like talent like assembly
like window washing whatever then it
morphed into
[Music]
mobile vehicles the autonomous vehicles
and in doing so
computer vision became a big part of
robotics
but then at some point computer vision
by itself
had offshoots like visualization like
human computer interaction
so today i think of computer vision as
robotics but there are some people who
think of that as computer vision
they don't think of that as robotics per
se
right so
robotics today is very diffuse uh so in
my mind so
people ask me like you know what's the
definition of a robot that's
whatever you want it to be it's the
definition of a robot
it's like defining pornography you know
when i see it i'll tell you
so when i see a robot i'll tell you it's
a robot and i
robotics or not and how do you see
the robotics work in the us um as
compared to
robotics for example in japan or europe
are there
particular differences i so there are
differences but there are also i i
see the work in the u.s and in japan
both being extremely exciting
uh the difference
is that in japan there are
companies that are committed to robotics
and
they develop technology which
at a superficial level doesn't look like
it has applications
but it has many applications i'll give
you an example a
company that developed a piano playing
robot
now if you think about it it is a
non-trivial process to develop a
completely articulated system that can
play piano
pretty much similar to a reasonable
level expertise and
of a pianist right a company that
developed
robots as pets i thought it was rather
futuristic for somebody to think about
that instead of having
chia pads which are these plants i could
have
a robot pad which could have some
emotional interaction with me if it's as
simple as blinking a light
to smiling to right uh pet dogs
you would not see that happen in the u.s
at least i don't know of any company
that would do it
in the u.s but there are these japanese
companies that would do it they create
markets like the sony uh
dog robot i mean that created a market
sony made good money
so i think that's the difference so at
the university level it's pretty much
similar type of research but
when it comes to some really creative
far-round
far-out applications the companies are
willing to spend
i also think that when it comes to
mechanical design
japan focuses a lot more
on the mechanical design of these
systems than the us does
and what about europe or now korea is
kind of korea right so they're all in
play right now so i think the real
big impact of robotics is going to be
what we at carnegie mellon called
quality of life
where can we improve the quality of life
of either disabled people
or older people using these technologies
and some of these technologies don't
look like robots anymore so for example
a
camera sitting in my mother's house in
india so i can monitor if she's okay or
not
sitting in the u.s you might think it's
not robotics it's camera but
the technology of computer vision of
processing that information was created
in the context of robotics do you
maintain your
um connections with iit and have you
seen the diffusion of robotic technology
in india i have not seen
as much there is clearly more robotics
in india today than there was
30 years ago which was non-existent but
as i look at india today
the focus is not as much on robotics
technologies it's on
web-based systems it's on i.t systems
it's
a whole lot of staff but i don't see
great mechanical design
research in robotics being done in india
i don't see the next generation software
architecture research being done in
indian robotics right
where it's been done in japan it's been
done in the u.s it's been done
europe especially france and germany
those are the two
major players i think
oh sorry go ahead do you have a
follow-up no it's not a fall if i was
going to change too
is this going okay
so you mentioned that you were on the
advisory committee for the combat
foreign i was that that program died
they killed the program so um but um
what do you see as some of the social
and ethical issues about army
autonomous robotics and
i don't know if i see uh
ethical issues the way we use robots
today because
even though we think of them as
autonomous systems
there is still a human being behind
the decision-making process they're not
just making up their own decisions
so what we have done is
remove the human being much farther from
the sphere of harm so if you think about
afghanistan and
the drones that we use to shoot missiles
right so instead of a pilot uh
flying the aircraft and have the
possibility of being shot at now we are
flying a drone with missile loaded on it
but back there somewhere is a human
being who's going to
make that decision on when to fire and
who to fire on right
it's not clear to me that there are any
ethical issues
in this context
removing the human from the loop and
having a fully autonomous system with
those
issues uh i
can see us human beings as
becoming nervous i can see
us human beings as thinking here's
another society of
intelligent inorganic systems that could
take over
and we have seen that in many sci-fi
movies but
sometimes i wonder about if i look at
the evolution
of this world right it's clear human
beings were not the first creatures
in the evolution process there were
single cell creatures
multiple cell creatures and so on and so
forth
now i don't know much about whether
these creatures can think or not
i don't know what they felt when human
beings came
into being and started hunting and
destroying these people so there is a
philosophical argument where some of us
believe not necessarily me that human
beings are
the highest level of evolution not
clear to me because by that definition
evolution would have to stop so if i buy
evolution i don't know what the next
generation is going to look like so i
don't know this is an expert would you
be comfortable with
having autonomous um
i don't know autonomous vehicles a lot
of different kinds of autonomous
applications
um out there both in you know in
civilian applications also in
military applications yeah actually i
would be so for example in civilian
applications the darpa
urban challenge right so if you look at
driving more people die
in road accidents than for example die
in
airplane accidents in this country and
across the world
so dying and then more people are maimed
and
disfigured and disabled because of road
accidents
why would we not want to use
technologies
that have been demonstrated to be safe
and not cause any harm to a third party
that would improve our
accident ratio i mean that would reduce
the number of access to zero why would
we not want
to have a goal of zero accidents on the
highways
if autonomous system if we can create
autonomous systems technology that
leads us there i think it's spectacular
you know we as
uh brothers sisters parents
would never have to worry about our
family member
not coming home on time and wondering if
he or she was in an accident
i see that to be a great thing i have no
ethical issues
chuck was just telling us about one of
their early
demos the autonomous land vehicle and it
had
they had put his three-year-old son
right in front of this thing as it was
going of course there was a person
inside there
um but do you think that there's a
possibility
of you know coming to the point where
you could
could have that kind of technology going
with people you know passing through
absolutely
no human controller
in a sense i can see it when do you
think
that i know it's really
so i think i think there are two issues
one is
does the technology exist to demonstrate
that
and is there enough of a wherewithal
for us to accept that right those are
two different barriers
i think the technology exists
today it could be made robust enough in
five years
that on highway uh at cruising speeds we
can see autonomous vehicles
right but for us human beings to have
100
confidence that nothing ever is going to
go wrong that bar is higher
so that bar uh
i think in about 10 years you would see
it on highways where we would have cars
in fact we already see cars with
infrared and radar
on the
for obstacle detection in the car lane
changing systems
these are all incremental technologies
that are helping us make driving safer
and there will come a time and i think
it's less than 10 years when these would
be integrated
and a car would be autonomous and i
would be in the car being driven by the
car
reading my email texting a lot safer
situation than me driving and texting
[Laughter]
and do you think in japan there's a lot
of
what i see is kind of deliberate work
being done by
you know the government by um by various
companies to present robots as this
thing that we can have in our daily life
right um do you see that as at all being
something that's also being consciously
done in the us uh
yes what about these vacuum cleaner
robots you know they look like these
dust balls that run around collecting
dust it's a very simple technology
right but just like the example of cars
you take somebody does the vacuuming for
you
then the dishwasher is smarter the
refrigerator knows when there's no mail
and you kind of incrementally add it all
together
and now you have a house where you could
be sitting here
and on your way home your grocery list
could be generated you would approve
it some third party would deliver it
right i mean there are companies that
deliver groceries
and your life your quality of life will
be very different i think a lot better
because now
if you're interested in social sciences
or in filmmaking you would have more
time
to spend on that instead of worrying
about groceries worrying about driving
worrying about a whole lot of other
stuff
and for a young person who's interested
in robotics and
having a career in robotics what would
you tell them i tell them go for it
it is as good as it gets it is very
exciting it's very open-ended and you
get to define your future
i have one quick question that goes way
back but
just because um it stuck with me but i
mean cme obviously made a huge
commitment to robotics that's you know
different potentially from any
any place that i've seen do you have a
feeling for why they decided to make
that commitment
yeah actually it's a
so so that's a really good question it's
actually part of cmu strategies so
in the last 40 years
there has been no other university in
the country that has
come up so far and so fast
like from literally a regional school to
an international
university ranked on top 10 in the world
right now
this rise has not existed in any other
school
and this started with dick syrette who
was our president in the 70s and he had
this philosophy of comparative advantage
where the idea was look we are a small
school
we will not do everything but whatever
we pick and choose to do
we would do it so well that we are in
the top few in the world in that area
and this then led us to pick and choose
areas and make big bets on these areas
uh robotics institute was one of them
developing or creating a school of
computer science was the other
in a different field creating a data
storage system center which is the
premier center in the country
doing data storage research so there's
hardly a disk drive that's manufactured
today that does not have technology from
that center
more recently creating scilab uh when we
created scilab
we had no professor doing research in
security there was one who left
so we started from scratch and we built
an operation which is the largest in the
country in
less than five years we had that
operation built because we made a big
bet
and we've been fortunate that we've
always won on these bats we've made an
impact one of these days one of these
bets will go wrong
but that's a risk we are able and
willing to take
and you know why robotics in particular
at that point
because raj ready had a vision of what
this could do
just like i had a vision before 9 11 of
why
cyber security was going to be important
and nobody was talking about it
just like mark kreider had a vision for
why storage technology was going to be
as important as computing
right so you've got to bet on people and
on their vision
and just make it happen
about his vision oh yeah raj is a great
guy
great great are we good yeah is there
anything like that
this is great it just reminded me of my
life
that's a good thing yeah i actually you
know i never
think about who did what for a project
and how did it go
my career evolved