so we're just gonna be asking about your
career and
experiences in robotics great i'm
happy to talk about it so can you start
by telling us
where you're born where you grew up i
was born in nigeria
and only lived there for six months
my family was there my parents were
there as teachers
and a little village called
kenny so we
and it's always amazed me that they
decided to have a baby
in this remote village that didn't even
have running water when they first got
there
but my father was an engineer
metallurgist and my mother was a teacher
psychologist and so they we lived there
until for about six months then my mom
said we gotta get out of here
and uh they had netting over my bed
because there was spiders and scorpions
and all kinds of stuff like that so
to keep them off of the baby and then we
moved back to
columbus ohio my father did his phd
there at the
at ohio state and then we moved to
bethlehem pennsylvania which was a steel
town
he was a steel and good research and
steel
so that's where i grew up so it's
a little town of pennsylvania where did
you do your undergraduate studies
so then i did undergrad at penn which
was actually
where both my parents had attended as
well and i studied
i started out actually doing metallurgy
like my father but i realized i didn't
like it
and so i ditched that and um i
wandered over to the ee department ended
up
doing majoring in electrical engineering
but
one part of that story well i took a
year off i did a year abroad
and my junior year in scotland and
that's when i discovered
actually the way it worked there you had
a they had a fair
different types of courses you could
take and i was wandering around there
and there was a guy
sitting at this table with a.i
and so i took a full year course in ai
had aspects of robotics and all these
different parts of it and that's when i
sort of discovered robotics but when i
came back i never forget i was walking
in the
electrical engineering building at penn
and i saw a sign for the esp lab
i thought the esp they're doing research
and esp here i just have to check this
out
so i went and it turned out to be a
robotics lab
and esp stood for experimental sensory
processing
and it was rojana baichi's lab so that's
she was the one who
started me off on during research and
robotics and i did as an undergrad i
just hung out with the grad students for
the last two years i was there
so then after undergraduate where did
you go
um then i went to carnegie mellon and i
worked
with two faculty there
mark rayburt who was doing the the
running machines
and he was also doing tactile sensing i
was very interested in
how do you give robots a sense of touch
so there are always interesting problems
it's still not a solved problem actually
is
the having a reliable sensor human sense
of touch is incredible
that we can detect a very microscopic
little ridge if you run your finger
across it you don't have anything that
can do that
as a robotic device and but when i was
at carnegie mellon i met matt mason
and he was doing work on the theory of
manipulation
and much more of the mathematical
modeling and geometry and i
really got excited by that and that's
what i ended up doing my dissertation
so what was your dissertation project
distribution was um
was on the mechanics of manipulation
so what we looked at was the problem of
orienting parts grasping and orienting
parts
and in particular what we looked at was
a very simple gripper
the parallel jaw gripper and that's the
simplest gripper
actually that's one it just opens and
closes and just binary on off
just but you can pick things up with
them in fact we argue that you could do
remarkable things because if you look at
someone eating chinese food
with chopsticks right that's you know
parallel jaw gripper basically
we started looking at how you can use it
to orient parts appointment factory
assembly line
and we noticed the first thing was that
the
um that you can parts could get caught
in certain
unstable configurations and this was due
to
frictional properties of the gripper and
it turned out that if you could remove
the friction
you could avoid these unstable
configurations the problem was then when
you tried to lift it
off the table the part would slip away
so we ended up coming up with this
i had this mechanism that i designed
that was just purely mechanical
that would basically have low friction
with linear bearings
but would have high friction inside so
that as you lift to the part it would be
stable
and so that was um we i got a patent for
that
we followed the patent models at
carnegie mellon then it was issued a
couple years later
but my dissertation was on the
mathematics of orienting parts
in the plane and so we looked at the
idea of how these grippers could
go through a sequence of operations and
have the part end up in a unique final
configuration
even though you never sense the
orientation of the part
and here's here's the analogy is let's
say
i i want you to put a ball there's a
ball someone in a room
and you need to put that ball in the
center of the room now you're not
allowed to
um i'm not going to tell you where the
ball is
it's a very interesting problem right
because you first think then it's
impossible how can i get the ball to the
center of the room but the answer is
you could imagine where you just you you
sweep
from the outside toward the center and
if you do that carefully at the end
the ball will be at the center of the
room even though you've never sensed it
and that was sort of the idea we used
with the geometry of grasping and we we
were able to
show that you could generate these
sequences of gripper angle
grasps on as long as you knew the shape
of the part
you could go through these operations
without ever sensing what was going on
and the part would end up in a unique
final orientation
it was like a a sort of mathematical
puzzle
and then we showed that you could
compute that for any any shape any
polygonal shape
and that there was always a solution for
every polygon
so that was the dissertation topic
great now what was the next project you
worked on after that
well let's see i did a little work also
on stochastic
manipulation when i was at cmu which was
looking at um probabilistic models
so you didn't know again you didn't know
exactly what's going on
i would say the fundamental issue just a
step back of
in robotics is uncertainty that the
world is just very uncertain
we are very very graceful and adept at
dealing with that
you know ever we do it every day all day
long
and we just have techniques to to to
naturally handle it but robots get very
stuck because as soon as that something
isn't where they expected
there's a there they get they basically
get jammed
so how do there are different ways of
modeling this and at that time i was
really interested in using
ideas from decision theory which is a
branch of
of statistics so i took courses in the
statistics department
and we started to apply them to doing
things like
stochastically orienting the part so in
other words you wouldn't do it perfectly
but with probability
you could succeed
that was also carnegie mellon should i
tell you about the art projects we were
doing
yeah okay so the other thing we were
doing was uh i was really interested in
in making art and in some ways it was a
struggle for me because i wanted to do
art
and not and my mother was very practical
um said to me look you can do all the
art you want but first get your
engineering degree
and uh so i'm actually really
always appreciative thankful for her of
doing that because
i now know how brutal the art world is
but
the nice thing was that because i came
from engineering the
engineering background i had a very
different
approach to making art than most artists
so i started but i was able to keep
doing art and at carnegie mellon we
i had a number of friends in the art
department and
we at one point rigged up a robot to
paint
so we put a brush on it and of course
you know obviously you can
you can have a plotter that would
perfectly reproduce but i wasn't
interested in that i wanted to see
how the the brush the sort of the
imprecision the improbabilistic
the probabilistic aspects could come
into play by having a loose brush that
had a spring on it
and it would dip into paint and slopper
with paint all over the place
and it but it was um the resulting
paintings
were not so obviously from a robot the
problem was they were terrible paintings
and it was really interesting to me like
i had a show by the time i graduated i
had a show in the student art gallery
and they gave me the whole gallery and
had tons of these paintings
they were all terrible but what
everybody was doing everybody was in the
back of the room watching the video we
had made of the robot doing the painting
and that got me excited because i
started to realize that it was the robot
and the act of the performance of it
that was much more interesting or
compelling than the paintings
so i started moving more in that
direction so i've since then i've done a
number of installations involving robots
and now actually i'm interested in
involving
people who work with technology and
oftentimes there are physical
bodies so i'm working on a piece now at
the contemporary jewish museum
that's about uses computer robotic
camera if you will
but it will basically track the motion
of visitors in the gallery very
precisely and then direct sound
right to where they are so that was all
going on it's kind of that was my i
i would say i had my my day job and then
at night i would come in with the paint
brushes and
we'd rig these things up and the
professors um were very tolerant
although i did
one night the robot went a little bit
haywire and spilled paint all over this
guy's
um actually it wasn't my lab and there
was a lab next door
and he came in and he was a little bit
miffed that uh that i had been
uh you know painting in his uh in his
lab but that was a great thing about
carnegie mellon they were very
open to all kinds of
hijinks and unorthodox experiments with
robots and i think it's part of the
reason it's such a great place to do
did the robot robotics any logic to the
way it was flashing paint around or was
it just
random or no i should say so what i
would do is i would draw these pictures
right so they were pretty figurative and
i and i i
would not claim to be a great um you
know a great figurative painter but
i would i would do them with um with a
mouse so that that was the code
then the robot would execute those but
it was always very approximate
and uh so yeah and i really wanted to
get this kind of gestural kind of
abstract expressionist painting so
actually
this is one from a later project
but all the oranges were generated by
the robot
were painted by the robot so you can see
if you look
now you can see a certain similarity to
that even though no two are exactly
alike
but that was a joint project i did with
an artist in la when i moved to
usc so has the
your research in or not your research
but your work and aren't
fed into your working robotics
oh absolutely here's the best way when
i was when i was a
when i started at usc i was
so i i graduated from carnegie mellon
and i went um
i was very lucky to get a position at
usc with george becky
who i hope you interview because he's
fantastic and actually they have a
number of great researchers there it's a
real
robotics i would say center of robotics
and there i was still doing this work on
manipulating parts and geometry and all
those things but
um i had a lab and i was i did this art
project where we built this big
crazy robot that was gonna paint on a
big scale like a
20-foot scale that's that's what this
came from
and in the course of that i had a number
of students who were kind of a little
bit you know unorthodox some undergrads
and
characters and one day they they came to
me and said you got to check this out
and um
this was 1993 and they walked me down to
the lab and they showed me the world
wide web
and i was like wow this is wild
and then they said um listen we want to
do something we want to do something
with this and we started talking about
having a robot on the web and so
um and we said let's make an art project
we're going to where you know
i couldn't didn't seem to me like this
was going to be a research advisor but i
could definitely see art
and so we started working and it was a
nighttime thing where they were coming
in
like all hours of the night and we had a
sense that
someone might beat us to it so we work
like crazy from
january and we launched it in august
1994
and it was it got all kinds of attention
it was in the
london times newsweek all the it was in
abc news
and a week later another team launched
one in australia
and it was like they were that close to
to to have to be you're that close to
being scooped
the reason it was interesting was
because it was i always saw it as an art
project in fact we worked
there was a whole story around it that
we had a sort of fictitious story that
there was a
there was a radioactive it was all from
a radioactive zone in the desert
people had to come in and help us
excavate this radioactive site
of course you couldn't tell the scale
but you were operating a robot and
you're blowing air in the sand
and so you were doing this thing and so
it really attracted
interest partly because if i just said
oh assemble the water pump
you know that might not have been very
compelling but
i was really tuned into the idea of like
making an art project that i knew
how to get people engaged with something
like that have a narrative around it
and so after that um we that
we we got it was so it was so much fun
to see people coming in and operating
our robot
and literally was online 24 hours a day
and you could hear it from my office
upstairs
all day long people driving and moving
around ten thousand tens of thousands of
people were doing
so then we got some funding and we did a
next version called the telegarden
that was uh the garden that was
operating controlled over the web
and that was much bigger because we had
a better robot colored camera
but there we had plants and so people
could move around look at the plants
they could water them and then they
could plant seeds
so that so going back to the question of
research
in the context of that i really first of
all i started getting excited by the
idea that there was there was some real
engineering involved
in how do you design a system that could
be operated by non-experts
from anywhere in the world it had to be
robust in fact that garden was online
for nine years
pretty constantly and so we had to think
a lot about how to make it bulletproof
and do a lot of error detection and um
but the other part was that the whole
field of networked robots
um started people started talking about
that so we formed an ieee society
under robotics automation um the the
society
the the technical committee on networked
robots
in about 95 or 96
or so and pretty soon we have 200
members
we publish a book published two books
actually uh with mit press
about them and there were all these
technical issues about the time delays
and
you know how to do them and now it's
it's actually interesting and that
uh it would be hard to imagine a robot
that's not on the network
so i was very it was a really
interesting
arc for me and that i would probably
have never gotten interested and worked
on that
that area if i hadn't been doing art so
who else was with you
in the founding of the technical
there was a um a person roland secret
seaguar who
hopefully will also um interview the
can i get the book
oh sorry sorry
um is it beyond webcams is it sitting
over there it should be
here um
but roland was the guy
[Music]
today yeah oh you did you guys
so you have done a little bit of
background research so yeah am i telling
you things already know
oh but this thing is not important okay
yeah
what's on the record okay okay all right
okay all right well just stop me if i'm
going off to
utah um so but roland was
the um a co-author on that a co-editor
and he was doing working on network
robots in europe
and so we started doing this and then we
brought um
we brought in um wolf wolf from
bergaard and so
and then over time it's evolved and
there's a number of new people dez song
my student is now
one of the co-co-directors and
nori hagata from from japan and korea
was involved i have a whole list online
but it was
really interesting to see that um we
gave a talk
on um the uh robot controlled by the
world wide web at a
ieee conference where we talked about
all the technical aspects and how it
worked and
we i was amazed i never forget when we
walked in and the room was just
packed i mean it was just like so
exciting because it was brand new that
the web was new to everybody and
everybody was just trying to figure out
you know what you could do with it what
was the first one called
the radiation the radiation called the
mercury project
and that's online there's archives on it
um lots of information
what was the australian project the
australian
was stacking blocks it was a it was a it
was kind of
um a guy named trevelian
who's actually a really well-known
researcher and one of his students
i want to say taylor i can look it up
for you in fact it's probably in that
book i think they wrote a paper one of
the chapters
did you have a feeling for how people
used your systems
those teller robots and network tele
robots sure because we had
tons of documentation that was the other
nice thing about it is that
um when i did art projects that were in
a museum environment
they we we would come in at the end and
you'd see like a guest book or someone
might have written a few things but you
really didn't have
any real contact with the watching
people
use it but on the web you could you
could you could have all these logs at
the end of every day i could see who it
visited where they heard about it how
much time they spent so we had all kinds
of information about what they were
doing with the garden there's all kinds
of stories i mean
we there there were there were um
a fed interesting aphid infestations
where
and actually the biggest one we made
this huge mistake which is that when we
got the um the garden all set up and we
brought in
plants to sort of inspire people so
because it would start out just bare
she brought these plants and then um
like about two weeks later they all died
and then they're like it's a mysterious
fungus or something so we brought in all
the new plants again
and they died and like after the third
time i called in a friend of mine and i
was like
you know what's going on here and he's
like well
you know tell me about the system and
he's like
when do these lights you know what's
your life cycle here and i was like
you know it's 24 hours because people
are using it all over the world and it's
like
you idiot plants need darkness as much
as they need light
and we were burning them up i mean
because uh and i just never occurred to
me but
so you needed a dark cycle and a light
cycle so then we put headlights on the
robot so
we could it could be dark at night and
they could still see what was going on
so there was that and then there were it
was an early version of like a a chat
system
so there was a village square and people
would get together and talk and there
was actually one
relationship that began people got
married that had met in the garden
and uh the best part the the thing i was
most surprised by was a
an email i got one day from a student
who said
what how do i know there's really a
garden here
and it was it was one of those questions
that completely caught
caught me off guard because we were down
there doing the plumbing and
running all the cables and all of this
and and i have to say there's a
fantastic group of students um that were
involved george becky was also on that
team
um uh stephen gentner and jeff wigley
were the two
uh most active uh who really put their
amazing amounts of time into it and um
so there was no question for us that
there was a pardon but this kid was
asking a very interesting
question and i kept thinking because
there were fake systems out there around
that time people were making lots of
hoaxes and there's lots of
disinformation as there is today
so how do you know and i thought
actually it could fake it right we could
have just taken a camera and taken
pictures
and pretended the whole thing and just
indexed into pre-stored images
and it would be it could be faked so
right around that time i moved up to
berkeley and i was offered a position
here
and when when i came up i
learned that one of my heroes hubert
dreyfuss
the philosopher is here on campus um his
brother was actually in my department
and i went and i said all right you know
how can i find
cooper dreyfus and he said let's have
lunch with him
um and so we i i got to meet my this
absolute hero
and not only that but he was super
interested in
robots and technology we had this great
conversation and i told him about this
this story he said well this is a this
is a philosophical question
so this is epistemology and so it's
about knowledge
so we started this this four-year
project together where he introduced me
a number of
philosophers and we invited them to
write essays and we began of calling it
tell epistemology
which is what can you know at a distance
and how the internet was actually
re-engaging that that question the same
way the telescope engaged that for
descartes
so and then we're actually now very good
friends we meet about once a month
and we've taught a course two courses on
philosophy and technology
um most recently of course on highbacker
that's it i think you may i sent you
that article but that was around
telerobots and a bigger question about
how technology changes
and shapes our our behavior
so has your questioning to tell
epistemology gone any further
after that yeah because then that became
the central question like
sort of guided my art projects through
for the next 10 years and so we had
we had a a laboratory where people
burned hundred dollar bills
we did a ouija board that was in the
whitney biennial
in the year 2000s and a number of
projects that
tried to look at that from different
perspectives always trying to
question what was knowledge what is
knowable
how deception and agency or interact and
also the question of the the physical
body which is something that's always
interesting to me because
robotics is at this interface between
the computer the abstract
digital world of pure representation and
then the physical world
that we live in so robots are always
trying to bridge that gap in various
ways
and so i'm really interested in that and
how embodiment
plays a role and also so in your
project with dreyfus what kinds of
things did you
did you look at or do did you build any
systems for that
trying to think he was involved i mean
we talked
a lot about the the projects we worked
on
he um i would say probably the best
outcome was the courses
that we taught that had to do with we
looked at
questions about efficiency and
flexibility
and how the technologies of the 20th
century are all about efficiency
and the technologies of the 21st century
are about
flexibility so these are things that are
very very adaptable
think about nanotechnology or stem cells
new kinds of polymers they're all not
they don't just do one thing they do
many things
and computers another example of this
and of course robots too
and also i mean in terms of tele
epistemology
how do you see the current since
currently there's somewhat of a rising
craze and
telepresence from what i can tell right
um
do you think that that research is in in
any way
kind of building upon some of the stuff
that you did or
have they really developed further or is
it
in some sense just to finally come into
a
sellable form without really challenging
kind of how we know or how we can embody
ourselves
i think those questions are are still
there i mean one of the
things we we tried to do certain things
but the the technology was really not
um ready um at that time and we weren't
we we did not develop our own video
codecs and things like that we were
using
lots of off the shelf tools and then re
combining them
and so at that time you know having high
speed video was just
not practical there was no wireless when
we were doing our experiments in the
around 2000
so like wi-fi was just it wasn't
commonplace
now it's there a lot of the tools are
much better
and as you know the the g4
with the iphone gives you really very
very respectable um
teleconferencing video teleconferencing
so it'd be interesting to try and do
some of these experiments again
and see but the idea i still think that
the idea of being able to
navigate to essentially be present
remotely
is very compelling and very interesting
what do you think
some of the ways to achieve that would
be so i mean for example in your
telegram
you know one of the things that we
talked about is that you were very
embodied in your life you were obviously
constantly interacting with the system
and the system was reacting to you yeah
and so how reactive was the system to
the users
when they were i mean could you see
yourself around
right yeah you could one of the things
that you're getting at is um
we were very interested in how the
people how people were
responding it was the in a way the
social network
was almost more interesting than what
was going on at the robot they were
interacting
with each other through the robot so
the robot system had to be designed in
certain ways that was
that that that was compelling for them
and kind of
i would say realistically reflected what
they were doing
so that they'd want to spend time there
but their interactions with each other
was really interesting so we developed a
robot
system this was with eric paulos
and john canny and a few others that we
called the tele actor
and here the idea was we wanted to put
robots into much more complicated
environments in other words we would
think of this evolution we started with
digging in the sand which was like
hunting gathering and then we moved into
agriculture with the telegarden and then
we wanted to go to a social environment
like we wanted to bring rope
these robot systems into like a cocktail
party or an art opening and all kinds of
things like that so
we were gonna we looked around and the
robots at that time at least were
just not reliable enough and
they were just too clunky so we said
well listen we're not so
concerned about that why don't we just
finesse that question and we'll replace
in this case you know robots have been
replacing humans we'll replace the
robot with a human and so we have a
person
who puts on these helmet and goggles and
um basically and then every if
all the information is transmitted back
to a
uh back to the net and so people
anywhere in the world can observe what's
going on
and then the key idea was was
collaborative tele operation so you had
many people
basically providing input basically
where they wanted to go and other things
by indicating with
markers on the screen and then all this
information would come down to the tele
actor
who was at the art opening or the
cocktail party or
at one point we said let's we wanted to
be invited to a dinner at the white
house
right and then she or he would would see
a display and then make a judgment call
about
what the group wants me to do and and
the idea was that this the rope that the
tele actor is not a slave
you know they're not just following
orders um
yeah we were that's a whole other story
but
it was about the italian actor like an
actor taking direction
but then obviously having to improvise
in these very complex situations
so we did that for we had an msf grant
we did that for three years and it was a
very fun
project we took it out into all kinds of
environments and
she was on stage at the opera house in
san francisco at one point
and so and so i think that kind of thing
could i still think that could be could
be interesting i mean how often do you
want to
there's some event you're you have
conflict or you have to be two places at
once
but if you could you know imagine that
this could be worked out and if you
could just dial in and you could
essentially go to that party or that
other event and really not just watch a
video after the fact but actually be
present talk to people etc
um i think that could be i think there's
some i think people would be interested
in that
what do you think is one of the
limitations either technical or
um or actually
within the human psychology of being
feeling
present in a remote space
well because being present is
an incredibly rich experience right when
you're in
embodied in immersed in a physical
environment there's so many things that
you
consciously or unconsciously are part of
that
so the sound of the fan that we're
hearing right now that you'll probably
get rid of with room tone uh afterwards
and the
the fact is there's a million things
that i won't even pay attention to but
there but if they would suddenly change
you know something would get scurry
across the floor i would notice
and so this idea that you're immersed in
this kind of
really complex thing your brain is only
giving you the tip of the iceberg
but your body is being sensitive to so
many of the other things
so i think that's going to be very hard
to reproduce i don't think
i'm not you know not one of these sci-fi
fanatics who believes that
you know this is going to take over and
it's going to be better than being there
i still
maybe call me old-fashioned but i still
want to go to that party if i can
but if i can't and i have to be home
babysitting or something i
would be great if i could have some
approximation it's a reality
but you mentioned technical i would love
to talk a little bit about some of the
other
um just going back to the day job part
of this this robotics
work because i was i've been very
active in ieee and very it's been a
great organization for
for my work and i published almost all
my my work within my tripoli
we did a lot of work we did my students
and i got involved in these questions
about manufacturing
assembly lines part feeding which is
orienting problem and fixturing
which is how you rigidly hold parts and
one of the things we did in almost every
case was abstract through the problem
into
we made certain assumptions so that we
could provide a nice formal
mathematical model of what we were doing
and then we could start to
prove theorems and rigorously analyze
what we were
the the contexts so we had some results
in part feeding
we generalized the results from my
dissertation to curved parts algebraic
parts parts with friction
uh conveyor belts so we had a number of
things there
also manipulating very tiny parts so
parts of the micro scale using vibration
and then we got interested in this
fixturing problem which is
kind of a dual problem which is how do
you now hold a part rigidly
using very simple primitives and so we
had a
an algorithm that lets you um configure
fixtures you draw the part and the the
system will
automatically compute the fixtures and
in both those cases we now have
projects online um applets
in java so you can go and draw your own
part and then it will generate the
the the illustration actually animate it
for you
and i don't know if you can edit
something like that in later on
on the video but it's very it's very fun
to to see it i
can show you on a computer but um so i'm
really interested in
the idea of how the network can also be
used as a way to illustrate research and
share results and let people i was
saying a friend of mine recently that
i did magic i wasn't very good but i did
magic shows when i was a little kid and
part of what i
i think is that um i'm aware of the
the many different ways that magicians
work tricks and that
how perception is so malleable
and that in a lot of ways i really think
that
research when you show a video of a
research project
it's so easy to essentially
show only the good case and i i'm really
i always want to say well i want to try
it myself let me try to
you know see how good it will really
stand up
and a video won't let you do that but if
you have a website where it's
interactive you can go
in and somebody can try their own
version and that's much more compelling
so we've we did these websites and then
i've gotten interested in the last um
the last five years in medical robotics
and
that's been very exciting because here
the environment is very uncertain
i mean it's bad in a factory um
but in a medical environment it you have
just a lot more uncertainty that
you have very good imaging often and
actually imaging is getting better and
better
with mri and very very sophisticated
ct scanning but the body is very
deformable
and changes so as soon as you come out
of the mri machine
everything you thought where the tumor
was has shifted slightly and now you
want to
get radiation say is a really
interesting challenge is how do you
account for that so we um
we've been applying techniques from
robot motion planning that actually are
relevant for human surgeons as well as
as as robots but the idea is how can you
try and model and anticipate
deformations
and errors and then plan paths for let's
say
needles that can go around organs and
deliver seeds to the desired location
so that's been really exciting we have a
project with johns hopkins
and nih on that we actually just got a
patent
on something we call steerable needles
which is a way of
basically controlling a needle from
outside the body by steering
steering it at the base so who are you
working with
or at john's weapons
allison nakamura who would also be great
um
russ taylor greg georgian and
noah
and we also work with uc uh sorry uc san
francisco
with medical physicians and then i'm
also just starting to work with
a robotics expert
jacob rosen at uc santa cruz
and a medical roboticist um
doug boyd who's at uc davis almost all
these projects are collaborative and as
you probably have heard this in the
interviews but like
one of the other great things about this
field is that it's it really is about
people
working together and because robotics is
so
so cross-disciplinary you need to have
computer scientists and
mechanical engineers and
sometimes materials experts and many
different backgrounds
and what what often happens is um
that becomes so so second nature that we
don't think of ourselves as
i'm an electrical engineer you know i'm
a roboticist which means that i'm
comfortable doing things and i'm
comfortable talking to mechanical
engineers
and computer scientists and industrial
engineers so i think that's an
interesting aspect of that
of that field and but it seems to be
really a trend now
i see here at berkeley and i think a lot
of universities
have cross-disciplinarity people are
really feeling much more comfortable
working outside their area of specialty
and working with someone who can
complement that
in interesting ways one other thing i'll
just mention is that um
another ieee story is that i got
involved in the um
around 2002 the
president of the robotics and automation
society dick bolt
emailed me and said we we
we are having a challenge that there is
not enough
this the robotics automation society is
too heavy on the robotics and we don't
have enough of the automation
so how do we get more of this here and
he he said we have a hunch of maybe
maybe if we had a a second journal like
our journal is very theoretical
um but it's almost all robotics work how
can we attract the industrial folks
to get involved maybe we'll have a
second journal it's just about
industrial
and he said i want you to head to
committee to investigate this
so not knowing that it was going to end
up taking
three years of my life we i took that on
and started
basically getting committee he did
almost everything on the internet but we
had
thousands of emails to basically try and
do a real study of what was out there
and what was going on and one of the
things we came to pretty early was
we didn't want to have two classes of
journal where one was going to be like
the theory journal
and one was the practice journal because
i thought nobody will want to publish in
the practice journal it's just not going
to be as sophisticated
but instead we said let's have a
robotics journal
that does theory and applications and an
automation journal
that does theory and applications and
they would both be
the the very high quality archival
journals
rigorous and that idea so we we worked
on that
and articulating that and telling that
story for for another
about for about two years and then we
proposed it and was voted unanimously
by the society and then went up to the
ieee and they approved it
and so in 2004 the transactions on
automation science and started
publishing
and so now it's in it's about sixth year
and
it's just tremendous it's now as in fact
it was interesting i think it was last
year it was bigger than the robotics
transactions and has all the rigor and
all the the quality of the robotics but
it's
much more dominated toward practical
industrial
problems what do you see as the biggest
applications for automation right now
oh actually it's a great point because
one of the things that we all another
aspect of this was we didn't want it to
be only factories
i mean factories are the biggest place
where
people think about automation think
about assembly lines and automated
systems and that's still a huge area but
we wanted to say let's not limit
automation to factories
so there's so many other applications
for example is the
fast lane system for traffic right
that's an automation system
and we started thinking about automation
for
um so for all kinds of transportation
problems
for areas of health care
not only for robot robotic surgery but
robots that could transport food trays
and prescription drugs around the
hospital that's automation
a robot that would monitor the by the
bedside
in much more sophisticated ways than
it's being done now that's all
robots that would monitor patients in
the home or senior citizens and if they
fell or something like that
could tell um security is a huge area so
in almost every kind of environment
there's
security needs for systems that can
monitor the environment not just
passively cameras but
actively detect when something unusual
is going on or maybe
send a robot out that would be able to
check around if they saw something
suspicious
so there's that there's systems
um in uh
you know even games um you can imagine
aspects of automation so the way we
characterize the difference between
automation and robotics was that
robotics is really where you're trying
to do proof of concept you're trying to
put something in environment that
there's someone is a super high degree
of uncertainty
and then automation is where you've you
understand the environment pretty well
and now what you're trying to do is make
something that's very very robust
and reliable and cost effective
so the criteria are different the
problems may be similar but the kind of
approaches are oftentimes
look different
well i was just going to say that the
process of working with i truly spent a
lot of time
in meetings and learning the policies
and how a journal
is born i mean there are 50 journals 50
societies i think there's
probably each one has a journal a
technical journal and a magazine
so is this huge publishing you know
behemoth and i visited i'm sure you guys
have been to piscataway
new jersey have you been there
i've been through missed out away but i
haven't come to volunteer oh i see yeah
well
it's a mecca you know it's like this
huge uh
operation operations like a university
campus lots of buildings all kinds of
offices
and that's where they actually put
together and assemble all the technical
publications
so it's a really huge operation and we
got to see
how these decisions get made and how
carefully it was
you know when we first put the idea of
the journal other societies
had to be you know had to we had to make
sure we weren't on their turf and
negotiate and all the levels of approval
financing you know and how we launched
it
and we had to you know develop our own
kind of
posters and brochures and anyway so it
was really fascinating but i have a lot
of respect because
the the quality is extremely high and
there's this emphasis on rigor and it is
it is it requires huge amounts of time i
mean i'm really
the great other thing about ieee is it
is how much dedicated
volunteer time goes into it people
around the world are just
you know as we're having this
conversation thousands of hours are put
in
um man hours women people person hours
are put in doing just reviewing
reading writing organizing i mean
that's one great thing about the the
organization because
i've had a chance to work with some
incredible researchers who are not only
great
minds or great theoreticians and
engineers but they're also extremely
generous with their time
and organizing and planning and taking
care of every detail down there where is
the box lunch gonna be for the banquet
and
where's parking gonna be you know
there's a million of those kind of
things and people don't get paid they're
doing it all for
you know for very good of the society
so what kind i think from from what i
understand of your work
would you say that most of it is in
robotics rather than automation
or no it's really straddles both because
um
the all the work i was describing with
the um
part feeding and fixturing and those
things are automation
and then the robotics i would say the
telerobotics is in that category and
then also
the medical robotics is kind of more on
the robotics
yeah i would definitely say it would be
more on the robotic side so what are all
the
just so that we kind of have a list of
these what are all the
robotic um platforms that you've worked
with through the different robotics
projects so we have we came to the
we had the telegarden we kicked medical
robotics do we have stuff after that or
have we missed anything in between
um no i guess i mean we've
i've i've worked with a number of
industrial robots
like this adept arm and
those systems are always very impressive
because they're so well engineered and
they have to work
round the clock for years at a time so i
really developed a respect for
how those systems are made and brian
carlisle in fact actually the president
is also another great part of this whole
robotics
field and um and then we built these
a number of these kind of systems that
were oh i guess
what i didn't mention was we've been
using building robot systems for
monitoring the environment
so we've been using um robotic cameras
that have pan tilt zoom capability
and putting them into environments to
monitor
birds and wildlife where in one case
there was an issue about global warming
and they're not sure if the bird
population has shifted because some
tropical birds have moved north
and there was anecdotal evidence but
there wasn't um
field documentation of this and it's
very expensive because it's a remote
area of texas
so we put a robotic camera out there and
then we invited people to come in and
operate it and they could they could pan
tilt
zoom in on things and take pictures well
it's amazing that people have spent
thousands and thousands of hours on the
system and documenting
huge amounts of material we have
photographs of
hummingbirds not to mention at night
raccoons and people are on this thing
you know all the time and
they've documented and they found three
of these extremely rare birds that were
not known
in this area 20 years ago so it's
confirmed some of the hypotheses about
climate change in the area so and those
those systems again we've often in
almost all cases we've used
off-the-shelf components
but then we build all the software in
the interfaces
and when did you do that the
the bird project or i should say the we
called it cone
collaborative observatories for natural
environments
and that was 90 i'm sorry 2003
to 2008 or so and in the course of that
while we're doing it we heard about this
ivoryville woodpecker
do you know the story about the bird
that was
presumed extinct it's a it's this grand
bird that was
at one point they talked to that it
might be the the national bird of the
united states it's a
beautiful giant woodpecker and
hasn't been seen since 1940
and then there was a sighting by a group
from cornell and it was very it made the
front page of
the new york times and the thing is that
bird watchers
are you guys bird watchers no in fact
when i usually when i ask you know
around
california nobody and new york nobody
but in the middle
there's 40 million bird watchers it's a
huge a huge activity
and and this is us alone so it turned
out that
everybody if you're a good watcher knows
about the africa woodpecker it's kind of
a
it's like the holy grail and so they had
spotted one and it made
you know this is a huge deal well the
problem was they they couldn't
they didn't have any good uh footage
they had one little piece of video
and so we thought this is a perfect
application for our system so
we went to arkansas we rode out with
these
with these experts into the backwoods
and you know they showed us what they
were doing with their you know
mosquitoes and
alligators and this you know it's a
terrible environment and they're trying
to
monitor so we built this song um
did most of this work he's now at texas
a m and designed a
robotic system that would monitor um
and the key idea was that whenever
something flew by that was
big enough and basically had somewhat of
a signature it would take a video clip
and then we would um actually it was a
collaborator who would
row out on a rowboat a couple of times a
month
and pull out the disk drive and put in a
new one and then we'd go back and
analyze the images
and we found a number of woodpeckers
the pileated woodpecker which is very
similar to the one we're looking for but
much more common so that was didn't make
any front page headlines
and then lots of other birds but never
we didn't get not get any footage of the
ivory builder which we later did some
back of the envelope calculations and it
was not surprising that even if there
were a couple out there that
the odds of us catching you know
actually getting it were pretty remote
but it was a good it was really
interesting experience for us to develop
a system that could really work in a
really remote you know hostile
environment
so from a theoretical perspective it
sounds like a lot of your work is
based on mathematical models where there
are other sources of inspiration
intellectually or
theoretically well the one thing i'm
getting very interested in right now is
um is robot learning and statistical
robot learning
so we hired someone here peter abeel
who's a young researcher who did his
dissertation at stanford
on controlling helicopters and not just
controlling him to sit
to hovers or move up and down but
he wanted them to do acrobatics
radio controlled helicopters and you
know there's this whole sub-genre of
people who do these incredible tricks
it turns out there's only like you know
a few dozen people in the world who
could do these
really crazy things like have it hover
upside down
and so he wanted to automate that and so
the way he did it was he said
we have no idea how to program this it's
too complicated what we'll do is we'll
observe the human expert and then
we'll try to infer statistically
what would be the idealized control that
the human expert is doing human can't
even tell us right they just know
they feel it instinctively and peter
applied
these beautiful techniques from from
control theory
so common smoothing and expectation
maximization
he was able to show that you could
observe a human doing the trick
10 times and every time was slightly
different but he could align them all
using a technique from speech
recognition
called dynamic time warping and then he
could
use common smoothing to basically infer
a smooth trajectory
then he could execute that and he could
autonomously repeat
these incredible maneuvers so
and the videos of this are fantastic
anyway he joined the faculty two years
ago
and we started working together on
medical robotics can you apply this to
the operating room
so we have some results and having a
surgeon
perform very simple not not you're not
going to do the whole
gallbladder operation but the suturing
step
subtasks we call them so that the an
expert would do the suturing
with a robotic system like the davinci
robot
and then the system would observe that
and record that
and then do this this time alignment
and smoothing to basically come up with
a idealized trajectory
so this is a whole new area for me i
don't i don't have a background in it
but i'm learning from peter and it's
extremely
exciting we have a number of new results
in the past two years
what we call superhuman performance
were you ever influenced by the the
writings or the ideas of cybernetics
i'm interested i mean i i followed
i've always been interested in the ai
and some of the the
the work on intelligence you know chess
playing robots and
that work in particular i would say i
was somewhat of a
skeptic um uh
just by nature i i felt that
you know to some degree to me the
the mind is is amazing that it's still
as mysterious as it is i mean we
understand about quasars and black holes
and things of the
you know million light years away but we
don't
we don't really know how you know
something right here
works and it's it's amazing because we
can
we can take we can we can analyze it
with uh with scanning electronic
microscopes we can look at it from every
different direction but we don't know
how it works
and i i actually find that extremely um
satisfying that there's that mystery
exists
so you know i was not happy when when
deep blue
beats kasparov because i was the one who
thought those masters are something they
have that that a computer just can't do
but
it turned out it was wrong computer can
do brute force is really powerful
for a lot of problems but not for not
for for everything and i don't think
so i think that there's going to be many
avenues of
creativity of deep understanding that is
that computers are not going to be
able to do in the near future and same
for robotics
and i'm so i'm very interested in this
idea of the
which problems are really resistant to
to the techniques
to to to modern science
so what would you say those would be
well like the ability to
i'm not sure i should just realize oh
okay
um you mean the things that robots are
not going to be able to do in your
future yeah like what are the limits you
know it's
there are so many things where robots
are
are and i have to be careful here
because i know my colleagues would
probably uh not like me to say but
i think that there are many different
areas where robots will be able to
perform
certain certain
tasks but i think that all of the nuance
the kind of
gracefulness and subtlety that humans
have
i think we're very far from seeing that
i think you can fake it you can sort of
have a robot that looks
kind of human-like for a little while
but i think that the idea of
that really be having a robot that could
really keep you company
i think i think that's pretty far in the
distance
at the same time things like you know a
robot being able to change a diaper
i mean that would be i have two kids i
would love to have one of those
um i think that's uh
you know i actually i think the the way
it will be is it will probably
when that gets solved and i'll say when
because i do think it will
um it'll be because we'll change the
diaper completely
you know it won't look like what our
diapers do now or maybe we'll change the
baby i don't know
but the idea is that there'll be
something just like you know
i think what we'll do is we'll adapt to
the world of robots
so people say when i want robot to to
load the dishwasher after a dinner party
the answer is if you're willing to have
plates you know they're
square heavy plastic with with markings
and handles and everything else that's
going to help
you know you can get a robot to do that
maybe people will be happy with that
in the next decade that will
our house will start to be more robux
compatible and that i believe
it will have sensor you know markers
around and things like that to help the
robot navigate
so it won't have to come in and just
deal with the crazy clutter
of the average household but the
household will sort of meet the robot
halfway
do you see any that there is there are
any dangers
in terms of you know that humans are
so inherently adaptable and potentially
willing to adapt all kinds of things to
the limitations of robotics
oh well don't get me started so this is
my whole other eye
sorry about that yeah
no i i think that technology is this
really powerful force
so i you know this is what uh i've been
talking with dreyfuss about is that you
know
that technology is this extremely
compelling thing it's not just about the
mechanisms and
the gadgets and the you know devices
because those are those are fascinating
on themselves but the whole it's a
it's a it's a mentality a mode of being
um in dresses terms that
it really is taken over and is is very
very powerful it's not something that we
can even be conscious of
so we're immersed and it's something
that makes
we want to make the world more and more
available
and so that transportation from the jet
you know to the railroad to the jet to
telepresence are all ways of of doing
that
um but so many other things where the
world wide web and
digital radio and all of these are the
video cameras right these are all ways
of
making the world available and what
what i do worry about is this idea that
we'll start to turn it on ourselves
that we'll see ourselves
what we want we'll want ourselves to be
more and more available
and i i was talking to a friend about
this and he said you know
do you think it's some some or some sort
of big brother you know is this going to
be imposed on us
and i don't think i think and he said
maybe is it more of a seduction
like we'll want it because it's very
very compelling it's very
appealing and i think he's right i think
that that's that's the way it will
happen
that we want to be available there's
something very very
irresistible about facebook and
twitter and all this things we want it
because we get this
my wife is a filmmaker and she she talks
about the oxytocin
effect that when you you know you give a
hug you get this
this chemical boost and it happens when
you have true meaningful communication
with somebody
so i think people are really were very
prone to that
and the danger of course is that i think
it it can
it can it can take over and we don't
know how
to resist it and so we don't we lose a
kind of ability to have
privacy and um sort of internal
um moments and and one-on-one
connections that that i worry
are we going to be able to sustain that
in a world where the technology becomes
more and more compliant
i hope so if you can thank you for just
a few more minutes i was just curious
just to
kind of wrap up um two things one
what are your kind of goals for the
future
um in terms of your research and then
also where do you think robotics is
going
in the future
no it's good um it's it's uh those are
hard questions i
i i'm very interested in
right now i'm starting to be interested
in social media
and so we have a system that we call
opinion space that's a way for people to
interact and have conversations over
distance now it doesn't look like
robotics but it involves geometry
and a way of visualizing this map
so you can see everybody in a
conversation at a glance and the most
insightful contributions get highlighted
and at least insightful get pushed back
so it's a way a very sort of
i guess effectively
scaling up conversations to large
numbers of people and
so we've been working with the state
department on a project
also now with a number of companies and
so i'm really interested in that
that direction i mean how can we take
social media and
visualize it um and then the second
question about
the field in general it's interesting
the big thing that everybody's talking
about you may have heard is uh
robotics 2.0 so the analogy here is that
just as
web 1.0 was about
a one-way flow of information read-only
information
web 2.0 is when it suddenly became a
two-way dialogue people are
participating
so when you come to a website you expect
to be able to provide
feedback or contributions to it and so
you know there's comment lists and all
kinds of things like that collaborative
filtering
um and there's something that's
happening in robotics right now that
we're just starting
just labeling for robotics 2.0 and
there's a number of people i'm sure
you're interviewing on camera christian
said i'm involved
but the white house is going to make an
announcement in
sometime in the next month or two um
about this white paper that we've been
working on
and the idea is that robotics 2.0 is
robotics 1.0
which is up the first 50 years is really
robots replacing humans and robotics 2.0
is about robots cooperating
collaborating with humans so the idea
being that we're put it's much more
human-centric we're realizing that the
human is very much a part of the system
so robots have to be designed very
differently than you know when they're
working autonomously
you have to be sensitive to people's
presence they have to be
safe i think because i was directing the
center for the media here
yeah so this whole thing about what is a
medium yeah has
been a fascinating question so i agree i
think robots are media
absolutely um thanks you guys
thank you for just fun time for this
absolutely