begin to start by telling us where you
were born where you grew up and went to
school
i was born in new york city new york
manhattan i was raised on
long island rosslyn north shore
and i did my undergraduate degree
at the university of michigan my
master's degree at stevens institute of
technology in hoboken new jersey
and my phd at the university of
massachusetts in amherst
what did you study as an undergraduate
two things uh
i have two degrees in chemistry which
many people don't know
uh the first was uh at michigan it was a
bs in chemistry but also
i got a dual major in applied
mathematics which
shows my age because they didn't have
computer science there uh
at the time so i had a good advisor who
told me
that you're taking all these mathematics
courses so maybe you should take a few
more
and i did i spent an extra summer term
there and it gave me the dual major
which
gave me the insights into the computer
science career that i have
but i also ended up
eventually working in the chemistry
industry for
a significant number of years uh first
at a dystop company
and then as a pharmaceutical r d chemist
both in r
d uh with my bachelor's and then i got
my master's while i was working
full time so i like to call myself a
serendipitous roboticist i kind of
stumbled into it for uh in an odd way
not like most folks who have had this
passion for
robotics for their use i've had a
passion for science from my youth but
not for uh
specifically for robotics so your degree
from steven's point was
that stevens institute of technology was
in organic chemistry
and then what made you decide to go back
to do a phd
there's a an interesting convolution of
events or a configuration of events is a
better way uh to put it
what uh happened was i was working in
this
r d laboratory working with uh
chemicals that were labeled known or
suspected carcinogen
on a daily basis uh fume hoods of course
and the like as well too
and i love the joy of discovery of
chemistry which was one of the beautiful
things about chemistry
a heavily plowed field but i also
recognize maybe there's not a future for
me
uh if i keep doing this and there were
accidents occurring uh
in the plant that i was working at uh
because we took it from
um research bench all the way out into
a process in the in the plant and uh
i kind of said drawing on yes i was a
hippie
uh i actually had long hair at one point
in time
so we decided to move up to new
hampshire uh my wife and i who've been
married for 41 years now to
investigate what kind of work we had i
had bought 50 acres of land up there
prior on a previous
visit so moved up and took a job at a
small college which is now defunct it
was called hawthorne college
in antrim new hampshire uh taught there
for i think close to
10 years with only my
master's degree in chemistry as a series
of events occurred where the florida
institute of technology came up and
bought the college as a campus for them
because they had a flight program and it
was boring to fly in florida and we had
interesting terrain
up there and since it was a institute of
technology they were interested in
starting up computer science and as i
had some background from applied
mathematics
i volunteered to set up the program i
served as department chair
and after a while uh i started
thinking maybe i should get my phd uh
if i'm going to be a department chair uh
and so this school was very
accommodating
let me go uh take the time off not
completely i was still teaching and
alike
uh and it was a teaching school and went
and got my
degree first i went to the university of
lowell
and i took a robotics course there
because it fit my schedule not because
of any deep-seated commitment for
robotics
then i found and i was disturbed because
i had to take the gre
even there and i said i'm a chair of a
computer science department why do i
have to take a gre
well i took it and they give you a
couple of places to apply to and i
umass lowell was actually umass
excuse me amherst was further away
uh but it was faster because it was
freeway i didn't know that time so i
gave my score
i was accepted because i applied there
and when i was interviewed there they
assumed i was going to be a roboticist
because they saw i took a robotics
course
at the university of lowell and they
lined me up with all these folks
at umass wendy leonard and others as
well two were in the ai group
and it stuck so
it wasn't a conscious choice and i would
say i'm fairly
unique in the field for two reasons
one is a serendipitous aspect of being
led i kind of feel like
jonah being vomited up on the shores of
robotics
from a calling from god if you will
perhaps
so that was one of the reasons and
the other is my scientific background i
have
formal training extensive formal
training in the scientific method and
i think that's something that in the
computer science area in which i
am housed within not as many of our
colleagues
have that kind of basis for a deep
scientific
exploration and understanding uh that
they
as they should and so again those two
things i think distinguish me from some
of my colleagues
and so when did you do your phd work i
graduated
in 87 i was able to complete it in three
years
which was unusual but i had four kids
living at home
at the time so i had unusual pressure to
do so
i spent um i would go down
after the courses were done and i was
able to stop teaching
uh for a while i would go down after all
the qualifiers were done
in a year year and a half i would go
down on monday
spend monday night on campus come back
late on tuesday work from home on
wednesday
go down thursday stay overnight come
back friday
and then rinse wash repeat or whatever
the order is
for about a year afterwards as well too
so
it was a bit grueling but it was very
satisfying as well too
and who was your thesis advisor and what
was your thesis on
my thesis advisor was the late ed
reisman
at the university of massachusetts he
was extremely well known in the
computer vision area and i worked in the
what's called the visions laboratory
which dealt with
interpretation of natural scenes uh
dealing with very tough problems
and that's where much of the schema
theory that has permeated my work
has come from i also like to say that
michael arbib
was my mentor uh in this case he was
there when i first got there
but he left for usc uh shortly
afterwards but
his work uh has had a profound influence
on the way in which i do robotics
can you explain that a little more was
it some of the the cybernetics
that you've done yeah it was the the
metaphorical brain and other things the
relationship
of biology to uh intelligent machines
and his formal language that he's
developed and used in many ways from
everything from explaining how the brain
works to how manipulation works
called schema theory which permeated
umass at the time as well too
in many different groups and the ai
group was a
the lingua franca if you will it made
a big difference in the way i thought
about these particular types of problems
and how were you applying the scheme of
theory to robotics
well at the time the behavior-based
paradigm was emerging
if you look at the early work by brooks
and others as well too
it tended to be rule-based
schemas themselves were concurrent
asynchronous processes
which could be multi-threaded and run
independently
there are also things those are the
motor schemas perceptual schemas were
the same
so it became an interesting question of
how you schedule and manage these
particular processes
kind of like one could argue the brain
does as well too with all these
different processes and how do you make
a sense of it my thesis title if i
remember correctly was
towards cosmopolitan robots
intelligent man-made intelligent
navigation in man-made environments
extended man-made environments
so we're looking at insides of building
and outdoor structured
uh sidewalks uh and the like as well too
but not the wilderness or something
like that the key factor i think which
distinguished my work
was and the advantage of using schema
theory was the ability to create what i
call hybrid architectures hybrid
deliberative reactive architectures
the early work in behavior-based
robotics
i would say through the baby out with
the bathwater
all the higher level reasoning and
high-level planning
no longer played an effective role
my work uh recognized that maybe we do
have a cortex and there is some value in
being able to deliberate about certain
things
and the schemas themselves gave us the
ability
to instantiate change parameters reason
over these things in ways that other
forms
of uh reactive systems if you will uh
were not as easily amenable
to so the real focus of my dissertation
work was hybrid deliberative uh
architectures the schema theory got most
of the press because that's
what made the robots actually do the
cool stuff that they were doing
but the deliberative side was what
configured those particular control
systems
can you give an example of how that
would uh play out in terms of
a robot making a deliberative decision
sure we used
initially used a star path planning over
uh maps of the world both that were
a priori available available ahead of
time
as well as those that were sensor
acquired
so we had both short-term and long-term
memory those would create
legs for a robot to complete if i wanted
to go from
here for example to a plenary session
and i had a map of this i could generate
automatically using a planner a set of
waypoints
but for each of those individual
waypoints instead of
following those in a rigorous i'm
following this trajectory
we would use behaviors the reactive
component which would be selected by the
deliberative system and parameterized by
the deliberative system based upon the
context
to be able to move from uh one location
to the other uh
goal directed while avoiding obstacles
including a little randomness which we
called reactive grease
at the time and other aspects which
would be able to complete and accomplish
the mission
uh in ways that are taking advantage of
a priori knowledge
when it is available but still being
able to function in the absence of it
which was
i think an important contribution and
did you implement it on a physical robot
yes we did we had
one of the very first i think the
denning robot number two
uh we called it harv uh was the acronym
uh which we came out uh initially we
called it harv
because it was short for
harvey wallbanger uh which was an
interesting cocktail as you may know
um and the other uh reason we had those
dutiful computer scientists
must come up with an acronym for harv so
we called it harv because it was hardly
autonomous robot vehicle
at that point so we've come up with good
ways in the whole field has come in good
ways
since then but it was hard it was very
hard in those days to be able to
given the limited computation and
perceptual power
to be able to make these machines move
in real time
uh in a safe and an effective manner and
i think some of the architectural
uh capabilities and the decomposition
the
parallel distributed processing which
was present and that
helped the field move forward
did you stay at hawthorne after you
finished your peers no i actually
left hawthorne after i completed my
qualifiers
so i knew i couldn't be working full
time and getting my phd that just
is a a impossible situation so
i quit uh about a year and a half before
then and devoted myself fully for my
phd studies at that point in time and
then i
started looking uh as people do when
they graduate
looking for an academic position uh
interviewed a number of places
uh georgia tech was the one that finally
landed me made me the right offer
my family was very happy to move out of
the cold winters of new hampshire
where we were living at the time
and i've been there now uh 28 years
it was a good decision one of the things
that has kept me there all this time
is the strong influence on
interdisciplinary research which is any
real roboticist knows that is absolutely
crucial we can't know everything
in every field and we need to find the
right people to be able to
team with and drawn when we need
additional knowledge so i went down
there
i was the first roboticist in the
college in the computer science
group there was one other roboticist
that i can recall there at the time
which was wayne book
in mechanical engineering he's now since
retired
he was working on flexible arms and
alike
so a lot has transpired now we have
almost 60
or so different roboticists institute
research center
in robotics a phd program in robotics i
won't take the credit for that but at
least i got to witness it happen
uh as well can you tell a little bit
about the history of how those things
emerge do they come out of the cs
department or
join the center
itself and the phd program came out of
the
computer science program uh whether it
was the college of computing or
information or computer science
and i have to give credit to the hires
that we made
the young faculty or the young turks's i
would describe them
folks like tucker bulge and frank
dillard
who said we need to do this
we need to do that so both having spent
time at carnegie mellon
and wanting to create a similar
structure to be able to do that
we also had at the same time i arrived a
president
uh who was the former provost uh
patrician
at carnegie mellon university who pulled
out
the computer science department from the
college of science and liberal studies
and made it a standalone college which
was amazing i mean because that was the
model that they had
at carnegie mellon and it gave us a lot
of freedom and opportunity to try things
basically
enough rope to hang ourselves uh with
and
uh so uh now that we didn't have to
answer to the dean of science and we had
our own deans uh and the like
we could explore things in new and
different ways
the younger guys got us old guys uh
worked up a little bit so we moved
forward
with it and then we
after time the first thing was a uh
i believe the phd program came first
then i had the opportunity to do
what technically i should call a
professional leave with pay but
effectively it was a sabbatical um in
the royal institute of technology in
stockholm i spent a year with
henry christensen working with him
watching him in his second year there
setting up the center
for autonomous systems that they had
there helping him
with that and through a series of events
later i was even serving on the board of
that center
but he called me up once one evening and
said
i'm leaving and he was very upset he was
not happy with the way the swedish
government was treating
them at that point in time i said i
think i'm going to germany uh
i tried actually at length to talk him
into staying because that was an
advisory board meeting but he convinced
me that he was going to go
um after i don't know 45 minutes an hour
or something like that on the phone
uh with him at least it seemed like that
to me and uh
he uh i said well
maybe you should think about applying
here and
i did he was well received and now
has become the center director for what
was first
the robotics and intelligent machines
center uh rim we were going to call it
the center for robotics and intelligent
machines but our provost
uh jean lucimo who is french noted that
that
crim is criminal and french so that
would probably not have been the best
one although we thought of wearing
striped t-shirts or something like that
to
show our commitment in many ways
but we after we got that center going
after a number of years we kept growing
the mass
and eventually under our new executive
vice president for research steve cross
we were able to stand up what's called
at georgia tech
and institute a interdisciplinary
research institute
there are only i believe eight or eight
to ten of those on campus
where it's basically the fields where
the
entire georgia tech is gonna put
priority in and research
for hiring and other aspects space
things like that as well too
um and so henrik deserves the credit for
doing that leading the national robot
uh the national uh roadmap for robotics
among other things as well too playing
very very well
on the national stage things i know
he has talents for which i saw firsthand
before and they're not my particular
strengths uh to be able to do that
uh as well so i i think the aggregation
of people was right
uh the timing was right and uh we're
very pleased with where we are right now
great so we can go back to when you
first arrived at georgia tech
what were what how did you set up your
lab oh it's interesting my
my very first lab uh was a uh
it was a point problem for mobile robots
the dennings were big
in those days they're about this big and
about this high so i got a denning robot
and my lab consisted of a small faculty
office uh
at that point so it was really just a uh
a point problem like this
so then we moved into the corridors and
eventually i recognized that
um that's not enough but there was no
space
for in the college or the school it was
actually school at that point in time
the school of information and computer
science at georgia tech ics
so i started looking around for
interdisciplinary research institutes
uh there was one oh gosh i should
remember the name
of it uh man that wasn't a manufacturing
research center
it was something like that a precursor
for that
and so i moved into a different building
surrounded by engineers
they were happy to share their space
with me
and so i actually had more laboratory
space
then after a while they put up a new
building on campus called the
manufacturing research center
i got a really nice lab at that
particular location with
adequate space for students and offices
and
for robots but then later the best lab i
got i still have it right now was
in the technology square research
building another new building we built
across the freeway which was a new area
it's in the basement
and i think it's one of the most
spacious laboratories for mobile
research
mobile robot lab research in the country
uh
so we can reconfigure the space in
interesting ways we can
do human subject studies down there and
the number of robots has proliferated
dramatically
in many many different ways initially i
did a lot of work
uh after the architecture stuff in teams
of robots and multi-robot scenarios
and so we were able through nsf funding
and darpa funding
to buy significant numbers four or five
of different types of robots to show we
can
maintain formations and do coordinated
activity
and create software where average people
could be able to program these kinds
of systems and did that sort of build
intellectually on
the scheme of work and behavior-based
work yes there's an actual a very
natural progression to the kinds of
things uh that i have done in my career
initially starting with uh single robots
in deliberative reactive architectures
then
we started looking at teams of robots
first working mostly in the reactive
side and people were asking you know why
it's so hard to make a single robot do
anything why are you working with
teams of robots well the real answer to
that question was it changes the way in
which you
approach the solutions to problems it's
like distributed computing you can do
things in fundamentally different way
with distributed sensing and distributed
actuation so
it made good sense maya matarich was a a
colleague of mine as well too at that
point
in time not at georgia tech but working
in this particular
space and then after that uh years later
we started looking at human robot
interaction recognizing that
teams not only mean teams of robots but
teams of robots and people
so we were concerned about how we could
integrate human beings into these kinds
of teams and leverage
their deliberative capabilities with the
reactive capabilities
of robotic systems and then after that
came the quality of the relationship
which spoke to the ethics work
that i later did in other words well if
humans and robots are working together
as teams
how should they effectively work with
each other as teams and what should be
allowed and what shouldn't be allowed
and under what sets of circumstances so
that's been the progression i'm still
doing things now in formal methods
trying to establish guarantees
for the particular uh robotic missions
that we develop
for humans before they deploy them in
particularly dangerous situations such
as counter weapons and mass destruction
we've spent a lot of time looking at
heterogeneous
robot teams where you may have small
robots and big robots and flying robots
and how we can deal with the issues
afforded by heterogeneity in terms of
doing things better and differently and
how you they perceive the world through
different sensors and how you can
reconcile
those sorts of things and getting back
to a theme that permeated all of my
research which goes back to arbitmen
and others as well too which is the
bio-inspired work
we're always looking to nature for the
ways in which
intelligent machines can function
because one they are
proof of uh existence of intelligence
and
i use intelligence in the broadest sense
not just limited to human intelligence
but
equally to ants and their domain and
wolves and frogs and
all sorts of species have uh provided us
the ability to analyze
and apply uh these uh techniques and two
to get a better understanding how we can
back project some of this
into the biology community uh in certain
cases as well too generally i don't do
that
but i am not reluctant to let biologists
say certain things about the relevance
of our work
whether it's from a cognitive
perspective or whether it's from a
ethological uh or which is an
observation of animals in their natural
environment
point of view how that can influence
their studies as well too
and it's always been extremely fruitful
for me
to work with folks in other domains
and now that has been more recently
extended to lawyers
and uh and philosophers and others as
well too
that's the thing i find most exciting
about the field is the ability to grow
in new dimensions and getting back to my
chemistry roots chemistry has been a
relatively
well plowed field for hundreds and
hundreds of years going all the way back
to alchemy robotics is i won't say
virgin territory but it is
a wide open frontier and i've often said
that in the early days including myself
many of us were more like
cowboys and cowgirls uh just showing
proof of concept of anything
and you know if you make the robot get
from here to the door
wow that's an amazing thing the field
has progressed tremendously since that
in terms of
the requirements for uh experimental
methods and other things as well too
which i think is very very good
um just to touch on some of your points
about the
ethology and looking at animal behavior
he did some work with
in the eye that's correct yes i did i
worked with uh sony for ten years uh
initially
on ibo we have a patent on that work
uh also uh with curio uh which is the
small humanoid
uh that they had i developed a model for
them
i think it was i studied human behavior
and i studied dog behavior and that's
the important thing that's quite often
neglected is understanding the
environment in which these systems
reside you have to understand
what do pet owners really want and how
do they interact with these
uh with these systems and what are the
underpinnings of
a dog's a trained dog's uh behavior or
domesticated dog's behavior
these 183 behaviors were compiled into a
model
using methods from timberlake and lucas
cognitive psychologists to be able to
deal with some of the coordination
mechanisms
the scott and fuller ethe who
orthologists
spent their lives studying dogs and
others like fox and others provided
the basis for the enumeration of the
kinds of
behaviors we developed this fairly
complex
architecture to be able to allow the
system to act
more like dogs than might be thought
otherwise
we included things like humping your leg
and chewing the carpet
the mal adaptive behaviors as well as
the normal ones
sony did not implement those but and i
can't say i blame them but
i tried to be as complete as possible uh
in this
we did not publish a lot uh there are a
few papers out there two or three and
there's a patent i mentioned as well too
part of this was the ip issues and i
worked with them as a private consultant
which made that feasible under those
circumstances what was the patent
actually
the patent was uh for etho ethological
and emotional
architecture if i remember correctly i
mean your patent language is incredibly
obscure but it dealt with uh both the
organization of the behaviors
that were present within ibo and the way
in which
what they referred to as the instinct
emotional model
which was more ethologically correct
referred to as motivation but
i lost that battle as to how these
things could coordinate
and manifest their behaviors in
interesting and meaningful ways that
hopefully would align with the owner's
expectations
avoid such things as what's called
behavioral dithering where you get
behaviors going back and forth
if you're hungry and you're thirsty at
the same time so you take a bite of food
and then a drink and then a bite of food
and then a drink
as your fresh internal thresholds go
down or should you be able to have some
hysteresis in this that stays
depressed and but part of the trouble
with obesity is sometimes
that depression stays down and people
don't understand that
uh i'm still hungry i'm still hungry i'm
still hungry and i should have quit
earlier
so we have to find ways in which to set
those particular parameters and the like
so that's what that was
involved with interesting um did you do
other
consultancies with companies i did a
significant number of other
consultancies i
can't say i remember that was the one
that was the most durable again they
they bought virtually all of my time
that i had
over the 10 years again as a faculty
member you're allowed the equivalent of
one day per week
to consult to do that
i didn't i wasn't able to spend all that
time on the project but up to that
particular point
in time and i've
worked with lockheed martin and other
companies as well too
uh essay i wasn't i can't remember you
have to check my vita
to get the the specific ones georgia
power i remember there was some
work uh recently completed some other
work for
a small startup uh uh incipient startup
a variety of evolution robotics uh as
well uh which is now part of irobot
um so a variety of different uh
samples i guess but none was uh to the
extent
uh and the duration of the the sony work
that worked with samsung
as well too uh but that was through
georgia tech partly through the
samsung technology advanced research
center that we have there i have two
patents on emotional
affective models of behavior for for
robotic systems
with my former student uh lilian
moshkina
and some others that were involved in
the project
dealing with traits attitudes moods and
emotions and how we can deal with
time varying effective phenomena such as
encoded in circadian rhythms and moods
specific responses based on your own
predisposition
of your personality which would be
genetic in your particular case
and how these things modulate themselves
both in the presence of
time specific things that you might have
attitudes towards particular individuals
versus uh particular objects you know
you might like some food and you'll be
happy to see it
you might dislike some food if it's put
in front of your plate you'll be unhappy
to see it
these attitudes also are present as well
too we developed what i think still
to date is the most complete model
for affective phenomena which integrates
all these aspects we couldn't find a
single cognitive model that
spanned this particular space so we had
to synthesize
these and we're still using that work
now and some work
in the management of early stage
parkinson's uh disease
uh uh and care uh
in the national science foundation grant
that we're using
uh were you ever part of a startup or
trying to do your own startup
no there's been points where i've almost
gone
into those uh i know i'm not a good
businessman
and i think the key to successful
startups is finding the right business
guy
to work with i have received great joy
out of curiosity
driven research i have been well
compensated over the years
in any case
i admire those who have the wherewithal
to be able to
to do that but to me it might have been
more
a distraction as opposed to
a goal in my case and uh
i have enough money i guess is one way
to put it at this point in time i'm
comfortable with
with what i have and i think we are
there's many different ways to achieve
impact uh and one of those is through
the university and private consulting
and other things as well
and who are some of the uh grand uh
people who supported your work oh gosh
uh all sorts
uh the national science foundation uh
was at first i still remember
the joy of getting my first grant i was
just uh driving home i was just all
smiles
on that took a while but uh as you learn
you have to deal with
rejection as well as success in this
field if you can't do that you shouldn't
be
in it darpa i've had multiple
darpa grants the army
uh applied aviation directorate uh the
ethics work from the army research
office um as i mentioned samsung i've
had
work with the department of energy uh
i'm gonna give short shrift on
many of these um onr has been a major
sponsor of me over the years
and also navair out of paccident
air force base
army research laboratories as well too
the mass
cta program has been part of it
and i know i'm neglecting some at this
point but
a lot of dod sponsors because most
u.s roboticists find that
a an easier pool and they also
do provide pure basic research as good
as 6-1
6-1 money is as good as nsf money in
many respects
i do not have a security clearance
although some people try to tempt me to
go after that as well too because
i like the freedom to be able to talk to
people about anything and i think that's
paid off
handsomely in my ability to have
discussions with folks such as yourself
uh internationally um
and uh uh
so yeah that's basically the uh the
sorts of
things i can't say that's a complete but
it is a representative example and
if you check my web page you'll find a
more complete version so in many ways
the
the military-funded research was more
freely accessible than the
sony or private company oh far so far so
i i couldn't talk
i saw a non-disclosure agreements with
both samsung and
others as well too so and in general
the dod work is for the kind of stuff
that i do
is wide open there have been some cases
that
we've worked with i guess it was
lockheed martin or other
companies as well too when you're
subcontracting there may be some
provisions which make you a little more
uh careful in terms of what you can say
due to non-disclosure
agreements but by and large
i started working in the manufacturing
domain when i first got to georgia tech
that's where the
where the money was and i've often said
i'm happy to play in anybody's backyard
you know if you can provide the
resources to do these sorts of things
and
my goal has never been to build specific
applications per se but rather to
explore the fundamental questions
underlying machine intelligence and
robotics
and how we can use those in ways that
have not been fully understood
uh previously whether that's in dealing
with
internal organization of action and
perception whether that's dealing with
coordination of teams of robots working
together
um whether it's dealing with the
importance and the role of communication
in these sorts of agents whether it's
dealing with human
machine interaction as well to like to
think that we explore
solutions in the more general way and as
i said
if you're paying for it we'll consider
the application that you're considering
as well too but deal with the
fundamental advancement of knowledge
which i think is what academics should
do
right and have you had you mentioned
your work in stockholm
have you had other major collaborations
outside of georgia tech yeah
uh again the other sabbatical
i worked i took uh professionally with
pay to be more accurate
uh deals was uh first four months
were at sony as well i was in japan
for that but after that i spent almost a
year
in toulouse france which is a wonderful
place to spend almost a year it's a
wonderful place to spend many years i
would contend
it at cnrs las
working with raja shatila and rasheed
alami
and others as well too
it was a wonderful time to be able to
finish my book among other things as
well too and to
deal with the questions of human robot
interaction in some cases flying robots
that they were working with
uh i grew greatly uh because of that but
i have to admit the
french lifestyle was an extreme uh
attraction
uh as well but raja is a
not only the current president of of our
society but
is a has an incredibly long and rich
history in robot architectures uh
which is something i am very much
interested in and rasheed
has worked extensively and continues to
work in human robot interaction which
had common interests with the sorts of
things
that i were doing so that was the other
one i have worked with the folks at the
university of pennsylvania
on multiple grants and onr vijay kumar
for example have worked with the
colleagues
at the university of southern california
guerrero sokotomi
as well
and i as the list goes on and on i'm
currently working with a colleague at
fordham university who i went to school
with uh
did my phd uh damien lyons
as well and some of the formal methods
we both have this same schema theoretic
framework which we got
from that particular time um
the work i'm working now with folks at
tufts university mathias schuytz
on the national science foundation work
that we're doing for early stage
parkinson's care management
using robotic systems so the real key i
think for a successful robologist
is finding best in class to partner with
uh
many times those are people that are
right around me at georgia tech uh
magnus segerstedt and
uh tucker and tuckerbalch and others as
well too
our great uh partners which i've worked
with and continue to work with
but in many cases you have to look both
nationally and internationally
if you want to do the best research you
have to find the best people and
um what about some of the grad students
or phd students who are continuing
work and sure uh my very first
i just had lunch with her today my very
first phd student was robin murphy
uh she has just recently completed a
book in
disaster robotics it's a texas a m now
as a chaired
professor uh working heavily in
search and rescue um very proud of
her accomplishments another student is
tucker balch
who is now a colleague he is at georgia
tech
a professor there
i have other students alan wagner a more
recent graduate
we've done some very interesting work
together in robot deception
he's at gtri uh the georgia tech
research
institute as a research scientist there
um
as is zolt kira who was a
previously was located at
sarnoff doing work up in the princeton
area
and had decided to come back to the
georgia tech research institute as well
too for the environment
it provides eric martinson
is another student who
is now at the toyota out in mountain
view
working on particular projects for them
i should know more erica
rogers erica rogers was my second
student she
ended up as a professor at cal poly san
luis obispo
since retired you know she when your
students retire before you what does
that say
cal poly san luis obispo um and uh
done fascinating work in human robot
interaction uh and the like was probably
the first student i had in that
particular space
actually working in the medical domain
uh initially an interpretation of
imagery and
developing very significant irb
protocols to tease certain factors out
of
how people process this imagery from a
cognitive
point of view and i feel i'm missing
several others lilia mushkina has worked
at the naval research laboratories
now is out with her husband it's
interesting when your students
marry each other find each other in the
laboratory struggle maybe with having a
basement laboratory i don't know
but are now happily married and she has
several kids out in california
um
i don't know those those are the ones
that leave uh leap uh
to mind and i think there's a couple of
others as well too but uh
a postdoc i had i worked with was
jonathan cameron he's at oh but here's
i knew there was others jonathan cameron
is at nasa
uh working on last time i
talked to him was aerobots which are
these robots that float up and down
in the heavy atmospheres and uh
khaled ali is a student of mine at the
jet propulsion laboratories
uh as well uh he was part of the rover
group
uh that is out there uh not sure exactly
what he's doing uh at this point in time
but i have to check my webpage if i left
anyone else and apologies to those that
i might have
missed i feel bad about that but i have
not produced a very very
large number of students some people
produce large numbers i
have been i
have been uh careful i guess
in terms of i'm not saying those are not
careful if they produce a lot
but i i try and i produce
a relatively small number compared to
many other people
and now you're also the vice provost or
assistant no no it's not vice president
associate dean
associate dean for research and space
planning in the college of computing
that's enough don't give me the vice
promise position i'll get that too
um and has that allowed you to further
develop the robotics research in
georgia no
i don't i again when you're an associate
dean for an entire college you
especially for research you can't have
favorite children
uh in this case and it has enabled me to
interact with
on behalf of the irim interdisciplinary
center that i talked about
at the evpr's level and intercede
accordingly i'm spending a lot of my
time right now we have a new high
performance computing
building being built on campus which is
going to be
very large work is working with the big
data people
data visualization folks and trying to
assist
the research enterprise within the
entire college
and part of that is the robotic side but
i wasn't hired to
just forget about everybody else and
focus on that that's part of the job
any of you also held leadership
positions in the ieee and other parts
yeah and in several places i've served
uh
two terms on the ieee robotics and
automation
society administrative committee
the board equivalent of board of
governors i was
one of the three founding co-chairs of
the
ieee ras technical committee on robot
ethics
i have served on the human rights
committee for
ieee robotics automation society i think
i'm still serving right now if i'm not
mistaken
and we don't have a whole lot of
business but it's interesting when we do
visa issues and other things like that
didn't
come up i've also served in a different
society the ieee
social implications of technology
society society for social implications
of technology
i served on one term on the board of
governors
i am now a distinguished lecturer in the
ieee
ssit society i served as the
chair of the distinguished lecture
series as well too
for a while so yeah i put in a lot of
time
with ieee uh and uh i'm glad i did
it's uh it's an important service uh
that you do uh
but you do have to balance it with all
the other things that are going on in
your in your life
professional and otherwise as well too
because it can be consuming but
i think it's an important volunteerism
uh is an important aspect
of being a professional and giving back
to the community
uh i think is an important thing to be
able to do
um so you want to tell us more about
your work on the ethics
and well there's different aspects of
the work
that i've done in robot ethics the first
and the most the most significant
project i guess
was the smallest dod grant i ever had it
was less than 300
000 for three years it came about in a
very odd way
i was at a army research labs planning
meeting where they were laying out how
what should we invest in in the future
and i had been thinking about
all these issues in the background i
said well what about robot ethics
well i didn't write it into their
program but the uh the chief scientist
the chief roboticist at the time said to
me uh
why don't you submit a proposal okay so
i was on sabbatical i wrote it in japan
i sent a proposal in and i got it back
it was kicked back to me said who do you
think we are
darpa because i was asking for like
three quarters of a million a year
uh which that's what i was used to in
these projects
these were more nsf-like so uh i rewrote
it scaled it back
and cut it down to 300 000 just under
300 000
for three years which as many people
know and this field is a relatively
small
amount of money so it was partly a labor
of love
as well and so i got the funding
uh the first thing we did was a survey
uh which was part of our
uh statement of work uh we wanted to get
an understanding of
how people felt about military robots so
we
surveyed different demographics uh and
tried to understand how
the general public but the most
important demographic or the most
significant responses we got were from
the roboticists themselves uh
it was very interesting seeing the
responses and people weren't happy
with lethal autonomous systems uh
whatever that meant we did define it
rigorously actually in the survey so
that people could get an understanding
of that and then the second part of the
survey was
that how if these systems were going to
be created we could develop software
which could potentially enable them to
uh perform better than human warfighters
under these circumstances so i created
an architecture
a deliberative reactive architecture
actually building upon pre-existing
architectural work
integrated in several new components one
was something called the ethical
governor
which is a bolt-on piece of software
that basically restrains the system to
perform within
predefined bounds as dictated by the
relevant laws of war and rules of
engagement we had
another component which was the ethical
adapter
which dealt with one of the emotions and
we had a lot of work under our belts and
emotions at this point in time
looking at the moral emotion of guilt
and how that can be used to restrict the
behavior of the system when
it recognizes the fact that what it's
actually doing is different
than what it is expected it thinks it
should it should have done
in those cases which would restrict the
use of weapons under those circumstances
um the other one was a responsibility
advisor
it became clear in my discussions with
philosophers among others
uh that it was crucially important that
we be able to attribute responsibility
to human beings
in this particular process so that no
one could hide behind the robot and say
the robot did it uh and we made
explicit in a prototype and a proof of
concept type of system
how we could uh establish that
particular connection through a gui
basically in two ways one was
pre-mission where in other words before
you send the robot out
you better say i'm taking responsibility
for sending this robot out to carry out
these particular actions
or don't do it and second is also in
the use of overrides where the human can
step in and either equivalent to hitting
an emergency stop butting don't shoot
but don't shoot or do the reverse
allow the human to say when the robot is
resisting firing to come in and say oh
you should shoot and
take responsibility and apply the lethal
force under
those uh sets of circumstances another
component that we had
was uh the behavioral design
which actually tried to integrate the uh
ethical behavioral control
within the behaviors themselves we did
not get as far as i would have liked to
in that case but
i was significantly underfunded to be
able to do that
but that doesn't diminish the work that
the other pieces did
we developed and tested scenarios that
we found
from real world events as much as we
possibly could
and showed at least to my satisfaction
that certain things are potentially
possible
using these kinds of systems which has
led to great
debate worldwide and
like i say as i've often said
the discussion that my research
engendered
is as important if not more important
than the research itself
and i have never made the claim that
this is
the way to accomplish these particular
goals i say it is one possible way
but i also say that others should be
investigating this to find if there are
better ways to be able to do this was
just a first foray into this particular
space
and it has led to all kinds of
interesting uh and sometimes bizarre
interactions through
commentary in the press uh being in
front of the united nations and having
the ability
as you did to speak uh on
behalf of what you believe is the right
way to approach these particular
problems
presenting in front of the assembly of
the international committee the red
cross which do the geneva conventions
to talking in front of pacifists who say
the only ethical robot
or the an ethical truly ethical robot
would throw down its gun
to begin with so
i have waited way up
to my in some cases up to my neck in
terms of the controversy associated with
that
that's because i feel it's important and
passionate i feel passionate about it
because i do believe it can lead to the
saving of non-combatant lives
which is if you not necessarily shared
by all
or even if they did believe that there
may be attendant risks associated with
that which would
say that even if you could do that it
might not be the right way to go
the debate is important it is ongoing
and it needs to continue uh and
i tend not to be prescriptive
uh as some other folks may be i'm just
saying this is one possible
thing that can be done i have stated in
print
and in public i am not necessarily
averse to a ban
but i think we may be able to do better
with respect to non-combatant casualties
and that's my hope
that's my hope that we can potentially
save lives through the use of this
technology and that's
where many of us part company in terms
of the risks versus the potential
benefits which
still are unknown and we're still
wrestling with the definitions of what
is an autonomous robot let alone a
lethal autonomous robot
what is meaningful human control who
wouldn't want meaningful human control
everyone wants meaning equally to
control but at what level
and what is the appropriate restrictions
that should be put into those systems
those are the sorts of things and i've
extended that work more recently
into the work that i'm doing with
management of early stage parkinson's
leveraging the ethical adapter and the
ethical governor uh
into that work i think i might have
mentioned this earlier but
the point being that we're trying to
deal with a new problem
which is a five-year nsf grant that we
have joined with my colleagues
at tufts university in terms of
maintaining dignity between human human
relationships
most of the time you hear about human
robot interaction it's between
the human and the robot and how are we
going to interact
and we want to maintain dignity in that
as well too but quite often when humans
interact with each other
in certain cases the relationship
goes poorly for whatever reasons and an
early stage parkinson's disease
due to this phenomena called facial
masking where people lose the ability
to effectively express emotion on their
face
and their voices become raspy so they
lose the ability to express emotion
through their voice a phenomena referred
to as stigmatization occurs between the
patient and the caregiver
even if the caregiver knows that that's
the case they just don't feel
that this person is being responsive or
uh caring about what they're feeling and
the
patient becomes more ashamed and
embarrassed by
this lack of congruence between the
beliefs of
of the two and so we're hoping that
through the use of a
robotic companion or a presence in the
room
we can enhance the environment through
the use of non-verbal cues
by this robot through both kinesics
and proxemics body language and facial
expressions among other things and said
distance of separation
that will assist in restoring and
preserving the dignity in this uh
relationship so we're leveraging some of
the work in this particular space
expanding the moral emotions as i
mentioned earlier from guilt
now to incorporate uh shame and
embarrassment
and also potentially empathy and foster
empathy in the caregiver
and try and reduce shame and
embarrassment uh in the client
in this case so you mentioned some uh
run-ins with the press and the media
and i know the ethical governor has
gotten a lot of
media attention and so has the robot
deception
work that he did if you could just talk
a little bit about what it's been like
trying to get
robotic research out into the public
i've i always say that if
after interacting with the press whether
it's a film
form or whatever and hopefully this as
well too
if we get 80 percent of it right we're
doing well
there's no way it's going to be 100
right the best article i ever had
was 95 i would contend which was from
the new york times
they came and visited our laboratory
twice they did fact checking they sent a
photographer down
they invested heavily in that in many
other cases things are
fact checking seems to have been thrown
out the window in recent times
which is a bit disconcerting but i guess
it's the
rush to press and uh
even worse is when articles are done
by like the game of telephone you know
they read one article and they yank
something out of that which
looks controversial or insignificant
throw a picture of the terminator on the
page which i give talks on
how to not build a terminator i'm not
interested i don't think anyone should
build a terminator the terminator is the
wrong thing to be building
no one should be building that thing but
nonetheless here we have those sorts of
things so
uh the way in which uh a kind of
it's like a fire running along the
ground takes root but
the important thing is even with that
you know i could take the choice and say
uh because of the misrepresentation
quite often that occurs uh from the work
sometimes deliberate from others but
independent of that
as i said the discussion that the work
engenders is very important so if it
gets people thinking about it
and talking about it and trying to
understand and get their head around
that even if it is sensationalized which
is
a problem with robotic sensationalism
occurs um
getting to the deception work very
interesting things have happened
in that space as well leading to
articles we created this simple article
appeared in the international journal of
social robotics
we had many limitations saying this is
not the final word this is the first
steps more research needs to be done
before we can say anything really
substantial about that
and then some articles say basically the
sky is falling chicken little is
uh is here and it's the end of the world
that
our names will live in infamy because we
taught a robot how to lie
uh uh and the like which seems a bit
extreme
and the same time we win an award
uh from time magazine is one of the top
50 inventions
of 2010 in this case it wasn't really an
invention either it was just a paper
describing an algorithm per se
so i mean it can go off in both
directions some cases like the new
scientists got it right and talked about
how important it is to get a deeper
understanding of theory of mind which is
what we were using
to be able to understand when and when
not uh to deceive
but the other side of the coin is when
we started doing work in squirrel
deception
not human models of deception we did we
did some studies of the eastern gray
squirrel
and we found that they basically uh
used evasive behaviors to hide their
nuts from other squirrels
so that they would patrol empty caches
when someone was observing them
we got lots of press on that as well too
but everybody thought that was really
nice
because squirrels are pretty and you put
a picture of a squirrel on the page
everybody loves it
i'm just saying there's so much nonsense
maybe
or irrational reactions to these sorts
of things the way it's uh
approached that it's a little
disconcerting uh
at times but you get used to it uh
you try and be selective in terms of who
you're talking to
and you try and find what the spin is on
the article before you engage
and you don't talk to everybody so you
don't get
burned more than you would normally get
burned all those things are important
lessons and i would contend that
media training is probably a good idea
for most roboticists
that go into this forum and this even
happens when
i'm asked to comment on someone else's
article as well too
even then things are taken out of
context in some cases
so dealing with the press is an art
in and of uh itself and most
roboticists are not trained uh or ready
uh to be able to do that
so just as fair warning if you're going
to do that
be prepared for uh for things that you
might not normally
encounter and at all costs avoid ambush
reporters as well too who lure you in
with the illusion of one story and
they're actually trying
uh to tell another the most interesting
anecdote um
was uh when the daily show called me up
uh
wanted to do a piece on uh the military
robots the ethical robots as well too
i'm a big fan of jon stewart but they
told me that samantha b
uh who did a field piece uh was the one
that we would be speaking to
they the producer of the show called me
up they wanted to fly me up to new york
uh i got like three calls from the
producer and i said
i love your show. i also know
how samantha b does field pieces and the
like as well too so
keep keep me laughing and enjoying these
things but i can't participate
in these sorts of things so it's
important to know at least from my point
of view
that not all press is good press uh
contrary to whatever that uh old adage
is that as long as they're talking about
me it's a good thing
i don't think that's necessarily the
case and it affects public opinion
uh and alike but like i said in certain
cases
where we're dealing with life or death
issues in intelligent robotic systems
and lethal systems
it is important that we engage in a
discussion in a public manner
and i've been involved in numerous
debates and i will continue to be
involved in numerous debates to make
people think about these particular
questions i'm not
the one that should be telling them what
to do and i would get no individual
perhaps no individual organization
should be able to tell them
what to do but we should present our
viewpoints and let people make a
decision
as to uh what they think
is the right way to handle this new
technology and that's not just in lethal
systems that's in
sex robots uh or intimate robots as i
prefer to call it
healthcare robots there's a i there was
a very interesting study from the
european commission which came out two
years ago which showed
what kind of robots should be banned uh
and they
interviewed 27 000 people in 27 nations
in europe
and i believe the number was 47 percent
said
that elder care and child robots should
be banned
which just blew my mind and basically
that indicates to me that
we as a community are doing a very poor
job of educating people as to what the
risks are
of certain types of systems uh military
was down like it's 17
or something health care was like 30
percent i don't know the exact numbers
but i'm just saying
when i saw this it was actually in a
healthcare in a healthcare workshop that
i was attending
i was just shocked and so i said i got
to see the report and it's really
available on the internet as well too
most interesting
okay so what are for you the biggest
sort of theoretical questions or
technical issues facing robotics
well there's two different aspects of
that what is what are the
issues preventing
large-scale uh dissemination of the
ideas that we in
academia and elsewhere are creating and
as i've often said
uh many of them are not the intelligence
of the robotics questions it
is things like safety engineering
reliability engineering power
these systems if we're going to get
broad public acceptance have to be as
reliable as our
automobiles or laptops at the very least
even if they do crash from time to time
uh you need 24 7 operational
capabilities if you really want them
to work that's important but that's not
so much the robotics questions
systems integration is absolutely
crucial to be able to put these things
together in meaningful ways a better
understanding
of human behavior again going back to my
roots
of ethology and neuroscience
if we can elucidate more uh how the
brain functions
which we are beginning to make
significant progress with new
instrumentation uh
that is available um how can we put that
to work
uh in intelligent robotic platforms
that's an important aspect
as well computer vision
remains daunting it has for decades
significant progress is being made but
we're still far
far away from anything ever approaching
general vision
ethical reasoning i think is an
important uh aspect as well too
keep in mind that i don't believe we
need full
human moral faculties to do the sorts of
things that i'm advocating in this
particular case
i'm dealing with compliance rather than
reasoning
those are important aspects of the
problem machine learning is something
i'm more worried about
perhaps than most i
don't want robots necessarily
to learn in unexpectable uninspectable
ways except in
bounded environments as well too
tremendous progress is being made with
new technologies in machine learning new
algorithms in machine learning
where they will lead it's hard to say
and then it gets to the questions of
regulation for domestic use
whether they're for drones and privacy
issues and all these other things as
well too
if we can create it will we be able to
use it as an important aspect
as well so those are the kinds of things
i focus on the good news is cpus and
processors and the like new sensors are
rolling out
at a highly uh in significant rate
uh we can expect new sensors to be game
changing things like the connect and
other sensors
were flash lidar will make a difference
as well too
and the cost will come down so new
applications uh
will unfold so
there's hope for many significant
advances but the field as i think i
mentioned earlier but the field is
goes in fits and spurts
quite often there is a fundamental
advance or a paradigm shift that occurs
within the field
and then for the next 10 years or so
people catch on move that forward and
make steady progress
with it enhancing it and then another
paradigm shift
goes on some of these things were
behavior-based robotics
uh then probabilistic robotics and
now we're in the phase where we're
making uh continuous
advances but slower and so i always
wonder
what the next big computational or
intelligence
breakthrough will be which will change
the paradigm i don't believe it will be
a positronic brain i'll
uh azimov but i do believe uh they will
continue
uh to occur and uh
they are completely and utterly
unpredictable as scientific research
often is
so is there any other projects or work
or things that we missed
oh there's dozens of projects but the
most recent ones are the
the parkinson's stuff uh we're doing
work
uh in the deception stuff that we talked
about as well we're looking at other
deception right now as well too
which is deception for the benefit of
the mark so that
you can tell me after this that was a
great interview
ron whether it's true or not i'd
probably feel better
uh because of it and the same sort of
thing if someone
is bleeding out on the floor uh you
don't want to say to him
oh you're all messed up you'd rather
perhaps try and reduce the risk of shock
occurring by saying help is on the way
somebody get help
to do these sorts of things there are
times
if you're a non-kantian if you're a uh
a consequentialist in this particular
case uh where you're judged by outcomes
lying maybe or deception may be
appropriate
and other schools of ethical reasoning
that is utterly and completely
false and that's part of the reason why
ethics is uh
is so hard so there are four projects
that i'm actively involved in one is
dealing with mental rotations and
primates
and the the formal methods as well too
for the defense threat reduction agency
encounter weapons of mass destruction
where we may be
it's not a question of if it's a
question of when we find ourselves
under attack by a biohazard a chemical
weapon
a radiological weapon or a nuclear
weapon
and we may have the opportunity to
deploy robots or robot
to address that i refer to our robot in
this particular case as jack bauer
from the 24 series we're trying to write
chloe i guess
to make sure that before jack goes in
there there's a high probability of
success
or don't send them in at all and we
always grab up with this question which
is
what's your advice for young people who
are interested in a career in robotics
get a solid foundation it's a question
of how young you're talking about but if
you're in
high school or are younger pay attention
to the basics
pay attention to mathematics pay
attention to
chemistry and physics don't be
seduced by the quick pull to
creating cool little robots that are
moving around you can do that that's
okay
but you need to have a deep
understanding
of the fundamental principles upon which
robotics
which is a science is based upon pay
attention to that
and work hard and you have to have a
passion for it
if you don't have a passion for robotics
uh in almost any
field for that matter as well too you
probably shouldn't go into it
don't although you can make a good
living in it it shouldn't be about money
it should be about
joy fulfillment and creating new
knowledge
great well thank you very much you're
most welcome peter thanks