well it's okay okay so my name is
hiroshi shiguro
um i i am a japanese and
i born in asia prefecture in japan um
and i studied
you know computer science when i was
undergrad student and
you know i studied the cultural theories
and artificial intelligence
when i i was in the student of
the graduate school in osaka university
and
but i was not so serious student when i
was
undergrad students and um i tried to be
a professional painter but
you know eventually i gave up to be a
professional painter
and then you know i well they
started the computer science and
artificial intelligence very hard
and now you know i became a roboticist
and who was your advisor well the
advisor in graduate school is professor
sabro tsuji in his pi one of the pioneer
of
computer visions in the world i think i
believe
and he was very famous professor in the
areas
and rather you know
when i was the who was in his
laboratories
um um i i saw the computer vision is
over
right we need to extend the the dart
areas
then you know i developed a vision
guided vehicle
and i i i and then i move into the pure
robotics
but but intelligent robotics not
traditional robotics
so what was the first robotic platform
that you worked with
um it was a very simple uh
robot it's a combination of a camera and
a mobile platform
right um but after that you know i have
developed the
robot that has the four cameras each
camera individually moves
like you know
and what were some of the early projects
that you worked on with the mobile
robots
um the purpose is uh well the uh
today the people say it's the slum
simultaneous knobby navigation mapping
and navigations
so i know that around 1990
uh we studied about the um
some sort of a sram right but we use
this
for navigating on robots and and and
same time i have developed the armor
injection
cameras and and by using arm reduction
cameras
you know i i have developed the um
another simple robot right
okay or the navigation was that your
phd thesis project yes yeah and my phd
was
uh was for the active visions and the
army direction visions
and panoramic visions the uh
well the activision is in how we can
control actively control
the cameras for observing environment
and panoramic or in army direction of
vision is
you know how we can use the 360 degree
views
for uh localizing the robot or
navigating robot in the environment
so i you know the focus on that kind of
two different types of
the vision systems did you integrate
those into the same platform
um honestly speaking it's no
right and but before the my project you
know there nobody
they see nobody seriously think about
how to move the camera
right so in my phd thesis
i focus on the two types of movement one
is
tracking the person or tracking
something and by
actively controlling the cameras another
is you know just
you know um panning the cameras
in order to get the 360 degree views
right
so so that is our introduction or you
know panoramic
vision was there a specific application
that you were thinking of at the time
right uh actually you know i think
i could you know they i i could develop
a pretty interesting technologies
you know by using army direction cameras
the well the robot does not robot could
develop the
uh environmental map and localize it
itself without any internal in
sensors like inheritor sensors you know
there i think i like that work very much
but uh
you know when i submitted that per paper
to the uh
the top class conference in computer
visions you know the reviewer said this
is a kind of a daydream
and rejected but you know and eventually
i have submitted the
uh the that paper to the i crowd i i
remember microwires and
and uh you know well i got i think you
know
i got a pretty good evaluation right and
after that you know i started
to write many many papers on that
how did you decide to be in professor
tucci's
well um well
you know because he is the most famous
and professors in the computer vision
area in japan so that
and how did you get interested in
computer vision um
you know well as i said you know i try
to be oil painter
right open well it's closely related to
the computer graphics and computer
visions i was
interested in how the human recognized
the visions so even early on
your work with robots was inspired by
thinking about human
yes yes always you know um
as the oil painter you know i was
thinking
you know what is human how we can
represent ourselves on the campus
right so i i believe you know i'm doing
the same things
did you continue your work and vision um
yeah and after when i was undergrad
student uh sorry the
graduate school right and and brother
recently you know
i was not so seriously working
in a computer vision areas but you know
i i
i'm interested in more human robot
interaction right
and so after osaka you went to kyoto
university
yes no um after getting a phd in osaka
university and i became the uh
assistant professor in osaka university
i'm sorry
yamanas university and in osaka
university i back to the osaka
university and spent
two years and then i became associate
professor in kyoto university
and then i i was in
ucsd one year and back to the kyoto
university
again and and moved to the wakayama
universities
and became a professor uh in that
university and back to the university
so did your research projects changed as
you change locations
yes right yeah i have some par i have a
policy
you know three years a maximum for one
project
the first year we prepare something the
second year we
run the uh experiments and so and for
our third years you know we write papers
right then you know we should start
something here right you know to start
something new
the the the best strategy is to change
the environment
right you know if we move to the another
university different university
you know the people expect me to do
something new
right now i can i can have a good
pressure from the environment
then you know that i can do something
right so always i'm thinking
you know so that is not a typical
japanese
like american but i like that kind of
you know the lifestyle
and so what was can you tell us a little
bit about the environment in the
different places if you start with
yamanash university
who was there what kind of things um
yeah yeah actually you know
when i i gave up to be a professional
oil painters you know i
was the one well
well i couldn't decide which robot is
good or not right
but eventually i i have met the
professor mori
and he was you know developing the
vision guided robot
and and i like that project and
and but you know his laboratory was not
so popular and therefore
i could join and you know um
i seriously started to study the
computer visions
and and after that i moved to the you
know osaka university
um so in your first right after your phd
um what kind of work did you do after a
phd
well you know i continued my work
a little bit a couple years maybe three
or three years
and when i was the assistant professor
in yamanas university in osaka
university but
when i moved to the kyoto university as
associate professors
you know um in japan the associated
professor is not independent you know we
belong to the
professor's laboratories and the
director of that laboratory was
professor ishida and his very famous
professor researchers in ai artificial
intelligence
he asked me not to write a
journal paper or conference paper
anymore you know
before joining to the kyoto university i
have written so many papers and
every month i have published a journal
papers and
you know he said it's good but you know
that isn't enough
right now what i have to do is to change
well to develop something new that can
change this world right
so that is you know the very important
mission role
of researchers and you know i
started started to think what i can do
with a robot
right with computer visions then you
know i started a two project one is
the project with interactive robots the
other is the you know
um the sensor network multiple camera
system
right um after that you know we um
i i'm always doing two things one is for
you know ubiquitous computing or you
know sensor network and the other is
you know interactive robots how did you
first approach
interactive robotics first pro approach
was
you know um when i started the uh the
project in kyoto university we didn't
have a
a good platform so that you know i have
developed the uh
a little bit a big robot um
i forget the name of that robot it was a
commercial product
they made in u.s and we have imported
and then you know
i i have designed the uh pantered
camera controllers and and put it on
their robots
and and you know i have developed the
some sort of
sim not so simple but the
mobile platform with a pantered camera
and and then you know the um
we started work with that robot and
you know but well the rob a rollover
robot was
not just the uh move around the
environment but
you know to interact to talk something
with people
right and we focus on the interaction
with people
and so what kinds of questions in terms
of interaction are you initially
interested
yeah well you know how robustly uh
the robot interact with people right
and at that time you know the camera was
well the quality of the camera was okay
but the computer was not so powerful
therefore you know that we have
obviously we had the limitations about
the vision process
right and for example today the
digital camera has the uh function of
face tracking
but you know we didn't have that kind of
function at that time
right and but still
robots need to get some information from
the cameras and take some of the action
and and the question was how we can
continue the
interactions between robots and humans
right
and the and the role of the robot was to
guide the
person to the destination right
but you know they're doing the guide
guidance the robot need to interact with
the person right right so we try to
develop that kind of
you know the guide robot and what were
some of the things that you found the
guide robot needed to be able to do
well um the
first of all the the robot need to find
the person but
you know the computer vision the cam
well um you know and the computer vision
technique was not so rich
right therefore you know um we needed to
develop a kind of
situated the uh intelligence right
um we need to put some knowledge about
the environment you know
and we we needed to limit the
well how can i say the possibility uh
for analyzing the
divisions right so i call
called it the situated visions and we
carefully design
that kind of you know the building
process for the robot
and what kinds of aspects of the
environment was the robot
well it was just you know the simple
corridors in the universities
right and well i
i wanted to have some uh robot that
moves
in out the door but uh it was not so
easy
right so you know and and
we could develop the robot that guy the
people
the in indoor or in a building
and was this project funded by the
university or outside
um basically you know the university
does not have much money in japan you
know usually we need to
submit a proposal to the government like
nsf you know we call it
the gst in japan you know and jsd and
jsp
jsps you know we i
wow i don't remember exactly but i was
get probably i was getting some money
from the jsps and yesterday
did you were you related to the robo-b
project yes
okay the robot project i have started
that
when i developed the interactive robot
in kyoto university
um i wanted to develop much better
robots then you know
i went to the atr and
i started started work in hr
um and actually you know my group was
i started the my group with just another
two people
and we are three right and and i have
developed a lobby by myself
and after that you know the atl robotics
group and keep growing it
right and now i i don't remember but uh
probably we
have more than 40 or 50 people there
and who were some of the people that you
worked with at the time
um professor imai and from kerry
university and
professor owner from hokkaido university
now they became the professors and and
they
they have own laboratories there
and so what were some of the things when
you were developing
um robov that you
did to kind of improve on existing yeah
the lobby when i developed the robbery
what i wanted
i i want i wanted to have is uh
some robust you know the platform
interactive platform
with the two arms and of course you know
the two eyes and
necks and two arms and tactile sensors
and cameras
ultrasonic sensors you know i have just
gathered the
you know very reliable sensors and
actuators and and
and i have designed or developed
that the robot and i think it was good
you know and the robot
we didn't have any problem with the
rubber band robot could work uh the four
hours with the batteries
right there and we could have many field
tests with rubber b
and after uh building on robberby what i
did is
you know the field test um and
we um we had a field test in elementary
school i think
well with my memories i that was the
first challenge in the world
right and the um
but and we we checked the you know
student reactions right
nobody was speaking in english right
and you know and the roboview was a kind
of
the foreign student and you know
and by speaking
and they know we interact with the uh
japanese
uh the students uh uh in english and
and and then you know the there i i
guess you know nobody could give
uh could give a motivation to study the
english for
japanese students so interacting with
robots interacting with humans seems to
have been an interest of yours from very
early on
but i mean perhaps at those early times
it wasn't as common
why did you have the idea that robots
would need to interact with people
um okay um right
um the the when we said robotics usually
you know
you know we developed a robot that works
in the uh
industry the machine shops right in the
world in the industrial
industrial robot was our you know the
main focus
as application of robots but uh um that
is over i think
you know at that time you know that we
needed to find some new
application areas and then then i
uh i was seriously thinking about that
right and as
the you know professor ishida said you
know how we can change this world with
my technology
and then well um and usually
um as a new applications the u.s people
and some
european people they were thinking to
develop a military application with a
robot but
in japan you know of course we don't
have a military and
it's illegal to work on the military
applications
and but on the other hand you know as
you know the japan has a pretty good
technology for assembling a small staff
for you know
the and many companies that build uh
making their good products
uh for their electric equipment
so that i thought you know they are um
uh
in the near futures in electric
equipment and they're gonna
be kind of robots right and then if we
improve the interface more and more
you know that's gonna be interac a kind
of interactive robot
so that was my ideas therefore you know
i started the
internal interactive robot project
when did you first first start thinking
about humanoid robots in particular and
was that a response to
what you learned about early yeah well
the robo-b probably is also the kind of
a human-like robot
but i had i
i had a basic idea that the human brain
has a
function to recognize humans not for
other
you know well for example computer um
the monitors you know right and and just
text that they are not so good
right and someone cannot read the why
japanese uh
well i mean probably you you cannot read
the japanese so that then even if we
showed some
you know japanese text stones on the
screen well
it it is not a good interface right
rather you know that well even if we
cannot speak
uh even if you cannot speak in japanese
you know by
taking some gestures you know they well
we can communicate right so that right
so that means that you know the human
need to have a human-like interface
right
the problem i i i have designed a kind
of human-like robot
right see so that is that was a robbery
right
but um the way i have developed the
robbery
and and and then in the mitsubishi
industry
they made a copy of roboby and they
developed the
guacamole the one kamal is a kind of a
brother of a
brother robot or roboby and but i didn't
like the
design right you know i asked to
give more human-like design to the
guacamole but uh you know
that famous industrial designer uh you
know
they gave us a kind of an insect-like
appearance but
it was okay as an industrial design but
you know for me
you know i wanted and well my policy is
the robot need to be more human like
probably you know that we
where we um we have some strong reason
for you know for this kind of human-like
appearance
and then you know i have decided to
start the android
project or enough to understand the
importance of
you know human appearance right well you
know of course you know the my
the purpose was not just me make a copy
of humans but
you know to understand the important
factors and for the human robot
interactions
importance of a human-like appearance
you know
we have we are getting much influenced
from the very human-like appearance
you know so we have well as you know you
know when i developed the gemini
it's my copy and we we did many studies
on that
and and we could find that kind of you
know
the human natures and where did you
find funding for the android project
well the first funding was
you know the what's coming from my gsps
it was a very small
amount of that but in 2004 we had the
world expo in ig and one of their focus
was
robotics and then on the government they
are you know give us a good
amount of money and then therefore you
know i i could develop the
female android first android a month
sorry it's a second android and the
first android was my daughters
right and and after that uh well
i guess i could establish a kind of
research areas for
the android and and then you know and
like
i could get another the big amount of
money for
building their gemini
gemini and another geminid and what
motivated you to make
replicas of specific people yourself um
well i started to have a kind of a
badger budget
appearance virtual humans you know it
was the
average phase right and then there is a
face
i don't know the exact english name of
that
society the japanese society for the
face
right and you know they're studying
about the you know human face and
um how and human facial expressions and
they have
the computer graphics for uh taking the
uh average of many humans face and
and you know well interesting things is
if we take the average of a human face
that face is going to be very cute or in
the hands of the
smart right and then you know i have
decided to
you know use that in the bachelor phase
but actually
android was not so human-like i mean you
know it was like a door
right so um you know
we don't know how how we can represent
human likeness
in detail right therefore i gave
give up to use the in that kind of
virtual face and and
and and you know i have hired
professional persons who can make a copy
of
the humans and actually there were she
was working in hollywood and
um and you know and she was pretty good
for making a perfect copy of humans
right and and you know and that is the
reason why
you know that we are using the uh um the
uh
uh the human the the model
um and the important things is not to
finish the
uh uh the uh the android right so we
need to
um uh well we need to put the real
person here
and android here and and and we you know
and that professional person
that world doing some
you know perfect makeup for the for the
end right
but i will say again we don't know
what what is the important the factors
for making the you know the human
likeness whatever they know well
uh well um right so
you know um for example
um if we have a single picture we can
make
android of the pictures but that is not
enough
right now the picture does not contain
enough information for representing the
human likeness
actually we need a real person there
right then we can have a perfect copy
you know so um therefore you know we are
always using the
real human model for the android
and how did you go about trying to study
facial expression and movement and
replicate that
systems um steering on the facial
expression for the android is limited
but uh um
we um that is a
kind of a very important know-how for
the companies you know we worked with uh
coco
it's a company you know uh the building
and uh
who are building the android and you
know they have
they have many know-hows how to use the
silicon materials
therefore uh for representing the uh
animal skins and human skins
and then you know always we are you know
we are improving
they have to represent the more better
the human expressions and
and so that is you know um
i always say that that is just a
know-how
and we are improving the know-how every
time
and you've done um also a lot of
research on
kind of human presence so how you can
show that with robots and a number of it
is with these
with teleoperated robots do you think
that's necessary
to have a human kind of put there
no you know the most important things is
uh
for representing human presence is a
subconscious movement
when we are sitting we never stop so my
eyes are always moving and
shoulder is also moving head is moving
you know subconsciously
slightly right and if um you know the
completely stop like this right it's
quite strange right
that means a dead person right so
subconscious movement is quite important
and the the another important is you
know the actions
when i touch or someone touch them to my
hand you know usually i take some
reactions that kind of reaction is quite
important then
you know to have that you know to
implement that kind of reactions i have
developed the
very soft sensitive skin sensors and in
the beginning of
in the early stage of android project i
have spent
more than one million usd for developing
that but it was good right there
and we could develop a very sensitive
soft skin
sensors and and then android became more
human-like
and so that that is the basic you know
the human likeness
right and in addition to that you know
the an android
it's better if android can talk right
um well i under does not need to talk
like for example recently
we are exhibiting the android uh for the
show windows
you know um probably you know the our
activities
in hong kong and and takashimaya
it was the shinjuku right
and my idea was you know the um the
mannequin is not ideal
uh and for the short windows you know
what
we want to do with the show window is to
to show the somewhere
futuristic scene right and
a kind of the elegant life right we want
to show that
right but not to show that you know and
of course we want to use a human but we
cannot use it
right human cannot stay in a window
right so that therefore you know i think
an android
is much better and better than the of
course the
human and the mannequin right um and
then we have implemented that kind of
subconscious movement and and
we have also implemented the simple you
know interactive behaviors
not talking about you know and the
android you know if
someone look at android and and and and
and
well and we can smile back like you know
that's kind of you know
well the number of uh interactions
but um so no that subconscious movement
and number of our interaction is quite
fundamental and it's important you know
to be a human right
and and then in the icon to that you
know that
well you know if android can speak it's
much better
and did you realize the importance of
the subconscious movement
through showing the android to people or
was it from
reading certain theoretical words
i don't remember exactly but
i had that kind of intuition right
um you know when i saw the
static android i thought right this is a
strategy
this is a kind of a dead person we
should make it move
right were you thinking about masahiro
mori's
uncanny
[Music]
i faced to the uncanny body issues
when i developed my first android
with my daughters um notice their bodies
and you know
um at the time i didn't have enough
money for
for buying the enough number of motors
and actuators therefore you know
i have just installed eight motors on
the head but i didn't
install any modules you know
my purpose was to have a human-like
appearance
right and and well it was okay
appearance was okay but in a moment when
the head is moving the body was
shaking like this right um it was a you
know
like a zombie and that is the uh you
know uncanny valley
right so my daughter's copy was in the
bottom of the uncanny body
and then you know they i found well
eventually from the and some i guess
then someone gave me
some important information about their
uh professor morita and getting bodied
and then you know we started the study
about that
what was the most surprising thing that
you learned
from your early evenings well
um yeah when i developed my daughter's
copy you know
well of course the uh movement was jerky
but
in that point we just put it on well
against there
right now okay
i'm gonna explain like this so the
suppose this is my daughter's copy
i tried to give a hug right and then you
know
and i get close to my face to the
android phase at that moment i got a
snare
on my daughter you know it's a bunch of
things but you know
well now and probably you remember that
did you try that kind of things with my
android and female
you know you know there's a human like
appearance you know
they they well they give us some
you know the memories about the snail
right you know
just giving a hug just you know closely
watching the
face of the daughter's android i could
remember the snare
of my daughters but it was
not remembering you know it was quite
realistic
you know and well they are feeling
so you were actually experiencing
basically herpes
can you talk a bit more about the skin
sensors that you developed and what the
challenges were and how you used
it now challenge was you know we didn't
have any and the soft skin sensors
that are before our project so
um you know as you know the well this
skin is a very flexible stretchable but
still
in addition to that this is quite
sensitive the question was how we can
develop
this kind of a very you know soft and
stretchable and
and and sensitive and dense
sensors and idea was my idea
was to work with the material science
professors and he had a good technology
for
the printing right but i know his
printing technology is stretchable
right and and and and
and and then we use the silicon
uh uh silicone rubber plate
and we printed the uh the wiring
and then we put the small piezo
film sensors you know this is a kind of
stretchable electric circuit
right and and that was the you know the
our the skin sensor and
yeah but you know the honestly speaking
we are not using that sensor
today because you know the mass
production
is different issues right again so
um there is no market for that the
sensor yet therefore you know
we couldn't reduce the cost it takes a
cost a lot
so the androids that coco produces um
are not they don't necessarily have kind
of
sensors in the skin yeah they need you
know and whether we are using the
you know just um
well the commercial products of
uh pierson theorem right
our sensor is is good it's good but you
know it
take a lot of cost for producing that
when did you make the sensors um i think
uh
i started that project the in very very
early stage of android project
probably i studied android project 2001
or
2000 or 2001 i guess and
and i spent six or seven years
for having the final products
were there any other technological
things that you needed to develop
while you were building your android of
course you know
we needed to develop the control uh the
the programs for the pneumatic actuators
right and this is how the model is quite
easy to control that but the pneumatic
actuator is a non-linear
and and and it's also the compliant
right it's quite difficult to control
that
and in addition you know the well for
example android
the female android
has 43 actuators
pneumatic actuators if we move the just
one actuators
the other object also you know moving
right so we
that means we have to control all
actuators simultaneously
therefore we have developed the various
well the computer program for
controlling that
um you know they will we try
the several ideas like neural networks
or you know
traditional control theories and you
know
and it was the student project
there are other um at osaka university
there are some other
also robots that use pneumatic um
um gemini gemini ah
right right you remember that okay yes
right right are they all
connected with yours or did everybody
develop their own actually you know i'm
developing a robot for
the baby robots right um the
six years ago and professor assad you
know he started the big project
and the i and the professor knew she
was a horseholder and another you know
the neuroscientist
there we are the member of that project
and
each of us we had a group and my group
in a lot of my group was to develop a
platform
for baby robots and you know the study
about the
uh social aspect of the robot i
said it's interactive aspect interactive
function of robots
and at the time uh in the project you
know
i have developed a small baby robot and
big and
baby levels cb squares the big one you
know
i i used pneumatic actuators but that is
a kind of full spec
android right and i have installed
around 60 in pneumatic actuators it was
so complicated
and you know the and and one of the big
challenges
how to control the 60 pneumatic
actuators
right and the honestly speaking i think
uh well
it's almost impossible and but you know
we also have
so many muscles how we can learn
to stand up and to work right so usually
you know the
the caregiver need to have that right
therefore you know
we we took the same strategies you know
how the robot can run the uh based on
the
help of caregivers right and and
actually caregivers that lift up like
this right
by holding the two arms right and then
you know nobody could understand which
actuator is important to stand up
and nobody could run something and that
kind of
understanding of behaviors and working
behaviors
with the help of our caregivers
um also the university there are quite a
few robotics groups
how much do you interact with each other
or are your labs very separate what is
the
um well the merit of osaka university we
have two faculties
and one is engineering what the other is
engineering science
and both the faculty has the mechanical
engineering and control
theory groups and artificial
intelligence groups right
and
that is a unique aspect of university
usually you know university has a
science faculty of science and faculty
of engineering
but we have you know i mean engineering
science right
and so you know
i think the we have good
activities and they i don't remember
exactly
how many laboratories we have for the
robotics but maybe we
have more around the ten yeah
and then you know each group has a
little bit different advantages in
different areas and then you know well
we can always you know try to
collaborate
and try to get the bigger their funds
and their
grant from the government by working
together
and when you went to ucsd what did you
do there and who did you work with right
i worked with the uh uh
uh mohan tori berry and he was in
in tennessee and he just they came from
the tennessee and joined to the ucsd
and
i also work with ramesh jain
i mentioned that he was a very par
famous people in a computer vision
and the mo hunter rebate professor mohan
river was a famous professor in
the robotics you know an ambassador in
the
my host and i also the works
with people in the cognitive science and
neuroscience
and uh ucsd has a pretty good uh
the groups for the neuroscience like
cenovsky the professor sanofskis
and and cognitive science group is also
very very traditional right and they
have
uh don't know one they had right and
there are many friends there so
you know the basically
the uh my uh role was to
do the thermosync with the robot and
with
a multiple camera systems and
you know the actually the mohanto rebate
he prepared the
laboratory space and student for me you
know
my stay was just one one year about
steering on
and he could and he kindly they prepared
the
pretty good the environment research
environment for me and and
they're probably not i could do the many
things in utsd
and same time you know by the way
talking with the uh
different areas uh people in different
areas neuroscience and cognitive science
you know i could start that
interdisciplinary
uh that works there and then right now
i'm hiring the
neuroscientists and cognitive science in
my group and you know because that is
quite important for studying the android
and
and the geminids right and you've
developed kind of a concept
called android science that com that is
this interdisciplinary
space can you tell us a little bit about
that and how you decided yeah
um actually you know there are two types
of uh two ways
that how we can understand the human
cell right neuroscience and cognitive
science they
their approaches is a kind of bottom-up
approach you know they
try to identify you know the very brain
functions
and some sort of you know cognitive
functions right
and try to build up the big pictures
about the humans
but uh it takes you know even if
we did the textbook of cognitive science
and neuroscience
you know but still we have many unknowns
right
robotics people cannot build a robot
based on that kind of textbook
but in order to add the
develop the robot we need to have some
the other the ideas
or and the system level hypothesis about
the
mechanism of humans right and you know
of course we don't have a perfect
knowledge about humans but we have some
you know the uh knowledge about the
system or robots
right therefore you know if we um
we can try to uh you know we can try to
build a human-like robot
right and with our hypothesis and with
their hypothesis of course you know
and then you know if a robot became
the human like that means you know well
we could verify
the hypothesis right and then we can
improve the robots more
and live and and neuroscience and
cognitive science people they can use
the our robot
therefore as their platform right
you know they by by by using the android
they can have some
you know the experiments and they can
brush up their ideas and
then they can give us the another
hypothesis
so that kind of you know and
collaboration is quite important
then therefore i say it's it is not just
engineering right it's a kind of science
and android is a term uh terminology for
the engineering
right but science is a science right so
therefore i call it in android science
um and what kind of uh who are some of
the maybe cognitive science or
neuroscience people that you work with
um the many of them well for japanese
japanese professor hiragi from tokyo
university
and the professor itakura from kyoto
university these people are very very
good uh cognitive scientists
neuroscientists
wow um well for example uh
uh dr kawada from atr right now the
recently we are running the
collaborative project
in in adr and osaka university and we
also have some other you know the
neuroscientists in osaka university
and you know the uh neuroscience and uh
for example javier marbellan
you know diploma ucsd and he is also
running the very similar project
to our you know professor of the project
actually um we i helped to develop the
uh
diego sun you know his robot's name it's
it's a cop
a kind of brother robot of a cb2 right
and i have to develop that robot and
well that means we are sharing the kind
of a common platform and
and he is he was in uh
carnegie mellon um the department of
cognitive science and economy and he
moved to the ucsd
and he's not pure according to scientist
and neuroscientist but he's pretty good
for
neuroscience and cognitive science and
from the 2000
and when i was the i was in the ucsd in
1999 and 2000 and
after that you know we were always
working together and we always exchange
information
and you give a good description of how
you the cognitive functions can kind of
map from cognitive science into robotics
but
how do you see the the neural models
sort of mapping do you think you can
directly model these kinds of neural
structures or do you just get
inspiration
um wow for example you know um
we can learn something right well
for example how can i say that you know
there is a model
the human brain model to control the
bodies
it's based on the uh central pattern
generators
and you know by using a neural network
we can
decompose the central pattern generators
and you know and we can
use that model for controlling the
android behaviors
so that is just one example but you know
um
there are many works about the uh frame
functions
and they have some model but still it's
difficult
to implement that kind of a human brain
model into the robot
it's quite limited especially for the
human human like robots
right and they're very different and
there is a you know the
very uh um this is a challenge
in in in utsd it's uh uh
not so consistent it's a neural
institute and there's a professor
i thought i forget the name of that and
it's quite famous
their project they're they're
neuroscientists
and and on on on the other hand they're
also they're building robots
and what they're building is you know
just their mobile robot
and with cameras and maybe the two
cameras
and the sensors but it's quite simple
robot
well for that kind of a simple robot so
we can directly
install the brain model but
[Music]
and honestly speaking the robot cannot
do anything like a human
right it's quite basic they're the
behaviors
right and so therefore you know i mean
um well and you know to have a more
human-like robots
still we we can get some ideas
we can we can get some hints different
neuroscience as i explained right um but
uh you know
we cannot freely install their you know
the model
into the robot i think
in what ways of acting and performance
helped you understand
human robot interactions acting
performance what
uh
[Music]
is also the android science for me you
know
um psychology and the neuroscience the
cognitive science
they're just focusing on the particular
behaviors
are kind of common behaviors in all
situations but human is associated
right you know the psychology the
their you know the observation about the
human is
it's maybe quite general but human is
not so general
we are very situated right so actually
you know the
um director has much knowledge about the
human and the director knows how
you know then they well um
you know for example when we're
interacting
so how much the time interval we need
you know when you say something if i
quickly respond to it
right and and okay
well let us constitute two cases in a
quickly
response or you know i take some time
right
just uh well three seconds and then five
seconds
you know if i take a time so you think
i'm thinking something right
by by changing the time interval then we
can
you know we can change the meaning right
so the theater director knows
that kind of thing spam a lot right and
but
you know um cognitive scientists and
psychologists now their interest is more
how can say more general not so straight
you know they cannot explain well the
human behaviors
according to the situation but the other
director can't explain that
right what we want to have is a theatre
director knowledge
right because we want to you know they
use a robot in an illustration
and we want to know how you know well
what kind of a human-like behavior you
know that we can install to the robot
right now i'm running the many things
from the seattle directors
and actually you know we are submitting
many patents on that
it's a kind of meta level knowledge
right
and cognitive science and the you know
the psychologists they're focusing on
the very small
the lower level the staff but you know
and and the theater theater director is
looking at
more global you know more higher level
the behavior of a human and then you
know we can learn many things that is
the motivation that is
why i'm working for the under the
theaters you know
that is not just profound and who are
some of the directors you've
collaborated with um
he's very famous most famous see
the director for the small theaters um
the theaters up to their
200 people or 300 people and
he he's also the professor in osaka
university professor of the
communications
you know and he knows and because you
know
he he has deep knowledge about the you
know
human human communications and and
cognitive science
right so and he's also very famous in
korea's and france
the most important things is to get the
good reputation in france
then you know and artists can be a real
artist
what about actors do yourself actors
yeah actors
well
he has their own theater companies uh uh
named the sane and sanenda means just
young people group
and they're not doing well but the
problem is uh
you know it's quite difficult to survive
as the actor or actress right
if they appear in the tv right you know
they can survive but you know
therefore the small seizures you know
it's quite difficult
therefore you know usually they're
taking some part-time jobs or you know
i know not not for everybody actually
you know
that theater will be supported by
government
right not so much like france france is
spending a lot of money
almost same as the research grant you
know you know that you know
the francis supporting the artists quite
well
very very well you know if you want to
survive as an artist you should go to
the france right
but in japanese you know the japanese
still support is maybe the one of ten
you know france
but but still you know the sane india is
they're getting
probably enough support from the
government you know because our
professors or is a hero
i'm just i have to just run up and get
the next person but
i'll continue
i said it on the workshop on the
performing robotics on
monday but um can you tell us a bit more
about how you're using robots and
theatrical performance
um yeah you know the well what we want
to do is
we want to change the this historical
the traditional
the theaters you know the software that
we result in the
um theater should be played by a human
but that is not true right sometimes the
robot
android can represent human likeness
more than human
right so that was our challenge right
and
and you know um there is some there are
a little bit different activity about
robot theater
and they all it looks like uh
the puppet robots you know but you know
um
our field is different right our field
is
the real sealed i mean
like traditional
theaters right um we want to have a
professional
the activities with robot and android
and not
just you know a kind of exhibition and
actually we could we got a
good success in europe and france
especially in
france and we got a pretty good
reputation therefore you know we are
getting many many offers
from european ages and we are always
traveling
so great and
i forget your questions probably
i could answer someone well what are the
biggest technical challenges
is uh um um not
not not to stop the robot
i would need to keep work keep working
and you know
we can not uh well
um well once the robot stopped
and robbed that problem right we need to
stop the theater there
but yeah it is it's too bad
so we need to have a very reliable robot
and we need to have develop a very
reliable computer program for
controlling that
you know so we need to have defined
uh the right they have to develop the
very robust
robot and robot the computer compute uh
the robot programs so and and
my staff to work on that maybe the two
two years and
always you know well now you know well
it's okay you know we can trust our
robot but
in the beginning of the theater
activities you know all
everybody was worried about the
you know and when the robot stopped
carrying
we were careful watching that but it was
okay now
the next step is to make it fully
autonomous
i mean you know um well you need to have
a fully autonomous route we need to have
a sensor network
right and as i said you know that
i i i'm developing two types of systems
one is the sensor network one the other
is a robot
and we could establish the good
technology for the sensor network
and now we are preparing for the sensor
network for the android theaters and
robot theaters
and then you know that we can well we
don't need to have any operators
to monitor the syst and the whole
systems you know the computer
and the monitor the whole activities and
the department perfectly control the
robot
how are you building the sensor networks
um
our idea is to use the laser scanners
very tiny laser scanners and is a
scanner is much better than
cameras like we're going to combine
laser scanners
and cameras as you know they connect you
know now you know
the uh many robotics people are using a
kinect but you know
um and we don't need to use the key the
kinect has some
limited range there for the scanning but
um one dimensional
the laser scan has much longer the
range for the scanning and well
you know we have we have spent more than
three or four years for developing that
kind of robust systems
that we can use in the daily situation
so otherwise they're mostly
tele-operated
i'm not the operations you know and but
uh someone need to
watch that monitor that right and uh you
know
um and someone need to push the button
to take a timing
not every time right maybe every 10
minutes or 20 minutes or something
but still you know it travels somewhere
like
we want to make it perfectly autonomous
and do you give your robot actors
speaking lines
um speaking right they speak or do they
just uh
robots after the you mean the robot or
robot is yeah speaking here actually
it's important to
speak something right it's a theater
right yeah
and you have trouble regulating that
speech or is it just pre-recorded
i'm pretty recorded but for the robot
and we are using the synthesizer voice
synthesizers
and for the android you know
we cannot use voice exercises the
quality is not
enough right the androids need to have a
human voice and are they better at drama
or comedy
um now we are going to have another
drama
right and then now you know android
theater is it's just 20 minutes and
and well this april we we extended it to
the
40 minutes but uh this september we're
going to have
90 minutes android theaters and android
you know we want to use the uh uh we
cannot answer it
i cannot tell the data but you know we
uh unders
is is moving around on the stage
yes and uh when you decided to make the
gemini that looked like yourself
was it did you consider this a kind of
self-portrait
soft self-portrait um
well it looks like a twin brother
not like a portrait and a photograph
right because you know it's moving right
and it's shocking you know if
someone control the geminid it became a
little bit different person
right but you know it has my appearance
that is a twin brother
yeah and the more important things is
i don't know my face voice
behaviors you know that it you know the
your face
you don't know that and every every
morning you're watching in the mirror
but the mirror is a budget right you
know
if we pretend you know the photograph
is your face everybody watching this
face and human face is not symmetrical
therefore
you know the photograph and the mirror
and these are completely different right
so the mirror is not your face it's a
virtual face
but still you know the by by watching
mirrors
you know you you you're confirming the
you know
your uh the hairstyle and
the face uh it looks okay but it is not
okay
it is not your face right same thing for
the voice right
um if we record our voice you know it
sounds like a different person's voice
right so human does not know the
appearance and voice
and and behavior also it's quite
difficult to have a perfect knowledge
about ourselves therefore
you know we just believe when we see the
very similar person there
right i can i can guess that it's
you know my copy that but
i i don't have any strong feeling that
that is myself
um and who have been some of your most
successful students
who have worked continued working in
robotics my students
um obviously
well dr khanda
he was you know now he's a group leader
in my project in idr he's quite
successful
and he's a new young
young leader in human robot interactions
conference and societies
and i'm going to retire and he is going
to take my position i think
but you know i'm i'm interested in more
recently i'm interested in more artistic
work or more
in a different science and so that
i think he's going to take the my role
and
and i and i i can go to the different
areas the main student i have you know
many good students you know there are
many people students
and they contributed to developing
android and and
robot and what do you see is the biggest
challenges to human robot interaction
going forward
biggest challenge is well to make it
commercialize first
but then unfortunately japanese
companies
were not so good but in the u.s
for example in touch heroes and vehicles
and the
railroad garage you know they're
building a
pretty good platform for the
teleoperated robots but but
that is the that is for the interaction
right
all right so um what was the questions
uh
so the the biggest challenge is the hri
going
okay all right the biggest challenge is
uh
um is to develop a new market first then
you know
the company gonna spend more money and
well
and we can improve the robot more more
and more
so the meaning of improvement is is to
make it autonomous
you know well um well if we develop the
market for the interactive robot
by using the artillery operator robot
right
then you know the company can spend more
money
or make it autonomous and you know the
trade operation give us a lot of data
and and based on that data we can make
it autonomous
right so i call this approach is
sustainable robotics right how we can
continue
now how we can keep developing new
robots right in order to do that
we need a market right we have spent
there so many so much money so far i
think it's time to
develop the market with the uh
the technology that we have developed
software and otherwise
you know the probability the people
cannot trust us right
and government right and but i
um i believe you know we can develop a
real market
and then you know and as we are
in improving the computers every every
year you know
we can have more rapid and improvement
of robots
then robots are going to be autonomous
soon right
and without any uh practical use
it's quite difficult to spend much money
for developing better robots
what's your advice to young people who
are interested in a career in robotics
well my advice is to have a fundamental
question
right don't watch the uh
simple applications or you know don't
focus on the particular areas particular
research areas
you know and if those the students
have very fundamental questions like of
human likeness or human presence
of human consciousness you know and then
you know
they can take some interdisciplinary
approach
and they can develop new research areas
so that is the
research science and that is a true
world the scientific approach i believe
so my message is they have a
fundamental question
okay um is there anything we missed
well so now another message is it's
quite important
to enjoy don't watch the professor
don't hear the stupid the senior
researchers opinion
just have your own opinions and let's
enjoy it
and then you know you can develop a much
better robot
and what do you consider to be your
greatest accomplishment in robotics
wow
let me see so many good works
right the robot is a kind of integration
of
important the studies important results
but the the in the sense of integrations
the most
successful robot is honda ashima
it's almost a commercial labor
right but uh now you know
the u.s government started the humanoid
challenge
i want to see what's happened there
right
and probably you know you may have much
better
integrations for for building humanoids
i'm not sure but you know that's going
to be a good target
you know or you know that motivates many
people
right and uh but anyway so so far
i think uh ashima is about noticeable
the good result
the result of integration and and i hope
to to use the that kind of platform
for my studies also well actually i'm
working with horn
but you know we need to have some common
platform everybody can use
it right so so in the near futures
i hope you know i want to see them that
kind of a human noise
in our daily life and then you know
we may we can change the our society
again
you know computer changes our society
very much but
next term is robot robot may change
our style lifestyle more so that is the
possibility
how do you see the the relationship
between academia and
industry within japan we have to change
that
all right you know technology
development is
getting the faster and faster
right so we need to erase the boundary
between
industry and academias um for example
the media lab in
mit and wow they're doing very well you
know
they're getting a new questions new
issues from the uh
uh the industries or and from the
you know our daily situations right and
then they're developing some
some new devices and so
we we need to quickly catch the the
research issues from the
industry or in our real life and then on
the
university of academia will provide some
answer
provide some solutions to that so that
the important things
is to erase the boundary and and
and but still you know i'm feeling some
some gaps some values between industry
and academia
and so and of course we need to keep a
traditional
fundamental research areas but you know
and for artificial intelligence and the
media technologies
you know we need to take a little bit
different style
the uh well for the collaboration within
the
and for the industry academic
collaboration
but i'm not sure yet right
and what kind of you know the much
better the collaboration we can have
more and more what is more embar better
situations
therefore working together but you know
that is also another the innovations we
need
some well some of they innovate
the new universities or new college
for doing that but i'm not sure we have
to change the university more and more
always you know by changing the uh the
topics in the university you know we can
catch we can develop
our futures right so that is the role of
universities and
working in university national
university in japan
and i'm also running the global coe
program
center for excellence program and that
you know central exchange program is
quite important
for changing the you know graduate
school
in in university and for having better
futures what do you think the most
promising applications for
robotics is human society there are so
many so many applications we have you
know
um um well for example
well um so my interest is
in interaction of communication okay um
well how can i say how about this idea
the convenience stores right um the
the staff of convenience store i think
you know humanoid and android we can use
there
and the educating machines well the
problem of
the current railway station is that we
can it's quite difficult to find the
stuff the human stuff right and when we
want to ask something right so we need
to access to the uh ticketing machine or
some
the display but that they are not so
good for the
elderly people you know or people
just want to talk to somebody but if
if we use the android or humanoids there
right then people can easily
they they talk to ask questions
so then we can improve the you know our
current
system more so
and another example is oh um this is a
secret i cannot tell the detail but
you know um okay let me how about this
example
that we we can have a more realistic
museums
of movie stars by using android
we got that kind of offers but you know
how about this idea if i get old
if i suppose i need to stay in hospital
right and in such case
um i definitely use my android for
teaching my class
okay so and
robots are terrified android they can
change our lifestyle
especially for the elderly people right
of course the robot
they support the address but you know
the robot can work instead of
you know errors not completely right
they
either can teleport the robots right
so either people has a much knowledge
and experience right
they're going to be good teachers but
you know the problem is that they cannot
visit the
school in such case you know that they
want to use a teleoperated android
so i think that is very possible
applications and now i'm building the
another my copy and that that is
that that that's going to be more
portable and
so then what i'm going to do after this
is
i'm going to just send android to the
foreign countries for having this kind
of interviews right
i can stay in japan or you know
i i i put my the android in my office
and and i never go to the university
anymore
i just travel around
so well i'm glad we got you the real you
on video
i'm i'm very serious yeah yeah but we'll
come back and interview your
android later i just send android next
time
thank you is there anything else you'd
like to add oh