so if you could just start by telling us
where you were born and where you grew
up and went to school
yes i was born in paris uh went to
school
in my neighborhood like in fifth
district
latin quarter and then i went to
engineering school called the ecole
national superior
telecommunication so it's the
telecommunications school in
france i stayed there
three or four years uh specialized in
artificial intelligence
and um i wanted to do um
a phd at that point i mean i was kind of
clear i wanted to
do research in ai or robotics um
that's a kind of childhood dream anyway
uh but i was not so sure where to do it
i mean i i was
planning to to uh to go to the um
to the university um and then i got the
opportunity of
being involved in the
setting up of the sony computer science
lab in paris
which was the the only fundamental
research lab
that was out of japan at that time
and when i discovered the lab it was i
mean you should remember it was a time
where
sony was not doing computer science
there was no like via
computers or i mean sony was really well
known more for like hi-fi
products and i was not so sure about
what it would mean to do
to do research in computer science at
sony but
i met luke steels who was the director
of that lab and he was someone that was
i knew from my textbook i mean he was
with ronnie brooks
one of the founders of the new ai
uh embodied intelligent paradigm i mean
really
shifting from a traditional symbol
processing uh reasoning kind of
artificial intelligence to much more
collective embodied approaches and
and so so i was very lucky to to meet
him by chance and
i started i think as the first french
employee and
freshly
created sony lab and i stayed there
10 years basically
what year did you start there when
i started there in 1997
and uh started by doing my phd
on the
multi-agent system and the evolution of
language i was like
modeling the origin of language that was
the theme of the of the laboratory i
mean can we do with
with computers simulation on how
language are evolving
uh can we tell what's happening when you
introduce newcomers in a large
population of
speakers and that was a very exciting
time whereas
it was a branch of artificial life which
was more focusing on
simulation of biological processes but
we were doing that for cultural
processes and we were like
there was just a couple of labs in the
world doing that and so so that was the
main topic of my phds already very
interdisciplinary in the sense that
there was like linguist
paleontologists very very exciting time
and the project
that was like the main project for my
phd was actually creating a population
of robots that have
had to agree on a common language i was
talk
the name was a talking ads experience
basically we had like um
in various places in the world couples
of robots
that could look at um at a wide board
with geometrical figures and they had to
agree
on on names for describing what what
they were
seeing uh typically like a red triangle
they would use like kofuba
or like an invented word for that of
that
red triangle and the other robot would
maybe not know that words and would look
where the first robot was looking kind
of joint attention
and make guesses and use the word in its
own term
in return and and we did like an
experiment that lasted
something like six months so kind of
really big
real time experience where a population
of something like several thousand
robots were
were exchanging bodies but they were not
like seven thousand physical bodies
there were like
20 or so different physical bodies but
they were like swapping bodies
and we were simulating this
the creation of a new language among
robots
and very interesting experi um
things were observed like uh police semi
for instance between words we had
situation where
um one words like um bosu pai i think
that was
that was the it was a word name used for
large objects in a sense um
that for let's say half of the
population
robots were looking at at the white
board and when they were
seeing an object with a lot of pixels a
big area they would
they would use bosu pies as a saying
this is like
a big object ozobot and uh
the other half of the population were
using that word before
wide objects they were just looking at
the horizontal
dimension of the object and um
what was interesting is that this
ambiguity was staying in the population
because there was very very few long
and thin objects and so when one one
robot was saying oh this is bosu by the
other one would say oh yeah that's
that object it's bosu by two although it
was kind of two different notions and so
when we discovered that
we introduced um a long and thin object
and and then the ambiguity between these
robots
were revealed and there was a kind of
reorganization when one word was used
for
wide object and the other and a new word
was recruited for
for big objects essentially and what was
funny is that although i mean these
robots they have
nothing in common with us i mean they
they were
very simple body i mean their perception
system was very different
but that type of ambiguity between large
and wide objects is
is something that emerge and it's
something that is actually happening in
in natural languages
like in french typically we use large
in a sense of white objects whereas in
english large
is is for bigger larger surfaces
so it's it's intrinsically ambiguous in
the evolution of language and that
reappeared in the um in the experiments
itself and so that was very encouraging
saying that in the future so that was
like
1999 we would be able to say something
about cultures
based simply on on artificial or robotic
experiments
and it's in some way the topic i've been
pursuing and i'm still pursuing today
um in with various type of other
experiments but that's
one of my big um
conviction that is that uh human
sciences
are going to gain a lot from the arrival
of new kind of experimental methodology
involving ai and robotics
in some cases so that was like
my phd project i published a book based
on that
by the time i was finishing uh i got the
chance of
being involved in the beginning of
the eyeball project which was the the
sony robotic dog um
[Music]
it was like at the beginning that was
really like a
rather secret project inside inside sony
uh only done in
in in a japanese lab and they had the
first
they came to us in paris and showed us
the first prototypes
and um because i was in some way
by chance becoming specialist of
language and robots
i i got invited to to stay longer
in the japanese lab in tokyo and i'm
started
to to go very regularly that essentially
to learn
about this first prototype and being
able to bring back
one of these first uh
[Music]
prototype to europe there was actually a
whole adventure to pass the costume of
that thing
um anyway but so i i went to work
directly with the
the japanese team building the robot and
that was really really exciting because
it was a time
where like a couple of people in tokyo
were just
bringing together uh
all the necessary uh components that you
had to
to to do to build a robot that at that
time nobody else
has done any four-legged robot as
convincing as the ibo was
it was not yet called the eyeball but
who were those people
so masaru fujita was the chief architect
and project manager on on this
on this project and that was like
he's an amazing man i mean he's really
like i rarely seen all my life someone
mastering
so many engineering techniques and
incapable of supervising such a complex
project as building a robot like this
one and he was like heading a team
um where they were like very patiently
uh prototype after prototype making that
thing real and the
spirit behind that team was toshidoi who
was like a very important engineer
in sony he was one of the maker of the
cd
at that point and he was the person
convinced at that time at sony that
um robotics would
become something as big as the as the
computer game industry essentially that
was the idea that sony was making a lot
of money with playstation
and they they thought there would be an
opportunity
for for doing the same
in a new market that would be like real
life essentially and
and the playstation of the future in
their
their idea would be a robot so that was
really i mean
a sense of pioneering something really
new in that team and
and i have really really good memories
of of spending time with the engineer
there and there were no
almost no documentation i mean you had
to learn most of the things just by oral
history and go
sit by each engineer to understand how
the vision system was working how the
operating system
because it was like a custom sony
operating system underneath
um etc etc and so i learned all that
so i was i was just lucky to be at the
right moment at the right time i mean
but came back with the first
uh ibo prototype
in europe and started in paris then
to develop all the interaction part and
the
linguistic part on on that robot and and
so we started to do
experiments a bit i would say naively i
mean we were trying to
to teach the robot some some words about
everyday objects i mean the whole idea
on the eyeball was that it had to be
entertaining in the long run and it was
obviously entertaining when you were
seeing it i mean it was like
moving very convincingly but the whole
idea was be
can we make that device stayed
interesting
for more than a couple of minutes i mean
can it stay interesting for several days
for several weeks for several years
that was really the challenge and the
underlying hypothesis we were making is
that
it's going to be by teaching it's going
to be by spending time with the robot
uh it's got it's going to be on in a
situation where you would see
how the robot change based on your
teaching that
you're going to have really a value for
that robot our objective would be
i mean typically sony was making a hi-fi
system
if sony would come to you and say hey
uh i take your old hi-fi system and i
give you the latest sony
hifi system most people would say yes
our objective with the ibo research was
to say that if someone come
to your eyeball and say hey i give you
the latest eyeball with all the new
features you say no i mean i want to
keep
my my old eyeball because hey it has all
this
unique live trajectory all this time
we've been spending together that's
absolutely
um more valuable than the couple of new
features you just added to your product
so that was like the kind of underlying
assumption so we tried to work
on on making the robot knows about
surrounding making it
understanding its everyday environment
and on something very concrete i mean
making it recognize every day object
that you would show it
so so um a typical day at that time was
uh
was like you would go in the morning you
would show
objects like a ball a small
plastic cucumber and you would say bowl
bowl
bowl and cucumber and then you would ask
what is it and it would say oh
cucumber for the bowl so you would start
it again and doing that every day
because in some way there was no way of
cheating i mean you had to show that
ball
on every lighting conditions on every
type of
backgrounds so that progressively it
would
it would learn um so
that was kind of working but very very
it was very exhausting to to teach uh at
some point i
i i was lucky to have like an internship
and something we're doing the this
teaching job for me
but still it was like um difficult
uh and we started to understand why it
was difficult and one of the reasons was
that it was very difficult to share
attention with the robot the robot has a
lot of different behaviors he was
attracted by different things and so
in many ways when i was saying the word
bold typically it would when you
look at the actual data it was using to
try to to reduce
the relationship between the word and
and its perception perception
you would discover actually that uh it
was not at all
there was nothing in common i mean there
was like uh you would
look at the wall when you hear the word
ball and then the data was so corrupted
in a machine learning sense that making
sense
of doing statistical inference on that
was was like worthless
so we we say oh yeah but that's obvious
because in fact what we were showing to
the robot had
no value for the robot we have in some
way to got
how robot interested in in what we're
showing to him
and so we started by building some
um uh triggering um
of attention based on movement typically
like i remember like
one summer i built an algorithm where
which was the idea of
okay the robot is intrinsically
interested by movement and then
if it sees like a reoccurring movement
you would build a model out of that a
kind of very bottom-up approach
in which we were trying to push things
in the world of the robot that was
really the idea
of that the robot has its own world and
that there's not necessarily an
intersection with your world
and and we were trying to make that as
intersection concrete it's really the
notion of umvelt
which was introduced by uh uh
the warnock school in germany uh
60 years ago um this notion of what is
relevant in an environment
for a given species and for for the
robot it was clear that they were this
intersection was not
sufficiently it could be i mean if we
would program the robot to be
interesting in the world then okay the
ball would exist you could name it
but if you're not programming the robot
in advance for being interested in a
specific object
then it was very very difficult to have
to have the what interested
um so we tried different things and
after
a couple of years or so um i started to
work with one of my colleague pierre
vuder
who just finished his phd at that time
on the new idea
that was that we could that the main
problem was a motivation problem
the robot was not interested in what we
were showing
in the object we were showing so we had
to work in some way not in
the perception level but one step
deeper i would say at the motivation
level we had to have a robot
intrinsically interested in some aspect
of the environment
and based on that we would be able to
teach words for the thing you would be
interested in that would be easy but the
difficult part was that
generic intrinsic motivation system
how we will call it and there were a
couple of other people working on that
idea at that time
but that was kind of still a new a new
idea in the sense that
most of the research was on
reinforcement learning with the idea
of an extrinsic motivation typically you
many people were teaching robots
what was good or what was bad by by
giving treats
for good things and and and saying no
bad for bad things and so
implicitly there were these robots that
could never go beyond
this type of shaping that was really
coming from an external point of view
and first it was very
that was making the old learning
dependent of the
on the um of the teacher
and also it was in some sense it's very
difficult to tell
what's what's easy to learn for a robot
and what's not easy to learn so all
our idea was to say we should
build an architecture where the robot is
modeling how it is learning
that's not so difficult in some way i
mean a robot can predict
can learn to predict the effect of its
action i mean it's based if it's moving
its arm
it could say i think my arm is going to
be at that position in a couple of
seconds
and then it could see whether it was at
that position whether it eats an object
or not etc etc
and based on that he has an error if the
error is
is small then that means he's predicting
very well
what's happening to him if the error is
high
that means he has no queue whatsoever
what's happening to
uh to him and typically if you would
look like a television
with like
the snow on the television i mean that's
just like unpredictable
and many things in the world are
unpredictable so
the whole idea was to say what the robot
should try to do
is find the things which are not
completely understood not a very small
error not
impossible to understand with high level
error but things which he can learn
which basically means things in which
situation in which the error is going
down
so it would measure the derivative of
the error curve and look
for situation where the error is running
is going down that means situation way
in which it
is learning so the whole idea is to say
we built a detector inside the robot
that detects
the condition of learning and then we
reinforce
the behavior of the robot based on that
that means that the robot will actually
decide
the action that optimize the learning
that means that in any situation you
will say what can i do now
to learn the most so i will avoid things
i already know
i will avoid things that i know as far
as i can see that i can't learn anything
because it's too complex
i will focus on what we call progress
niches
situations which are optimal for
learning at a given point because
obviously imagine a situation imagine
two situations
which seems to be promising for learning
one
if you focus on one progressive he will
he will learn everything he can on that
and reach a plateau and then
do not receive reward anymore and then
focus on the other one etcetera so that
was a very generic principle
that we actually discovered i mean it
came as a discovery really
by by deduction but then once we in some
way discovered that we discovered that
actually
many other people have said something
very similar already
quite a long time ago i mean about the
search for curiosity
if the search for intrinsic motivation
the balance between
between easy challenge and difficult
challenge actually in many different
domain
there were um there were ideas like that
we had just rediscovered that in our
exploration
[Music]
and so we built this kind of generic
learning engine and
and try to apply it to it to many many
many things like
we had the robot learning how to walk
based on base with that system that
means that you would just like
make random movement at the beginning
and discover at some point some rather
movement were
actually managing to go in one direction
typically going backward and then our
robot will go ahead
for one hour backward until i mean going
backward was not surprising at all
anymore
and then you would discover that a small
shift in the
in this parameter would go a bit on the
left and it would
start to go on the left on the right for
another hour and typically on the third
hour discover that
he still has not tried to go forward and
that if you go forward
then there are other things to do
typically exploring spaces sensory motor
spaces
that could be as simple as the working
spaces
are as complex as learning how to use
object and we did
a whole series of experiments where we
basically almost reproduced some of the
piaget steps
on developmental developmental skills
scales showing that how robots could
learn to apply generic
schema schemata on on
on a particular kind of objects without
having any
inter internal notion of the schema or
something like that i mean just
by progressively developing a
developmental pattern a developmental
sequence that would just
emerge about as a result
of the robot confronted with its
environment and
that was very exciting again because
it's um
we were showing for it experimentally
that
the body had as
an importance that was as high
as as the software i mean typically the
software inside that robot was always
the same we built this generic
engine and then we were just like
coupling a body
attached to the engine and imagine just
you had a robot with slightly long
longer legs
then the size of his legs would actually
change the whole developmental sequences
because some think some type of walking
would be learned before
others then some other type of skills
would be built on
on top of that so we were showing that
in fact you could code behavior
into very concrete morphological
uh dimensions and that
and that in some way uh understanding
developmental pattern
had to be done in in that way not as a
kind of
program in the dna that is just like
unfolding itself but that actually the
size
of a finger the size of a head the
global morphological
consideration may influence later on
some other aspect
to what extent did you draw on
biological and psychological models
and the actual design of your
architecture so we were never
i mean we we read a lot in psychological
model we were
definitely aware of
of of the literature and but in some way
i think we always
try to have an engineering approach in
the sense that we were
trying to think of the problem as
as problems to receive solution and
that's how we
we were creating the algorithms not
trying to be biologically inspired
for the sake of biology claims but i'm
personally not convinced about
imitating nature i'm i'm much more
convinced by trying to
understand what the real problem are
find original solution and then
discovering
usually that indeed these solutions in
some way
are presented in the brain or in
other animals or gave inspiration what
to look for
but i always see like engineering as a
rather autonomous process of discovery
that is then having a lot of influence
and create
tools for fault for all the other
disciplines and
situation where you just look at how
animals are
what they are doing or just look at the
internal structure and then try to mimic
that
there's actually not so many examples in
the history of technology where this had
a
significant influence so so
it's a bit i mean radical and many other
researchers are exactly defending the
opposite view but
but but my feeling is that it's it's by
trying to
to ask the right question and build the
machinery
using has as many possibilities as you
want
that usually makes the most interesting
discovery that then
can be applied or give ideas when you
look at animals or look a biological
system or look at cultural systems
yeah and so you mentioned that
the size of the body affects how you
learn interaction and things
so what other aspects of the body are
crucial to learning especially for
perception and language
it's it's your body is shaping
[Music]
all your skills
in the sense that uh especially when you
see
development as as a sequences as a
cascade
of of events
then it's just like built one on top of
another so
so so it's your relationship with the
world that serve
as as the basic material and then that
serve metaphorically
to to learn other things later on so
it's it's really um
it's really the everything is embodied
there's no
notion of abstraction there's no notion
of internal representation
or symbolic representation or whatever i
mean everything is based
on the initial sensory motor
relationship that you have with the
world and and
these relationship is what you use to
to actually build all
the abstract thoughts that that one may
have later on i mean there's there's
actually a very strong argument for that
based on all the work that has been done
on metaphors or the word of lack of
george
laker in particular including his work
with nunes on mathematics i mean he's
showing that even
what may seem the most abstract or
disembodied thing is mathematics
is strongly um strongly linked
with basic sensory motor skills and that
it's
it's essentially based on that so so
that's also why i mean
i've been basically starting from
question about language and going back
to question about
joint attention how do you how typically
would a
one-year-old child learn and and
stepping back to a three-month child i
mean
we've been just going back and back and
back
to more fundamental sensory motor skills
because in some way if we don't
understand them
jumping immediately to a higher level
skill is
very unlikely to work it's actually an
intuition that
curing ad in his uh in his famous um
1950 article i mean on computing um
on the german test and he was saying
that if actually
a machine would ever succeed the turin
test it has to learn like a baby
and that was like a just a note at the
end but
it's it's actually phenomenal and you
have to take that very seriously in the
sense that learning like a baby is not
like mimicking our babies
learning is coming from this this very
unstructured world
uh in utero to uh
to to to building skills which all
builds on one on top of another
which are highly uh embodied and
and uh luckily i mean you can do that
now with robotics and that's
it's one of the main contribution
robotics can do is making
the the body viable i mean making it
something you can change
making something you can you can like
have situation where the software stay
stable and the body change and then
understand for the first time
the influence of the body on some
specific processes
no other scientists can do that i mean
because
they can just say that people that have
different type of body
have different uh scales or whatever
but they can't like use this very
um uh
trigger on on the on the body structure
as a
as a variable and that's uh one of the
main
potential of robotics
what was it like working with luke
steeles that was
extraordinary years i mean working with
luke steele is
i mean he was really like a master for
me i mean uh
in term in the way he was writing in his
very large approach to science
in the way he was using arts
also in as part of the research process
i mean half of the things we were doing
were in museums
because there was places where where we
were just trying new things and
and he had like a mix of
fundamental research and and ambition to
be in a real world an ambition
to contribute to to
problems larger than just like the ai
problems that was i mean
amazing to me i mean and he was like
also he had um
i would say a very um
trustful relationship with his students
i mean the sense that he
he was not like using the students as
slaves for doing
his own research i mean he has his own
research in some way and he was really
like letting us grow and
becoming independent scientists so that
was like like great use definitely
um in what made me
so i stayed ten years at sony what made
me change essentially
among other things is that sony decided
to stop the eyeball in
96 in 2006
after seven years of commercial
exploitation and
although i mean that was absolutely
great to be in the only company
manufacturing
entertainment robots uh for all this
time although it was much less
interesting i mean
uh to to be in the company which which
this robot wants to
not make anymore and so i thought that
was a good time to move
and i wanted also to to do things
maybe more concrete more in in the real
world more applied
and paradoxically because all this work
in the industry
was actually very very from from the
mentor in terms of our
type of research and we were dealing
with with absolutely
a strongly fundamental research question
and i thought okay i want to do things
more applied let's go back to the
academia
and i went to epfl um on the invitation
of pd labor uh to join like
a growing lab it was a craft laboratory
which at first um may have
been like a big shift because actually
it was not in a robotic lab per se i
mean it was
a pedagogical lab where the idea was
can we make furniture
new kind of computers that would that we
could introduce
in in in learning environment as tools
for students to learn and um
[Music]
yeah and i was uh really thinking at
that time
that the future of robotics would not
necessarily be
a small robot like the eibo i mean
although the ibo
was a success in some way i mean it also
showed that
i mean it was probably not the only way
to
to to have robotics coming into the home
and my hypothesis
which still my hypothesis now is that
the
wonderful technology we were building
for robotics especially for
human machine interaction for
understanding context for
being able to act in a real world which
no other computer can do
then we can just translate them and
apply them to things
that we will not call robots when we
receive them but
which in fact we would integrate the
robotic technology
so robotic objects in any way and i
started
exploring that in that new context at in
switzerland at epfl
building interactive tables movable
furnitures with a lot of very exciting
projects also working very closely with
industrial designers
knowing how to make these objects and
one of the project
which was very important for me was
the try to make a robotic computer
really my
conclusion was that robotic um that
computer science had stayed very
conservative
in terms of interfaces for quite a long
time with the mouse and keyboard
and we were at that point there were no
multi-touch yet i mean when i started
with that
type of experiment but we were feeling
that there was com
that new type of interfaces were growing
and i really
thought that robotic had a role to play
and so
with the help of martino de esposito who
was an industrial designer
um i met actually during the zoning
years where we we worked on
on building a robot playroom
with new objects and i was really
impressed i mean of the creativity of
that industrial design i could have in
terms of building things
we started to think about what could be
the shape of a robotic computer
essentially we got inspired a bit by the
imac publicity where you have like it's
an old imac publicity where
someone is passing in the street and the
the imac is
is following that person start to have a
purely non-verbal exchange just by
by moving its head and
although that's just like marketing that
was showing the force of non-verbal
communication and what you could do if
you had a computer that would
move a bit not move in the house but
just like be essentially a robotic
screen
that was the guiding ideas and based on
that we got lucky to be
chosen for an exhibition at the museum
of modern art in new york
and that really launched the project
because we had to produce
in something like six or eight months an
actual robot
prototype that would stay there with
three months
in an exhibit and so we worked like like
like tremendously for all that
[Music]
i mean passing from what was just like a
drawing on the paper
to actually an actual working prototype
and and that first prototype called the
wizkid
was really the starting point of an
interesting adventure
which is this uh the the exploration of
the space of of
of robotic computers so so that was like
like a screen
with um the camera in front of it and
and then a robotic arm so it could like
move in different directions
and at the museum people would just pass
by it would turn
detect faces turn towards them then you
have a gestural interaction system where
they could select things on the screen
at a distance
go like a kind of augmented reality
system
and it was interesting because the
museum of modern art was an art museum
there was no like scientific explanation
whatsoever so it had to be
very fluent in terms of very intuitive
although at that
time nobody had done gesture interface
or that sort i mean it was
you started to have the first
multi-touch screens
and people were kind of understanding
that maybe that's something to do like
on
surfaces but but like gesturing in the
air was something very new
now people are uh more are more used to
that but
and still all these visitors at the
museum of modern art
they got the idea quite rapidly i mean
they were there got very
attracted by the robot gesturing some
were understanding
some were not but but still we were very
very
impressed by how well that was working
and so based on that we decided to
create
a company because i felt i mean
i mean the project benefit a lot from
the pfl
ecosystem so that we could go to that
level of an actual working prototype
in a museum but to go to the next level
like industrial production even like
small series
that was like yet another step to take
so we created a
company named ozui or zwe
to commercialize that product
we tried as much as possible to make it
more looking like a sports car
then a gentle
friend because there was like obviously
a price problem and so we thought i mean
we should make it serious
it should be like this is this is not
like a toy because we can't price it
like a dog there was already a problem
with ibo that was actually
priced looking like a toy priced like a
computer we thought
we need to be priced unfortunately
because of swiss quality
and all that and we need to be priced
slightly upper so we need to have a
design that goes with it and so we came
up with a second design and another name
called qb1 which was much more like
monolithic it might look really like a
sculpture when you would see it from the
from an external point of view and um
and then that sculpture was
actually um animated
in a sense that it would turn towards
you
and and you could have a whole
relationship with it was very very
exciting we also built our own 3d camera
there was no like off the shelf really
camera
at that point so we built a new system
based on first infrared
where basically the the well
this device could have depth i mean it
could actually in a scene perceive
not only faces and objects but actually
the distance and that was changing
completely all the vision problem
i was dealing with for
15 years uh making them so much
different
i mean that means though we had a
computer that was a gesture-based
computer
that was really understanding its
environment i mean he was understanding
who was there he was understanding what
type of object was there
so we started to build applications uh
for for that new settings um like
what i call real scale media i mean for
me this is a new type of media which are
not
like immersive media like like your
smartphone or your tablet or
books or television where you go look at
the screen and you're basically
leaving the space to go to that
particular
fictional space or virtual space robots
make you stay in the same space i mean
when you interact with a robotic dog
it comes to you i understand the
relationship between you and the dog
and then you see that i'm interacting
with it we stay in the same space when
it's not like if you were reading a book
and you basically
go somewhere else so that notion of real
scale media which is intrinsic to
robotic
that was really what we're trying to
push with that device
and uh and showing i mean what it could
mean for some different type of
application
i was really personally convinced
that the number of applications you
could do
is as big as you what you can do now on
an iphone or whatever
there's a market there if you if you
obstruct it from robotics which is which
is as big
in fact and which is things happening in
the real world at real scale
so we started for music for various type
of augmentation
typically you're in your kitchen you're
finishing
[Music]
tomato sauce that on your pasta the
tomato sauce is is finished you you just
show it at the distance to the device it
recognize the
the the shape and the and the type of
source it is
put it automatically in your next
shopping list eventually suggest you
that maybe you should try the arabiata
source the next time etc
and then you put it to the bin i mean it
gets basically
it goes in the fabric of our everyday
interaction it's it
it's it's much less interrupting than
than going to a smartphone or switching
the computer or whatever i mean all
these things are still
things that that are blocking however
the interaction here it's as smooth
as another person in a room where you
say hey by the way remember this
launch that music have a look at this
this type of thing
um so we produced that that computer we
sell we sold it before
everybody else i mean before microsoft
came with
with its um kinect system i mean there
were like they were sold in
2009 so that was really probably the
first computer sold with that technology
uh it got the attention of many big
groups actually and we
and among the first to buy uh there were
a couple of
real like customers but really
that was more the big groups that were
interested so logitech bought one
and and quite soon after um samsung got
interested and
also integrated by technology in their
lab
of course you never know what happens
when they when they start integrating
the big industrial lab and i've been
working at sony and so i know
i know how hard it is actually even if
if you you
to actually successfully bring
an object like that into a line of
product into a big company but
but we were very like happy of that
that that at least some people were
getting the idea and we've been
continuing
demonstrating showing it and
that was really a landmark and
still i mean how many of them do you
think you sold
oh i think we never went like
something like 30 or something i mean
not not not so much
clearly i mean still like a very small
production in many ways
they were costing uh 10 000
so which is a reasonable price in terms
of
when you start a product like that uh
obviously for big market i mean that was
i mean
that was not at all competitive but um
so um so that was like um
that was um yeah that was an
important step and uh
[Music]
and well let's go back a little bit um
can i can i interrupt you a second uh i
just need to check
uh my other visitors so that i send
someone to somebody one
yeah yeah um so i'm just going to ask
you as oh
well so first was the um was that the
design and the elastic mind show
yeah designing the elastic microphones
yeah many interesting things were
that uh at that exhibit i mean it was
stuff with interesting things so
yeah so uh so that was like a great
opportunity also to
to um yeah to show things in the making
and things that were making bridges
between
design and science and we were really i
mean i mean
martino was was like industrial designer
and i was like a
computer scientist or something um okay
well i'll
yeah since we're limited on tomos the
important ones first and then we'll jump
back to some of the historic ones um
so what would you say on the sort of
road ahead are the biggest technical
challenges
for robot objects and or
collaborative learning with robotics i
think
it will i mean the big thing is the
integration of robotic technology in
everyday life
and something which is changing now
is is the fact that well technology
becomes available
it becomes available as as kits or as
element which permits
of people to properly tinker kind of new
system of some sort
and the big big new thing is the cloud
now i mean the fact that
you can benefit from a limited
uh computer science computer power
you can aggregate the learning on the
experience
of thousands of of cameras robots
we can we start seeing that with with
like google goggles for instance that
start actually do really really
interesting
result on the problem on the very
problem i was
tackling like 10 years ago when i was
trying to teach
my own robotic dog the name of an object
i mean if you have like
all the cameras in the world all the
systems that
con that actually collaborate in in
sharing their databases
of objects you can start and you have
like unlimited processing power
that type of problems becomes becomes
possible
so viewing the robots really as
interfaces
to the cloud like what are computers now
the computers
turn now to be just interfaces to the
cloud what counts
is the possibility of reusing the same
data over different interfaces
robots are one of them and they have the
possibility of acting in specific way in
the world and they have the possibility
of
of capturing some elements specifically
in the world
a good example i think is how i mean
you know for instance google
street view which is in some way
type of robotic technology i mean i have
this perception technology
well you put a roomba
or like a
robotic element in your house you put
the similar type of of
of camera on top of it and you have
almost 10 a search engine for the home
which means that i could just type and
find back an object at my place having
that type of system
just like monitoring where are the
things
these are typically the type of frontier
where the the mobility is important the
real
scale aspects are important the fact of
having the possibility of dealing with
the mess of the real world is important
this is
this was not possible on small system
because of the computing power
now with with the fact that there's
basically a single computer
which is a worldwide computer we're
going to have robotic interface attached
to it
this is for me the future of robotics
and for young people interested in
careers in
robotics what's your advice to them
well well there's obviously you mean
very interesting academic uh journey i
mean
many difficult programs to solve very
interesting phd to their own
etc for the one who wants to contribute
to the real world
it's sometimes more difficult because
there's no so many uh
successful robotic company out there i
mean there's a couple
but it's not yet i mean that uh always
anticipated
uh boom where robotic is gonna be like
computer science
so um so well
i think it will come
from small companies that would know how
to
actually trigger that the power of the
of the cloud and be spar smart on doing
some for robotic interfaces but maybe
simple
and find the right equilibrium between
what it costs
and and the benefits it gives and and
that will typically come from startup i
think
and so the obviously interesting um
um yeah opportunities for young people
who are more interested more in the like
real world real
real application thing not so much in
the academy aside to go and find one of
those startups and try
try the adventure with with them uh of
course with
i mean there will be one percent of
success only among all the startups as
usual but
but among them they're going to be i
mean the future
key players in this new type of media
which are robotic media
what and you've done a startup yourself
so what are the big challenges
for starting up your own robotics
company well
the challenge is is just listening to
the market the market the market the
market
and trying to forget
about your dream robot that you want to
make i mean that's fine for academia in
many ways
you may have like big big things to
solve and
and challenged and a big challenge you
want a big problems you want to
to to explore but once you're in the
market you should listen to the market
all the time that means
trying really to solve the problems
people really have and be completely
agnostic
on forms i mean a robot should have the
form to perform its function
but something very tricky when you start
a startup is that the press
for instance always over-evaluate i mean
you got a lot of attention all the time
uh because the press is always
interested in in about the future of
robotics and
always interesting about the fact that
they're gonna be small humanoids hanging
around
but when you look at the history of
successful robotic product it's always
been
product very pragmatic dealing exactly
with a
well identifying problem and solving it
and
part of the difficulty when you start a
robotic startup is being pragmatic
and saying well of course i would love
to to do that special type of robot of
course even the press encouraged me to
do that i will get a lot of attention if
i do that
but a couple of years later well i will
just be
out of business because there's no
market for that so
so well just listening to the market and
being of course inventive but
inventing programmatically and you
mentioned you did some other
museum work with uh when you were in the
sony lab
yeah what were some of those projects oh
the we worked with the
project called laboratorium essentially
there was like extension
of this robotic culture idea i mean the
idea that
you can have robots creating their own
cultural phenomena
interacting with one of them having this
special thing where
what i've been creating is more than
the sum of all the robots in the sense
that if you suppress progressively
the robots one by one from that
community and you replace them with
new robots that don't know anything then
they still learn
from that language typically which is
something nobody is actually
possessing and that was of course a
fascinating idea as a
from an artistic point of view and from
a also a
philosophical point of view i mean what
are our plays i mean what is the
difference
between between what make us human is
that culture but whatever i mean this
stupid machine seems to have culture
also
that means that maybe animals have
culture too et cetera et cetera so that
type of question
and so we've been exploring that in
different settings essentially as
extension
of the talking head experiments as
different type of dialogues between
machines about things there was a system
where
where people were just like getting a
snapshot of their
of their eye and then that was the
subject of conversation of the robots
we had other systems where people can go
and and change what was on whiteboards
and the robot would talk about it and
and so and what was interesting is that
all the systems were connected with one
another i mean there was one system in
antwerp
one system in cambridge one system in
tokyo and and that was like the whole
idea of
of installation that goes beyond
installation that are like
kind of phenomena from which you have
like a point of view
and i like a lot that idea and that's a
place where museum can play a role
showing that yeah it's more than just
showing one thing it's showing
something which is happening at a
worldwide level and that's actually the
difficult
thing for people to understand is how we
shifted
in 20 years to a situation where
technology is worldwide
there's a single machine which is a
worldwide computer accessible from
anywhere
and robots what they do is at a
worldwide level
and so artistic installation they should
have this worldwide aspect and it just
should be in communication with other
artistic installation uh trying to grasp
this globalization which which was still
not having completely realized what it
means
um and what do you think is the the
place of or the relationship between art
and
robotics i mean
artists and scientists have similar
issues in the sense that they want to
say something they want to demonstrate
something and they have
in some way to convince or to point
about specific things happening in the
world
they have evaluation criteria which are
different
i mean in the sense that scientists are
judged by the peer and and have to
publish and where artists have to
to to to be recognized in other ways but
apart from that
the questioning and even sometimes the
methods are
a lot of similarities so i would not
like say like meeting between art and
science and blah blah blah i would just
say
they are like similar
concern similar type of discourses
different validation processes but apart
from that
it's purely permeable and
and uh it's about saying something about
the current society and
and deciding something about its future
i mean in that sense it's a it's a
there's a lot of motivation i would say
which
that's why i mean it's so easy to make
in some way a transition
between good good artistic installation
and good scientific work i mean it's uh
it's as long as things are done
seriously that's fine
and um you mentioned a few but who are
some of the collaborators you've had
over the years
oh so many
so most i mean so after i worked with um
with
[Music]
luke skills i worked with one of
his students which is pierre boudier and
we worked a lot
together and built a lot of system most
actually of our paper is our co-sign i
worked with verena hafner
a german
[Music]
woman we worked for several years
together she's now in berlin
also doing robotics
arriving at epfl um i work with
on the pedagogical aspect and
integration of
technology into school orchestration
that that notion
of that actually the teachers like
orchestrating in real time what's
happening in the classroom and
and that technology should be there to
support the teacher not to replace him
that's a strong idea from pierre but
we've been exploring together
and all my phd students here and
my colleagues it's a huge team so
so i won't start naming them because
it's it's going to be too much and
martino de esposito on the on the
industrial design part and now
my new venture i would say on my new
topic of exploration is done with lauren
bolly who is a graphic designer
a graphic industrial designer and we
work
on book as machines i mean i'm convinced
actually that uh
the other
machining revolution that will come in
the coming years is
linked with books in some way i mean
books have been
going on the first machinisation process
with
the printing press and then offset etc
etc
going into more regular forms this has
had tremendous impact on what we
are on the on how our eyes are reading
on our sensory motor space i mean we
know i mean that
we've been profoundly changing the way
we see our world not only in terms of
ideas but in terms
of of reading skills and so in terms of
our relationship
a very intimate relationship with the
world since
the arrival of books and what's
happening now
is a full mechanization of the book i
mean it happened
to the to the maps for instance um
map are interesting cognitive tools that
you use for
for coordination there's a history of
map which is
very similar to the history of books i
mean there was just they were hand
produced and they've been mass produced
and they've been in print but made from
digital data i mean all the digital
parts actually
were done for the preparation and then
they were just like printing
the the map okay um
and now they've been translated into
full digital media
like google earth where you can zoom you
can have like other aspects and what is
interesting that transition
is not so much that it's more powerful
is that before with
a printed map you can do you have to
learn how to use it but you can do also
many other things with it i could wrap
something with a map or i could use it
to protect myself from the rain or i
mean
basically interaction with the map was
conventional but not
mechanized by the time it actually
become fully digital
all what you can do with google earth
typically is absolutely
predefined i mean there's a lot of
things you can do but
outside of that you can't do anything
this is what's happening to the book now
it's a kind of a second revolution
or third or whatever to the book i mean
we start to to
to a situation where the book is fully
mechanized from the production process
but its usage is still very free i mean
you have like with paper book a lot of
freedom in what you do with it
and you come to a situation where it got
fully mechanized that means that you can
do
interactions much more interesting with
it much more
rich like a robot i mean like like a
full complex computer system but also
it means that interaction become
regulated become part of the machine
that means it has integrated
the second part of its uh not only this
production rule but its usage rules
so this is basically what is currently
happening uh it's a long process i mean
people don't always look at
wow whether the kindle is more
interesting than the ipad i mean we're
not talking about that i mean
typically after the printing press for
50 years the books that were produced
they were exactly the same
than the old medieval manuscript and
they were just copying for 50 years
nothing changed in terms of the user
but the system terms of machines
and in terms of the way the book was
produced was changing and that had a
tremendous
impact on our culture so one of my topic
now is really
seeing book as machines and again
translating
all the things we've been learning about
robotics about
sensory motor skills about embodiment
with the topic of books it's something
which is not yet
discussed a lot and that's why it's
interesting to me and i've been starting
to talk
with lauren volley a new venture an app
called book app where book applications
where we explore that both from a
research point of view and then from a
very pragmatic point of view adapting
existing books and
and and seeing what's what will happen
on that so it's uh
robotics leads to everything great
uh anything else you'd like to add i
think that's that's fine