okay so i need you to start by
introducing yourself
telling us uh when and where you were
born
and where you went to school yep member
statements
born in newcastle near cologne
in 1936 january 4th
and i went to school in little staff a
neighbor neighboring village
but my father was a teacher
and then i went to the gymnasium
starting in 1947 in ports
which is now part of cologne the city of
cologne at that time it was a
city of its own and of course like in
germany we went to
gymnasium for nine years and then after
that i did a practical
work in industry which you had to do at
that time if you wanted to become an
engineer i wanted to do
aerospace engineering and this i started
1956
what kind of industry were you working i
did my
my practical work here in near munich
in with dorney aircraft engineering
and another part which was
really a
written reason what is it called in
english
um mining the the the auras are
mining yeah well now it's if you process
metallurgy metal metallurgy yes right
they said in toast off
which is close to cologne again okay
and then and then in 1961 i meet my
diploma from engineer and i went to
the deutscher at that time to actually
focus on zeus and
which is somewhat like german nasa at
that time
in mulheim and one year later in 1963
moved down to uberwaffenhofen where it's
still located
right now and from 1962
to ninth to 1971 it was essentially
associated with the aerospace
research and i did a
study on nasa um fellowship
at princeton university in new jersey in
1964 65
essentially control engineering and
aerospace applications
and when i returned i started working on
a optimization
project numerical methods for trajectory
optimization
and uh with this i did make my phd
in 1969 at the technical university in
ahren
re-entry vehicles and turning the
orbiting plane of re-entry vehicles by
dipping into the atmosphere and
finding out again at that time this was
one of the future
maneuvers which i thought would be used
possibly in the space shuttle
but it turned out that the shuttle was
so delicate with respect to heating
that they never used maneuvering when
re-entering
so this was more an academic work
but uh being in in the us i
met one of my co-students from achen he
was with
nasa huntsville and on a drive down
to a seven launch in 1971 when i was
uh no in 1964 when i was in
princeton um i met him and
we kept this
connection for the next 15 years or so
and he invited me after i had my phd to
come for a
postdoctoral research associateship
to huntsville alabama which i did in
1971-72
and there i worked on a shuttle
arbitrary entry
this was the field i had been working in
with my dissertation
so we did some investigations and
several publications in this area
and then i came went back to germany but
at that time
i think all the activities in this
direction of future
um launching systems in europe
they were very delicate so there was not
enough money so they abandoned it pretty
soon
and that's when i started working with
satellite control and my group did the
what you call the launch and positioning
of the first european communication
satellite symphony
in 1974 and 75.
and after that at the first of january i
think it was in 1975.
i became uh acting head of the research
center
over an orphan of dlr
and i was in charge of
about 700 people and i don't know half a
dozen
up to a dozen institutes and smaller
installations at that center in 1972
so one year earlier the federal army
uh university university visited
was founded in neubee buck which is
close to here and i got an invitation
to give a presentation and
possibly become a professor in the
aerospace department
of this university and after
some hesitation i decided in october
1975 to go there
during my activities at the oberhaven
wolfen research center of dlr i've
become acquainted with
satellite imaging essentially for
remote control not remote control remote
sensing
and being a control engineer
i had the impression with the progress
which was you could observe
in increasing computed power
per volume per performance
unit and also the prices
so that it looked like you could afford
uh sufficient computing power for
real-time image processing somewhat 10
to 15 years down the road
so i decided to
do all the installations at the
university in the direction of
developing vision systems
for mobile systems and
i had the application areas of aircraft
space vehicles and off-ground vehicles
and with this topic we started in 1977
the first dissertation started in 1977
so we did the pole balancing first and
then we did a
satellite control i can show you in a
movie if you if you like
this was an air cushion vehicle floating
on a table two by three meters
and doing by reaction jet control doing
docking to another satellite on the
table
and after this had been successful we
decided
to go for grants and that's when
the development of our vision system as
a
relatively big line of development
started here in germany
so we were approached by time to events
but
we could do this together and we decided
to cooperate with the daimler
in developing the sense of vision for
vehicles at that time
possibly you should know that 1986
was the 100 year anniversary of the
first car being built
by daimler and benz they approached us
and said there was a
big european framework the eureka
framework where industries should
cooperate
in order to compete with the us and the
japanese developments in these areas so
daimler proposed to do a
large scale research half a billion
marks at that time
for the development of technologies for
the second
century of car developments and
vision became one of these issues so we
were
successful in eliminating buried cables
for driving autonomously on the autobahn
and we said we can do it by vision
and we got this contract and uh well
there were
several steps in between i think i
skipped these you can see them in the
video
and we made a final demonstration of
this project which run for seven years
from 87 to
94. at the paris
demonstration in october 1994.
for this demonstration daimler had
equipped two
s 500 cars
with vision systems and
we drove in normal traffic around the
airport
charles de gaulle three lane traffic
with speeds up to 130 kilometers doing
autonomous lane changes with guests on
board
and so this was the first demonstration
i think in public to a large audience
in total or two vehicles and over
several thousand kilometers on the
ottawa one
in france
and the next step was a continuation of
this by european funding
so we switched to the next generation
the first one was done with transputers
and then we switched to the next
generation
of transputers we hope but this didn't
didn't realize
the nine series never came to existence
so we switched to conventional
processing with the methods we have
developed
which have been derived from control
engineering recursive estimation methods
quite contrary to what has been done in
the
ai and computer science community
where they looked at image processing
essentially as a cause aesthetic
image evaluation and by evaluating
sequences of embedded images they wanted
to arrive
at an interpretation of the motion along
the timeline
our approach was quite different we said
we want to do
a high evaluation rate right from the
beginning
i made the set the limit at 0.1 seconds
so
10 frames a second and then we
decreased the complexity of the task and
we wanted to increase the complexity
when more computing power became
available
and of course what you could observe was
that
every i say phd generation four to five
years
the computing power increased by a
factor of 10.
so within 15 years we started 1977
from 80 to 95. this is a factor of 1000
in computing power and then at that time
there was
sufficient speed and the the systems
were small enough so we could switch
to off-the-shelf systems up to then we
had custom design systems
but not as in the us developing
systems completely new with hundreds and
thousands and ten thousands of
processors maybe single bit processors
but we decided
right from the beginning to go for
conventional microprocessors because
we saw that there is a big difference
between the biological systems
which was one of the um
yeah the the the uh
the predecessors where the development
in in your country in the us
they looked at and said the technical
systems should look similar to the
biological eye and quite a bit of the
development was
going into the direction of coming from
pixels to
[Music]
processing structures behind that and
we said there is no need for combining
image taking with processing because in
silicon you have
10 to the 5 10 to the six times higher
bandwidth
so you can send not just two three four
but dozens of images
in one frame time so there's no need to
do this and we could stick to the
conventional microprocessors but of
course the
evaluation has to adjust to these
properties
and this of course could be done with
the recursive estimation technologies
which have been around since 1960 since
the kalman filter
but not as has been done in the computer
science community
by looking at static images but by
looking at
real image sequences with models
for motion of 3d objects
so these were 4d 3d
motion of 3d objects in time over time
that's what we called our approach
before the approach and i can show you
afterwards
the basic idea of a view graph which is
systematically built up and
you can see and this allowed us to
achieve
maybe 10 times the performance of
autonomous vehicles with
0.1 or 0.01 computing power on both the
vehicle
so the community was
quite surprised when we demonstrated our
first running on the autobahn
in 1987 at speeds and the maximum speed
of our vehicle of armors
at 96 kilometers power when in the darpa
project
the alv and the nav lab they crawled
around
with two three five kilometers per hour
and
they evaluated say for one image
between one to ten seconds and
this was only possible by developing
these methods which we called for the
approach
and if you like we can you can look into
this yeah
so let's let's um talk a little bit more
more about that
so how would that relate to say um
james gibson's theories of perception or
david mars sort of uh he has this 2d
two and a half d 3d so how do you
construct reconstruct three dimensions
out of two dimensions is what you sort
of
think so the vision problem do you buy
do you endorse that perspective
no it's not we we
reject it because we feel that
you have to have available all four
dimensions
right from the beginning and your models
should have the time component and the
3d space components
and then we talk about motion of 3d
objects with features
over time in 3d space and we are looking
for features
essentially we were looking at corners
and edges
that we could extract relatively simply
and
from these we have to reconstruct the
objects
in addition to the edges it is important
in order to make distinctions between
objects to have the
um what is it the gray value
at the side of the edges so this we took
in addition
and then we were able to look at motion
from one frame to the next we made
predictions
and what we did right from the beginning
was prediction error feedback so
according to the model
we assumed a 3d object
with a 3d motion and then we made a
prediction how should this develop over
time
but we did not just take the model as
the nominal model but we also used a set
of linear models around the nominal one
and then in the next frame we
compared the prediction with what really
happened in the real world
and that's why we call it prediction
error feedback and this was
a breakthrough in real-time processing
so most of the computer science people
didn't believe the results when we
first showed them and of course it was
very simple it was a wet alliance
and dark too bright transitions at the
side of the road
we didn't have to have white white
markings lane markings on the road
but we could do it with a normal
transition from
macadam to just grass or whatever
and then of course we have to be very
careful
uh to check whether the
initial assumptions about the world and
the objects was correct or not and
it turned out that this seems to be
very similar to what the biological
systems are doing and i think right now
the developments you can observe
in the theory of mind is according
exactly to these models because uh what
the
spiegelner what is the neurons what they
do is essentially
store developments of appearances over
time
and from this uh you can see
directly not just a snapshot but
if you see a snapshot for you in
perception it's
a snapshot of emotion so you perceive
the motion
and that's that's a big difference to
reconstructing
2d two and a half d 3d i don't believe
in that
2d plus time we have to to retrieve 3d
plus time
and of course you do have the continuity
conditions over time which is very
essential
and you do have them in the in the
interpretation right from the beginning
are you familiar with james gibson j.j
gibson's
um and that's theories of affordances
because it's a lot about this
that action is what gives you that third
dimension as you move through
space but um but that wasn't influential
on your work no
no so no we directly came from control
engineering
so this is the kalman filter approach
and we said
what has been done wrong in the computer
science community is
to look at images and say the edges move
linearly
or maybe non-linearly so you add some
noise in the image we say
no what you see in the image is a
prospectively distort
mapping of the 3d world and the
interpretation is directly done in the
3d world
not in the image not in the image plane
it's directly done in the 3d
so in terms of recognizing say obstacles
that are
off the ground versus just a colored
patch on the ground
so is your method then a lot more
efficient than oh yes of course
and the good example a nice example you
can do is
if you stand on a road with a camera in
the in the vehicle
and you look at the road in front you do
have a certain appearance of a
like a pencil and
if if you take an artist and he puts a
large
sheet of paper on this on one of the
locations say
three meters in front of the vehicle and
you paint
the outside world exactly as it is as it
appears to you at this position
then there's no way of making the
distinction between the real world
and the painting but
if you move your vehicle moves along the
road
and the sidelines on the picture
they move to the side while
looking at the real road they shift
along the road
so over time you see the discrepancy
between
aesthetic interpretation which is wrong
and the 3d interpretation versus right
which is correct so this is very
essential
were there other sorts of visual tasks
that were difficult for this methodology
that you encountered
that were well of course obstacle
detection uh is
uh one of the difficult ones and the
most difficult one we met
later on maybe 10 years later these were
negative obstacles
which has been part of research in the
us in the military field
uh not so much in the in the university
uh also not in the darpa activities of
of uh urban challenge and grand
challenge but
the u.s army research lab they had
projects going on and we've been
cooperating with them since 1987-88
so there you do have the big difficulty
that
in order to understand depth
of course you have to have stereo vision
or some laser rangefinders
but if you drive at higher speeds then
you don't
detect the obstacle
early enough to do a close reaction
so we did an interpretation with um
stereo interpretation and laser
rangefinders
and with image interpretation and we
came to the conclusion
that you should have a combined system
both laser rangefinder
and stereo interpretation and the the
visual interpretation because if there
is a
hole in the ground you do have some
certain properties of this
of the horizontal surface and you have a
usually a completely different
visual property of the vertical part of
the hole
so what we can do is by normal vision
detect
the um small vertical part if you look
at 10 meter distance
on a hole which is maybe one meter wide
and one meter deep
you just see a small small upper
fraction
and if there are a grass um
in front of this you're not able to
detect it by
at least at that time you're not able to
detect it by stereo
and also uh the laser rangefinders they
do have proper
problems detecting it so we made the
combination that
the first detection is being done by
normal vision and then if you close in
then of course you're lost with vision
and then you have to look at
the stereo interpretation or the defined
structure
which you get from laser range finding
so this was the most difficult
what about like water filled uh potholes
or uh
well of course uh the um even more
challenging there's a mirrored
and uh it's the same like for humans i
believe it's just a shallow water
or it's a deep water you you cannot
detect what it is
uh quite a bit of discussion has been
going on in this field
uh my feeling is if you are on a normal
road and
if up to then the road has been
used well almost flat then there will be
puddles
and usually you can assume that you can
drive through it so
if there are these um
appearances we just assume it's not
critical
if we go cross-country which we did in
the late 90s and early
2000s the field is completely different
then you have to stop and you have to
look what it is yeah
yeah it's a good assumption on german
roads but we have a lot more holes in
american roads i'm afraid
um so what about um
moving obstacles so if if a bicycle or a
pedestrian comes out in front of the
vehicle
how do you try to deal with those sorts
of no you see how the the features shift
from one to the next you see the
the the uh set of features which moves
in conjunction
and then we come to the conclusion uh
that this is an obstacle
moving obstacle in the first
investigation we did was the pro
balancing
maybe you should have a look at the film
so you really understand much better
what
what is going on i do a 15 minute film
which gives a survey on the approach
with all the applications we've done i
don't know whether you have
time or you would like to to do it it's
also on the on the
on the cd yeah we can look at that and
uh shall we do this first
um well i think i have we think we can
go on for now but i'm looking
um but as far as uh so then from the
control perspective
was it ever a problem to figure out when
you should be braking when you should be
turning
how do you maintain the stability of the
vehicle if it's a
if you're trying to avoid a collision at
high speed
of course these are difficult problems
but usually if you do the
the 4d representation then
you also make predictions of not only
how you are going to move but also how
the other
objects are going to move and you do
have a representation of all objects
and in parallel and then you see and
that's a
straightforward analysis will there be a
possibility of a
collision and then of course you have to
react and
well car control it's it's simple for an
engineer
that's that's being done for 100 years
that's that's not
not a new field no but you didn't have
any issues with
like hunting or over steering or
anything like that
well this you see essentially if those
people
deal with car control who start from
static images
and who don't have the notion of
maneuver i didn't
emphasize this but this is very
essential
if you do control application i've been
working in
aerospace maneuvers
extending over not seconds and minutes
but hours
so you always should have in mind what
is the overall maneuver you are doing
what are the maneuver elements you are
applying and then you have a list of
maneuver elements
which you have to work upon one after
the other
and then you to have to find the
transitions from one maneuver element to
the next
but within the maneuver elements there
are control engineering feedback loops
or
if you want to make a lane a turn off uh
there is
a feed forward control you do this for a
lane change
so so this means you do a direction
change
by car it's an upcoming steering all
cars and then if you do the other the
other side you means you do have a
an offset to to the side but go in
parallel if it's symmetric
so this is again it's relatively simple
and
if you do have some errors most people
i've seen many people that try to
correct this immediately
now we say i'm in a maneuver and
how do i have to change my maneuver in
order to achieve my goal
so the control corrections the magnitude
of the correction is much smaller
i've seen very nervous control
applications
in these application to ground vehicles
and
i sometimes had the impression that
there's another magnitude difference
between
what you see in many approaches and what
you see in our approach
interesting and in terms of organizing
the high level maneuver are you using
um a hierarchical control system are
these
um at what level is um
are these being represented i guess
maybe i should i prepared some some
information here for you so let's see
this is the the overall
system and it's it's hard to explain
just by word i think one should look at
the
at the different uh realizations
so these are the the maneuvers
the maneuver capabilities action and
behavioral capabilities
this is a lane change this is the
superposition of
non-linear feed-forward control and
linear feedback control
and then here you do have the network of
capabilities
which indicates which other
subsystems you need for realizing a
certain behavior
and usually like in say vision control
you do have
one motor turning in your and the other
one in pan
and then you can combine this in order
to do all the different maneuvers and
circades and whatever you like
and you asked for the hierarchy of the
system this is
the vision system so first we do feature
extraction
just on the overall image without any
meaning behind it
this is edges and corners and then
we immediately jump to the assumption of
objects
and motion of objects and then we can
set on top of this
looking for special features if it is
this type of object i should see
and then we look for that and either you
see the udon
and then we write the spatial temporal
coordinates
in this uh what we call dynamic object
base
and here is all the information on the
on the objects which are relevant for
the tasks you have to solve
and of course the data you have to to
handle
is two to three orders of memory to less
than in the image plane
down here and on top of this
there is the situation understanding
level it looks at the
combination of objects or subjects we
call a
an object which is able to sense the
world
and do action according to what it's
it's sensed this we call a subject and
of course we assume that we do know uh
reasonable behaviors of other subjects
see
they would they want to survive and they
behave essentially like like i
control my car so this is being done
here and the overall control system
that's how it's called in in the in the
loop the overall control system looks
like this
so we do have vision control which is um
with multiple cameras usually um
here is the capabilities of the vision
system capabilities of the vehicle
this is the hardware of the vehicle and
based
on the collection of
sense data we come to the conclusion of
a situation
we do have a mission element over here
and then there is the decision for gaze
control and the decision for
vehicle control and this is the loan and
there's of course an overall goal which
i have to achieve
this one shows the number of feedback
loops
it's more well just an
abstract representation which we close
and it's half a dozen about half a dozen
on the different levels both in image
processing and feature extraction
in feature interpretation in object
control
and stabilizat stabilizing the
interpretation of the situation so this
is very essential to do this
well i think that's that's about it more
than a minute okay
yeah that was very helpful um so in
terms of
other kinds of applications so how how
robust is that methodology would you
think for say a
a robot that wanted to walk through a
crowded plaza
instead of drive down a road because on
a road you have rules that you know you
stay on the right
there's traffic signals no i think it's
it's
completely applicable and as far as i
can see it's
the best method available to do any kind
of control tasks of mobile systems
we did applications in satellite talking
applications in aircraft landing
i can show you we've been recognizing
the relative position to a runway within
a real flight
in 1991 with experiments in bronze week
and in the hardware and the loop
simulations we did
helicopter control around the airport of
brunswick
with different aspects and finally
landing on top of the helicopter age
this
marked on on one of the the taxiways so
it
recognized it maneuvered to it
hovered above it and then came to a stop
and if you do have of course if you do
have many
objects moving well not so nicely like
usually cars do
but more erratic like like
pedestrians you need computing power
we started looking at pedestrians in the
early 90s but at that time we were not
able to do it in real time
so i do have a video film showing this
but since 1990 to now
it's four it's two decades it's four
times 5 so
10 to the 4 is the increase in computing
power available now
so i'm pretty sure we could do it now in
real time
with a dozen other subjects or objects
but of course the the difficult tasks
and there we do have
did have quite a bit of discussion with
the industry
what are the proper sensors in order to
do this
should you have 20 or 40 cameras around
the vehicle
or should you develop a system like
the like biology did
moving the hair and moving the eyes
and we came to the conclusion that maybe
the best compromise you can find is
having fixed focus systems but
three to four cameras in conjunction and
then do gaze control
in two angular directions
and by this we say maybe 120 degrees
under 30 degrees viewing range should be
sufficient
in horizontal direction and maybe 20 or
30 degrees or 40
in vertical direction and then you can
do the rest by
just looking where the interesting
things are
and the same one big discussion was
well we could afford nowadays with the
cost of cameras to put several hundred
cameras on
on the vehicle and we do have high
resolution everywhere
and then we scale down to a low
resolution
but again my impression was doing the
navigation
through all these images it's much more
complex
than developing an eye and have a wide
viewing range
with two cameras with an overlapping
central area where you can do stereo
interpretation
and then direct the gaze control to the
area where you want to
see something more closely and we
propose to have
a color camera with medium range say 100
meters range
for the wide angle cameras to have 20 30
meters
good resolution and then on top of that
in order to be able to have a resolution
similar to the central part of the human
eye
we say we should have another camera
with a higher focal length
say at 300 meters we should have a
resolution of about five centimeters per
pixel
so these combinations of focal lengths
that's what we call the
emv uh vision system marvin
multiple
oh hooray
i can tell you lately my memory is
vehicle vehicle eye and we call the
vision system
expectation based multi-focal
um psychotic vision system that's that's
the one
we developed and i think this is a good
compromise between the different aspects
realizing the basic functioning
of biological systems in silicon
on a silicon basis not doing all which
is necessary
because in carbon you are not able
to get high-speed data transportation
but you're able to
make thousands and ten thousands of
crosslinks that that's what we have in
our head
which is very difficult in silicon at
least with the technology we have
available
so we decided to take these simple
systems
but then use the high data transfer rate
in order to organize an overall system
which does have the same functionality
on us very similar or if not the same
a functional basis like biological
systems but
realized completely different yeah then
that was a feature of the first design
with the transputers
when you didn't have very much
computation power even
yeah well the first system was even one
to two orders of magnitude
less than the transputers so the first
drive we did in 1987
it's hard to believe but this was 80 80
86
processors with two megabytes
cycle rate um
basic frequency that two megahertz two
megahertz
two megahertz yeah and
uh of course the world today is
completely different now we are
into two gigahertz and um
you do have uh even higher much higher
data communication was a problem at that
time too we were not able since we were
not able maybe i should mention this
since we're not able to
to look at the entire video image the
colleague of mine falcograph
he developed a system that i had
proposed he realized it
where the video signal is transferred
line after line and he developed a
system that was able to grasp
those pixels which were in a certain
window
marked by the upper left and the lower
right corner
and we were able to define about a dozen
of those windows
so we could disintegrate the image into
12 sub windows
and we just extracted it from those
windows but
we were able to switch the window from
one frame to the next
uh so this was very they were analog
video
they were analog video of course at that
time the cameras were this
this size yeah so you had
you had to choose the rate the hurts of
the
scans and interlacing and de-interlacing
and drawing yes yes
we had to work with american and
japanese and german cameras and we had
to switch between 30 hertz and 25 hertz
and
all that stuff ntsc and or dvd
pal
so so
how did who what was the inspiration for
the the work and vision like did you
come
you said you saw that work was
accelerating very quickly in that field
when you were doing control systems and
you saw that convergence coming
were there particular researchers that
you collaborated with or people whose
work
that inspired you no at that time
i was uh just picking up the
the less school um professorship for
control engineering at the
university of the business yeah and
i met my colleague who's who's
been working in measurement technology
measurement science
vocal graph and he was also in
interested in that and
we decided since we couldn't get a
system on the market which which
satisfied our requirements so he said
well i think i do have an idea
and he came up with a dozen
microprocessors intel
industry microprocessors and he
developed these
systems grabbing these windows and
integrated into a system
and the first relatively high dynamic
stabilization we did was
balancing a pole and you know depending
on the the length of the pole the
frequency goes down so we started with a
one meter pole
and we were able to come down to 50
centimeters i think 30 centimeters was
not possible at that time
and we did this probe balancing on an
electrocard
3 kilogram and acceleration of 0.8 g
8 meters per square second with
about 1.1 second
frame time with
four intel 8085
8-bit microprocessors and when we came
in 19 when was it in 1981 we went to the
first international conference and
showed this
people didn't believe us that's
impossible
because if you look at the at the the
edges you get from
from the pole you're balancing they of
course you have to integrate over time
in order to get the image so you have to
to look at the front part of the edge
because there was quite a bit of
blurring going on
yeah and uh myself did this dissertation
and
together with haas who was the other phd
student with photographer
they really developed a fine system and
this was the basic start for our method
what we learned from this simple
application
the next application was the satellite
talking as an air cushion vehicle which
of course was a very slow
system by jet propulsion from from
compressed air
but these were relatively complicated
hexagonal 3d satellite bodies real
bodies in 3d and the
system had to move around you can see it
in the video and
it decided which corner to look in order
to get the best
interpretation with a limited amount of
features available four or five
so it decided now i should look here and
then when it was a self-occlusion
something called catastrophic event at
that time in the literature
it decided well yes this is going to
disappear so i have to abandon this and
go to another one
and looking at the uh
the matrix of of the the
linear approximation
what are the oh my my memory follows and
the names
jacobian matrix if you look at the
jacobians from the jacobian matrix
you can set up a
feature which tells you which
combination
of features to select in order to get
the best interpretation
and this is what wintry developed and
he's now my
my follow-on at the university of the
bundeserian so he's
he's now working with uh the
modern uh autonomous vehicles
which are now vienna uh vw tourx
okay yeah yeah so um
did mercedes-benz make any practical
applications
from your work and yes that they use on
their vehicles
um the start
how to proceed was
well it was quite some discussion
because the question is
vision is so complex and you have to in
to invest quite a bit in order to get
started
before you get the first result is it
really worthwhile doing it for just one
function
or should we wait until we have
developed a system
which is able to do four or five
functions
and then you can you can justify the
cost of the system
but there was a divergence
with respect to application to world
application
in in production line around the mid 90s
when industry decided we should go very
simple functions like lane departure
warning
and we should install very simple
systems in the car
and then do it on a minimal cost basis
uh and that's what what was started and
these systems
have been around since the early 90s and
you can buy them on the on the market
right now you can even buy them in
middle class
cars right now next first was
the lane departure warning the second
one was distance keeping
and industry decided because the
number of computations you had to make
and the computer systems you had to have
available on board
were much lower for radar systems so
they decided to go for radar in distance
keeping
what is called adaptive cruise control
but then of course in radar you usually
you don't see the road
and uh it took some time until we
realized that maybe
and when computer availability of
low-cost computers became better
then the idea came that why don't we
combine radar with vision and these are
the systems
which have been developed in the late
90s early
2000s and they are coming on the market
right now
i don't know whether there is one on the
market i'm retired since nine years
well the mercedes has this uh attention
assist
so uh it tries to monitor i don't know
exactly how the control system works but
it's supposed to indicate if you're
getting sleepy but it's looking at the
driver input as well as
this is right this is different there's
a different system yes there are many
many
different systems which have been single
line different systems which have been
developed
what has been demonstrated in 1994 also
in paris was traffic sign recognition
but in order to do a real-time
demonstration they had
they had a separate van with separate
computers on board and they were able to
analyze just two types of systems
passing not allowed and i think one or
two
speed limits so it was not possible to
integrate this
into a system uh at that time
which was sufficiently small to put it
into a
passenger car in the meantime
it's it's completely different you can
buy these systems reading traffic signs
and showing them
in the in the display on the on the
dashboard
so these are available and what else
i think in preparation are also
warning systems for if you access
a crossing to see whether there are
vehicles coming from the site
so first single um investigations
have been done started during the
prometheus
project but these were cameras mounted
looking to one side and camera mounted
to this side and one camera looking
ahead
and of course if you look at all of this
in a in a real
world system which you want to sell
somebody
costs are so high you could do it and
that's why we came to the conclusion
we should develop a
vehicle eye with the properties i
mentioned before and then
develop the entire software around it to
be very flexible
and since industry was not willing to do
this or they
didn't expect getting the money so we
decided
to leave this type of civil development
and
join the more military developments
which were going on both in germany
and in the us and since that time since
1995
we did have a cooperation with several
uh american
institutions like sound of research and
national institute of standard
technology
and we developed these systems the the
ems system we developed in
connection with these and the german
ministry of defense
so what were some of the military
applications or have they made
applications out of it well of course
vehicle guidance on road running
this is one and detecting negative
obstacles
is another one what is really available
in application i guess it's very very
little
it's still too early it's too complex
to be introduced and to be competitive
so it's still in the research area
right um
so as far as the autonomously driving
vehicles
were you ever riding in the vehicle
while it was driving itself
sure it's a strange feeling the first
time
i got into the there's a five-ton van
and when this uh accelerates
like a human driver really with all its
power it has
uh the first ten minutes uh you were
very
uh anxious what's going to happen and
then you see well
he behaves like a human behaves and
you get accustomed to it and we always
had a safety driver in the seat
and he was able to intervene when
something happened but we've been
driving
not only in the car since 1986
but we've been driving in public traffic
since 1992
in on the autobahn on bundeslars and on
small roads
so there was uh
well we had to have a special license
from the german military
organization to do this but after they
have observed this
us for i think about 10 years
uh yeah somewhere like 10 years they
said well yeah you can do it
but they have to be two or sometimes
three people in the car
so we were not allowed to to have a
fully autonomous vehicle nobody on board
running in public traffic because uh
i think the they assumed that those
people in the car they want to survive
and they'll be very careful
good insurance policy but you took it
down to the autobahn at
very high speeds okay well on we
in in the european project uh cleopatra
this was in 1995 96.
there were other groups one danish group
that hit
schweissen in shipyarding shipbuilding
um we call this schweizen
fuse plates of of iron
oh well done welding yes and there was a
meeting in oregon
in denmark which is 1600 kilometer
kilometers
north of munich so they decided two
people decided
after in april or march
the cmu group which was our competitor
in the us
they had done the hands off through
america so
the longitudinal control was done by
human driver but electric control was
done
autonomously and then we had a meeting
in
i think it was september october and we
decided
to do a fully autonomous drive to this
meeting
up there and we wanted at the same time
to collect material in order to see
what are the most essential parts which
have to to be improved
in the third generation vision systems
this was the final of the
transputer system we knew we had to
switch to a different
computer system so we wanted to know
what are the parts which have to be
improved
and during this ride in the northern
plains in ludobuga haida
the top speed driven autonomously was
180 kilometers an hour
that's pretty fast also pretty fast
that's 110 miles an hour
yeah yeah um were there moments where
you were
unsure of the robot or uh well of course
of course you sometimes see a
misinterpretation of the situation
it needn't be dangerous one one thing i
recall is
there was one student looking at
other cars as obstacles and obstacle
avoidance
and the system was looking on the
autobahn two lanes and
one car in each lane and it suddenly
came up with a hypothesis
uh didn't see the left card didn't see
the right car but
it interpreted one object in between of
the two
so and looking for for some time of
course they moved to slightly different
speeds
and then after a second or so or maybe
one and a half seconds
it made a decision though that that
cannot be a correct hypothesis so it
started from a new and it
got two objects and then it converged
towards two objects
of course it always takes some time
maybe
a second or so in order to recognize uh
until it's
uh sure what uh what it interprets and
this was one of the conclusions
we took from uh from this test drive
to the north we should have system
capability
on all levels both the feature
extraction level
the object interpretation level the
situation interpretation level
so that the system itself comes to a
conclusion
how sure am i about the data
are my predictions correct are they
getting worse and then there should be
warnings so this type of
of recognition
and then calling for help it's very
essential and one of the interesting
points was that
if you look at statistics of human
trafficking accidents
there is a surprising lack of
people that got a heart attack while
driving
so when people said something must be
wrong
and then somebody else came to the
conclusion well those people maybe they
feel that something is coming and so
they
slow down and maybe stop at the site and
then they looked at the
statistics and came to the conclusion
well if we include these
cases uh statistic is okay
and this is the same conclusion we came
to the system itself
should observe its properties its its
performance
on all the different levels and then
come to a conclusion am i sure that i
can go at all
should i stop or should i slow down and
of course
in the auto on the autobahn with the car
you can always slow down if you are in
the air
it's a little bit more difficult fewer
obstacles though
fewer obstacles that's right um so
how do you foresee that everybody
eventually like all cars are going to
have autonomous driving capabilities or
the majority of cars
i think if we think in decades or
centuries yes this is going to come i
tended to make prediction that maybe by
20
in 1980 85 or so 80 at the end of the
80s
maybe in 2010 or so 2015
i thought we did have enough computing
power that this would be the case but
seeing the difficulties it is
it is a long way and i'm pretty sure
because of one meeting we had
with the organization which is
responsible for developing
the traffic guide rules in germany
there's a special institute and we had a
meeting
with the legislative side the automotive
clubs
some legislators and the press
and then of course the same question was
asked
and of course industry said
we are willing to do this but there is
one precondition
everything what's happening in the car
should be
written down or should be a protocol so
that later on you can you can see
who did which control input and then the
people from
the the car uh
area c what is the car clubs
representing
the drivers they said this is never
going to happen as long as we
are in charge and have something to say
so
there will be no full documentation of
what's going on
in the car not on a legally
on a legalized basis and i left this
meeting
with the impression that maybe the
technical problems will be solved before
the legislative problems will be solved
and this of course has to do something
with the with the processes
uh if if there's an accident then the
situation of course is much worse
if you can nail down what really
happened so
yeah and this is the same situation
right now people are
very hesitative to do full documentation
of what's going on
but uh if this is being done i think um
it's maybe a few decades in order you
can do this
do you think the the robots will be
safer drivers than people
yes in the average yes they don't drink
alcohol
they don't get tired so i think
these these monitoring systems and
that's the first application
this will be very useful and they will
work
first for the first one or two or three
decades as monitoring systems
and then when maybe on the military side
there's quite a bit of experience with
real autonomous driving
uh it may also come into private cars
but it's a long way to go
are some of your applications the being
used for actual satellites in space
yes we did some satellite docking we
looked at it
and the only application we've done was
uh with robert
with herzegovina grasping a
free-floating object in space
this was in 1993 on board of space
shuttle columbia
but this was inside the d2
box which is inside the shuttle so
there we did the remote control because
computing power was not sufficient on
board
we had to transfer the images to the
ground and this went through several
satellites through
i don't know how many kilometers or
thousand kilometers of cables
uh then there was a computation done in
over 500 and of course
there was a three second time delay when
the computer started working
and of course the signal going back to
the satellite was another three seconds
so overall we did have six to seven
seconds time delay
and we were able to compensate for this
by
predictions because in outer space of
course there are nice newtonian
conditions
it's a second order system except for
the
corrections which are being done on the
shuttle but the system itself
it's just newtonian motion and we've
been able to show that
there's one film also on the cd
where he you can see that one was march
203 1993 that was the first grasping
in the meantime i think some activities
have been done both by the japanese and
the us and the russians
but this was the first one real
application i think
wouldn't be too difficult so the docking
with the slow speeds
it's just a question of would you do it
uh
because you lose one more justification
for having humans on board
and
maybe some agencies are very happy that
they have justification for humans
yeah yeah um so who are some of your
intellectual influences
as far as what inspired you to go into
control systems or what inspired you to
to do some of these vision control
systems for for self-driving crimes
well i was uh intrigued when i first
learned about differential equations
when i was 16 or 17 i really i recollect
that
learning differential equations and what
you can do in describing the world
this was really a big event to me
and then i decided to become a control
engineer
and to do a team tour engineer of course
the most difficult
things to control were aircraft so i
decided to do
aircraft engineering and after i've done
it in germany i
went this one year to princeton where
i looked in depth into control
engineering
and there it was dunstan graham and
heckle
he wrote a book on dynamics and flight
mechanics
and then when i came back as i mentioned
we did
this um symphony
positioning on the equatorial
orbit geostationary
and then the rest was just uh my own
idea
i've seen what you could do with this
recursive estimation
part which is nothing but
gaussian's idea of least squares
approximation but gauss did the least
square approximation to batch
numbers measured and what kalman did was
he
re-defined it for a
sequence of measurements coming in one
after the other for
space applications this was very
essential so he did it in a recursive
way so you now should have
dynamical models again there are other
differential equations
and this was the breakthrough so
i tried this and i don't know it was in
the early
80s when we had a meeting somewhere in
kansas i think it was
people from computer science
i didn't believe in it they had heard
common
filtering but they didn't realize
that what you have to do is formulate
your models
according to the real world in 3d space
and time
everything in 3d both the object shape
and the motion
and correct in time and then take the
perspective model
and use linear approximations to these
strongly non-linear processes so this
was the breakthrough
in the mid mid 80s
so essentially at that time um maybe
it's more the
discussion with the people who didn't
believe in it i recall when we first
made a presentation in 1987 at santa
cruz in
california there was a not advanced
study institute
at that time they had these institutions
where they brought together people from
physics from biology from physiology
engineering in order to exchange
what might be necessary for developing
intelligence for vehicles and
when i first said we've used common
filtering for
interpreting emotion image sequences so
they said well forget it and we've tried
this for years and
you're not going to get far with that
and then the day after when i showed my
films
already with the car running
and well that was a big surprise
so uh several names i don't want to to
call them here but
they said well that's surprising so
then we continued developing this and
one of those in america which we had
quite a bit of exchange
who was the most open one from all the
american colleagues was takio canada
so i really appreciate the way he
handled all of that
it was different from other experiences
we had that
video equipment was gone when i had to
to give a presentation things like this
but
so he really he invited me to give a
talk at cmu
and we had discussions and i could
discuss with the students and
they were open and they started some
students
started in the same direction and well
they
essentially said yeah
looks like this is a good idea
and when i was at caltech as a
visiting professor in 1998 i recall that
one italian guy came back from a
conference
in hawaii where he said
well it looks like all the developments
of these special architectures for
vision systems based on
this multiple 110
10 000 million fold single bit
processors
it's going to be abandoned it's
no more seen as the way to go and they
switched back
to the what is it what do you call this
there's a there's a short
term uh general purpose process gpp
um and development which then just went
from 16 to 32 bit or
on 32-bit the next step anyway the
communication also was high enough at
that time that they came to the
conclusion
we can do all of it with conventional
microprocessors
and this is the way how things develop
i don't know whether you were aware of
of the first
developments in the darpa project on
autonomous vehicles
they had about a dozen different
architectures
specially designed architectures for
vision one of the most
well known and widespread was a thinking
machine
and i always had the impression that
handling numbers and
getting to an intellectual level is
different from just working bottom up
you have to have the high end component
and very influential to me i think you
asked this question
was were the thoughts of tropenhauer and
of course
way back [ __ ] [ __ ] has been
misinterpreted here by the german
idealists
and schweppenhauer put them from the
head on the feet again
he made the distinction between the real
world and
the world within our haiti alzheimer
that was the title of his main work and
i think this
this gave me some um
confidence into the approach
and i suddenly thought maybe
this approach is not just good enough
for technical interpretation
but you can you can really um
recognize or explain quite a bit of
biological systems in the of the human
mind and this world by going this way
so you have to start from a high level
representation
and how this was generated what was the
the hardware and the time involved
millions of years and
i don't know how many hundreds of of um
generations of development
what it boils down to is that there has
to be
some preconceived notion of
a spatial temporal world and then
you can combine sets of features
with object object classes
individuals of objects and of situations
how to behave in which situation
and this is what we did then i had up to
20
phd students in peril and
so this was very interesting this period
and we made quite a bit of progress at
that time people came to visit us from
all over the world from japan china
u.s of course also all of europe we had
the common project prometheus was the
european project
they were somewhat like i don't know 20
or 30 universities involved in this
project
and we switched around from one place to
the other and had discussions
so this was very very interesting
um were you much influenced by the works
of the cyberneticians of
norbert wiener or warren mcculloch
well of course robert wiener he was
essentially
a control engineer and said that the
idea of feedback
that you use information in order to
derive your proper control in order to
achieve a goal
that's the basic idea which
has been followed right from the
beginning yeah
i think that's very essential but then
of course uh
when all the predictions have been done
what you can achieve with this very
general approach
people note it especially in the
computer science community
that it's very hard to represent
background knowledge on this level and
that's when
the separate computer science artificial
intelligence direction developed
and they started with more or less
static frameworks
but this was and i i've seen that
right from the beginning this was not
going to solve the problem
what we need is both parts we need the
one side and we need the other side
and what i could see was that what is
necessary
in order to get powerful systems
is to combine quasi-static
representations
but not as static facts but as
models for dynamic processes
and if you do have articulated bodies
you do have certain laws how they can
move
and all of this you have to represent
and once this is available
you can expect what the other guy is
going to do which
you observe if you've seen a situation
like that
so this was very essential but you need
in addition to venus work you need the
static representation but not of course
aesthetic states but of
how what is the the collection of models
which you need in order to derive at an
overall system
which can be goal directed and based on
momentaries data you measure do you
think those models are learnable
yes they have to have we learned them
this way
i don't believe that there is some
insertion from
another world also so this is all
the biological systems discovered that
they have some data
about the world by all the senses and
vision
and they tried to get a good
interpretation
and from my point of view maybe
schwitzen to say it a little bit
over
what you get an understanding
of why the notion of space and time is
necessary
and available in all animals right from
the beginning is if you look
at the way how the inner ear works
and how the eye works and on one of
these natto
advanced study institutes i noted and i
learned
that ndi you do have a lag
until the interpretation of the world is
there of several hundred milliseconds
in the inner ear you do get the derived
measurements of accelerations and turn
rates
with say order of magnitude millisecond
time delay
and in order to get these two combined
you have to have a notion of time how is
this going to be developed
and one of the nice things is if you go
on to
yarmouth we say like the oktoberfest
there are certain carousels
where you have separate motions around
separate axes and what happens there is
that the visual impression doesn't fit
the inner ear impression
and then you get your
yes and this is one of the one of the
actual inputs where i came to the
conclusion
the internal representation of time
is due to these different time delays
between the inertial part of the inner
ear and the eyes
and if you get these two in conjunction
and you get them consistent
then you feel well if it's not the case
you get dizzy
and of course that's the very nice thing
that the
inertial sensors from the energy sensors
you get the impact
of perturbations right on the
acceleration basis
right on the on on the lowest time level
if you want to interpret an acceleration
by
some perturbation from vision you first
derive
vision is on the second integration
level so you look at states
you have to have the first derivative to
get the speeds and then the second
derivative is accelerations
so again the combination of both is very
essential
and that's why we propose you need a
4d model in order to combine this and
this
i consider this is our main contribution
to dynamic vision that's why we call it
dynamic vision
yeah that's very interesting um
one of the machine science phone
forester built was the dynamic signal
analyzer for sound
but he simultaneously did these multiple
derivative analysis of sound processing
yes if you have one sensor you are not
in the situ in the same situation
uh and if you do have only sensor data
and you have to work with it you're also
not in the situation so what the
the the biological systems had to come
up with
some representation of the outside
worlds which allows you
to combine the large time delay from the
visual system with a very short time
delay
at least on your system and what helps
you also is
that in vision relative motion between
two bodies you can make a distinction is
he moving around
and again here the visual the inertial
system helps you because
your own motion is being sensed here too
so you get another input
and you can make a better discrimination
and from my point of view
this is the starting point for
developing higher integrated systems
with the notion of 3d space and time
integrated
that's great um
wow okay
so
do you think um in your systems do you
ever try to extract that
three-dimensional representation and
save it as memory i mean you're
obviously doing
uh over time spans you have this you
know there's
one dissertation and that's maybe the
most complex representation where one of
my students
did a generic class of car models and
by combining certain parameters he had
about 20 parameters i think
by selecting parameter value ranges for
these parameters
he could make distinctions between a
passenger car sedan and
a combi and a
van a bus and
as i mentioned 12 different types of
cars
and we were able to
show that this works with the real
systems
so models and the
uh adjustment of model it's the central
part
generic models so we do have models
which are
quite adaptable and then you have to
have the
procedures available to adapt it to the
corresponding situation
and we mostly worked with edges and
corners and we very soon came to the
conclusion we have to have some
area-based
information like texture and color and
but computer power wasn't there that's
why we couldn't
couldn't uh start with it but nowadays
it's available and one should do it
yeah and once this is available i think
we pretty soon
achieve the performance level of humans
i guess it's one or two decades away
tomorrow
so what do you think the biggest
transformations in robotics have been
during your career apart from the
increasing computing power
well of course i hope that the
transition from course aesthetic
representations to
spatial temporal dynamic representations
is one of the essential parts
on the other side from
no i think all the basic laws have been
known
motion laws have been known perspective
projection has been known
um realizing
that systems in silicon
have to be because of the properties of
the
of the substrate in carbon you do have a
reaction time in the range of
milliseconds
in silicon you do have ranges of
nanoseconds or
so this has been recognized from my
feeling where
money went for developing systems only
by the late 80s early 90s
so this was a very essential part and
now of course once this is going to be
developed
you can see that with progress in
electronics also now you can make
subsystems
which you combine with conventional
computer systems
uh maybe i should uh mention at this
point
when we did this it did have this
cooperation in negative obstacle
detection with the us army
uh there was the task of detecting these
uh
ditches and in 1921 i think
in 201 i think we had a system available
which was 30 liters
like this that was the first system
capable of doing
full video frame full video raid stereo
interpretation
like this so we did some developments
over here and we showed that it worked
it
also worked in connection with the ems
vision system
and then they said yeah well maybe we
should proceed and they got a funding
that
the system was developed further and one
or two years
later we had a europe card sized
16 by 10 centimeters
electronic card which you like
something like this one here
which you put into into the
into the normal pc system and
we could show that we could do full 3d
interpretation and detect these ditches
with something just put into one of the
four pcs we had on board we had four pcs
with
two dual processors eight processors on
board
and this in addition this unit which
they had was somewhat like
80 billion operations per second so it's
a
tremendous tremendously powerful
and this development of course is going
on and
you may have half a dozen or a dozen of
this or similar systems
in future simple conventional
computers i recall when we started in
the
late 80s i think it was in 90s first
internet international discussion the
goal was
one coffee pot 2 liter size
for the entire computer system on board
of an autonomous vehicle and we are
pretty close to that what influence do
you think that the darpa challenges had
on
uh autonomous driving systems and
research and development well i'm
sorry to say but i don't believe in the
first one because
this was just vehicles
pulled through the desert by gps weight
points
and uh they didn't have to look for
negative obstacles
all they had to do is drive along these
gps paths
and avoid positive obstacles sticking
out of the world
yeah this they did well i think the the
uh robustness of the system was a very
good demonstration
but with respect to recognition of the
world
it wasn't too much and the second one
the urban challenge
also is if you if you look at it from a
perception part of you
it's also a little bit more complex but
not the real task because the entire
information
on the road lane markings on the traffic
signs available
on crossings where to go how to move
from one direction into another on
crossroads
everything was stored on a cd which was
put into the computer and in one of the
the the focus areas the the uh
talk what is it uh what the people
exchange
one guy asked we do have a vision system
on board
and if i notice that the
lane marking is different from the one
given on the
cd which one is valid
the answer unfortunately was the one on
the disc
so again i would say
with respect to perceiving traffic
situations
there's not so much progress
good progress has been made in detecting
obstacles stitching out of the ground
and
adjusting to obstacles and also to a
relatively complex situation
at crossings but also with vehicles only
and again the paths they drove were
given by gps
so this is a torn
through the city by rather strict
predescribed routes
uh it was impressive how they could do
it
but there were some cars without any
vision and i think even one
did finish without vision so
not like visual perception what not too
much of like visual perception what has
been done there
yeah do you think it's a general
strategy having these big competitions
is a good way to try to
well no no no you have to look at the
the justification and
the justification was if we have to do
what we call this mission support in an
area which has been
um not covered which has been
um where the army has a
rule over the terrain where they know
the roads are good
and you want to move um support what is
the maintenance stuff
from one location to the next and this
is the type of
of tasks which could be shown that can
be done because
then you know that there are no
obstacles no negative obstacles and you
need
a relatively small perceptual capability
for this it was good
but if you look at the general task of
perceiving
a 3d world there's quite a
quite a way left to go there's some
progress
but it's not the general task which we
would like to solve
what were some of the other major
breakthroughs in the last 20 or 30 years
of robotics that you think are really
influential well with respect to
what you can do with human-like or say
articulated bodies
i very much admire the work of
herzinga here in germany and also in
what is it near salt lake city
there's another us institution they also
did some very
very nice developments uh this was
yeah this was uh really good what else
impressing the most impressing to me was
these small vehicles on mars this is
really something
where i have the feeling
that things
are able to perform much better than
engineers they're saying so they
predicted the lifetime of what six weeks
or something like this and it worked for
four years or five years it traveled i
don't know how many
kilometers and it's really amazing what
has been achieved there
so that's that's really very impressive
and i'm not so aware of what's going on
in um
say outer space with the suns which are
sent to other
planets that may be maybe similar things
but this one here is is really yeah i
think to me this is the most impressive
one
that's great um
what else would you like to add any
other thoughts reflections on
robotics and where it's going where it's
been
well i think the main part is that one
should
try to get a better understanding of
robotics that it's not going to be
competitive with life development
that it's going to abolish life after
some time
as you can see in some of the science
fictions
one should really try to promote the
idea
that the cooperation between technical
systems
and biological systems is
what we are looking for and that life
also for
humans and other biological systems may
be improved quite a bit
if we do the proper application of these
technologies
so there's quite a bit that can be done
in household care in other people care
and developments are going on in this
direction maybe i should add that also
what would impressed me well quite a bit
was the work which has been done in your
country
with respect to support in hospitals
by robots so but then
you see there are always some some
individuals that
try to fool the system i don't know
whether you
have learned about that these marks for
electronic positioning have been removed
and one of the
the robots toppled down the staircase
and things like this so it's it's
amazing but
uh um apparently this is part of the
nature of humans
of some humans humans that they would
like to
to show that they can fool systems
biological ones and technical ones
and as long as you have to deal with
these system vandalism i think it's
called
and you always have to be aware that
those people tending to vandalism will
be around
and this is one of the main difficulties
for
technical systems because it will be
almost
impossible to predict which type of
vandalism
will be possible with these systems and
if the manufacturer
is going to be made liable
for a product even under these
conditions progress will be
rather slow
yeah i think that's a very good point um
what do you think the importance of
modeling robotic systems after
biological systems is
how important is it to really pay
attention to how biological systems work
well i think the functional functional
idea look at the functions
and how it is being solved like
instead of having hundreds and several
hundreds of eyes or even tens of eyes
like
like some animals even do have today
nature developed
the head and neck and and the eyes so
the function
is important and we can learn from that
but
we should make the the mistake to
to rebuild similar systems on a
completely different
technological background as i mentioned
carbon systems
and silicon based systems are completely
different and
you are not able to transfer the
solution
in carbon-based systems like our bodies
onto silicon this is different and this
has to be recognized
they have been big projects in your
country and in my country also
where they wanted to develop a
electronic eye
because it's known that there are
somewhat like 120 million
light sensitive sensors in the eye but
the number of
of lines of
communication lines going to where the
vision process is being being done in
the back of the
it's only 1.2 million so there's a
factor of 100
compression so that was
the basic idea why people started
looking at combining
sensors with processors in the first
stage in order to reduce
data communication after that but
silicon that's
that's silly there is no problem
take take all the information you want
you need to have and just send them
because bandwidth is so high
don't fool your system around with
requirements which
are not due to technical necessities but
due to
some um example
based on a completely different hardware
or what way or whatever you call it yeah
yeah i think that's very interesting um
in terms of robotics research in europe
and then germany specifically where
do you see the major research centers
being and in robotics in robotics within
germany and
and europe well i think one
of the the best ones in europe
is herzinga dlr in over
north and there are other ones
in france and
good ones also in sweden
i'm not so aware of what's going on in
england because we didn't have
so many connections they pretty soon
dropped out of the
cooperation in prometheus so i'm not
aware of what's going on there maybe
there are some activities going on in in
oxford
cambridge uh maybe either but i'm not
sure
we did have some cooperation early in
autonomous vehicles but they have been
abandoned
as far as i know in england with respect
to autonomous vehicles
finland has been doing uh interesting
work
every now and again which labs oh it's
close to helsinki
hazel yep so i think it's cool
um maybe also
but i'm not sure yeah what were the labs
in sweden and france in sweden it's
the what is it
in stockholm and in
and uh what is it
no it's nisha berlin shopping
me shopping there's there's a university
and
lean shopping link shopping it is i
think yeah
and in france
and of course to lose the
uh ah what's uh what's the name
well the odds the big well there's the
no that's industry
i thought of the the research
institution there's a
large research institution which has
been in in paris first and was then
moved down to
to lose and we had quite a bit of of
cooperation or exchange with those
um i'm sorry but my memory with respect
to names it's not too good anymore
so i forgot the names but but you find
if you look at those okay and some
activity have been in
in the southern part um
also in rio institute sofianti police
but this is 15 20 years ago so i don't
know what's going on right now
okay i think that covers
most everything that you could think of
unless you have more to add
no maybe you should have a look and look
at some of the videos