so if we could start with your name okay
i'm ray jarvis
uh i was born in rangoon burma
9th of january 1941 my parents had sort
of british background and some asian
background mixed
we came to australia in 1947
landing in fremantle on the west coast
and i went to a christian brothers
school in a suburb near where we were
and finished up going to
university of western australia where i
did my undergraduate
degree in electrical engineering and
subsequently completed a phd in
electrical engineering
following that in 1968 i spent two years
at purdue
in contact somewhat with professor thu
who was one of my cohorts
but i actually worked with professor ed
patrick
who was working on pattern recognition
those days
went back to australia
to join australian national university
as a senior lecturer
and was essentially given the task of
setting up a computer science department
which began
as an embryo group within department of
statistics
and a few years later we became a
separate department but we're still
strangely in the faculty of economics
which is a bit odd
so i worked at the australian university
for about 14 and a half years
went to a chair of
electrical engineering at monash
university which is in melbourne
in 1985. um
changed the naval department to include
computer systems soon after
and established in
uh intelligent robotics research center
in 1987
and been director of this group ever
since
so the group essentially included other
academics from in the department with
backgrounds in computer systems with
several different interests in robotics
so my colleagues had
complementaries to mine and we tended to
work in a fairly democratic
way collecting a few students
roughly 10 to 12 between the four of us
expanding up to about 24 when the
funding was good and dying back to about
10 when it wasn't so good
my own robotics interests started off
with a combination of
global search as it was called in those
days and some pattern recognition
background drifting into image
processing
and then computer vision and robotics
so the computer vision robotics about
1982
so when i established the center at
monash
my main interests were things like
robotic hand-eye coordination
various range finding methods stereo
depth and then i
switched between the hand-eye
coordination to
robotic navigation of
vehicles generally wheeled sometimes
tracked
and mixed indoors and outdoor
experiments so
about halfway through i suppose my time
at monash
i got interested in larger outdoor
vehicles that could
operate in rough terrain those that
sense the environment in sufficient way
to carry out either tele-operated tasks
before the automated tasks and
i suppose the biggest project i worked
on in the last
five years we completed a year and a
half ago was
working with the country fire authority
who were interested in
robotic vehicles that could assist with
putting out
forest fires and that
work split up into several different
approaches one was vehicles that could
go into forward
scouting to check whether some area was
safe
others that were capable of clearing the
path for
another vehicle and then finally the
vehicle that itself would carry water
and extinguish flames and the idea there
was
never to be too concerned about the
degree of autonomous control but just
make sure that the
human operator was at a safe distance so
the
concentration there was on rich sensor
feedback
and very simplistic control of existing
mechanisms like steering braking
accelerator without modifying the
vehicle more than
absolutely necessary i guess another
project
which i had several goes at was
creating a semi-autonomous wheelchair
and the idea there was to allow for
an adaptive shift between the wheelchair
being fully under the control
of the user to taking over a certain
cinematic control where that user
showed some signs of fragility either
from foresight
or tremor or tiredness and the
i guess the key factor there was to be
able to shift
the responsibility between the user and
the
robotic equipment on board so this
vehicle is able to
avoid collisions but also give advice as
to
safe directions and sometimes
pull the sort of the vector of intended
direction towards the safer
and to change the parameters so you're
switching between
the fully autonomous to fully human mode
i had several goes at that the first
vehicle was simply one where you
moved the joystick and had indications
of where you could go safely and
shifting around the second version of
that was where you had
an eye gaze tracker looking back at the
user
so that severely disabled person could
go
wherever they like simply by looking so
just where your eyeballs not even your
head necessarily
keep your head still and look sideways
and that turned out to be
very successful technically but
incredibly expensive so no one's going
to work with that
then i had a variant of that which was a
walking frame that was motorized
actually converts into a wheelchair
again with the same thing the last
version is using the kinect
sensor which now brings the price down
one connect sensor
feeding forward to work out the obstacle
space one for you back to see what the
user is doing for gestures
so i guess i've mixed my interest
between
path planning vision robotic navigation
and generally have been absolutely
convinced that experimentation is then
going to go i have
developed a few theoretical ideas on
clustering on distance transform
methodology for navigation and
complex hull sort of geometric some
computational geometry
so i've done a few theoretical things
that have solid
founding but most of my interests are
making things
work in physical space and to
show i suppose the relationship between
concepts and
reality and adjustments that are
required to
take the
existence in in a clustered space into
account
my ongoing interests remain in the area
of
human machine interactions and
particularly in thinking about the
different types of intelligences
required for
human robot interaction and i had this
idea that
there were three types of uh
intelligence
highly related first i call spatial
intelligence
which is about how the robot can find
its way around without bumping into
things in an efficient way
how it can recognize and handle objects
this is all about things in space and
what they are
and how to move the second i called
transactional now some of the computer
scientists would
call this dialogue i call it transit uh
transactional
because it's about the way the human and
the robot communicated
but beyond that how the robot came to
understand
what the human required but within its
constraints so the negotiation took
place
to get the robot and the human to make a
deal about what was going to be done
and that's a little bit more than just
the gesture or just the voice or just
understanding of diet the third level i
call social intelligence is how to have
robots mix with humans
in such a way that humans feel
comfortable so it's not just fulfilling
the task efficiently
it's about having some
notion of cultural limitations about
proximity and approach and interruption
and and timing and also to include
perhaps some aspects of
i suppose you call it entertainment and
flexibility and a bit of softness in the
whole pressure and i think that's the
one of the hardest areas particularly is
extremely hard to get funding in that
area because it seems too much like
social science but i think in the end
we'll have to master that part as long
as the other two sort of layers
so that's the work i'm doing now i've
sort of been semi-retired
and hopefully we'll get an emeritus
status to stay linked with my
equipment and colleagues at monash
but my mentor's interest i continue
this work about multiple robots in a
human interactive environment
sensor rich mainly interested in vision
and laser range finding
navigation gesture recognition voice
and somehow protocols for
human cultural
limitations on behavior
that's what probably brings me up to
date
that is a great summary so we're going
to
um to try to delve into some of those
episodes of
into the different projects and some of
the um
the details of things that you were
interested in and the kinds of
technical and conceptual challenges that
you were dealing with
um so if we could start with when you uh
well before we start with your phd i was
curious
how did you decide you wanted to go into
electrical engineering in the first
place oh okay i i guess i was interested
in making things
as as a kid i was always making things
like
projectors for film for photography or
models of things and i always had this
notion that i would like to be in a
situation where i could make things but
i didn't really know whether it was you
know
bridges electrical things or mechanical
things
and i suppose i didn't decide that
strongly in term i was in my final year
high school
and it seemed to me that electrical
engineering
appealed to me more because it looked
like it was going to
explode into lots of different areas and
as i got more and more into it
i think i was reasonably pleased with my
choice so i never
really seriously considered moving
sideways into
mechanical civil or chemical engineering
i went to a school i went to at
university of west australia
had this common first year which many
eastern
schools had but then i kind of split up
into specialities but in fact
there were still some common uh courses
that were run
right up into my third year it was
actually a five-year degree
so the specialization only came halfway
through the fourth year and then
continued the fifth so i'm actually
looking back
enjoyed the mixtures of topics for
example
in addition to electrical engineering i
learnt a lot about
mechanical design about surveying
about strength of structures material
science
and a great variety of things which i
suppose at the time we
did not think were relevant and in most
areas of electric engineering would not
have been relevant but in robotics
happened to be incredibly relevant
so even knowing that surveying was done
is a huge benefit when you start to
think about
how to localize a robot in environments
and use of theater lights and
lasers and all this kind of stuff
i also remember that
we were taught to sketch things a lot
and i did technical drawing in high
school as a special
subject which wasn't taught by my school
actually night school and
technical drawing because i like the
idea of visualizing so i think that's
where
i started to think as visualizing things
as being
the thing i really was interested do you
remember any
um things that you build when you're
younger that really stand out like any
specific
problem well i i guess i did build
uh a number of different versions of
photographic enlarges and
projections and i built models of boats
and things like that and
my brother went into chemistry and used
to make explosive things i enjoyed
making things that shot
rocks out of pipes and things like that
slightly dangerous things but
i was generally interested in in things
that
one could make with very few tools and
inexpensive
bits and pieces so i don't suppose
there's any particular thing
i i just made whatever i could with the
things i could find around that cheaply
so
i didn't really know too much about how
to go to a disposal store and buy things
i also had a sort of hobby interest in
radios
i happened to have an extra neighbor at
one stage who
had an interest in this and used to give
me old rages and my ground valves and my
crystal sets and twiddle around trying
to hear what was happening
so i had a general interest in gadgets
of one kind another and those interested
in gadgetry has actually
stood in good stead throughout my career
so i think if any of my colleagues
were asking what sort of captures what
i've been interested in
i suppose you could say it's a gadget
someone kind of
which i've accumulated quite a few
so after you got to the university and
during your
ee degree in undergraduate did you were
you working in the lab at all at the
time or
was that in your pm no there wasn't much
opportunity to create things at that
stage but i do know that our
engineering course was very very lab
based
we did many many hours of laboratory in
fact
for most of my classmates the laboratory
work was considered dull but i actually
enjoyed it because i i just liked
to see how things work so if anything
i had a greater interest in the
experimental laboratory work than i did
in the more theoretical site
and i think that
this is partly of course why i've always
considered experimentation as the way to
do things although i
perfectly happy to slot in you know
theoreticians and
people who have more philosophic views i
i guess i'm just saying yes we need all
that
but the thing i do best is
experimentation
um and then you when you got to your phd
what kind of environment did you get
into
and who are you working with okay now
now when i started my phd
i had recently developed an interesting
computation
this is a 1960
364. um
the university at that stage had a very
small computer it was an ibm 1620
and i with a help
of friends developed some programming to
optimize
the circuitry for
switching circuits for
binary switching circuits that was my
first introduction to actually using a
computer
so as my phd developed i got interested
in
hybrid computing which is combining
analog
computation with digital computation
in those days it was considered that to
build
dynamic models of differential equations
on a
hybrid website on an analog computer
allowed it to assimilate physical
systems at maybe thousands of times
faster than you could with a digital
computer that day
but as uh digital computers became more
powerful
then the need for the analog side
dwindle so i was in that in between
stage where there was still
benefit of combining two types so i had
an analog computer with a
link to the campus computer which
upgraded from
plural ibm 1620 to uh digital equipment
corporation
deck 10 which is apparently the second
time shared machine made by this
company that was outside the us
continent so
the uswa was the first outside the us
actually acquired one of these
seemingly very advanced machines where
multiple users could all
clatter away on their tele types and
share
memory and do their
computations by either mixing
with their colleagues at during the day
and sometimes in
blocking large chunks to be hands-on
single user at night to do the heavy
computations
and i had a long cable between the
physics school where the computer was
housed
and the electrical engineering lab i was
working in
about 200 yards apart part of the
problem was how to get
digital signals and backwards and
forwards without noise and this sort of
stuff so
i built the interface between the two
computers i built models of different
dynamic systems on the analog and i got
interested in using the digital computer
as a means of optimizing
parameters so that's where the
optimization
of parameters came from on a hybrid
computer so my thesis had a very long
time
about using a time sheet computer and a
hybrid
system to do global optimization
um in those days there was always
that well people still today realize
that if you have an optimization problem
which is
monotonic that is there's only one
valley if you're looking for the bottom
of a valley
you could use steepest to sense slide
down the hills and so on but as soon as
you had multi modality
it was a different problem so i was
deliberately trying to work out
what happens if you have a parameter
space which has
almost chaotic multimodality
and it turns out that you have to
consider how to
spread your testing as well as
concentrate simultaneously
and so that was called global
optimization and uh
that's where i first got to know about
kingston ferguson
because one of his students had done a
version of this using a probabilistic
method
so what you try and do is have a method
that was able to
randomly select where to sample
but also possibly had some local
refinement going on simultaneously so
that was the idea of how to
work on two levels at one time so my
thesis came out with this sort of
randomized
search methodology for global
optimization it was still
fairly
theoretic in the sense that the types of
multiple
modal surfaces i was working on were not
from industry they were just
things i constructed myself and then the
question became what the shapes of these
valleys
shifted in time so the idea was how to
have an ongoing real-time process that
could
track shifts and particularly those
shifts
meant that the valley that was the
deepest went up in a
valley over the hill went down you don't
have a direct path
how do you keep a watching brief to
continue so that a lot of it had to do
with
strategies for keeping a watching brief
and what was happening in different
parts of space and then
with some idea of not wasting time
because obviously if you've spent a long
time
on the hilltops looking at the next
valley you're not going to overall
minimize so that that was my thesis work
and then
that's where i knew about the work of
king san fer so
when i graduated in 68
i was attracted to go overseas and it
was fairly natural for me to say okay
purdue is a good place to go i'd never
been outside australia except when we
migrated initially
and so it was a great uh kind of excited
exploration to go to the united states
and spend
i intended initially to spend a year
stay two years
but ironically didn't actually work with
kings unfair work with ed patrick
instead
and that's where i got interested in
pattern recognition who was your phd
advisor
i was dr
brian leary leary l-e-a-r-y who
did not have a strong interest in my
topic
but what he did have was a faith that i
could do it so he would say i'm not
quite sure what you're doing
but i think you can do it and it turned
out to be the best attitude
because i didn't want too much
advice quite frankly i had an idea and i
didn't want
someone to divert me from what i wanted
to do
so that that part worked out well
because
if i had no real sort of drive
then the supervisor
not being specifically interested so he
was generous enough to say
i'm telling you now i don't know much
about it i'm willing
to act as a supervisor because you need
one formally
i'll help you where you can to get
resources but don't expect that and i
took that as being very honest and
made the most of it basically so that
was good
so he would not be embarrassed
to hear that uh i reported his
non-speciality because he was really
upfront about that and i
appreciated that at the time because i
find
sometimes when i have my own students
and they start drifting in
areas that i'm not familiar with i tell
them outright either to
get advice from my colleagues who are
specialized or i tell them look if you
want to move in that direction i can't
help you
may be better for you to find another
advice i think it's good to be up front
because
it's not good for a student to believe
his advisor
is an expert and find out later that his
suggestions are almost wild guesses
and don't really reflect what's
happening on the field and i think
that's dishonest so i really like this
straight
forward approach so
and so when you got to purdue what was
what was the situation there like what
kind of
okay now what i found out was that uh ed
patrick
who had a very strong theoretical
background in pattern recognition had a
number of students working in this area
i got friendly with one of his students
and we worked together on a project
several two and one of them was to
build an interface to their computer
to do some image processing in those
days one didn't have sort of standard
cameras
so this was like a scanning method of
collecting data and analog so that was
my first
interest in the image processing side
and i also was very interested in
various pattern recognition
methodologies
patrick was very smart in
sort of high level mathematics about
probabilistic structures and the like
and he happened to have some good
external grants so it was really nice
working with a group where
there was funds for travel and there's
no concern about buying components and
things like that so for me it was rather
nice i did a little bit of lecturing as
well
in computer logic theory and stuff like
that so it was a pretty nice time
um we wrote a couple of papers together
uh i got registered in an area called
clustering which is
non-supervised pattern recognition and
i authored a paper which i
acknowledged his support as second
author
on on clustering methodology which
turns out to be more useful than i ever
imagined because
it was called clustering using a
similarity
based on sharing new neighbours now the
title actually tells you the method so
the method was just super simple what
you did is imagine all these points in
there
so you click in space you found
maybe k nearest neighbors okay maybe
seven or ten for each point
and you looked at the neighborhood list
of two points and you said
these two points could be considered
put together if they shared a lot of
neighbours it's a bit like saying
tell me your friends you tell me your
friends if many of them common you are
probably
friends and it's a very very simplistic
computation and it doesn't make any
assumption about underlying
probabilistic so it's
totally non-parametric and by changing
the size of the neighborhood and the
thresholds you could
adapt to many different sorts of
clusters clusters that were long
strings of points those are very
globular and so on
and then 20 years afterwards
say five years ago i just looked up
clustering and i found everywhere
they're referring to this jp algorithm
what the heck is the jp algorithm it was
java's patrick algorithm so i was very
pleased to find
it was in great use and by
pharmaceutical and molecular
people who are working in the field on
the edge of medicine
and molecular chemistry they're using
this as a standard method for clustering
their different types of products and
and making this clustering available to
their customers and the like
so uh video didn't patent it but you
know i was pleased to see it was used
and that's where the kind of mix came
drifting out of global optimality into
pattern recognition and image processing
that finally led to my interest in
computer vision and violence
did you while you were doing the
clustering work did you have a
particular application for it in mind or
was it
uh no i think i was simply
picking up the interests of the people
around me
uh patrick king saunders was a
international figure in the pattern
recognition here at the time
so i think i was just trying to absorb
the local specialities
at that time i didn't really think about
a slight shift
but certainly later on it was
interesting because that my old interest
in
optimality and pattern ignition came
together
so i picked up some of my
thesis topics again when i went to the
anu
and instead of
just oh i guess it was mixed
so instead of looking at at the question
of how do you find when the
minimum points have shifted in a way
that you cannot track them
i developed this idea that you could
have a pattern recognition algorithm
that tried to detect when there was a
sufficient change
to warrant a re-exploration so that's
where the patent reaction
and and the optimality came together
and then when i went to their new i got
more interested in
collecting image data and analyzing it
so i drifted away from the pattern
recognition side
but i did find applications for
clustering and image processing
on several different occasions so
probably uh that idea of mixing it
came to its peak when i went back to
purdue and studied it
um kind of felt a bit guilty going back
to the same place but
um there's still some of the colleagues
that were there before and
that was just a one year stint so i went
back and this time did work a little bit
more with
king son foo and some of his students
and there
the image processing and the pattern
recognition came together quite strongly
and what i was interested then is what
is now called image segmentation
so someone gives you a photograph and
you
show it to someone else and say can you
take a pencil and draw an outline of the
major things in this
then say can you get a computer to do
that it turned out to be really
difficult in those days partly because
they were computationally intense
took a long time to do and i tried to
use the idea of clustering
to group the pixels in the image into
conglomerations and i developed a number
of different strategies
that were based on graph theory
basically saying that
you could imagine every pixel in this
image as a leaf node and what you're
looking for is how to get
the leaf nodes to conglomerate into
large and larger pieces but not to
cut across the boundaries of different
objects and the clustering
helped there so by the time i went back
to the anu after that i had this
strong interest in the segmentation
and more and more concerned about how to
do it faster
and you need to do it faster if you
wanted to use a robot so i got
interested society of
robotics as a means of testing the
validity of
the computer vision and so about that
time i got interested in the 3d aspects
so until then just 2d imagery
now i visited i think sri and bob boles
was there i don't know if i met him that
time
but he had some colleagues david nitzan
and others who built a laser rangefinder
one of only one or two in the world at
that stage
and i was fascinated by the use of you
know time of flight
laser ranging
and when i went back after that visit i
was determined to see if i could get
some funding to build on
in my lab in canberra
the australian national university is
and
i applied for what's called an
australian research council grant
for mainly the equipment money i worked
out how i could possibly
buy off the shelf equipment and
put together a budget was mainly about
the cost of various things
and in those days the australian
granting committee was split into
smaller groups and they often visited
laboratories to talk to the applicants
and try and ferret out some of the
background information and whether that
was in fact a very good way of doing it
rather than just looking at what's
written on paper
and i met up with three of them and they
spoke to me
very openly and said oh we probably came
just to see a lab but we're also doing
this job for the australian research
council
and we only think it's fair to let you
know that we
have uh got some opinions about your
plan
by some experts in in that case of the
csro which is a commonwealth
research organization in australia
i wasn't given any specific names and i
said well the experts there believe that
what you
want to build is not possible under with
this kind of you know
under under plan you've got i thought oh
well at least they're decent enough to
tell me don't have false expectations of
getting the money and i had a nice chat
and they thanked me for the visit
and then they gave me the money so now i
was really under
some pressure they told me call me down
they said have a go
and to my delight i managed to get
working
and uh stabosis was the first laser
rangefinder in the southern hemisphere
which it was in those days that was sort
of 1982 it's very early
and that's where i really got interested
in the third dimension
because that used to be the thing that
stopped you doing
the segmentation and the handling
probably there was no point just
outlining figures and people on the
photograph what you really want to know
how to recognize things and to find
where they were in space
so about that time
i had this notion that robotic
manipulation was perhaps
the most honest way of testing whether
your vision was operated correctly
anyone could look at your sketches and
say hey that looks like what i would do
but how do i know that that schedule
that
segmentation has sufficient information
to actually permit
manipulation i thought if you could
actually show that manipulation was
supported by this
analysis you then had a way of saying
uh even if you have objections to some
of the philosophic ideas
i can demonstrate a practical outcome
therefore from an engineering point of
view that's a good
solid result and then i thought what i
need is a robot
and again with some funding i had
something like twenty thousand dollars
for a robot
and in those days a decent robot
such as the one made by
joe ingleburger's
stuff was called puma and this one
called the unimate the small one was
called a unimate 250
those cost about 45 to 50 000
the local agent for um unimation
i think what animation incorporated in
those days
said why don't you ring joe engelberg so
with nothing to lose i remember trying
to pick my time so it was sort of
mid-afternoon
united states and i spoke to joe
engelberger you know the great
the father robotics and all this kind of
stuff and he just said oh
look leave it with me maybe i would find
a refurbished unit and send it to
and indeed good to his word this
box arrived with a puma 250 at 20 000
so that was my first robot and
between the time i got it and the time i
finally
shifted to monash like over a three year
period i started doing hand-eye
coordination
so i was using my laser rangefinder and
other methods
vision methods to allow robot to pick up
blocks
so i scavenged a lot of the building
blocks of my kids play pens and stuff
like that
had all these different colored blocks
and i could manipulate them and
pick them off a table and put it into
different and i was quite pleased with
that
again the experimental side was seen as
critical for me rather than more
theoretical science
were you in contact with some of the
other people who were doing
manipulation and block world at the time
um no but
i certainly i knew the bloc's world
experiments
in great detail and in fact taught them
in my classes in computer vision so i
knew all the work
of winston and guzman and others and
also some work that was happening at sri
simultaneously in fact there were two
schools of thought about how to do it
the sri people were
dealing with one aspect and mit was
going the other direction and the
two actually came together later so i
was very intrigued by the box world
experiments but
i was really more interested in trying
to exploit the
the physicality of the 3d analysis
rather than the
notion of clustering vertices
which is the guzman approach of
sort of reasoning about block structure
from the appearance of
the edge data i said if i've got a 3d
analysis of things physically existing
i can pick them up even before i know
what they are
and what's more i don't need to know
which things are
attached or not attached to other things
because by picking them up i'll find out
so there was this is active this is now
called active vision it's how to
actually
improve your quality by saying the
manipulator
is allowed to change the world and
therefore the change can expose things
that you may not have known
statically so i was very keen on the
idea of interacting
so a number of other projects that
followed along this line
is all about what i called uh
post recognition manipulating first and
then recognize for example picking up an
object so i've got this object
now let me look at it and now decide
what it is
which is not what humans do but from a
robotic point of view
much simpler because once you separated
a physical object from a cluster
you now had it in isolation and you
could change the viewpoint
so one of the projects we had when i
had just established the intelligent
robotics research center was in fact
a hand-eye process by which we had
a 3d scanner there's not a laser scanner
but a stripe scanner collecting 3d data
and then
picking up unknown objects holding them
up to the camera and then recognizing
them
and that was pretty successful
that project about three years with a
few colleagues and so on with one that
we were hoping would be taken into an
industrial
application with a very large company we
had
as an industrial partner this company is
uh
bhp which is one of the largest
mining companies in the world combined
with the company called built and built
and
php bulletin is quite huge i give you
some idea of the size but of course a
great sense that last year they
posted a profit of 22 billion dollars so
it's big stuff
in early days they had a research
laboratory about
one and a half kilometers from my campus
so it's quite good to
work with them so we got interested in
the hand-eye coordination stuff
and also in mobile robotics
so they were talking about the
possibility of using mobile robots in a
steel plant and mostly the sensors that
we're developing not so much
either the demonstration with the
hand-eye coordination or
navigation but we were developing a
number of
3d sensors striped light and trying to
speed things up and get quality
and in fact one of the outcomes of this
research project was a
a strike light 3d system that was used
in the steel mill
to to test the smoothness of the surface
of a
rolling mill sheet coming out and bhp
claimed that that application alone
justified the
the money that went into the research
project so i actually took one of
probably about five ideas and at least
used that
within in-house uh bhb at that time did
have a
commercial wing that was meant to take
good ideas to commercialize
outside but they were a little bit uh
hesitant they didn't actually take this
product i think that wing of bhp
disappeared anyway so we're a bit
disappointed that what we thought were
uh usable industrial ideas were not
exploited except for that
particular range finance building how
much money was involved if you remember
uh i think it was a million dollars over
three years so it was enough to employ
about four people
um in the australian context that was
considered quite large
and it was one of those very early what
was called a
good gird was government's uh industrial
research development
so it wasn't meant to be pure research
was meant to be something you did with
an
industrial partner and the hope was that
by using
public purse funding you would encourage
the
industrial partner to exploit the market
with the
the outcomes and certainly bhp were
one of the key industrial partners for
quite a number of different projects
what year was this
[Music]
now that's a bit harder i think around
90 how about
i think i've been about uh 9192
i3 about then i have to double check i'm
not sure exactly
yeah it would be about then
about a little later i i joined uh one
of the uh
um panels of the australian research
council
i think that was about 82 92 to 96.
so i think the year this other project
was before then
that was interesting just to be a member
of the panel be like being on an
nsf panel or some such thing in america
so i got to learn i guess
what you could call the the outer
writing
submissions and how to do it and how not
to do it there's many many examples and
i got good write i got good at writing
proposals i think about that time and
so typically what this committee would
do is collect like 300
applications farm them out to assessors
subdivide it into the panel members to
act as the lead for discussion on
maybe 100 samples each and then
incorporate our own responses with those
of the referees
and come to a common idea about the
quality of project
and then more or less democrat would
decide who was going to get the money
that was a fairly uh intense process
so the pre-reading maybe took a week or
two and then you'd meet
together in a room and sit in that room
for three four days
and then take the results off to the
next committee
so i learned i guess three basic things
one is
is how to write a grant so that someone
reading it gets the
impact and sense of it in the first page
something is critical
we found some of the people who had very
deep and
narrow fields tended to carry on
about some theoretical issue that very
few people were interested
in and you'd get through 20 pages and
you come out of that saying
sounded right but i still don't
understand the way it fits
so i got that idea that when i read many
applications and people make good points
in the first couple pages i was much
more positively disposed to
reading the rest of the detail and fully
understanding the second was i really
enjoyed working with those people
because they were all top people in
their own field and you got to know
i guess their mental processes how they
thought about things how they approached
things that was very
and the third thing was i got to know a
lot about the research community because
in addition to seeing the proposals we
also did site visits
to talk to people saying how's your
project going are there any
bottlenecks have you write the right
sort of facilities to do this work
i found that useful because you had a
feel for what other people in the
country doing
were your own research fitted and how
essentially to to
tap the source so during the time you're
on the committee of course it's tricky
because
your uh projects have to your proposals
have to be sent
to another committee you probably don't
understand what's happening to avoid
that kind of in-house
inside knowledge aspect so that was a
bit tricky but
fortunately through that process the
projects didn't get lost in the system
and
in 1996 i think
uh after i'd been
yeah so i think i must have started in
the ask giving paid perhaps
four nine five and six or a little bit
around about 96
i was given a special grunt which is
terrific so what had happened is i'd
written a project
the project was regarded as good and so
good that they said
here's some money but you don't have to
do that project which is great
because you're saying hey that's
impressive here's some money do whatever
you like and i thought well i'm not
going to work on that project
i'll keep that one that's a good one
maybe i can use it again
and i use the funding for buying
equipment basically because i thought
i'll never have this opportunity again
so i had that flexibility for three
years
i think it was 96 97 98
again the dates are not clear so i had
three years in which i had complete
freedom over not a huge sum of money but
probably about
quarter million dollars buying things so
i tried to say look let me buy things
that are going to be good for 10 years i
got
lots of piece of equipment that i use
for 10 or 15 years what kind of things
did you get
robotic manipulators vehicles
different sensors range sensors of
various kinds and so on
so i got myself kitted up essentially
and was able to then do a lot of
experiment work from their own so
so that was extremely valuable to have
that freedom
i was curious you mentioned that you got
to see the lay of the land of
what was going on in australian research
what were some of the topics and things
that people were
very interested in oh okay um
because the particular committee i was
on was looking at
computer science uh various
parts of engineering as well as some of
the physical sciences
you've got a quite a wide variety so
some things i remember was a very strong
interest in
one universities in solar cell
manufacturing very
early work on silicon cells and they had
pretty well the most efficient cells in
the world for a while
and i remember feeling isn't it too bad
that the government isn't
exploiting this they're not just early
stuff and in fact i think they sold it
overseas eventually
that's the sort of thing there were
other people doing a pure artificial
intelligence you know this sort of
reasoning thing
uh i don't think the the idea of bayes
networks was there but certainly there
were people talking about more discreet
discrete
maybe extensions of the sort of stuff
you saw in the blocked world but then
lisp was the big language at the time so
it was all about reasoning about
spaces that might have been involved
with making decisions in industry or
selecting
diagnostic stuff of drugs and the like
remember that aspect
there are also people working on
microscale i suppose vlsi
circuitry at that time that was big
everyone was walking around
had been already walking around with the
textbook carver
carver me textbook was the bible for
vlsi designs a number of different
projects
building up in australia and that was
expensive exercise because unless you
were
getting your chips manufactured
somewhere else you had to have clean
room facilities and it took a lot of
expense i remember that being one of the
things i was quite uh impressed with at
the time
and i suppose there are also uh
let me think yeah i i guess the solar
work and the vlsi work
apart from robotics are the ones that
really impressed me there's not wasn't a
huge amount of robotics happening in
australia in those days
what kind of things were people doing in
robotics
um the best known research project in
robotics in those days was a sheep
sharing project
run by a friend of mine james trevelyan
t-r-e-v-y-l-a-n
and in fact rather annoyingly for about
10 years whenever i was at an overseas
conference
and that someone picked up my australian
accent they say
are you james trevelyan
and you did it
so james was doing this sheep sharing
project
and whilst this may sound very
agricultural it turns out
he was a master mechanical design in
fact you can ask any people like
osama khatib and others and people know
james as a very elegant designer of
mechanisms
and so some of the things he designed
for the shearing head on the robot
arm were exquisite and still have
a fair huge respect for that work
uh and he wrote a book about this called
sheer magic
so so that was what people understood to
be
the robotics impact uh in australia
about that time that there was
so my group start started up at monash
there was a little bit uh going on
with i guess a little later on anyway
peter corke was in csiro in queensland
so that group is growing
a little bit of work happening in
new south university so but they're
really only about five groups you could
identify each with three or four people
and then doing any robotics work
and then the big impact was uh when huda
and white
you know if darren white took up his
chair in sydney university and
established the australian
field a fee field research center
that was momentous because
hugh came with a lot of very strong
ideas
very very strong industrial links to
people
doing port automation mining and
built up his group to 40 or 50 60 70
people in the end
and absorbed a huge amount of funding
from
both governments and from
industry in fact in a way other robotics
uh
groups even mine really didn't get a lot
of looking because he was
so powerful in attracting the funding
and
legitimately competing so you and i were
hopefully going to set up a
field research center together but our
first attempt to fail
because most of the money was going to
big science
uh our projects were not monolithic
enough right so if you're going to spend
10 million dollars
and you could buy one instrument for
people doing
uh gravity waves or something then you
couldn't say well i could use it in
small pieces because then those other
people would never get a look in
so you and i were interested in writing
that up and we failed
and subsequently when the call went out
for special research centers
the money was relatively small and i
rang you and said should we go together
and he said no the money is too small
and we split
and what happened was that his group
got the centre for autonomous systems
and we got a second ranking
support for a center at half the funding
so we were called to sort of half
center but we did different sorts of
things so here
center was big scale stuff mining
ported automation and eventually into
underwater
and aerial on the big scale
our stuff tended to be gadgets small
sensors of various kinds
my colleague was interested in touch
sensing and thermal sensing others were
in
ultrasonics and i was working in vision
so the things we did were a really small
scale and he was doing the big scale
stuff in the meanwhile peter cook was
starting to do big scale stuff as well
so i think peter and you worked together
for a while
so the landscape changed quite a lot to
to go towards the group that hugh set up
there's still some work happening in wa
and summer
university new south wales and our
monash group
had a reputation but in sort of limited
size of moderation so so
hugh was keen to use the phrase that
the australian academic research
committee was
punching beyond its weight you know the
term
so it means where you know featherweight
has moved up the scale so it was true
because if he took the population
and the small number uh we were making
quite a good impact you know we'd find
lots of australians at international
conferences
much higher than the proportionality
would suggest
but nevertheless we never got very
strong recognition
robotics has never been you know a
national
national project in australia
they've been funding for various
particular things but no one's ever come
up and saying hey this is where we
really need to have
certainly there has been a strong
emphasis on alternative
energy sources on things like uh
medical issues and
mining and all this kind of stuff but
not specifically
robotics so given that we were all
scrabbling for our funding
from a bigger pool having to
compete against a bigger pool of
relatively unrelated
engineering and science we're doing
reasoning
so what kind of research did you end up
doing with here you mentioned you had
kitted yourself
um up
uh um and that was it monash yeah
already there
yeah that's right we already there um
so i started
well because i i was able to buy bigger
pieces of equipment i started to get
interested in
a lot of outdoor robotics um
so two of the big things i bought
both related to my visit to a
laboratory in finland i can't begin
the exact date what had happened was
that a
colleague of mine professor ann halm
who was running quite a big laboratory
in the university of helsinki
had a conference that i helped with by
reading some papers and became good
friends and
he introduced me to some russian
engineers
who had been part of
the russian space effort
they had a large laboratory just outside
of saint petersburg
on this first visit anyway to uh on home
there was a swedish
no no there was a finnish company that
built a big tractor-like
vehicle that was incredibly powerful
and had a very low impact on the soil
and they were presented as a
potential vehicle for robotic
agriculture because it didn't dig up the
earth
and they were able to show that this
thing could be driven with high
precision and did lots of nice things
and i got interested in that machine
finished up having one bought
at slightly reduced price around
fifty thousand dollars of their perhaps
and also i have visited
the um laboratory in saint petersburg it
was just a few hundred miles
train trip between helsinki and
petersburg
and uh spoke with these uh russian
engineers
and my wife who is half russian could
speak russian with them she thought it
was wonderful so we did a deal
over the making of a small scale
martian robot causing
so i
went there i did a deal with these
engineers and we finished up
drinking vodka and eating sausages about
10 o'clock at night in
the european uh
white nights area it was light until 11
or something
and my wife had a wonderful time talking
religion russian
it's always a lovely sort of feeling and
then subsequently this
robot was delivered without
the motorism to helsinki
where my colleague arn hall fitted it
out with
maxon motors which was seen as the best
of the time
and he then brought it as excess baggage
to a conference he was attending
straight at my invitation so in not that
big was this big
so this thing was something
that was just fabulous because it had a
articulated body that was made of
aluminium
and had slightly conic
wheels that had special serrated
sort of fins on it and this was made out
of titanium
titanium cannot be welded so every
connection was a rivet man-made river
and and i was told that the russians
were
really good at mechanical design but it
was better to get the instrumentation
than elsewhere so the combination was
russian equipment and and
finish setting setting up the motors and
the light
so that arrived and then i did a lot of
work on that machine
in a rough environment this thing could
climb objects
as big as its own wheels just go over
the top because it was like being able
to articulate over the body
and i don't know if he spoke to
raja chatilla but he had full scale one
of these for many years in a project
they had
called eden and he will tell you how
wonderful this design was he had a more
sophisticated one that could extend
the wheels as well but i had a lot of
fun with that over a number of years so
that so the
the big track machine and and the
martian
rover with two of my key experiments and
a lot of sensory stuff and then trying
to work out how to navigate how to
climb things how to build models the
environment
and i extended some ideas i had on path
planning
originally in about 1984 and used it
over and over again in different
circumstances america and talk about
that
this methodology is based on an idea
called distance
transforms distance transform started
out
as a way of analyzing binary images
and the two key people were
asleep rosenfeld and john phelps i think
with the university of maryland
identified the time but this became
famous
stuff on how to analyze binary images
and their interest was if you're looking
into a microscope
and you could see the outline of the
cell how would you
make some measurements about the shape
of that cell and its volume and so on
one of the ideas was to
use what's called a grass fire gonna
bring light
fire around the proof or you let it grow
inwards and you mark
the path of this trajectory towards the
inside that was called a distance
transform because
the outer layer was zero the next layer
would call one
two three and you got into the center
and you could do a lot of computation
on those numbers to work out things like
the area
uh the the uh amount of
the the ratio of the perimeter to the
area and these sorts of things
and general shape and you could actually
take the skeleton so
a kind of compressed version of this and
recreate the original song
so my idea was to turn the algorithm
inside out and say
how do i get this same type of
propagation arrow to work on the outside
of things which i call obstacles
so my breakthrough was turning that
algorithm inside out
and having something that propagated
distance in such a way that flowed
around
obstacles and it gave you a very very
simple way of
global path planning because after you
sort of
had a start point and grew distance out
from that all around the obstacles you
now had all this free space all the
places the robot
could go with a number on it saying
precisely how many steps to the gulf
and if you went down the quickest way
you will guarantee a global solution
so that was mighty in 1984 and i found
you could extend it to any dimension so
i actually had a three-dimensional in
fact
one of the very i think the first isr i
came and presented the three-dimensional
version of that
uh and this algorithm has been
reinvented
several times uh once or twice even in
my presence you know i've actually been
in a room where someone said i've
discovered this and
fortunately some you know gallant
uh colleague has gone up and said hey it
was done 10 years ago
so this algorithm turns out that you can
extend it so
one of the extensions i did was with a
student of mine about five years ago
where i said
instead of finding paths simply from
a to b what if you had to go from a to b
but be unseen
from most positions most of the time so
what i call covert paths
so had applications in in security how
to get from a to b
to be the least detectable and then
how to go to a for b if you knew uh
a century looking for you has entered
that door and it's going to move
probabilistically through the space
where should you go
to find a hiding place where should you
go if you need to escape from hiding
place
and then that extended onto you are in a
disaster zone
where should you look for victims of a
earthquake and again how do you find
paths that are weighted by the
expectation of certain things so it
turns out that these paths can
incorporate all sorts of other factors
about being seen or not seeing
potential for discovering things and it
also worked when you didn't know
everything you could say
for the time being i'll do this and then
i'll get a little bit more and i can do
that
shift a little bit so it's fast enough
to keep on applying so this idea
um i use for quite a few of my projects
and the simplicity i guess i found
appealing i didn't really want to get
into
very very complicated methods this
suited what i wanted so even my after
work
this distance transform was one of the
key
underpinning path planners i was using
and i had some students working on
rather related projects
on this so i'm still interested in
the visual side but
for the outdoor projects more in laser
range findings and cameras
and the combination of how to build maps
to plan how to move that was before this
idea of slam
had got any strength
the slam idea
grew out of the need to go and explore
new areas
and how to develop maps of new areas
and the only way you could do that
effectively before that point was to get
out of the other lights
do a survey using standard equipment
moving very accurately from point to
point and taking
readings of angles back to edges of
buildings and objects and
slowly but meticulously putting that
together and even
in early surveying this idea of closing
the loop
existed so i remember i mentioned doing
surveying and stuff in my undergrad
i remember there being a technique by
which you
started somewhere you started to measure
angles and distances
and when you went further and further
around the tendency would be small
accumulations of error
with each dependent on what you thought
was your current location that depended
on your previous measurements
and the idea in surveying had always
been that when you came back to where
you started you'd say
this is meant to be zero zero but all my
things are saying minus five plus ten
so you knew you were an error and then
there was quite simple ways of
distributing the error to make the
corrections
all the way along now the current
slam is really that technique but in a
much more sophisticated level
and probabilistic the use of common
filters and extended
it's all about how to distribute and
propagate the area to minimize the cliff
but you still have to complete the
closure
because i mean a lot of controversy is
how do you absolutely know for certain
you're back where you started
you make that mistake everything blows
up a lot of work in slams about that
now by the time i sort of realized how
important this area was there are
hundreds of people already doing it so i
decided not to do it
so i went in the opposite direction so
um
i started doing what you could call the
opposite of slam so
i bought a very uh with a colleague we
bought a very expensive laser
finder which you could put in the middle
of a city square
and over a period of two hours collect
every detail up to 800 meters away
right so you can just the whole volume
high precision
fractions of maybe half a centimeter
and with a high resolution camera get it
all painted as well
so instead of building your maps
incrementally i said well
you just put this down you collect
everything at once and now you now have
a
sort of cyber model of that whole space
and if you could not see everything from
one
position you could pick three or four
positions and then you can with a
can automatically but with a little bit
of hand help you can get the stitch
together of all these things so we did
this over a number of spaces around the
campus and also a
sort of a property i finished up buying
out of
out of melbourne to do some of this big
field work
and so i sort of argued that in some
circumstances where you have either
access to
existing plans or buildings or you're
prepared to spend half a day collecting
this information
if you're going to have a robot working
there for 10 years the cost of
collecting this
is of no significance so why do i need
this complexity of a moving robot that's
doing all this high
uh calculations of uh common filters i
just said if you were prepared to do
this once and do it
accurately there was no need for that so
so i'm sort of saying there was
room for slam you could do it this way
if you were being practical about
an existing relatively organized space
and then the notion was okay you've got
this very
exact map of things
could you now put a robot in that could
find its way easily
around this and one of my students
worked on the idea
you have a camera pointing up at a
panoramic mirror
and you could see right around all in
one hit
you could collect that image and then
you could hypothesize
where should i be to recreate that image
from
the cyber model right so what you're
really doing is saying i've got
an idea what's there where must i be to
get that same so that was not so you
could actually say i'm collecting this
live data
and i could do a search using particle
filters to work out where must i be in
the cyber space
for this same data to be visible and
that's where you are and that worked
pretty well
so you could just move this robot around
with its mirror and camera
and it could almost real time tell you
where it was
and plot it fairly accurately and it
wasn't necessary in fact to have a
robot you could actually put it on a
walking frame so this student actually
bolted on a walking ground and walked
around like this and you could see a map
of where he'd been and so on i thought
this is good so later on i got a second
laser rangefinder
called velodyne which is about the size
of a human head
and can be bolted on the top of the
vehicle
and that wasn't as accurate as the first
one which is called the regal if you
want any
details a lot of people bought these
things so what that could do is collect
range data very very quickly up to 120
meters away
so it could collect one and a half
million samples per second
so that gave the opportunity knowing
what's around you uh
quite accurately you know 10 times a
second or something you could spin it at
about 10 hertz
so it meant that you had live data from
your vehicle
and it covered from about plus two
degrees to minus 24 so it actually
covered
the volume that the vehicle may be
running through because you're not
worried about overhanging branches above
that north things of course under the
ground
so it allowed you to not only avoid
things
but we use that range data to now
hypothesize
where must i be in the cyber knowledge
to get this set of range measures
and you didn't really have to use the 3d
you just use a slice
so you've got the nice way of saying i
can avoid obstacles but i can also
and we found that in an area of about
150 meters by 150 meters you could find
out where you were to 13 centimeters
in rails i'll make 10 times a second so
i'm still working on that because i
think well okay if i have
a vehicle in the bushland setting and
i'm prepared to
say nothing much is changing of course
one of the problems are the trees grow
so but if you use your scan about where
the trunks are
coming out of the brush that's not going
to change that much as the growth above
that point so we found out if you take
something about a meter above the ground
that's reasonably stable and i've been
back for the same site
six months apart and still work so this
sort of thing
so i'm still keen to use this
combination of pre-scan data
and this fast scanner for localizing
robots so i'm
going back out to this property to do
this sort of work
i guess that's probably a good story too
i bought this property when i knew that
i couldn't do this work on campus so
there weren't enough
playing fields and stuff and then
students always come and gawk and then
it's dangerous because this is a big
robot they could run over their feet so
and there are all sorts of risk issues
so i got a bit frustrated so i finished
up buying a 20 acre property
after about three hours drive from
melbourne it happened
near some sort of foothills of some
reason looking mountain
and a water lake nearby and it's a
lovely spot so
um so we used it partly for mostly flood
work but
also enjoyed going out there and
so the big work that i finished up doing
with the fire
engines and all sorts of things were
done out there so what happened is that
i went to the country fire authority
and we had a project with them for five
years so the big stuff was done out of
this property
so in fact even today i have three large
fire trucks out there
so you're not in danger of forests no
the trouble is i'm not sure that
that there's even petrol in the tanks
yeah
they could be used for firefighting well
tell me you make up two of them
i'm just going to go check if rashida's
up yeah i've probably done my time if
not
yeah but if if you want to think while i
go check on him if
there are a couple of other things you
wanted or you want to be in there and
i think one thing we didn't discuss very
much is who your collaborators have been
over
okay oh okay yeah some collaborators and
um
i'm just gonna i mean if you guys want
to continue yeah i'll go find you
you know maybe some of the um um some
folks in the last maybe 10
because i think we got quite a bit early
on yeah maybe about the last 15. well
well
um i had uh
from 2003 to 2007 set up
this center that i told you uh
just slipped off the first rank when you
don't write got his center
that was called a center for
perceptive and intelligent machines
and i had partners in
three other institutions
the partners in melbourne were a group
of people doing
database analysis with artificial
intelligence methodology
a group in perth who were doing pattern
recognition
and computer vision and
a particular individual working at
australian national university
who had spent 20 years studying vision
and insects particularly bumblebees
he was very well known for that work and
he'd come up with some wonderful ideas
about
optical flow which is the process by
which you can work out
a three-dimensional structure by working
out how things move
as you move through the environment and
he picked out some
analogies from insect vision to do this
work
uh his name srinivasan is sort of a
world leader in this field he's since
moved
from anu but anyway it was his mirrors
in fact we finished up musings amongst
other things he designed wonderful
optics and they were the people that we
collaborated with but
the collaboration was very loose it was
very hard to bring
those four different interests together
on focus
in fact that was pretty well the
downfall of our center because the
uh the finances were subdivided
too early in my opinion but that was the
deal that i had to do
and it was very hard to jog people away
from their current
research interests they were
individually doing extremely well
publishing well getting recognition so
it probably was not
sufficient incentive to be more
centrally involved so
my job was to actually try and stir up
more and more
collaboration i i guess in the end i
have to say i failed
i couldn't get this to work so they
they're my
should be collaborators rather than my
actor but most of my work i've done
just with my students and to some extent
with my immediate colleagues i haven't
in fact done
a great deal of collaboration sort of on
the ground
except in very sort of ethereal terms of
saying
talking to people at conferences and
swapping a few things so
i found because my work was highly
experimental
it wasn't too good just swapping a
theoretical paper i actually needed
people who had know how to build things
and
right there on your particular device so
i didn't collaborate a great deal
so um in terms of well one thing that
selma
has been asking everybody is um and
things are gonna do a piece on education
too uh what are some
insights or some advice you might give
your students
maybe high school age yeah um
i guess to start with we found that
there were many
students in undergraduates who developed
a strong interest in robotics
partly because of a few demonstrations
we could do
but also because typically computer
systems people spend a lot of time
sitting in front of a computer and i
think the prospect of
actually seeing something move was a
kind of a refreshing thing
so i had no lack of interest and
for a while i was teaching courses in
pattern recognition
computer vision tumblr theory
path planning and they usually
specialized group of students
really interested in this i was always
excited about students who come up with
new ideas in small classes
so i'm very lucky to have small really
interested classes made the job much
easier
now some of our students who
had an interest in great work would come
to us but
i think our group attracted more foreign
students than locals in the longer term
a lot of students come from
some malaysian students some indian
students iranian students
and some locals as well
so i think the second thing to say is is
we told all our students um at least i
told all my students
uh two things to remember one is
don't read any literature until you
started to think about your project
that was controversial because everyone
said hey you've got to do it i said
don't do it.
i said the three things can happen
one you'll find your ideas have been
already done i put you off
the others you find your ideas have not
been done but for good reason
and the third thing may be that uh you
feel that you've lost your
ownership of something i said actually
it's better to think hard have initiated
about a problem and then when you've
written down some ideas then you're
going to look at the literature
check it out and that way you'll get
either affirmation or you'll be diverted
so the literature i said was actually
corrosive it would spoil your
your creativity and that was
controversial but those students that
did that actually i think got a lot out
of that
yeah i could see that because the other
side is even just
looking at a problem a certain way and
once you start seeing
yeah it could be just slightly different
you may have found something new
and that newness is kind of gone because
you've been distracted the second thing
i think
i've told all my students is that in
this center
we're only interested in theories that
can be proven
with physical experiment you tell them
right from the beginning
have good theories but we wanted to see
the thing move correctly
and that was always the the kind of
motto of our group in fact all my
colleagues
believed in that too so that was good
because it meant that
if you had someone that was just strong
in computer science but had done very
little
hobby type things they got told well you
better find out how to make something go
and talk to the
people in the machine shop get to make
your things know how to put the wheels
on and stuff like that so
our students generally were able to uh
were good at maintenance of their own
projects
all right so that that was the second
thing i guess the third thing was
uh once you've got this idea and you've
looked at the literature
uh it sort of aimed pretty high
right there's no use just incrementing
an idea by
epsilon aim for something exciting and
even if you don't quite get there well
you'll get a better thesis thing
okay yeah sounds great we really
appreciate it