yourself tell us your name
where you were born grew up and went to
school okay
so my name is jana koscietzka and i was
born
in slovak republic or here in former
czechoslovakia
i finished my equivalent of master's
degree there
so it was the engineering degree in
electrical engineering and computer
science
and then i came for graduate school to
united states what university is taking
i went to graduate school at university
of pennsylvania
in grass club that was the lab i was
affiliated with
and where did you do your undergraduate
masters slovak technical university
in bratislava which is the
capital of slovak republic
and why did you decide to come to the
university of pennsylvania
i was applying for graduate school for
so i applied to several universities
and i got offered a scholarship from
university of pennsylvania so that was
partially a deciding factor but
you know at that time it was sort of
fairly big laboratory so it wasn't sort
of a nice group of people and
so there was clear
possibility to have a lot of
opportunities to work on various things
so
and what did you study as an
undergraduate it was electrical
engineering and computer science
so it was kind of a mixed degree at that
time
the specialization in computer science
happened only
in the last two three years so it was
not just the core computer science but
we had also some basic electrical
engineering background
a lot of lectures and about
antennas and microelectronics and things
like that
and at that point were you interested in
robotics
no i was i was interested in artificial
intelligence in general so
that was the area in which i did my
final thesis
master's thesis and then i
at the time when i was deciding to apply
for graduate school
i worked as a research associate at the
university
so i had a research position and i was
working on
some of these more traditional ai
systems expert systems
for some diagnosis problems and
i went to a conference we had a paper
and
so that was the first time i saw a talk
on robotics and that sort of got me
interested in the field and
and then subsequently in the graduate
school and what was your master's thesis
master's thesis was
so it was not quite let me sort of
remember
i don't remember exactly the title but
the project which i was working on
was some sort of diagnosis of disease in
sort of some agricultural
application
i don't remember exactly like they were
trying to diagnose some
chicken having certain diseases
sort of gathering information about
certain symptoms so
sort of moderately exciting
but you know but this was many many
years ago so at that time
you know all these fields looked a
little bit different
oh well this was i mean
and then you so you decided to go from
the chickens to the robots for your phd
yeah i you know so when i came to
graduate school it was not
entirely clear to me what i will work on
but
kind of the real world aspects
of the robotic systems were very
interesting to me
and who did you work with so my advisor
was
professor baichi and
and she was the person whose talk i
heard so she was a visiting speaker at
the symposium
in slovakia when i sort of had my first
exposure to some of these robotics
problems
and was there any other robotics going
on in slovakia and not that the
institution i was at but there were some
people working on controls but
more there were other departments so our
computer science department didn't have
any researchers working in robotics
and so you went into the computer
science department and i went to
computer science department it was
called computer and information science
department
but the laboratory was a very
multi-disciplinary
laboratory so there was
faculty affiliated from electrical
engineering and mechanical engineering
so i also felt it was a good fit because
partially my background was in
electrical engineering
and so i was sort of interested in some
of these modeling
aspects of systems which sort of the
classical computer science or the
expert system ai parts of things didn't
really
offer those opportunities and who else
was
at the grasp lab oh lots of
wonderful people
so i at the time when i arrived
i i guess probably the best mentioned in
the connection with people who remained
in academia
and so greg hager who
was a postdoctoral fellow at that time
was also at the laboratory
and now he's a faculty at johns hopkins
another i think by now famous roboticist
who was
sort of my contemporary or he was
visiting there or he was doing his
masters there was
how he chose it he is at cmu
this is a lot of people from my
generation
ended up going to industry so
um i'm trying to think um
my era is also sort of at the boundary
of computer vision and robotics because
i work
in robot perception so a lot of people
who were my colleagues from graduates
school worked on computer vision field
so
i don't know robert mandelma mandelbaum
robbie's i think a program manager now
in one of the
i think lockheed mart martin i mean not
a program manager but he's a
head of a division and was previously
program manager
one of the one of the darkest programs
so
these are sort of quickly few people
like
come to my mind and what was your
phd project so i
worked on
so this was at the time where
i think the robotics was sort of
undergoing some revolution because
this was at the time when rodney brooks
came came up with his first
behavior-based robotic systems
and um they
sort of generated a lot of excitement
and also a lot of controversy because
they were sort of very much in the
contrary to the popular belief at that
time how a robotic system should be
designed
so um we with the professor baichi were
sort of very much motivated it's sort of
trying to understand a little bit
the underlying theoretical principles
of of these emergent systems which
generate these intelligent behaviors
so i was working on
the design of kind of a formal framework
where one could kind of specify robotic
behaviors
and then combine them in
various ways in order to generate
certain
sort of tasks and also be able to prove
some properties that this task will be
accomplished in a safe non-conflicting
manner and the goals will be achieved
so the field was uh i mean
the work was the inception of at that
time of
this emerging field of hybrid systems
where
we were basically trying to model the
behaviors using
automata theory but the behaviors within
the individual
states of the automata were modeled
using the differential equations and we
were trying to come up with some
analysis tools in order to be able to
predict the behaviors or proof some
properties about
these systems
what were the big challenges
well so the big challenges i think were
both theoretical and experimental you
know at that time
everything was incredibly slow and the
domain
in which we were trying to apply these
techniques was
visual navigation and tracking and
landmark-based recognition and so forth
and so being able to do experiments
robustly and reliably
with i guess with the state-of-the-art
cameras where
at that time there were no digital
cameras we have these like huge
digitizer boards and everything was slow
and robots were sort of not so readily
available so we had sort of various
mobile platforms where we were doing
sort of these experiments so there was a
challenge to to demonstrate that this
actually works robustly
but also i think theoretically it was
sort of not obvious how to do it so it
took some time
to kind of choose the right set of
problems where we can actually design
the design the techniques and analyze
the behaviors so
what sort of robots were you using oh we
had this um
we just had this omnidirectional a
couple of them were omnidirectional
mobile platforms
where we just mounted various sensors on
it and
um and then couple of them were this
differential drive robots so just mobile
robots
with sensors
mostly cameras and structured light
sensors
and when did you finish i finished
in 1996
and so then i went to
i got a position a postdoctoral position
at university of california in berkeley
and so after that i
initially went there to start working on
an
autonomous driving project because that
had sort of
some of those characteristics where
we had kind of visual sensing and some
control components
so we worked on this was
kind of a first federally funded project
for sort of automating the highways
and there were lots of lots of teams
involved and it was a
kind of a great fun project because
we got to driving these autonomous cars
and there was like a big show at the end
and demonstrating the feasibility of
some of these concepts
and and it was also kind of fun team
project
and it had both some engineering
challenges but also some
kind of technical challenges in terms of
how to design the techniques and
algorithms
and then i staged for two more years
so this was just the first year of the
postdoc
and our team kind of entered the game
very fast and we got the car from honda
with the two honda engineers so we
managed to
sort of kind of very quickly get up to
speed and have basically a working
system within the year of
of car which can change the lanes and
keep a safe distance from the car in
front of you
just using visual sensing so we had sort
of cameras mounted behind the rear view
mirror and
um and we also got to
learn how to have some sort of emergency
driving maneuvers for safety reasons
which was kind of fun because we had to
go for this training for
for race car drivers anyway
so there was like an extra perk which
which was part of the project
um and then i stayed on and ended up
working a little bit more
on the perception aspects on visual
sensing
uh as applicable to broader class of
problems not just
these problems where vision is in the
control loop
of of of a system of a robotic system
with some of the people you collaborated
with so
at the driving project i uh
so this um that first year the two
post-doctoral advisors were
professor jatendra malik who was a
computer vision faculty
and a professor who was a control
faculty and i worked with
cj taylor who is now a faculty at
university of pennsylvania
and phil
mclaughlin who was a former graduate
from oxford
but then he went on and he
sort of ended up working in some special
effects
company for for movie industry and
advertisement so
so these were the two other people on
the project
what were the perception projects so
then uh
at the time we
so you know i was as a postdoc i was
sort of
halfway in the computer vision group and
halfway in a control group
and being in the control group there
were a lot of people who were interested
in vision problems
and so we started revisiting some of
these
i guess at that time seemingly
well-defined problems
of kind of computing motion and
trajectories
from image sequences so this is this
well-known structure from motion problem
and then subsequently the 3d
reconstruction problem
so myself
professor sastry and another graduate
student
started looking at these techniques and
then we came up with some new insights
which
used some tools and techniques commonly
used in control theory
and sort of very formulated some of
these problems in
using some more linear algebraic
techniques as opposed to
projective geometry techniques which
were sort of dominant at that time
in computer vision and managed to come
up with some
new sort of well-performing algorithms
and so
that sort of again generated a lot of
excitement
and sort of led to several other
projects which
other students ended up working on for
say autonomous helicopter landing and
control
but then these techniques were also
applicable to completely general
settings where you can just take say
handheld video camera
and you can compute how the camera moves
and also recover the 3d structure of the
environment
and so we kept on working on
on this for sort of quite a few years
and
that part of the work led to a monograph
so we
wrote a book which sort of ended up also
being a commonly used textbook
in the field for for this class of
geometric problems
um 3d vision
from images to models but
let me i just want to make sure that i'm
not omitting some words
but yeah
um and uh so then what did you after the
birthday so then i was there for three
years so it was a long postdoc
and um so then i started applying
so at the time when i started postdoc i
was not sure
whether i want to be in academia or go
for an industrial
position but the experience was so
wonderful so i decided
that i would like to apply for academic
jobs and that's what i ended up doing
and i was not applying very broadly but
i sort of wanted to
remain at one of the coasts so i had
some geographical constraints at that
time
and so i ended up getting a faculty
position
at this george mason university were
there anybody else working on
robotics or computer vision so there
were
no faculty working in robotics but there
were a couple
other sort of more senior colleagues
working in computer vision
what sort of research were you doing
when you got there
so when i came i
ended up continuing a little bit in
using these techniques which we
developed
which were very general techniques and
trying to understand
how can we make them more robust if we
exploit some constraints of
specific environments where these
techniques are applicable
so we made sort of several variations
and improvements focusing on
deploying these techniques in in manmade
environments where you have a lot of
planar and orthogonal structures
and so i ended up working quite a bit
on the localization problems mapping
problems
structural recognition problems in these
environments
and and then more recently
we also started to be interested in
doing some research in
recognition and segmentation problems so
and were you primarily focusing on the
vision aspects or were you still using
some robotic platforms
so i think i have been
primarily focusing on vision aspects but
i was trying to choose the problems
where
those solutions would be actually
relevant for robot perception
and those choices were made by
making some conscious decisions of what
kind of data we use
how we acquire the sequences so
and then we also did some experiments
with mobile platforms but
um in that first stage or
right now we are sort of investing a
little bit more in sort of the hardware
infrastructure but we were mostly
focusing on
kind of working with videos and images
acquired by robot platforms but have not
done
as much in the control aspects as
before any
anything with regard to the interaction
and its
relation to perception so using the
movement of the robot
yeah so i found it a little bit
difficult to work on the control aspects
because now i
my position was at the computer science
department as opposed to before i was at
the
computer science electrical engineering
department so it was very difficult to
find students who have the right kind of
background to work on it
while before i had sort of a little bit
better background from my undergraduate
and also graduate studies being in the
lab
so this was also partially related to
the decision of focusing on the problems
which did not require such a heavy
control component
partly because the students who i was
working on
didn't have the right kind of background
and our electrical engineering
department
also didn't have sort of the right type
of collaborators to really fully pursue
this
but i think this is changing so i feel
that i'm kind of moving a little bit
more
closer to looking at some control
problems and
i guess being better equipped also so we
know now
how to do the perception side a little
bit better so
so hopefully we can close the loop
so i saw that you're a recipient of the
david maher
and do you have a perspective on the
ecological perception versus this kind
of
individual reconstruction well so the
the
award the work which we got the word
first was very geometric
okay so um
so i guess in in the mars philosophy of
how computer vision is structured into
sort of levels and sub problems it
really focused mostly on
issues of how to recover the 3d
information
i think everything is important i think
vision is a very complicated
sense and we use a lot of different cues
how to interpret scenes and
and we are incredibly good how to
combine these different cues
and priors on the tasks and environments
of how to
solve these really highly ambiguous
problems and i think we are still
quite far from being able to develop
systems which replicate some of that
functionality
so i think both aspects are important
the ecological
more definitely so and who are some of
the people that you've
collaborated with
i collaborated quite a bit with
my former colleagues during from the
postdoc times
um i had some collaborations
with on the vision side also so this was
mostly
uh yima and professor sastry
and stefano soato who were my co-authors
in the book i had some collaborations
with hanni farad from darth mode he
ended up more sort of working on purely
vision and image processing aspects
um and then
majority of the work i sort of ended up
doing with my students afterwards
and who are some of the students that
you worked with phds or postdocs
right so i had
i guess a couple phds so
i had wei zhang who was a phd student
both of them ended up going to industry
and he worked on some of the
kind of man-made structure recognition
problems i had another student who
we worked on some of the object
recognition problems
i had several wonderful postdoctoral
fellows so i had that postdoc from
slovak or czech republic who worked with
me for several years
his name was brano mitsushik i mean he's
you know
one of my cultures so it's easy to trace
down
he ended up going to kind of a research
institute in austria
i had another postdoc who worked
with me recently on the semantic
segmentation problems this is the cesar
cadena
and he moved on to be a research
associate in australia
in one of these national labs and
universities
i had also sort of quite a few visitors
which came
for a shorter period of time so
anna murillo was a great collaborator
um we work on a lot of the localization
problems together
so these were sort of my non-student
collaborators
who i've spent extensive time together
i also did a very long sabbatical
so i had some successful collaborations
during that time
so i had after my tenure
i got a sort of visiting
researcher position at google or
visiting faculty and so i worked
on the street view project at the time
where there was no street view product
yet so
i was sort of
involved and helping out a little bit to
get some of the
initial prototypes up and running
because
that project is sort of very directly
related to the expertise of
sort of taking sequences of images and
registering things
and computing the trajectories of
vehicles
and so i worked on the project for a
year
and then i had some
sort of consulting position at nokia
research lab
where we were also sort of interested
in these problems of
how to use all this data which is
collected
by companies which build maps so
nokia has their own map building
subsidiary which is navteq and
we're sort of interested in how to kind
of develop
tools and techniques which can help
better
um which can help pedestrian and
pedestrian zones navigate better
and sort of give directions maybe to
visually impaired
so there is sort of the central theme of
sort of understanding
these man-made structures environments
and how to localize and recognize
and recover 3d structure of them which
which was in a way common to to many of
these projects
and then the the rest of the time i was
spending as a visiting
faculty at stanford so i was teaching a
class and working with my postdoc
and visitors i took one of the students
with me
were there any other people you
collaborated with so
yes i collaborated with uh
christian theobald who was a sort of
vision slash graphic
person he was sort of interested
or we were sort of collectively
interested in motion capture
so this was kind of a setup where
you have multiple cameras
where you can record both the 3d
depth and video and were interested in
recovering 3d motion
using some of these 3d reconstruction
techniques we had some experience with
and sebastian thran was also partially
involved in that project
although at that time he was also
partially on leaf
but christian was sort of his visitor or
collaborator so we had some common
students and interacted a little bit
and then
i had some collaborators in nokia so
we have written couple papers with
radek very hard to pronounce last name
grasschook so
and some of some of his team
teammates so over the course of your
career where have you typically gotten
funding
for your research projects so i am
i was very fortunate to receive
some nsf funding early in my career so i
got sort of a couple nsf grants i got
the career award
which which was sort of wonderful start
and
i got another grant on sort of this
visually guided control and sensing
i have
so that was sort of the pre uh
tenure funding sources and i had a
couple small
smaller grants from the university and
then i since i came back
we continued some of the work
on this sort of semantics in
understanding which was supported by
army research office aro
and then some of the vision work
has been supported by irpa
national geospatial agency nga
also supported some of this work
so these are sort of the main sponsors
sort of say
and what's been uh your view of the
relationship between the robotics
community and
the computer vision um so i feel
they have been for
for quite a few years for many years
kind of quite divided and
and i always sort of tried to kept
one of my feet in each of the
communities and
but i think they're starting to sort of
re-engage a little bit
because there have been lots of lots of
advancements in computer vision
which i think the robotics
problems and systems and community can
really benefit from
and you know that creates both
challenges and opportunities
because the settings in robotics are a
little bit different usually
uh so that's i guess an opportunity
because certain things could be done
better more reliably
but then there are also challenges
because
we have time constraints and one needs
to sort of choose the right
task and and then also the challenges
are the
the type of data
you encounter in robotic settings is
often
not the the type of data you encounter
when you analyze
general photographs which are taken by
photographers from canonical viewpoint
so
there are a lot of biases introduced
just
by the way how people take photographs
which are the main source of data
computer vision people analyze
which do not exist in robotics i think
and so how to kind of
close the divide and sort of develop
systems which
which sort of perform well and are
robust
but you're seeing a deeper engagement
these days i see
because i think people in robotics are
first of all excited
about the advancements that a lot of
things
are starting to work really well also i
think the technological advances
in which are kind of facilitated by
these gpus and
being able to implement things much
faster
are also very exciting because the
techniques which could not be conceived
previously to be run in a robotic
setting can suddenly run in real time so
so i think both the sort of level of
problems but also at the level of
technology
i think there is a lot of communication
and i also
it's sort of emerging also at the level
of
sort of people trying to sort of visit
their respective conferences organize
workshops and have discussions of
what are sort of the common problems
where both communities can engage in
and what are the big challenges and how
problems going forward for robotics
so i think um so the detection and
recognition of objects and human
activities
it's still sort of very challenging and
i think in computer vision you know
people are more interested
in trying to make some headway in say
trying to parse youtube videos or
or you know other um sort of
depositories of videos
in uh in the robotic setting it is
of importance in the context of human
robot interactions and understanding
human intentions so
both the goals are different but also
the environments where these
videos are taken are different and so
there are different challenges
and and the same for object recognition
and detection i think
in robotic setting you have a big
difference because
you have this capability of moving you
can move your platform
change your viewpoints while in computer
vision setting
you typically all you have is that one
image which you're trying to understand
then you really have
no control of your sensing
capabilities so so that's sort of
what we are currently excited about
so can you tell me a bit about your work
with the ieee robotics and automation
society and how you became involved
so i have been mostly involved sort of
at the level of
kind of reviewer conference organizer
workshop organizer
and you know program committee member at
various levels uh i'm
also the chair of the
sort of the computer vision uh robot
perception
sort of area the robotics and automation
societies
the technical committee for this area so
these are mostly my levels of
involvement so we are sort of trying to
i guess promote the field and
also keep people informed so what's new
and what people are doing
and what are the new exciting and
interesting problems and you're also
part of this historic
uh all women organizers yes
yes so i guess
i am partly because yeah i've been in a
field for a while as you can probably
easily
count and yeah so i think it's really
exciting
that we have sort of sufficiently
large body of women who are sort of
strong and
motivated and
so do you think this is going to help
increase the participation of women in
robotics and what are the challenges of
doing that
this is i think a very tough question i
think
you know my personal view of this is
that
this is clearly related to a much
broader problem of
sort of engaging not just females
but students in general in these stem
related fields and
i you know having my children in school
now i
i feel that the problems
should be addressed actually much
earlier than
at the level which they're being
addressed
i think you know high school level
middle school level
i think it would be great if students
can get more exposed to this
to these fields and also you know i
think there are a lot of opportunities
to kind of change
the way how how this science and
mathematics is taught at sort of this
introductory levels
how would that change more i don't know
i think you know i had
i came from a very different educational
background and
that background was very effective in
preparing me well for the career which i
have
and i i think it's it's hard to attain
such a level of preparation and and my
background was in norway special i mean
so i mean everybody could sort of
attain that level of preparation
and i think it's just harder to attain
just the way how the
school systems are structured i mean
there are some sort of
fantastic institutions which devote a
lot of effort to these things but
i think it sort of takes a lot of
involvement of
individuals and parents to follow
through
and and so it's it's i think it's much
easier you may have a lot of
talented children which just fall
through the cracks
because of the system and i think it's
it's difficult
do you think it's a need for more say
rigorous mathematical training or do we
need more of these kind of robot
challenges i think both i think
and this is you know i have a very small
sample
set of observations but there is a huge
emphasis on language arts
and and much less emphasis
on kind of repetition and training and
all these things have to be really done
at a young age
i think it's it's hard to catch up when
you are not young
because you know when the more
complicated
topics come around there's just no time
to think about simple things they just
you have to just it has to just you just
have to have it in your system
so a question we always ask is what's
your advice to young people who are
interested in a career in robotics
um
well i think it's um try to be good at
math
i think it's it's great if you
know how to program and and also sort of
be good in system development but i
think you should just
everybody should just follow their
dreams i think if you're persistent you
can sort of
overcome the obstacles but i think it's
also very important to have
the right kind of guidance you know to
be able to choose the right problems
such that you can kind of learn along
the way
and become more independent and such
that you can formulate your own problems
and lead them to successful solutions
and
and i think they should just seek
mentors
and get involved in projects which are
well defined and
especially at sort of this younger age
i'm saying that because i
you know work with some undergraduate
students who are sort of interested
and um and it's sort of eye-opening i
think any kind of experience of getting
involved in some projects early on
is uh it's extremely valuable for them
did you have some questions
um and have you kept track of the
developments of robotics and
czech republic um well mostly
i have mostly through the sort of the
regular new channels
there have been a very exciting recent
developments
um where
sort of a team or a company of the
slovak engineers
joined some forces with some artists and
they came with this first flying vehicle
i don't know whether it made you know
there was a big uh
i guess there were a lot of news around
it
that i think it happened sometimes in
fall
and i think a nice example of this
is that it was a joint collaboration
between artists and engineers so i think
that
capability of imagination and dreaming
is sort of extremely successful
in you know coming up with new solutions
and thinking of new problems and
that's i think it's very important to
have this kind of well-rounded
education you know as long as you have
you know
so i would not recommend just focusing
super strictly on on some very narrow
area because
i think it's kind of easy to to get lost
in sort of technical details without
sort of being able to see
bigger picture and imagine sort of new
opportunities
because a lot of these very very
successful
researchers are
the way because they're very creative
and they can think of
new problems and possibilities
is there anything that we missed that
you'd like to talk about or any projects
um
nothing i mean i think it's just a
wonderful field i'm really happy
with the field i chose because it's uh
you know i think it's quite like you
will not run out of problems anytime
soon
which is great i think and
and i think now it's also a sort of a
good time because
some progress is being made i mean there
were these periods where there was a lot
of false promises and no deliveries but
i think
um you know it's changing a little bit
partly due to the fact that we've
learned a lot but also
due to these technological advances that
we can do things which we could not do
before and
they're making a difference