Projects
Human
body modeling, automatic model acquisition and motion capture using voxel data
A 3D
voxel reconstruction of the body in each frame is computed from silhouettes
extracted from six cameras. Human body model is described using twists
framework. An extended Kalman filter performs the tracking and with joint angle
limits guarantees physically valid posture estimates. The system also performs
automatic model acquisition using a Bayesian network for estimation of body
part sizes. The system has been evaluated on a number of different motions such
as sitting, jumping, running, dancing, walking, kicking, etc.,
These
are the models the system acquires for five different people. The process is
fully automated. The parameters of the Bayesian network are fixed.
Here are the
movies for a few sample sequences:
|
|
One of the
original camera views |
Voxel
reconstruction |
Model
acquisition and tracking |
|
Stair |
|
|
|
|
Dance |
|
|
|
|
Running/Jumping |
|
|
|
|
Walking |
|
|
|
|
Cartwheel |
|
|
|
AVIARY - Intelligent room project
An intelligent room project using cooperating networks of static
cameras, pan-tilt-zoom cameras and microphones. We are building the system that
is aware of the identities of the people present in the room and of their
activities. The system GUI enables remote participants to efficiently view
events in the room and also provides efficient summarization and replay of
events for later review. I have designed the algorithms for multiple people
tracking, pan/tilt/zoom camera control based on event recognition, head
orientation estimation, GUI for event summarization and replay. Kohsia Huang
works on face recognition and audio processing.
VoW - Vision on Wheels
I am working on the analysis of the
passenger posture for modulation of airbag operation. We are also developing a
system for analysis of the driver and his environment for modulating the
actions taken by the vehicle and the telematics equipment.
ATON - Intelligent transportation project
Moving
object and shadow segmentation. An algorithm for detection of moving objects
and cast shadows. Based on background model estimated from data and on the
model of color change under shadow.
|
raw video |
segmentation
results video |
past:
Video
Surveillance and Monitoring (VSAM)
My
contribution to this project is the algorithm for 3D tracking. The goal is to
extract 3D motion of objects in the scene using measurements from multiple
cameras. For every frame in the video sequence, a set of centroid locations for
segmented objects in each camera is available from the segmentation layer
(written by Erik
Sudderth and Edward Hunter). Tracker provides a list of
objects in world coordinates with their positions and velocities. The algorithm
performs associations between object centroids from different cameras and
association of measurements to tracks. Tracks are maintained with Kalman
filters. Our current implementation of the system (video acquisition,
segmentation, tracking and visualization) runs real time at about 10
frames/second on a PC. Below is an MPEG movie demonstrating the tracking
algorithm. Red crosshairs are centroids of segmented objects produced by
segmentation, and colored crosshairs are projections of track locations back to
image planes. Each object is assigned new track ID which corresponds to a
different crosshair color. In this sequence, each object has two crosshairs,
one for predicted and one for updated position.
MPEG movie showing the tracking
using four cameras
Best view selection using the Omnicam image as interface: Manual
selection and automatic tracking
Omnicamera
and four regular cameras are placed in the room. Omnicamera video is displayed
along with a video from one regular camera (see the figure). In the manual
selection mode, user can select a point in the omnicamera image and a video
from the camera that has the best view of the selected area will be displayed.
In the automatic tracking mode, user can select an object in the omnicamera
image. This object is tracked and the regular cameras are switched so that the
video from the one with the current best view of the tracked object is always
displayed.
|
|
|
Example of the automatic mode at two different time instants. A person in the green jacket is tracked in the omnicamera image and the best view from a regular camera is shown.
Segmentation and tracking of moving
structures in ultrasound images
I
used the snakes to segment the images. Optical flow is used to "push"
the snake in the right direction, resulting in a tracker that can handle very
large frame to frame displacements.
See
the demo page
