Unsupervised Content Based Video Object Segmentation And Tracking

Content-based representation of video sequences for applications such as MPEG-4 and MPEG-7 coding is an area of growing interest in video processing. One of the key steps to content-based representation is segmenting the video into a meaningful set of objects. Existing methods often accomplish this through the use of color, motion, or edge detection. Other approaches combine several features in an effort to improve on single-feature approaches. Recent work proposes the use of object trajectories to improve the segmentation of objects that have been tracked throughout a video clip. This thesis proposes an unsupervised video object segmentation method that introduces a number of improvements to existing work in the area. The initial segmentation utilizes object color and motion variance to more accurately classify image pixels to their best fit region. Histogram-based merging is then employed to reduce over-segmentation of the first frame. During object tracking, segmentation quality measures based on object color and motion contrast are taken. These measures are then used to enhance video objects through selective pixel re-classification. After object enhancement, cumulative histogram-based merging, occlusion handling, and island detection are used to help group regions into meaningful objects. Objective and subjective tests were performed on a set of standard video test sequences which demonstrate improved accuracy and greater success in identifying the real objects in a video clip compared to two reference methods. Greater success and improved accuracy in identifying video objects is first demonstrated by subjectively examining selected frames from the test sequences. After this, objective results are obtained through the use of a set of measures that aim at evaluating the accuracy of object boundaries and temporal stability through the use of color, motion and histograms.

Semantic Video Object Segmentation for Content-Based Multimedia Applications provides a thorough review of state-of-the-art techniques as well as describing several novel ideas and algorithms for semantic object extraction from image sequences. Semantic object extraction is an essential element in content-based multimedia services, such as the newly developed MPEG4 and MPEG7 standards. An interactive system called SIVOG (Smart Interactive Video Object Generation) is presented, which converts user's semantic input into a form that can be conveniently integrated with low-level video processing. Thus, high-level semantic information and low-level video features are integrated seamlessly into a smart segmentation system. A region and temporal adaptive algorithm was further proposed to improve the efficiency of the SIVOG system so that it is feasible to achieve nearly real-time video object segmentation with robust and accurate performances. Also included is an examination of the shape coding problem and the object segmentation problem simultaneously. Semantic Video Object Segmentation for Content-Based Multimedia Applications will be of great interest to research scientists and graduate-level students working in the area of content-based multimedia representation and applications and its related fields.

The 30-volume set, comprising the LNCS books 12346 until 12375, constitutes the refereed proceedings of the 16th European Conference on Computer Vision, ECCV 2020, which was planned to be held in Glasgow, UK, during August 23-28, 2020. The conference was held virtually due to the COVID-19 pandemic. The 1360 revised papers presented in these proceedings were carefully reviewed and selected from a total of 5025 submissions. The papers deal with topics such as computer vision; machine learning; deep neural networks; reinforcement learning; object recognition; image classification; image processing; object detection; semantic segmentation; human pose estimation; 3d reconstruction; stereo vision; computational photography; neural networks; image coding; image reconstruction; object recognition; motion estimation.

Object detection and segmentation are some of the key components of Computer Vision, which have wide ranging real world applications. The current state of the art techniques in computer vision are based on Deep Neural Networks and one of the key challenges is using the state of the art techniques in these fields on novel images, and videos in different environments, and classes. These methods require expensive manual annotations and transfer learning to make them work on domains different from their training data sets. In this thesis, we explore both domain adaptation, and deep learning techniques that don't necessarily rely on the idea of a class, to help with the annotation of private videos. We implemented the initial idea of domain adaptation for directly annotating objects and followed with using video object segmentation (VOS) tracking methods for propagating annotations. Their application to a novel video acquired in the GURU lab is explored as well as ways to improve their performance.

“If you have built castles in the air, your work need not be lost; that is where they should be. Now put the foundations under them. ” - Henry David Thoreau, Walden Although engineering is a study entrenched firmly in belief of pr- matism, I have always believed its impact need not be limited to pr- matism. Pragmatism is not the boundaries that define engineering, just the (sometimes unforgiving) rules by which we sight our goals. This book studies two major problems of content-based video proce- ing for a media-based technology: Video Object Plane (VOP) Extr- tion and Representation, in support of the MPEG-4 and MPEG-7 video standards, respectively. After reviewing relevant image and video p- cessing techniques, we introduce the concept of Voronoi Ordered Spaces for both VOP extraction and representation to integrate shape infor- tion into low-level optimization algorithms and to derive robust shape descriptors, respectively. We implement a video object segmentation system with a novel surface optimization scheme that integrates Voronoi Ordered Spaces with existing techniques to balance visual information against predictions of models of a priori information. With these VOPs, we have explicit forms of video objects that give users the ability to - dress and manipulate video content. We outline a general methodology of robust data representation and comparison through the concept of complex partitioning mapped onto Directed Acyclic Graphs (DAGs).

Performance Evaluation Software: Moving Object Detection and Tracking in Videos introduces a software approach for the real-time evaluation and performance comparison of the methods specializing in moving object detection and/or tracking (D&T) in video processing. Digital video content analysis is an important item for multimedia content-based indexing (MCBI), content-based video retrieval (CBVR) and visual surveillance systems. There are some frequently-used generic algorithms for video object D&T in the literature, such as Background Subtraction (BS), Continuously Adaptive Mean-shift (CMS), Optical Flow (OF), etc. An important problem for performance evaluation is the absence of any stable and flexible software for comparison of different algorithms. In this frame, we have designed and implemented the software for comparing and evaluating the well-known video object D&T algorithms on the same platform. This software is able to compare them with the same metrics in real-time and on the same platform. It also works as an automatic and/or semi-automatic test environment in real-time, which uses the image and video processing essentials, e.g. morphological operations and filters, and ground-truth (GT) XML data files, charting/plotting capabilities, etc. Along with the comprehensive literature survey of the abovementioned video object D&T algorithms, this book also covers the technical details of our performance benchmark software as well as a case study on people D&T for the functionality of the software.

Video segmentation has become one of the core areas in visual signal processing research. The objective of Video Segmentation and Its Applications is to present the latest advances in video segmentation and analysis techniques while covering the theoretical approaches, real applications and methods being developed in the computer vision and video analysis community. The book will also provide researchers and practitioners a comprehensive understanding of state-of-the-art of video segmentation techniques and a resource for potential applications and successful practice.

Video Tracking provides a comprehensive treatment of the fundamental aspects of algorithm and application development for the task of estimating, over time, the position of objects of interest seen through cameras. Starting from the general problem definition and a review of existing and emerging video tracking applications, the book discusses popular methods, such as those based on correlation and gradient-descent. Using practical examples, the reader is introduced to the advantages and limitations of deterministic approaches, and is then guided toward more advanced video tracking solutions, such as those based on the Bayes’ recursive framework and on Random Finite Sets. Key features: Discusses the design choices and implementation issues required to turn the underlying mathematical models into a real-world effective tracking systems. Provides block diagrams and simil-code implementation of the algorithms. Reviews methods to evaluate the performance of video trackers – this is identified as a major problem by end-users. The book aims to help researchers and practitioners develop techniques and solutions based on the potential of video tracking applications. The design methodologies discussed throughout the book provide guidelines for developers in the industry working on vision-based applications. The book may also serve as a reference for engineering and computer science graduate students involved in vision, robotics, human-computer interaction, smart environments and virtual reality programmes