Perception systems in robotics are typically closely tailored to the given task, e.g., in typical pick-and-place tasks the perception systems only recognizes the mugs that are supposed to be moved and the table the mugs are placed on. The obvious limitation of those systems is that for a new task a new vision system must be designed and implemented. This master's thesis proposes a method that allows to identify entities in the world based on motion of various features from various sources. This is without relying on strong prior assumptions and to provide an important piece towards a more general perception system. While entities are rigid bodies in the world, the sources can be anything that allows to track certain features over time in order to create trajectories. For example, these feature trajectories can be obtained from RGB and RGB-D sensors of a robot, from external cameras, or even the end effector of the robot (proprioception).

The core conceptual elements are: the distance variance between trajectory pairs is computed to construct an affinity matrix. This matrix is then used as input for a divisive k-means algorithm in order to cluster trajectories into object hypotheses. In a final step these hypotheses are combined with previously observed hypotheses by computing the correlations between the current and the updated sets. This approach has been evaluated on both simulated and real world data. Generating simulated data provides an elegant way for a qualitative analysis of various scenarios. The real world data was obtained by tracking Shi-Tomasi corners using the Lucas-Kanade optical flow estimation of RGB image sequences and projecting the features into range image space.