The aim of the presented work was the development of a tracking algorithm for a stereoscopic camera setup equipped with an additional inertial sensor. The input of the algorithm consists of the image sequence, angular velocity and linear acceleration vectors measured by the inertial sensor. The main assumption of the project was fusion of data streams from both sources to obtain more accurate ego-motion estimation. An electronic module for recording the inertial sensor data was built. Inertial measurements allowed a coarse estimation of the image motion field that has reduced its search range by standard image-based methods. Continuous tracking of the camera motion has been achieved (including moments of image information loss). Results of the presented study are being implemented in a currently developed obstacle avoidance system for visually impaired pedestrians.