Visual object tracking on real-time is one of the hottest research point in computer vision. It has broad applications and a wide range of future in video surveillance,robot navigation, human-computer interface, military application and so on. Based on the survey of existing methods and techniques, this paper gives further analysis and research, designs and achieves the real-time object tracking system. The contributions of this thesis are three fold: Firstly, in order to mitigate model drift problems, the tracking algorithm of collaborative tracker based on short-term tracker and long-term detector is introduced.It could make use of the characteristics of tracker and detector so as to mine the gradient information of appearance model in the short-term and the stable information for a long time and can handle the problems of object occlusion, disappearance-appearance, gradual change. The experiment results show that this method can be obtained robust tracking results in a complex environment. Secondly, advanced algorithm based on sample rank learning is proposed.The method expands the positive-negative pairs in original paper, and introduces stable positive template library,the zero sample between positive sample and negative sample which will generate the positive-negative pairs, positive-zero pairs, zero-negative pairs for effectively depicting the context relationship between object and background. Testing results on the public data sets show that this method can better deal with the problem of target occlusion and choice, with a smaller average central location tracking error. Lastly, we build up an active visual object tracking platform.By rational design of data structures and algorithm optimization, the platform can be tracking object robustly in real-time and be able to deal effectively with the target drift shift.In addition, the platform also has a strong maintainability and scalability.