Autonomous robots also known as Unmanned Ground Vehicles (UGV) is of broad and current interest, in both research and industry. Many studies have been done in stereo vision based UGV vision in positive obstacle detection and avoidance, but negative obstacle detection and avoidance remains less explored. In this paper, we propose a new method to detect negative obstacles incorporating the knowledge gained from stereo vision and detecting saliency on the ground plane. Experimental results show that our method works well on detecting negative obstacles.