An Autonomous Ground Vehicle (AGV) should be capable of self-navigating through various terrains based on priori data as well as self-configuring and optimizing its motion on the basis of sensed data. Research has been in progress in this domain to improve terrain perception for planning, execution, and control of desired motion of an AGV. There involve certain processes to achieve these goals. During the perception phase multiple classification techniques such as Bayesian Inference, K-Mean clustering, Artificial Neural Network and many others are used depending on underlying sensing technology for example LADAR and RGB Camera. This paper proposes a multilevel classification scheme for terrain identification and obstacle detection to improve self-organization according to the known terrain type. As a result the computation cost is reduced because of the use of multiple sensors.