Exoskeleton robots can assist humans to perform activities of daily living with little effort. In this paper, a hierarchical control scheme is presented which enables an exoskeleton robot to achieve cooperative manipulation with humans. The control scheme consists of two layers. In low-level control of the upper limb exoskeleton robot, an admittance control scheme with an asymmetric barrier Lyapunov function-based adaptive neural network controller is proposed to enable the robot to be back drivable. In order to achieve high-level interaction, a strategy for learning human skills from demonstration is proposed by utilizing Gaussian mixture models, which consists of the learning and reproduction phase. During the learning phase, the robot observes and learns how a demonstrator performs a specific impedance-based task successfully, and in the reproduction phase, the robot can provide the subjects with just enough assistance by extracting human skills from demonstrations to prevent the motion of the robot end-effector deviating far from desired ones, due to variation in the interaction force caused by environmental disturbances. Experimental results of two different tasks show that the proposed control scheme can provide human subjects with assistance as needed during cooperative manipulation.