Deep learning has the potential to enhance both the performance and efficiency of indoor positioning using inertial measurement units (IMUs). Many existing deep learning methods for positioning rely on convolutional neural networks (CNNs), which are designed with increased depth to expand their receptive fields for capturing long-term features from IMU measurements. However, this design often introduces more parameters and computations, extracting redundant information. Conversely, shallow networks struggle to extract global features, resulting in unsatisfactory performance. In addition, in the era of the Internet of Things (IoT), the lightweighting of models and the high real-time performance of systems can effectively enhance the usability of portable devices and reduce their energy consumption, thereby enabling broader application. In this study, we introduce a lightweight network named ResMixer, capable of linearly capturing channel information and sequential features from IMU measurements. Despite maintaining the number of stages of a ResNet-style network, ResMixer significantly reduces the number of parameters - approximately a 70% reduction. The experimental results demonstrate that the proposed network achieves similar accuracy as ResNet18 while offering competitive performance compared to other lightweight models in terms of parameter count and computations. In our self-collected data, ResMixer demonstrates improvements in positioning accuracy and model efficiency compared to advanced algorithms such as lightweight learned inertial odometry (LLIO) and IMU network (IMUNet). Additionally, we enhance the indoor positioning performance by combining measurements from additional single low-cost ultra-wideband (UWB), which further marks a 48.4% enhancement in precision in real scenario experiments compared to solely ResMixer.