Driven by the increasing interests of multi-tier computing architecture, this paper considers a hybrid non-orthogonal multiple access (NOMA) and frequency division multiple access (FDMA) assisted two-tier edge-cloudlet multi-access computation offloading. In particular, part of the computation tasks of the edge-computing user (EU) can be offloaded to different edge servers (ESs) via NOMA, and part of the computing tasks received by the ESs can be further offloaded to the cloudlet server (CS) via FDMA. We formulate an optimization problem to minimize the total energy consumption of the two-tier offloading system by jointly optimizing the multi-access offloading decision, NOMA-transmission duration, FDMA-transmission duration and bandwidth allocations. Although the problem is non-convex, we identify the structural feature of the problem and propose a decomposition-based algorithm to attain the optimal two-tier offloading decisions and the associated allocations of the computing/communication resources. Specifically, by leveraging the hidden convexity after decomposition, we propose a distributed algorithm, which is based on the block coordinated descent (BCD) method and Lagrange duality, for solving the subproblem. In our distributed algorithm, the EU optimizes its Tier-I offloading towards different ESs, each ES individually optimizes its Tier-II offloading towards the CS, and moreover, the CS optimizes its bandwidth allocations to the ESs. Furthermore, based on the proposed algorithm for the singe-EU case, the multi-EU scenario is further analyzed. To improve the energy-efficiency and fairness for the multi-EU scheduling, we propose a gradient projection (GP) based multi-EU scheduling algorithm, which leverages our single-EU algorithm as a subroutine, to optimize all EUs' energy-efficiency in computation offloading over a long-term period. Numerical results verify the performance of our proposed algorithms. For the single-EU scenario, our simulation results demonstrate that the decomposition-based algorithm can outperform some benchmark algorithms. In the case of multi-EU scenario, our proposed GP-based multi-EU scheduling can effectively improve all EUs' energy-efficiency compared to the proportional fair scheduling.