Permutation is a commonly used operation in many secure multimedia systems. However, it is fragile against cryptanalysis when used alone. For instance, it is well-known that permutation-only multimedia encryption is insecure against known-plaintext attack (KPA). There exist algorithms that are able to (partially) retrieve the secret permutation sequences in polynomial time with logarithmic amount of plaintexts in the number of elements to be permuted. But existing works fail to answer how many known plaintexts are needed to fully recover a underlying secret permutation sequence and how to balance the storage cost and computational complexity in implementing the KPA attack. This paper addresses these two problems. With a new concept of composite representation, the underlying theoretical rules governing the KPA attack on a permutation-only cipher are revealed, and some attractive algorithms outperforming the state-of-the-art methods in terms of computational complexity are developed. As a case study, experiments are performed on permutation-only image encryption to verify the theoretic analysis. The performance gap of the proposed KPA between artificial noise-like images, which perfectly fits the theoretical model, and the corresponding natural images is identified and analyzed. Finally, experimental results are shown to demonstrate the efficiency improvement of the new schemes over the existing ones.