Deregulation of the cell cycle machinery, which has been linked to dysregulation of cyclin-dependent kinases (CDKs), is a defining characteristic of cancer, eventually promoting abnormal proliferation that feeds tumorigenesis and disease development. In this regard, several CDK inhibitors (CDKIs) have been developed during the last few decades (1st, 2nd, and 3rd generation CDKIs) to inhibit cancer cell proliferation. 1st and 2nd generation CDKIs have not received much clinical attention for the treatment of cancer patients because of their limited specificity and high toxicity. However, the recent development of combination strategies allowed us to reduce the toxicity and side effects of these CDKIs, paving the way for their potential application in clinical settings. The 3rd generation CDKIs have yielded the most promising results at the preclinical and clinical levels, propelling them into the advanced stages of clinical trials against multiple malignancies, especially breast cancer, and revolutionizing traditional treatment strategies. In this review, we discuss the most-investigated candidates from the 1st, 2nd, and 3rd generations of CDKIs, their basic mechanisms of action, the reasons for their failure in the past, and their current clinical development for the treatment of different malignancies. Additionally, we briefly highlighted the most recent clinical trial results and advances in the development of 3rd generation FDA-approved selective CDK4/6 inhibitors that combat the most prevalent cancer. Overall, this review will provide a thorough knowledge of CDKIs from the past to the present, allowing researchers to rethink and develop innovative cancer therapeutic regimens.