Weakly supervised object detection (WSOD) aims at learning precise object detectorswith only image-level tags. In spite of intensive research on deep learning (DL) approaches over the past few years, there is still a significant performance gap between WSOD and fully supervised object detection. Existing WSOD methods only consider the visual appearance of each region proposal but ignore the useful context information in the image. This paper proposes an interactive end-to-endWSDOframework called JLWSODwith two innovations: i) two types of WSOD-specific context information (i.e., instance-wise correlation and semantic-wise correlation) are proposed and introduced into WSOD framework; ii) an interactive graph contrastive learning (iGCL) mechanism is designed to jointly optimize the visual appearance and context information for betterWSOD performance. Specifically, the iGCL mechanism takes full advantage of the complementary interpretations of the WSOD, namely instance-wise detection and semanticwise prediction tasks, forming a more comprehensive solution. Extensive experiments on the widely used PASCAL VOC and MS COCO benchmarks verify the superiority of JLWSOD over alternative SOTA and baseline models (improvement of 3.0%∼23.3% on mAP and 3.1%∼19.7% on CorLoc, respectively).