Neurofilaments are functional protein polymers that are transported along the axonal processes of nerve cells. Studying the movement of neurofilaments is important in applications such as diagnosing neurodegenerative diseases. Traditional methods have been largely relying on manual labeling of the neurofilaments in fluorescence microscopy images. In this paper, we describe an automated method for analyzing the motion of neurofilaments based on tracking-by-detection. The axon along which the neurofilaments move is extracted and represented by a parametric curve. The axon is decomposed into blocks, and the blocks encompassing the moving neurofilaments are determined by graph labeling using Markov Random Field. The leading and trailing locations of a neurofilament are then refined to sub-pixel accuracy. We demonstrate the efficacy and efficiency of our proposed method by experiments on real time-lapse fluorescence image sequences of neurofilament movement.