DecoderTracker: Decoder-Only Method for Multiple-Object Tracking

Abstract: Decoder-only methods, such as GPT, have demonstrated superior performance in many areas compared to traditional encoder-decoder structure transformer methods. Over the years, end-to-end methods based on the traditional transformer structure, like MOTR, have achieved remarkable performance in multi-object tracking. However,The substantial computational resource consumption of these methods, coupled with the optimization challenges posed by dynamic data, results in less favorable inference speeds and training times. To address the aforementioned issues, this paper optimized the network architecture and proposed an effective training strategy to mitigate the problem of prolonged training times, thereby developing DecoderTrack, a novel end-to-end tracking method. Subsequently, to tackle the optimization challenges arising from dynamic data, this paper introduced DecoderTrack+ by incorporating a Fixed-Size Query Memory and refining certain attention layers. Our methods, without any bells and whistles, outperforms MOTR on multiple benchmarks, with inference speeds 2.06 and 3.03 times faster than MOTR, respectively