Object detectors can be divided into two types, namely traditional two-stage detectors with good accuracy and emerging one-stage detectors with high efficiency. In recent years, one-stage detectors have exceeded two-stage ones in accuracy by adopting stronger backbone networks, improved loss functions, and optimized detection mechanisms, at the cost of losing their advantage in detection speed. Therefore, a method by setting anchors more in line with dataset is proposed to improve the prediction accuracy of objects’ bounding boxes and then improve the detection accuracy. Under this method, each category’s bounding boxes feature is reflected by a specialized group of anchors, thereby further improving predicted bounding boxes’ accuracy. This method is applied to YOLOv2 and improves its accuracy while maintaining its detection speed.