The technology available to assess sperm population characteristics has advanced greatly in recent years. Large artificial insemination (AI) organizations that sell bovine semen utilize many of these technologies not only for novel research purposes, but also to make decisions regarding whether to sell or discard the product. Within an AI organization, the acquisition, interpretation and utilization of semen quality data is often performed by a quality control department. In general, quality control decisions regarding semen sales are often founded on the linkages established between semen quality and field fertility. Although no one individual sperm bioassay has been successful in predicting sire fertility, many correlations to various in vivo fertility measures have been reported. The most powerful techniques currently available to evaluate semen are high-throughput and include computer-assisted sperm analysis and various flow cytometric analyses that quantify attributes of fluorescently stained cells. However, all techniques measuring biological parameters are subject to the principles of precision, accuracy and repeatability. Understanding the limitations of repeatability in laboratory analyses is important in a quality control and quality assurance program. Hence, AI organizations that acquire sizeable data sets pertaining to sperm quality and sire fertility are well-positioned to examine and comment on data collection and interpretation. This is especially true for sire fertility, where the population of AI sires has been highly selected for fertility. In the December 2017 sire conception rate report by the Council on Dairy Cattle Breeding, 93% of all Holstein sires (n=2062) possessed fertility deviations within 3% of the breed average. Regardless of the reporting system, estimates of sire fertility should be based on an appropriate number of services per sire. Many users impose unrealistic expectations of the predictive value of these assessments due to a lack of understanding for the inherent lack of precision in binomial data gathered from field sources. Basic statistical principles warn us of the importance of experimental design, balanced treatments, sampling bias, appropriate models and appropriate interpretation of results with consideration for sample size and statistical power. Overall, this review seeks to describe and connect the use of sperm in vitro bioassays, the reporting of AI sire fertility, and the management decisions surrounding the implementation of a semen quality control program.