Human milk oligosaccharides (HMOs) are important components in human milk. Understanding the differences between non-HMOs and HMOs is fundamental to develop infant formula with improved nutritional functions making it more similar to human milk to satisfy the nutritional health of infants. This paper reviews the types and structural characteristics of HMOs and their roles in regulating the intestinal microflora, improving intestinal health and promoting immune system development. Meanwhile, the differences between the composition of oligosaccharides in mammalian milk and HMOs composition are described, as well as the characteristics and functions of oligosaccharides commonly used in infant formula. The problems associated with the application of oligosaccharides in infant formula are proposed and future prospects are discussed. We expect that this review will provide a theoretical guidance for the development of new infant formulas.