Multi-Source Heat Pump (MSHP) systems are intended to achieve a higher system efficiency through the combined or alternate use of two or more sources for a heat pump. In this study, an MSHP system, designed for a house located in Michigan, is described. A water-to-water heat pump system is integrated with solar thermal collectors, vertical boreholes, and horizontal underground loops buried in a partially-insulated, confined underground region beneath the building foundation. Heating is provided through a floor radiant system, and cool air is supplied to each conditioned space through air handlers using the heat pump as the primary cooling source. Solar heat collected will first satisfy the heating requirements of the Domestic Hot Water (DHW) and floor radiant systems (for free space heating) in heating seasons. Excessive solar heat, if left over, will then be transferred into a geothermal buffer storage tank that is connected with the underground loop to allow for long-term or seasonal thermal energy storage in the underground regions of earth around and below the underground loops. When heating is needed by the floor radiant system, the heat pump will extract heat at its source side from either the geothermal buffer storage tank or the vertical boreholes, whichever can give a higher outlet water temperature. The horizontal underground loops, known as Thermal Battery, will be used to charge the geothermal buffer tank if the temperature of the tank is lower than a set point. This enables to maximize the system efficiency for space heating in winter, which is more critical for this heating-dominated building located in a cold climate region such as Michigan. This proposed system was modeled numerically with a series of simulations, whose results show that more than 20% energy savings can be achieved for the heat pump heating when comparing the proposed system and a conventional Ground Source Heat Pump (GSHP) system without using the horizontal underground loops as a heat storage element. The results also demonstrate the feasibility of using this type of MSHP system to achieve more energy savings as well as the importance of optimizing the control strategy. For example, this study indicates that a more than 10% difference in terms of heating energy usage is identified when looking at two control strategies, i.e., charging the underground region through the horizontal loops during the whole year or only charging it during summer.