Building energy simulation has become a powerful tool in evaluating retrofit opportunities, operation and control strategies for existing buildings. Calibrating the energy model to measured data is the critical first step to further applications. However, model calibration is usually challenging because of highly uncertain building system operations and user behaviors. This study investigates the potentials of using physics-based inverse models to calculate hard-to-measure infiltration airflow rate and people count to enhance the traditional model calibration process. The experiments indicate that the inverse models can further improve the accuracy when a model under calibration has already achieved convergence at monthly level.