Uncertainty in predicting occupancy patterns leads to discrepancies in simulated building energy compared to measured data. Typical simulation models represent occupants through identical schedules and repetitive behavior. However, users’ activity patterns comprise numerous variations, especially when focusing on occupants’ interactions on neighborhood scale. This paper presents a framework for modeling occupancy and consequent energy loads in residential buildings using measured data for calibration; it employs a clustering approach to profile energy use which generates inputs for Urban Energy Models (UBEMs). The framework is demonstrated on a residential neighborhood and reveals that the generated inputs can more accurately predict energy load patterns.