In recent years, physics-based building energy modeling (BEM) has started being used to evaluate the performance of buildings in the context of connected communities and on an urban scale to study their aggregated energy use, interactions, and impacts on the energy supply infrastructure and environment. The development of urban scale BEM solutions needs extensive effort. Existing attempts tend to focus on different aspects of BEM on an urban scale, such as collecting as-built building data from different information sources, integrating geometry modeling with geographic information systems (GISs), representing operational and occupancy profiles, automating workflow, processing and visualizing the results, and conducting large-scale simulations. Urban scale BEM development would benefit from multi-disciplinary research areas and from an open platform to adopt advancements on data sources and tools. For these purposes, this research proposes a modularized bottom-up model creation and simulation framework that is built on the state-of-the-art BEM tools and can accommodate different building stock data. This framework uses a standardized modeling inputs schema to describe building design and operational characteristics, and it can be instantiated from different building survey datasets with heterogeneous structures. The paper presents a case study of the proposed framework that demonstrates how thousands of surveyed buildings from the 2012 U.S. Energy Information Administration’s Commercial Buildings Energy Consumption Survey (CBECS) were one-to-one converted to EnergyPlus models through the schema and the model generation process, then simulated efficiently with distributed computing, and their simulation outputs aggregated to show satisfactory results.