Integrated Evaluation Through M&V 2.0

This post was written by Tim Guiterman, Director of Measurement and Optimization Solutions at EnergySavvy, as part of its membership in NEEP’s Allies Program.

The following post was adapted from my comments given at NEEP’s EM&V Forum Spring Meeting. I was invited by NEEP to speak and moderate a panel on whole building analysis and the emergence of M&V 2.0 technologies. I had the privilege of kicking off the panel and setting the stage for a rich discussion, where the spring meeting was buzzing with ongoing industry discussion about innovation in EM&V. These remarks are just a small sample from all the important issues raised at the spring meeting, but offer a brief glimpse into this exciting industry trend.  


Our panel today, while titled “Whole Buildings Panel,” is really centered on a core theme - the role of meter-based measurement in the field of M&V and, more importantly, within the practice of Evaluation, Measurement & Verification. That leads us into a conversation about M&V 2.0.

A defining criterion for automated M&V software is that it is continuously analyzes data as it becomes available

I’m going to kick off our session with a quick overview of the semantics and definitions around “M&V 2.0” and provide some vision for where I think the evaluation field can and should move. The industry is evolving as it forces the role of the evaluator to shift from a static check-off-the-box activity to one that can be central to the utilities. This shift is critical to support the changing utility business model, where the focus is moving away from reconciling programmatic achievements, to ensuring holistic reductions are appearing on the grid. Near real-time, meter-based measurement will be necessary to account for energy efficiency as a resource, and open new markets for energy efficiency investment. 


M&V Floating Names


There are lots of names, and some scattered definitions around this topic of M&V 2.0. During this panel, you will hear different definitions of M&V 2.0 that will correspond to the use case or application where our experts are involved.

Despite these various definitions, we see a convergence on some key requirements. For example, the state of NY, in its recently revised official EM&V guidance, emphasizes the continuous nature of an automated analysis. This is key in moving from 1.0 to 2.0.

Collaborative Study involved DOE, Utilities, Evaluators, and Analytic Firms This room is not the only place where this discussion is taking place. You may be familiar with the recently released RMI paper on the Status and Promise of Advanced M&V. As one of the co-authors on that report, I can say that this collaborative effort between key stakeholders recognized this same theme, that automated analytics providing continuous savings estimates is changing the field of evaluation.

But as you’ll see reflected in this panel, the paper also recognized that increased granularity of the data – from frequency to volume to end-use detail, is another key element and attribute of M&V 2.0. And importantly – it’s not just about how this fits into a final evaluation report – M&V 2.0 is critical to support program design, customer targeting and making energy efficiency a real, reliable resource that is trusted to reduce load when and where it’s needed.


But this is not just about technology. It’s about supporting program administrators and utilities in the modern era. I want to share some things that I hear from evaluation departments within utilities.

·         Evaluation simply is not fulfilling a valuable enough role to the utility. It’s not keeping up with where the utility and the grid need to go.

·         It’s a static checkbox, a cost center, and/or a regulatory necessity that isn’t providing valuable, granular and timely feedback to program operations.

·         It’s too wrapped up in its own language of potentially-false precision and certainty that just doesn’t translate to the grid planning and operations side of the house.

So how do we address this?


Traditional Evaluation Developmental Evaluation
Roles & relationships: Positioned as an outsider to assure independence and objectivity. Roles & relationships: Positioned as an internal team function integrated into the process of gathering and interpreting data, framing issues, surfacing and testing model developments.
Measurement: Measure performance and success against pre-determined goals and SMART outcomes. Measurement: Develops measures and tracking mechanisms quickly as outcomes emerge; measures can change during evaluation as the process unfolds.
Evaluation results: Detailed formal reports validated best practices, generalizable across time and space. Can engender fear of failure. Evaluation results: Rapid, real-time feedback; diverse, user-friendly forms of feedback. Evaluation aims to nurture learning.
Complexity & uncertainty: Evaluator tries to control design implementation and the implementation process. Complexity & uncertainty: Learning to respond to lack of control, staying in touch with what's unfolding and repsonding accordingly. 

Source: Patton, M. (2010). Developmental evaluation: applying complexity concepts to enhance innovation and use. New York, NY: Guilford Press.


RMI: The Status and Promise of Advanced M&V

"Advanced M&V can increase the value of evaluation, reduce costs through automation, enhance program targeting, allow for early adjustments to program designs & budgets, and increase accuracy of savings estimates to support EE as a resource."

If anyone here is not familiar with the works of Michael Quinn Patton – particularly his book on Developmental Evaluation, I highly recommend checking it out. Our industry is demanding that the field of EM&V take the kind of pro-active, responsive approach that Patton specifies. For example:

  • We need integrated data collection on a continuous basis.
  • We need to interpret this data as part of a team that is critical to the delivery and operation of programs and initiatives designed to produce energy savings and load reductions.
  • We need to do this quickly as outcomes emerge. Utilities, new measure developers, and implementers need rapid and near real-time feedback.
  • For example, at EnergySavvy we’re working with other technology companies to streamline the long process of formal evaluation for pilots, which is one of the main barriers to getting their energy-saving solutions from pilot phase into commercialized program delivery.
  • If energy efficiency is going to support non-wires alternatives aimed at targeted load reductions to defer distribution system upgrades, we need to move beyond after-the-fact reports and deemed engineering equations.
  • This becomes especially critical as we integrate energy efficiency and other DER’s such as solar, electric vehicles and time-of-use (TOU) rates into the mix. With these blended emerging technologies, we must understand what happens at the customer meter when these interventions occur. Which means, we have to start measuring impacts at the meter in near real-time.

An Example of Integrated Evaluation

But we can move from thinking about the bigger picture to the immediate next steps ahead of us.

Integrated evaluation is simple: by integrating M&V 2.0 and meter-based measurement into applicable program evaluations today, we can work together to enhance and support the delivery of critical performance feedback and savings estimates to utility and program administrators–evolving the evaluation field and the energy efficiency industry to meet the needs of the modern and changing energy sector. 


EM&V Summer Workshop: Cost-Effectiveness Testing For Next Generation Energy Efficiency If you’d like to join NEEP for the next M&V discussion, please join us at our summer workshop in Connecticut June 15.


Stay informed

Stay up to date with the latest NEEP and industry news, policies, and trends to your inbox every so often.