Integrated Evaluation Through M&V 2.0

Our panel today, while titled “Whole Buildings Panel,” is really centered on a core theme - the role of meter-based measurement in the field of M&V and, more importantly, within the practice of Evaluation, Measurement & Verification. That leads us into a conversation about M&V 2.0.

I’m going to kick off our session with a quick overview of the semantics and definitions around “M&V 2.0” and provide some vision for where I think the evaluation field can and should move. The industry is evolving as it forces the role of the evaluator to shift from a static check-off-the-box activity to one that can be central to the utilities. This shift is critical to support the changing utility business model, where the focus is moving away from reconciling programmatic achievements, to ensuring holistic reductions are appearing on the grid. Near real-time, meter-based measurement will be necessary to account for energy efficiency as a resource, and open new markets for energy efficiency investment. 

 

 

There are lots of names, and some scattered definitions around this topic of M&V 2.0. During this panel, you will hear different definitions of M&V 2.0 that will correspond to the use case or application where our experts are involved.

Despite these various definitions, we see a convergence on some key requirements. For example, the state of NY, in its recently revised official EM&V guidance, emphasizes the continuous nature of an automated analysis. This is key in moving from 1.0 to 2.0.

This room is not the only place where this discussion is taking place. You may be familiar with the recently released RMI paper on the Status and Promise of Advanced M&V. As one of the co-authors on that report, I can say that this collaborative effort between key stakeholders recognized this same theme, that automated analytics providing continuous savings estimates is changing the field of evaluation.

But as you’ll see reflected in this panel, the paper also recognized that increased granularity of the data – from frequency to volume to end-use detail, is another key element and attribute of M&V 2.0. And importantly – it’s not just about how this fits into a final evaluation report – M&V 2.0 is critical to support program design, customer targeting and making energy efficiency a real, reliable resource that is trusted to reduce load when and where it’s needed.

**

But this is not just about technology. It’s about supporting program administrators and utilities in the modern era. I want to share some things that I hear from evaluation departments within utilities.

·         Evaluation simply is not fulfilling a valuable enough role to the utility. It’s not keeping up with where the utility and the grid need to go.

·         It’s a static checkbox, a cost center, and/or a regulatory necessity that isn’t providing valuable, granular and timely feedback to program operations.

·         It’s too wrapped up in its own language of potentially-false precision and certainty that just doesn’t translate to the grid planning and operations side of the house.

So how do we address this?

 

Traditional Evaluation	Developmental Evaluation
Roles & relationships: Positioned as an outsider to assure independence and objectivity.	Roles & relationships: Positioned as an internal team function integrated into the process of gathering and interpreting data, framing issues, surfacing and testing model developments.
Measurement: Measure performance and success against pre-determined goals and SMART outcomes.	Measurement: Develops measures and tracking mechanisms quickly as outcomes emerge; measures can change during evaluation as the process unfolds.
Evaluation results: Detailed formal reports validated best practices, generalizable across time and space. Can engender fear of failure.	Evaluation results: Rapid, real-time feedback; diverse, user-friendly forms of feedback. Evaluation aims to nurture learning.
Complexity & uncertainty: Evaluator tries to control design implementation and the implementation process.	Complexity & uncertainty: Learning to respond to lack of control, staying in touch with what's unfolding and repsonding accordingly.

Source: Patton, M. (2010). Developmental evaluation: applying complexity concepts to enhance innovation and use. New York, NY: Guilford Press.

 

RMI: The Status and Promise of Advanced M&V "Advanced M&V can increase the value of evaluation, reduce costs through automation, enhance program targeting, allow for early adjustments to program designs & budgets, and increase accuracy of savings estimates to support EE as a resource."

If anyone here is not familiar with the works of Michael Quinn Patton – particularly his book on Developmental Evaluation, I highly recommend checking it out. Our industry is demanding that the field of EM&V take the kind of pro-active, responsive approach that Patton specifies. For example:

• We need integrated data collection on a continuous basis.
• We need to interpret this data as part of a team that is critical to the delivery and operation of programs and initiatives designed to produce energy savings and load reductions.
• We need to do this quickly as outcomes emerge. Utilities, new measure developers, and implementers need rapid and near real-time feedback.
• For example, at EnergySavvy we’re working with other technology companies to streamline the long process of formal evaluation for pilots, which is one of the main barriers to getting their energy-saving solutions from pilot phase into commercialized program delivery.
• If energy efficiency is going to support non-wires alternatives aimed at targeted load reductions to defer distribution system upgrades, we need to move beyond after-the-fact reports and deemed engineering equations.
• This becomes especially critical as we integrate energy efficiency and other DER’s such as solar, electric vehicles and time-of-use (TOU) rates into the mix. With these blended emerging technologies, we must understand what happens at the customer meter when these interventions occur. Which means, we have to start measuring impacts at the meter in near real-time.

​

An Example of Integrated Evaluation

But we can move from thinking about the bigger picture to the immediate next steps ahead of us.

Integrated evaluation is simple: by integrating M&V 2.0 and meter-based measurement into applicable program evaluations today, we can work together to enhance and support the delivery of critical performance feedback and savings estimates to utility and program administrators–evolving the evaluation field and the energy efficiency industry to meet the needs of the modern and changing energy sector. 

 

If you’d like to join NEEP for the next M&V discussion, please join us at our summer workshop in Connecticut June 15.