Your 10-Year Forecasts Need Hourly Data Now

ISO-NE Is Updating Their Long-Term Load Forecasting

Every year, ISO New England (ISO-NE) releases a Capacity, Energy, Loads, and Transmission (CELT) report—a 10-year forecast on the New England grid’s…capacity, energy, load, and transmission requirements. In particular, the CELT report exists to forecast system peaks—that is, the hour of the year with the literal highest total system demand. ISO-NE tracks a winter peak and a summer peak—these two peaks define the infrastructure requirements of the grid more than the other 8,758 hours of the year combined.¹ The 2025 CELT report is due for Summer 2025-ish, but ISO-NE started talking about CELT in September 2024, because they’re changing their methodology. The preliminary results match expectations², but the more interesting takeaway is that ISO-NE needed to restructure their forecasting in the first place.

Daily Forecasts Aren’t Enough Anymore

Prior CELT forecasts measured peak loads on a daily basis—model every day of the year, make a forecast on what the highest potential load (in MW) would be for each day, and from those daily peak forecasts, find the winter and summer peaks. The 2024 CELT report doesn’t specify what specific day or hour the peaks would be, because that didn’t particularly matter for long-term forecasting.

It will matter in the 2025 report.

The driver, primarily, is behind-the-meter photovoltaic (BTM-PV) systems—residential and commercial-scale solar systems, typically installed at the 10-kW scale for residential homes and the 100-kW scale for commercial buildings. These systems are installed by private customers and incentivized through net metering programs, utility incentives, federal tax credits, and predatory sales tactics.³ These systems are seeing good uptake, in part because of rising electricity costs, the IRA-sponsored federal tax credits,⁴ increasing public familiarity, and raw social signaling. The familiarity and social signaling are especially important because 1) they wear down permitting barriers for installing BTM-PV systems on, say, historical-district homes, and 2) they make solar panels more resistant to price signals. Few commercial customers would accept the ten-year payback periods of a modern solar system (or the underwater payback of a solar-storage system), but as Elon himself has proven, people will spend a lot of money to be based.⁵

The problem is that ISO-NE can’t see most of that behind-the-meter solar. These systems typically have production meters (relevant for getting renewable energy certificates), but they aggregate on a daily or monthly basis and do not necessarily interface with ISO-NE markets. Instead, ISO-NE sees a hole in their electricity demand, from which they reverse-engineer a “gross load.” They’ve made BTM-PV estimates for a while, but they only started showing those estimates in 2022.

As BTM-PV systems continue to proliferate, they make a progressively larger impact on the New England grid—which means that ISO-NE’s demand projections are increasingly skewed by small, distributed energy systems that generate when they feel like it and transmit no data back to the grid. And eventually, these solar systems will be augmented by behind-the-meter battery systems that will also predict winter and summer peaks, discharging in a distributed game of peak-load Whack-a-Mole that will again transmit no data back to the grid.

Daily forecasts aren’t enough, so ISO-NE will run hourly forecasts for their 2025 CELT Report.

In particular, ISO-NE will forecast 25 models—one for the day, one for each hour of the day—repeated across eight zones: three for Massachusetts, one for every other state. The daily forecast output serves as an input for the 24 hourly forecasts, and the eight zones will sum into a system-wide forecast. The daily forecast is a straightforward linear regression model, and the hourly models are built on neural networks. In both cases, the models are built on 9-10 years of historical data: mostly weather and calendar variables, although global warming has also been factored in somehow.

I have no idea how this will look on the final report.

Distributed Resource Methodology Tweaks

The above details focus only on the “base load” forecast for the CELT report—there are three other forecasts for:

• BTM-PV systems—a.k.a. “distributed energy resources” (DERs)

• Heat pump systems

• Electric vehicles

The challenge with these systems is calculating their impact on the grid without really knowing how quickly they’ll be installed in homes and businesses—the “adoption curve,” in industry parlance. This is partially because prediction those adoption curves runs into core contradictions:

• Solar and solar-storage systems are expensive, and their cost-benefit depends on a constantly changing incentive structure—in particular, net metering is a nightmare to navigate.

• Heat pump systems are expensive, and they’re more expensive per unit of output heat than natural gas boilers. However, policy goals assume everyone will replace their gas boilers and furnaces with heat pumps by 2050.

• Electric vehicles are compromised, particularly for light-duty trucks and commercial vehicles. However, Massachusetts plans to ban internal-combustion vehicle sales in 2035, and the Trump White House’s attempts to knock this down will only go so far.

At a broader scale, these contradictions are contributing to divergences in oil demand forecasts, in particular between the International Energy Agency (IEA) and OPEC. Some analysts say this is because IEA has gone political,⁶ but no one fundamentally knows how seriously to take decarbonization regulation. Between the shale boom and the Russo-Ukrainian war, the core energy map of the world has changed twice in the past twenty years, and those shifts change how much decarbonization policy will stick.⁷

To this end the 2025 CELT report will disaggregate load impact forecasts for these three key technologies, which will all drive peaks in different directions and times depending on how quickly each of them proliferate. By running these forecasts separately (and on an hourly basis), ISO-NE can track not only peak demand, but also “duck curve” effects, shifts in peak hours, and miscellaneous weirdness as the grid sees more systems that react to grid conditions but don’t necessarily report back on what they’re doing.

In particular, the 2025 CELT report has a new model converting heating and cooling degree-days (HDD/CDD) into load impacts. They’re no longer modeling the relationship between temperature and heating/cooling-driven load as a linear relationship—instead, there’s a saturation effect on particularly hot and particularly cold days. Given how much of New England electricity demand is driven by space heating and cooling, this will make a difference in the output model.

No One Knows What’s Happening Anymore

Now that I’ve been posting for a few months, I’ve found a through-line for Energy Crystals:

The grid is getting more expensive and less predictable, and everyone must figure out a new strategy before they get punched.

And that’s part of what’s happening here—the proliferation of electrification technologies that affect grid demand without reporting back generates increasing chaos in the system that in turn warrants more granular action. In the same way that ISO-NE’s reserve markets needed to speed up from seasonal to daily,⁸ ISO-NE’s peak forecasts needed to speed up from daily to hourly. The grid now moves too quickly for the old approaches to keep up, because when the electricity shows up is now a salient concern.

The future needs to think faster.

The future needs to think stochastically.

If you can’t differentiate between a 5th-percentile tail risk and a 1st-percentile tail risk, you’re screwed.

If you don’t build a data, forecasting, and AI strategy yesterday, you deserve to lose your shirt, job, and credibility in rapid succession.

---

This post and the information presented are intended for informational purposes only. The views expressed herein are the author’s alone and do not reflect those of their current or previous employers or any elected officials. The author makes no recommendations toward any electric utility, regulatory body, or other organization. While certain information contained herein has been obtained from sources believed to be reliable, the author has not independently verified this information, and its accuracy and completeness cannot be guaranteed. Accordingly, no representation or warranty, express or implied, is made as to, and no reliance should be placed on, the fairness, accuracy, timeliness or completeness of this information. The author assumes no liability for this information and no obligation to update the information or analysis contained herein in the future.

1. The higher of these two peaks defines how many literal poles and wires we need, but we need to track both because they involve different factors and thus will change in different ways. The summer peak (late afternoon in a heat wave) is driven by air-conditioning load and the wind-down of behind-the-meter solar, whereas the winter peak (early morning in a cold snap) is driven by heating load and a limited inflow of natural gas.

2. Interestingly, the new forecast pushes the date when New England shifts to a winter peaking system a few years forward, from early 2030s to late 2030s. Ultimately, the underlying numerical changes are under 5%. Call it noise, expect winter peaking around 2035, procure the countermeasures now.

3. Solar installers, generally, are about as honest as used car salesmen.

4. Now potentially at risk, but…solar installers hire good lobbyists.

5. Based : Gen Z :: Hardcore, wicked, sick : Millennials, Gen X

6. As opposed to OPEC, which has never played politics…

7. My nuclear take is that the two most important figures driving Western decarbonization are 1) Vladimir Putin and 2) the diving crew that blew up the Nord Stream pipelines.

8. More on that here: