Working smarter: the role of distributed AI and data collection in solar grid connectivity

This is to say nothing of regional variations that exist simultaneously. On the same day, the EIA reported that daily energy demand in New York ranged from 15.9-24.9GWh, while energy demand in the Midwest fluctuated from 81.5-122.8GWh, significant variations in regional energy demand made more complicated by the variations inherent to solar power, which is an increasingly integral part of the US energy mix.

According to the EIA, total US solar generation has increased from 89,199GWh in 2020 to 145,598GWh in 2022, a significant increase, and one that looms larger considering the US produced just 9,036GWh of solar power a decade ago, in 2013. While this accounted for just 3.4% of the US’ total energy production at the end of 2022, this is a growing sector, and ensuring optimal grid connections will be integral to the solar industry realising its potential.

In addition to the obvious needs of the world’s future energy grids, from building new grid infrastructure to optimising the permitting process for new projects, the use of nuanced, and in some cases automated, data collection and analysis could be of significant benefit for the sector.

Assessing how a grid is used, when its times of peak supply and demand take place and how this grid interacts with neighbouring power systems, could all improve the supply of power to people, companies and industries, and a smarter approach to data could be vital for the future of the world’s power grids.

Data collection and grid infrastructure

The importance of data in optimising grid operations is apparent not just in the US, but around the world. Figures from the EU suggest that sharing data regarding power demand and consumption across the EU’s power grid could unlock more than 580GW of flexible power generation, as the continent’s grids are better equipped to move power as supply and demand fluctuate.

“We need much better-quality data on the distribution grid, particularly at the grid edge where data is generally sparse, late, not contextualised and sometimes misleading,” adds Jess Melanson, COO of Utilidata, a US company which has worked with technology firm Nvidia to install artificial intelligence (AI) processes at power grids in the US states of Michigan and New York.

“We have to fix this quickly because the grid edge is becoming exponentially more complex as customers buy electric vehicles (EVs), distributed solar, batteries and heat pumps; outages are increasing; and the need to shift or shed energy demand during events like heat waves becomes critical to keeping the power on.”

Melanson’s comments are particularly significant considering the growth of power grids to meet the world’s energy needs. The EIA expects the US to add over 100GW of new power capacity in 2023 and 2024, while decommissioning around 30GW of coal and natural gas facilities, moves which will ultimately add to the US’ domestic electricity generation facilities, and require similar expansions in grid infrastructure and data collection.

“Walmart or Amazon aren’t guessing what’s in their warehouses; they know exactly where everything is and they focus their efforts getting it to you cheaper and faster,” adds Melanson, suggesting that the global energy industry needs to learn from other sectors, such as the technology industry, to better optimise its operations.

“In the energy industry, we’re still guessing what’s out there; where EVs are charging, who has battery storage, which customers have shiftable energy resources, or what is the real-time impact of distributed solar on power quality.”

Insights for the solar sector

This emphasis on data analysis, and data collection from distributed locations, could be of particular benefit for the solar sector, with Melanson suggesting that grid interconnections could stand to benefit.

“Take solar interconnection for example; because utilities need to estimate system conditions, interconnection takes a lot of analysis and assumes worst case scenarios, which leads to delays and costly infrastructure upgrades,” says Melanson. “Visibility into grid conditions and capacity in real-time, and the ability to interact with solar systems in real time, that would enable quicker and cheaper interconnection, less solar curtailment, and higher payments to solar for grid services.”

This is particularly pertinent considering recent research into delays in connecting new solar generation facilities to the US grid. Research from the Federal Energy Regulatory Commission found that inefficiencies in the US grid had left more than 1TW of generation capacity waiting to be connected to the US power system, of which 947GW was solar generation capacity alone.

As a result, more sophisticated grid operation could be a necessity, not just a benefit, of an energy mix in which solar power is more prominent.

“If grid planners and operators need to guess what’s happening and cannot influence it, they need to overbuild the grid, limit solar because of potential worst-case scenarios, and even revert to blackouts more quickly during resource shortages,” says Melanson, suggesting what could happen if grid operators do not integrate more effective forms of data collection.

“In contrast, a grid with distributed AI not only provides clear visibility, but can predict and react to various operational scenarios. This would improve planning and operations, save money by avoiding grid upgrades, and encourage both grid-scale and customer-sited renewables.”

The role of “distributed AI”

What Melanson calls “distributed AI” is among the key technological innovations necessary for solar to take on a larger role in the global energy mix. Utilidata and Nvidia’s smart grid chip is a component that, when added to existing smart power meters, can estimate a power grid’s capacity to take on more power in times of high demand, and improve the overall efficiency of an energy system by forecasting when components are likely to suffer from outages.

Melanson notes that Utilidata has already deployed the chip with the Lake Placid Municipal Electric utility in the state of New York, and the University of Michigan Transportation Research Institute, and plans to deploy more with Portland General Electric by the end of this year.

“The goal of these projects will be to show the power of a distributed AI platform, so we won’t just be looking at one use case but many use cases, such as improved real-time operations, increasing solar hosting capacity, and identifying and managing the impacts of EVs,” says Melanson.

A key benefit of components such as these is their distributed nature, enabling them to be used in newer or more remote parts of power grids, or on local-scale solar facilities that operate entirely independently from established energy grids.

This is important considering the predicted growth of such solar systems in the future. The International Energy Agency (IEA) expects global additions of distributed solar to reach 317-406GW between 2019 and 2024, a significant increase over the 139GW of distributed solar added between 2012 and 2018.

“Grid operational conditions at the edge will change in a matter of seconds, as passing clouds change solar output or utilities utilise virtual power plants. But it will be too slow and expensive to send all that seconds-level data back to a centralised system to manage,” adds Melanson, pointing to the benefits that distributed AI systems offer in terms of flexibility.

“That’s why utilities need more powerful distributed computation so the vast majority of that data never has to leave the edge, and some decision making can eventually move to the edge.”

The IEA also expects distributed solar account for between 45-46% of total global solar capacity by 2024, up from 36% by 2018, as the world’s solar mix looks set to be more balanced between utility-scale solar and distributed generation. As a result, investing in distributed AI systems for data collection and analysis could soon be of benefit not for a small part of the solar sector, but facilities that account for close to half of global solar capacity.

Challenges and step-changes

Yet, as is often the case with new technological innovations, existing infrastructure is often not designed to accommodate such new products, and Melanson suggests this could impede the large-scale adoption of distributed analytical technology.

“As an industry, we have generally not designed our infrastructure with enough consideration of data collection and software value,” says Melanson. “There are lots of reasons for this, from the way regulators evaluate grid modernisation projects to the types of solutions vendors offer to utilities.”

Melanson also pointed to “chronic under-investment” in data capture across power grids creating conditions where investing in such technologies is simply not financially attractive for grid operators. For instance, European electricity industry trade group Eurelectric concluded in 2021 that Europe’s power grids would need an additional €375-425 billion (US$407-462 billion) in investment until 2030 to meet the continent’s growing energy demands.

However, Melanson also suggests that this focus on work that is, first and foremost, financially viable, could be of detriment to the long-term health of the world’s power grids. Instead, Melanson argues that identifying what a grid needs, and working to deliver those changes, should be operators’ priority.

“While improving the business case for billing infrastructure is a move in the right direction, we’re not asking ourselves the right question, which is: what kind of grid edge data and computing infrastructure will be necessary to deliver safe, reliable, and affordable service on a dynamic, clean electric grid?”

Ultimately, the work of companies such as Utilidata could implement technological innovation, and deliver benefits in efficiency, for a global power sector that is likely to only increase its reliance on solar power. But, as Melanson suggests, implementing those innovations could require a change in how solar power, grid infrastructure and data analysis are perceived and understood.

“That lack of high-quality data was okay for the world we’ve been living in, but it’s untenable for the world we’re entering,” says Melanson. “We need to upgrade the data and compute infrastructure on the grid to manage this complexity.”