So long, and thanks for all the sunshine

But this darkness has proved more of an eclipse than a black hole over the past couple of years even as Mitt Romney (remember him?) attempted to keep this zombie issue alive during the presidential campaign. SolarCity's successful IPO later that year appeared – finally – to silence the critics and improve investor sentiment for downstream solar. As PV Tech’s senior news editor, Mark Osborne, points out, overcapacity issues seem to have stabilised in 2013 thanks to high demand and even equipment orders have increased. “The industry is ‘officially’ in an upturn and one that could easily turn into a major boom in 2014,” he writes.

However, the sun has taken its time to rise in this new dawn for solar. Are we on 2.0 or 3.0 already? Demand has driven the market along the rocky road, and promises a smoother ride with plenty of road ahead.

At the start of 2014, the US market is likely in excess of 10GW, most of which has been installed since 2010. So much for the impacts of the 2008 financial crisis and Solyndra. In fact, projects funded by the Department of Energy's 1705 loan guarantees have a 3% failure rate and account for around 10% of the total installed solar in the US.

While the utility gold rush seems to be all but over, residential installations appear to be on a seemingly inexorable rise thanks to third-party ownership models (at least until 2016). Last year, California's residential sector more than doubled its rooftop solar installations from 1,000MW to more than 2,000MW. But signs of robust markets are emerging in other, sometimes unlikely, states. Minnesota's solar industry again punched above its weight with a crucial win against natural gas last week. And with cheap natural gas from the US shale boom generating headwinds for the solar industry its place as a partnership fuel with renewables seems to be developing.

Even Duke Energy's new chief executive, Lynn Good, told the Financial Times this week that renewables and natural gas would lead new generation capacity in the US as nuclear was too expensive. That position is at odds with the new energy secretary, Ernest Moniz, whose enthusiasm for renewables seems temperate compared with his predecessor, Steven Chu.

In fact, even before he took up his post, Moniz promised early action on energy storage – a promise that sadly slipped until the end of last year.

Energy storage continues to be the Holy Grail for renewables – technology and deployment seems still stuck at pilot stage. But initiatives such as California's mandate for investor-owned utilities to procure 1.325MW should help stimulate a nascent market that could help re-energise commercial solar.

In the autumn of 2012, hurricane Sandy taught the local utility companies and policymakers a useful lesson about the value of smart grids and energy storage and how they could be used with PV to improve grid resilience. Most panels were not damaged, but they were isolated from the broken grid.

While in the western states, CSP has also taken a battering even as Abengoa's Solana plant added another 250MW in October 2013, and BrightSource's Ivanpah plant added 391MW. More nimble players, such as SolarReserve, have looked overseas for their growth opportunities. But it would seem that this generation of CSP, most of it backed by the government, has run its course in the US.

Appetite among developers and investors for mega-utility scale projects out in the desert has dwindled as federal support through the 1705 loans and 1603 grants have run down. Transmission and distribution costs are not cheap. So in the past couple of years, focus has turned to the rooftops over American businesses and homeowners.

The American Legislative Exchange Council (ALEC) failed in its attempt to crush or curb state Renewable Portfolio Standards [].

But the greatest threat to utility revenue is from the residential sector, and perhaps, in time, the commercial sector. Last year's running battles between Arizona Public Services, the state's largest utility company, and the solar industry culminated in November with the introduction of a fee for solar customers. The US$0.70 per kilowatt charge on solar net metering customers would average around US$5 a month, but neither APS nor the solar industry was satisfied with the decision from the Arizona Corporation Commission.

That's probably the result commissioners at ACC were aiming for. But the decision was seen as a testing principle for the rest of the country and battles over NEM are not likely to go away this year.

Utilities are seriously concerned that any demand growth – amid a sea of neutral or negative trends – will be absorbed by rooftop PV panels, sending them into an irreversible death spiral.

Duke Energy's Lynn Good is not the only one predicting “consolidation” through utility industry bankruptcies. We have seen it before in telecomms – could we see it again as the supply of electric power becomes more distributed and its management becomes “smarter” and more digitised? Those utilities that will thrive are those who tackle this issue head on and embrace solar.

Other players such as Google and Honda will continue to enter the market through the side door, for example through tax equity investments. Meanwhile some larger independent power producers, such as NRG Energy, have clearly made a statement of intent with renewables and have invested in an innovations unit, Station A in San Francisco.

While the utility and solar industries are often characterised as being at odds with one another, compromise and collaboration on both sides will be essential. This has already happened to some extent. Last year, despite protests against California's Assembly Bill 327, which also aims to charge solar customers a fixed fee, the solar industry ultimately welcomed the bill after eleventh-hour changes that provide greater certainty over NEM.

Meanwhile, SolarWorld continues to try to escalate the trade war with China by launching a fresh legal action to “finish the job”. But SolarWorld's minority view is likely to be a distracting sideshow, much to the frustration of trade associations such as the Solar Energy Industries Association. Most companies are happy to compete in a completely free market, SEIA believes.

But everyone can agree that there's still a lot of work to be done – with less than 1% of US electricity generated by solar. That's a challenge that Americans can't help but turn into an opportunity.

Last year's block party at Solar Power International was held in Chicago's Museum of Science and Industry, crammed with curios such as a replica of the Wright Brothers’ aircraft and installations including a Tesla coil. On the ground floor, guests could take a trip into a replica coal mine – a tribute to old technology that could gradually become obsolete thanks to solar, at least in the US.

On the bus on the way back to the hotel I met a psychologist who'd given up his New York practice to start a solar business. The following morning, I met an engineer from Egypt who was convinced that projects could still be developed in the country despite the challenging environment for investors because of political instability after the Arab spring. The only thing we three had in common was a belief that solar energy has the power to make the future much brighter.

It's been a privilege to have had a ringside seat watching the industry progress from its birthplace in Silicon Valley over the past couple of years. Wherever you are in this journey, I wish you well. Thanks for reading.