Comparing Production Estimates to Actual Production

[pleppik]

Inspired by this thread, I took a stab at comparing the actual production from my solar panels over the four months I have complete data against what the installer estimated the production would be. All the usual caveats apply, especially the fact that four months' of data isn't really enough to draw any conclusions from. But I found the results interesting nevertheless, so I figured I would share.

First Stab
So the first thing I did was simply compare the actual production in September through December against the monthly production estimates from the installer.

Solar-Analysis-1.gif

This shows that in September and October we produced a lot more power than estimated, and in November and December we produced less. It makes sense that we had less production in November and December, since we had a big run of cloudy weather those months, and one of the arrays was covered in snow for a significant amount of time during those months.

Second Stab: Snow Cover
On my house we have 16 solar panels arranged in two arrays. One of the arrays is over the garage and easily reachable with a brush on a long pole, the other array is three stories off the ground. Since the beginning of winter I've been keeping the garage array brushed off after every snowfall, and also keeping a log of which days the house array has been snow-covered.

Because I kept one of the arrays clean and the two arrays are monitored separately in our eGauge, it's possible to estimate how much production we lost because of snow cover (I used the measured production from the clean array to estimate what the snow-covered array would have produced if I had been able to brush it off).

Solar-Analysis-2.gif

This is showing that we lost a fair percentage of November/December production to the snow cover (about 25% and 20%, respectively). This is not surprising, and probably pretty typical for a Minnesota winter. Up here, after it snows it tends to get very cold for a while, and from what I've observed so far this winter it needs to get up to about 25F and sunny before the snow and ice will loosen and slide off the array. It's possible to go weeks after a snowstorm before we get weather that warm.

Last year before signing the installation contract, I asked the installer how he accounted for snow on the panels. His response was that he just applied a fudge factor and reduced the annual production by a few percent. That's not a very satisfying approach, but maybe the best you can do given how unpredictable snow cover is.

Third Stab: Cloudy Weather
In this post, sensij provided a link to a nearby weather station where I could download solar radiation data since mid-2007. This gave me a way to try to figure out how much of the deviation between our actual production and the estimate might be due to the cloudy weather we had in November and December.

So I downloaded all the insolation data from that weather station and calculated monthly averages for 2007-2014, and used that to estimate how much more or less solar radiation we received in each month of 2014 relative to the average of the prior seven years.

Solar-Insolation-History.gif

This estimate is fairly rough because (a) I only had seven years' of data for calculating the historical average (20-30 years is more typical for calculating climactic averages), (b) the weather station is 15 miles away so may have gotten somewhat different weather, and (c) I didn't even attempt to account for any time of day effects or anything more granular than the monthly average insolation.

But it did give me a way to quantify how much cloudier/sunnier each month was relative to the longer term average. So I used that to adjust the installer's original estimate, by just multiplying the production estimate by the percentage of average radiation received. So for example, in October when we received 112% of the average radiation, I multiplied the installer's estimate of 352 KWh by 112% to get an adjusted estimate of 395 KWh.

Solar-Analysis-3.gif

What Does It Mean?
Four months' of data isn't enough to draw any conclusions, so this is really more a fun exercise in data analysis than any kind of realistic assessment of how accurate my installer's estimate was. Nevertheless, here's a summary:

Month	Installer's Estimate (KWh)	Installer's Estimate Adjusted for Weather (KWh)	Actual Production (KWh)	Est. Production with No Snow (KWh)	"No Snow" Production / Adjusted Installer's Estimate (%)
September	548	502	654	654	130%
October	352	395	519	519	131%
November	198	178	161	212	119%
December	119	85	93	114	134%

It seems pretty clear to me that my solar panels are producing a lot more power than the installer's original estimate. In September and October, when there was no snow cover, we outperformed by a huge margin even though September was cloudier than average. In November and December we produced less than the estimate, but still a lot more than I would have expected given the cloudiness and snow. Even with all the assumptions and approximations I made in this analysis, generating 30% more power than expected is a big gap.

[bcroe]

Inspired by this thread, I took a stab at comparing the actual production from my solar panels over the four months I have complete data against what the installer estimated the production would be. All the usual caveats apply, especially the fact that four months' of data isn't really enough to draw any conclusions from. But I found the results interesting nevertheless, so I figured I would share.

First Stab
So the first thing I did was simply compare the actual production in September through December against the monthly production estimates from the installer.

[ATTACH=CONFIG]5479[/ATTACH]

This shows that in September and October we produced a lot more power than estimated, and in November and December we produced less. It makes sense that we had less production in November and December, since we had a big run of cloudy weather those months, and one of the arrays was covered in snow for a significant amount of time during those months.

Second Stab: Snow Cover
On my house we have 16 solar panels arranged in two arrays. One of the arrays is over the garage and easily reachable with a brush on a long pole, the other array is three stories off the ground. Since the beginning of winter I've been keeping the garage array brushed off after every snowfall, and also keeping a log of which days the house array has been snow-covered.

Because I kept one of the arrays clean and the two arrays are monitored separately in our eGauge, it's possible to estimate how much production we lost because of snow cover (I used the measured production from the clean array to estimate what the snow-covered array would have produced if I had been able to brush it off).

[ATTACH=CONFIG]5480[/ATTACH]

This is showing that we lost a fair percentage of November/December production to the snow cover (about 25% and 20%, respectively). This is not surprising, and probably pretty typical for a Minnesota winter. Up here, after it snows it tends to get very cold for a while, and from what I've observed so far this winter it needs to get up to about 25F and sunny before the snow and ice will loosen and slide off the array. It's possible to go weeks after a snowstorm before we get weather that warm.

Last year before signing the installation contract, I asked the installer how he accounted for snow on the panels. His response was that he just applied a fudge factor and reduced the annual production by a few percent. That's not a very satisfying approach, but maybe the best you can do given how unpredictable snow cover is.

Third Stab: Cloudy Weather
In this post, sensij provided a link to a nearby weather station where I could download solar radiation data since mid-2007. This gave me a way to try to figure out how much of the deviation between our actual production and the estimate might be due to the cloudy weather we had in November and December.

So I downloaded all the insolation data from that weather station and calculated monthly averages for 2007-2014, and used that to estimate how much more or less solar radiation we received in each month of 2014 relative to the average of the prior seven years.

[ATTACH=CONFIG]5481[/ATTACH]

This estimate is fairly rough because (a) I only had seven years' of data for calculating the historical average (20-30 years is more typical for calculating climactic averages), (b) the weather station is 15 miles away so may have gotten somewhat different weather, and (c) I didn't even attempt to account for any time of day effects or anything more granular than the monthly average insolation.

But it did give me a way to quantify how much cloudier/sunnier each month was relative to the longer term average. So I used that to adjust the installer's original estimate, by just multiplying the production estimate by the percentage of average radiation received. So for example, in October when we received 112% of the average radiation, I multiplied the installer's estimate of 352 KWh by 112% to get an adjusted estimate of 395 KWh.

[ATTACH=CONFIG]5483[/ATTACH]

What Does It Mean?
Four months' of data isn't enough to draw any conclusions, so this is really more a fun exercise in data analysis than any kind of realistic assessment of how accurate my installer's estimate was. Nevertheless, here's a summary:

Month	Installer's Estimate (KWh)	Installer's Estimate Adjusted for Weather (KWh)	Actual Production (KWh)	Est. Production with No Snow (KWh)	"No Snow" Production / Adjusted Installer's Estimate (%)
September	548	502	654	654	130%
October	352	395	519	519	131%
November	198	178	161	212	119%
December	119	85	93	114	134%

It seems pretty clear to me that my solar panels are producing a lot more power than the installer's original estimate. In September and October, when there was no snow cover, we outperformed by a huge margin even though September was cloudier than average. In November and December we produced less than the estimate, but still a lot more than I would have expected given the cloudiness and snow. Even with all the assumptions and approximations I made in this analysis, generating 30% more power than expected is a big gap.

With the method worked out, that chart might get extended indefinitely.

The first couple months your production pretty much tracks mine. The next 2 we in NW IL also
had nearly continuous overcast, but your output dropped about twice as much. Could be your
snow; we only had one, which was cleared immediately.

Might again try to run the electrical defrost experiment this winter. Like, set up a couple spare
panels by the back door, one vertical and one angled. After a storm, see if either can be
electrically warmed enough for the snow to slide off. Bruce Roe

[pleppik]

With the method worked out, that chart might get extended indefinitely.

The first couple months your production pretty much tracks mine. The next 2 we in NW IL also
had nearly continuous overcast, but your output dropped about twice as much. Could be your
snow; we only had one, which was cleared immediately.

Might again try to run the electrical defrost experiment this winter. Like, set up a couple spare
panels by the back door, one vertical and one angled. After a storm, see if either can be
electrically warmed enough for the snow to slide off. Bruce Roe

Bruce:

The difference between your results and mine could be the snow. It could also be that it actually was cloudier here than where you are since we are several hundred miles apart. It could also be due to the shading on our panels, which is substantial in the winter months. During November, December, and January the sun only barely makes it above the trees around the house. Our site assessment showed that in the month of December, one of our arrays is 47% shaded and the other is 88% shaded.

On the flip side, the TenK modules seem to be performing as advertised as far as being tolerant of partial shade. The tree which is causing the 88% shading on one array doesn't have any leaves this time of year, and that array isn't so much shaded as receiving dappled sun. We're getting a respectable amount of power from that array despite the partial shade--at least, on those rare days when the sun has been shining.

[pleppik]

Actual Production vs. Estimates for January

I extended my graphs to include January:

Actual production vs. Installer's Estimate

We generated 199 kWh in January. The installer's original estimate was 182 kWh for January, so we exceeded the estimate by a few percent.
Solar-Analysis-1.gif

Snow Cover

We had a few light snowfalls during January; a lot less snow than usual, but enough to cover the panels a significant amount of the time. Comparing the production from my two arrays (one is easy to sweep clean, the other is inaccessible), I estimated that I lost about 30 kWh of production in January because of snow on the roof. Since the installer didn't explicitly account for snow in making his estimates, I'm adding this loss back in to get more of an apples-to-apples comparison.
Solat-Analysis-2.gif

Weather

According to the data downloaded from a local weather station, January was a little cloudier than usual. The actual solar radiation received during January was 89% of the average for the past six years. This graph adds a line for the installer's original estimates adjusted for the weather data.

Solat-Analysis-3.gif

Putting it Together

Finally, I calculated our actual PV production as a percentage of the weather-adjusted estimate. In every month so far, our actual PV production has been 20% to 40% higher than expected, once you add back the production we lost due to snow cover.

Soalt-Analysis-4.gif

It seems fair to say that our installer's estimates for PV production were extremely pessimistic. 30% is a lot to underestimate by, though I suppose pessimistic estimates are a good way to prevent customer complaints when there's a long run of cloudy weather.

For me this is good news. 30% higher production means the system pays for itself by Year 8 instead of Year 10.

[sensij]

For me this is good news. 30% higher production means the system pays for itself by Year 8 instead of Year 10.

Thanks for sharing your analysis. It sounds like you are respecting the limits of the data but are still able to draw some high level conclusions about the system performance.

With respect to the over production, this is a good reason why sizing to offset less than 100% of consumption can be a good idea. Solar production will vary from year to year, and if the system is sized large then the extra in the good years is essentially wasted, instead of contributing to the roi.

[Solarnemo]

Thanks for sharing your analysis. It sounds like you are respecting the limits of the data but are still able to draw some high level conclusions about the system performance.

With respect to the over production, this is a good reason why sizing to offset less than 100% of consumption can be a good idea. Solar production will vary from year to year, and if the system is sized large then the extra in the good years is essentially wasted, instead of contributing to the roi.

Great evaluation- you mention eGauge as your monitoring system. Does this let you look at each panel or the entire systems output? The installation post advised that the panels have some electronic smarts to them.

[pleppik]

Great evaluation- you mention eGauge as your monitoring system. Does this let you look at each panel or the entire systems output? The installation post advised that the panels have some electronic smarts to them.

The eGauge doesn't allow panel-level monitoring. The eGauge only monitors the AC side, and the TenK design uses a DC bus combining an entire array to feed multiple microinverters. But I do have my two arrays with independent monitoring.

I've heard rumors that the built-in electronics of the TenK modules actually do have some sort of monitoring capability, but the company hasn't made that available to customers (yet). It's plausible, but I have no idea if it's true.

[control4userguy]

^Not entirely correct. eGauge can handle DC current as well as AC current and the voltage (LI & L2) that powers the 3000.

[pleppik]

^Not entirely correct. eGauge can handle DC current as well as AC current and the voltage (LI & L2) that powers the 3000.

Let me rephrase that: In my installation, the eGauge doesn't give me panel-level monitoring, just the AC side. I didn't mean to imply that the eGauge could not be installed for DC monitoring--I should have written that "My eGauge doesn't have panel-level monitoring."

Panel-level monitoring would be nice, but my installer didn't even suggest the idea. Given that it would have required two additional voltage sensors and 18 more current sensors, and the standard eGauge only has 12 inputs, I'm not sure how this would have worked. I don't know if eGauges can be stacked or extended somehow for additional inputs.

[control4userguy]

^Multiple eGauge 3000s can be used together. I run three here. If you had purchased the 64 register option then this would open you up to panel monitoring but it would come at a premium cost & wiring complexity. IMO, not worth it.

[Solarnemo]

Egauge monitoring.

Let me rephrase that: In my installation, the eGauge doesn't give me panel-level monitoring, just the AC side. I didn't mean to imply that the eGauge could not be installed for DC monitoring--I should have written that "My eGauge doesn't have panel-level monitoring."

Panel-level monitoring would be nice, but my installer didn't even suggest the idea. Given that it would have required two additional voltage sensors and 18 more current sensors, and the standard eGauge only has 12 inputs, I'm not sure how this would have worked. I don't know if eGauges can be stacked or extended somehow for additional inputs.

Sounds like there is not much written about the eGauge system. Saw some of the input modules on the eGauge web site, but how the reporting system works and what formats it can monitor would take a savvy experienced controls person to figure out. It appears to some kind of data logging system, but how to program the system may take a detailed manual and a lot of time to figure out.

I would expect that some of the large commercial sites should have the capability to see each panel output. Would not want to go to physically look at several hundred modules on a site.

[control4userguy]

Sounds like there is not much written about the eGauge system. Saw some of the input modules on the eGauge web site, but how the reporting system works and what formats it can monitor would take a savvy experienced controls person to figure out. It appears to some kind of data logging system, but how to program the system may take a detailed manual and a lot of time to figure out.

I would expect that some of the large commercial sites should have the capability to see each panel output. Would not want to go to physically look at several hundred modules on a site.

I'm not sure what you mean by "formats". EG3000 is a data logger looking at mV and V. The UI is a rolling line graph. That's it. "Programming" is not needed. Installation tab allows channel designation/naming and calibration. Takes all of 10 mins for 12 hardware channels. This was never designed to complete nor replace with the likes of Enphase or others that visually represent module-level performance.

[pleppik]

Sounds like there is not much written about the eGauge system. Saw some of the input modules on the eGauge web site, but how the reporting system works and what formats it can monitor would take a savvy experienced controls person to figure out. It appears to some kind of data logging system, but how to program the system may take a detailed manual and a lot of time to figure out.

Not really. The eGauge is intended for consumer use (with professional installation), and is pretty simple to set up and use. Once set up, you view/download the data through a simple web-based interface. Here's mine: http://egauge14368.egaug.es

The documentation stinks, though. When we set up the eGauge in my home, it took the installer longer to wade through the manual than it did to actually set it up. Programming the registers took maybe 15 minutes, but we spent at least twice that amount of time trying to make sense of the instructions. I'm convinced that the instructions were written by a PhD in electrical engineering who only speaks English as a third or fourth language.

[control4userguy]

Not really. The eGauge is intended for consumer use (with professional installation), and is pretty simple to set up and use. Once set up, you view/download the data through a simple web-based interface. Here's mine: http://egauge14368.egaug.es

The documentation stinks, though. When we set up the eGauge in my home, it took the installer longer to wade through the manual than it did to actually set it up. Programming the registers took maybe 15 minutes, but we spent at least twice that amount of time trying to make sense of the instructions. I'm convinced that the instructions were written by a PhD in electrical engineering who only speaks English as a third or fourth language.

As far as I'm concerned, the only ambiguity I found was how to treat the channel as a producer or consumer. And, even then, it's up to how you want the info displayed. Red or green, positive or negative value. If you have an electrician there, he'she will tell you what leg the clamp is on and explain how to measure a 240VAC circuit with just one CT. I have used tech support at eGauge a few times and they are very helpful and will remote-in to check your configuration if you need. Consumer grade? Well, if you love LabView I suppose you could use that but why? Anything else would most likely have a very steep learning curve compared to this.

One thing I have noticed and their tech guys raise a brow over is my calibration. I happen to have lab-grade I measurement hardware here and I can say, for the critical channels (inverters) you would do well to run a custom curve. Three instuments all reading the same and the eGauge not is pretty compelling.

[pleppik]

I happen to have lab-grade I measurement hardware here and I can say, for the critical channels (inverters) you would do well to run a custom curve. Three instuments all reading the same and the eGauge not is pretty compelling.

eGauge claims that they are "revenue grade" (whatever that means), so it's interesting to hear that the accuracy may not be all that.

For me, I'm OK as long as it's within 5% or so. I use it mainly to see how much power we're using to help convince the kids to turn the lights off, and also so I've got something interesting to watch on the solar panels. After all, solar panels are kind of boring and they mostly just sit there.