Originally posted by sensij
View Post
1) Is data also available from the SMA Sunny Portal, or only the Sunpower site?
2) Is data from the Sunpower site available from a direct API call, or are you scraping it from a page?
I use a perl script (running on a Raspberry Pi) that pulls the data from the SP site, massages it for how PVOutput likes it (notably for energy vs interval power, and also the date/time format), and then posts it to PVOutput using their posting API. I have a web front end to restart this perl script when it dies (which happens when the SP or PVOutput sites are down, or we lose the network (thank you Comcast...)); I am also making the script more robust as well, to get it so that it simply waits and re-tries when it encounters network issues.
I recently decided to go ahead and host the data directly, in addition to mirroring it to PVOutput, so the same perl script also collects it to local .csv files, and also aggregates daily/monthly/yearly totals. I then wrote a web interface that will download those .csv files, and present the results (both in a graph using SVG and a table) with interactivity, all using javascript on the client side. Since I'm hosting both the data-collection/cross-posting script and the web interface on the rPi, I needed to keep computation light, so I'm letting the client do the graphing, rather than doing it server-side. All modern browsers on reasonable systems (even my super-slow first-gen iPad) render the data in a responsive fashion. I'm pleased with the results.
Honestly, getting production and consumption from the Sunpower site, even with a 15 min delay, would be an adequate solution that would enough to publish to PVOutput, but if it is only scraped, it isn't likely to be very robust.
But I think I've had a breakthrough in the approach of monitoring the traffic now, using a man-in-the-middle approach, and will follow up in another post soon...
Comment