I won't go into the details of their posting protocol here, but let's just say that all the data from the WattNode net meter and the SMA inverter are available (as well as some meta-data from the SP supervisor). Of these, the most relevant values are the "Inverter Interval Energy Produced(kWh)" (i.e. the "interval production energy"), and the "Net Meter Total Lifetime Energy(kWh)". The latter is only useful when you compare t0 with t-1, and that delta yields the interval net meter energy (i.e. the "interval net energy"). So then the "interval consumption energy" is just the "interval production energy" + the (potentially negative) "interval net energy". Now with both interval consumption and production values, I've got what I need.
So I wrote a perl script (using the Net::Pcap and Network::Packet::{Ethernet,IP,TCP,UDP} modules) that can observe the traffic, and extract the "Inverter Interval Energy Produced(kWh)" and "Net Meter Total Lifetime Energy(kWh)" values, to produce the "interval production energy" and "interval net energy" values (the latter can only be had after the second set of data posts, since it is a delta-from-previous). I should then (haven't gotten here yet) be able to use that to trigger my cross-posting script (I do not want to do any heavy-lifting in the pcap packet callback, and perl doesn't really support multi-threading very well), which will then collect the data into local files for local hosting (see details in my previous reply) and cross-post to PVOutput. Worst comes to worst, I can simply collect the data into a file, and have the cross-posting script poll that file for new entries; this will still be quicker than polling the SP site, since I can use a much shorter polling interval.
Now the MitM approach to tapping/observing the packets is annoying, probably has performance implications, and will take the network down if the host goes down, so I'm not too keen on that. Furthermore, it requires two _additional_ network interfaces (since I'm using the main one for hosting and cross/posting, and the bridge requires dedicated NICs), which not only means two additional USB dongles consuming both USB ports on my rPi, but much greater (well, relatively) power consumption for the rPi... I'm looking into network switches that support mirroring of selected ports, or even a simple old-fashioned hub (if I can find one), since true hubs mirror all traffic to all ports, or even a simple tap. That way my rPi can coexist on the network with the SP powerline adapter, and not interfere with it, just observe the traffic. It can also use the same NIC for both observing the packet flow, and for hosting and cross-posting the data. The only caveat is that the NIC will be running in promiscuous mode to allow for observing the traffic, and I'm not sure if that will result in performance issues for the poor little rPi...
I'll post more as I make more progress, in case there is anyone still reading this...
I will not post the details of the SP protocol (or where the traffic is going), as it's possible SP might consider that intellectual property or something, and I am also keen to not piss off the "powers that be" that own/moderate solarpaneltalk.com. Sniffing the traffic and reverse-engineering the protocol are neither illegal nor unethical, but posting the details of what I've found is grey enough that I think it wise to avoid... 
However, I can say that it's fairly straight-forward to replicate what I've done, so anyone handy with the technologies involved can do so on their own.
).
) website only allows downloading the data that is currently being presented in the graph, and only 4 parameters at a time, and you have to manually click on the export button to get it. There is no URL to curl programmatically to get the data (the flash-based data exchange is tied up with their authentication model).
Leave a comment: