Announcement

Collapse
No announcement yet.

Mirroring/intercepting SunPower Monitoring Traffic?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by intipower View Post
    Anyone know when the software on the PV supervisor was updated again. I had not noticed it went from build 325 to "SWVER": "5.0.0, Build 327",
    My PVS doesn't appear to auto-update. They set the CT scaling wrong and when connected to fix it an update to 327 started.

    Comment


    • Originally posted by intipower View Post
      Anyone know when the software on the PV supervisor was updated again. I had not noticed it went from build 325 to "SWVER": "5.0.0, Build 327",
      My PVS doesn't seem to update automatically. The install company set the CT scaling to the wrong value and when I connected to fix it on 23-Jan the system updated to 327.

      Comment


      • Originally posted by ehampshire View Post
        Ack! For some reason I didn't get an email alert there were updates to this thread. I have lost my monitoring via the tcpdump/snooping method as well. My direct queries to http://sunpowerconsole.net/cgi-bin/d...and=DeviceList also changed the data format, so I'm pretty sure our PV supervisors got a firmware update. I've updated my direct query scripts to parse the new data properly, I will give an explanation below. I thought I'd share my PV's version:

        "supervisor":
        {
        "SWVER":"5.0.0, Build 325",
        "SERIAL":"ZT162585000441C1402",
        "MODEL":"PVS5"
        }
        }

        So, I'm not sure if they turned on TLS/SSL or why we lost all our data. I think it may have switched over to using the cellular connection - I can confirm my PV supervisor has a SIM card and is able to communicate over the cell network. I no longer see 140 or 130 messages coming across in the clear. But, at least direct query still works so I can get my production values (I still haven't worked out consumption on this). Here's the new format:

        PV monitor (140 line):
        {
        "ISDETAIL":"1",
        "SERIAL":"PVS5M508095c",
        "TYPE":"PVS5-METER-C",
        "STATE":"working",
        "STATEDESCR":"Working",
        "MODEL":"PVS5M0400c",
        "DESCR":"Power Meter PVS5M508095c",
        "DEVICE_TYPE":"Power Meter",
        "SWVER":"4",
        "PORT":"",
        "DATATIME":"2017,06,30,14,56,51",
        "ct_scl_fctr":"125",
        "net_ltea_3phsum_kwh":"-640.39",
        "p_3phsum_kw":"-1.8784",
        "q_3phsum_kvar":"0.5875",
        "s_3phsum_kva":"2.0794",
        "tot_pf_rto":"-0.9033",
        "freq_hz":"60.02",
        "CAL0":"125",
        "CURTIME":"2017,06,30,14,57,04"
        },

        Inverter example (130 line):
        {
        "ISDETAIL":"1",
        "SERIAL":"414051637006465",
        "TYPE":"SOLARBRIDGE",
        "STATE":"working",
        "STATEDESCR":"Working",
        "MODEL":"AC_Module_Type_C",
        "DESCR":"Inverter 414051637006465",
        "DEVICE_TYPE":"Inverter",
        "SWVER":"951007408",
        "PORT":"",
        "MOD_SN":"",
        "NMPLT_SKU":"",
        "DATATIME":"2017,06,30,14,56,21",
        "ltea_3phsum_kwh":"284.4779",
        "p_3phsum_kw":"0.107",
        "vln_3phavg_v":"247.3326",
        "i_3phsum_a":"0.4554",
        "p_mpptsum_kw":"0.112",
        "v_mppt1_v":"56.2324",
        "i_mppt1_a":"2.0239",
        "t_htsnk_degc":"30.75",
        "freq_hz":"60.042",
        "CURTIME":"2017,06,30,14,57,04"
        },

        So, because I'm lazy here's the copy/paste from my code showing what I used to be parsing (commented out with the # sign) to what the values now are:

        #insert_line.append(x["ac_curr"])
        insert_line.append(x["i_3phsum_a"])

        #insert_line.append(x["ac_power"])
        insert_line.append(x["p_3phsum_kw"])

        #insert_line.append(x["ac_volt"])
        insert_line.append(x["vln_3phavg_v"])

        #insert_line.append(x["dc_curr"])
        insert_line.append(x["i_mppt1_a"])

        #insert_line.append(x["dc_power"])
        insert_line.append(x["p_mpptsum_kw"])

        #insert_line.append(x["dc_volt"])
        insert_line.append(x["v_mppt1_v"])

        #insert_line.append(x["energy_total"])
        insert_line.append(x["ltea_3phsum_kwh"])

        #insert_line.append(x["heatsink_temp"])
        insert_line.append(x["t_htsnk_degc"])


        I've uploaded what I have for parsing to the capture folder of my project here: https://github.com/ehampshire/sunpower/tree/dev/capture . It's called sp_monitor_json2.py and has a conf file (sp_monitor_json.conf). The way I reach my PV monitor is through a Raspberry Pi in my garage hooked into the installer port on my PV supervisor. This Pi then bridges to my wireless network and I put in a static route on whatever machine I want to reach it through, like so:

        echo "Adding static route to 172.27.153.0 netmask 255.255.255.0 via 192.168.1.163"
        route add -net 172.27.153.0 netmask 255.255.255.0 gw 192.168.1.163
        I am presently looking into getting a SunPower Equinox system.

        Can anyone confirm if the above workaround still works? I did see some posts mentioning they changed something that made getting detailed stats impossible.

        Thanks!

        Comment


        • I also have a SunPower solar system - 12 panels that produce great energy in Southern California. I was trying to implement @@JJNorcal's solution to essentially intercept the traffic from the PV. I believe i identified it on the network because it's using the HF-LPB100 wireless NIC. Thanks to great instructions on setting up the Raspberry as a bridge, I got everything up and running fairly easily. I believe I have configured the settings on the application correctly but i am not seeing anything recorded. locally or at PVOutput.

          ehampshire, I am reasonably adept at Python so might give your approach a try. Thanks for documenting all of this. Have you been able to capture data via your methods?

          Comment


          • To confirm things on my end I reconnected my supervisor to WiFi and now see traffic to several IP addresses. Most appear to be DNS resolvers, but 50.112.130.148 and 52.7.213.242 are Amazon Web Services. One of note is this from tshark: 192.168.1.3 → 50.112.130.148 TLSv1.2 432 Application Data. This would imply that the traffic is indeed now encrypted. Also, on reconnecting to WiFi, my supervisor updated to Firmware 2018.4 Build 1092 which appears to be the latest.

            Comment


            • Originally posted by RedTechie View Post

              I am presently looking into getting a SunPower Equinox system.

              Can anyone confirm if the above workaround still works? I did see some posts mentioning they changed something that made getting detailed stats impossible.

              Thanks!
              This still worked for me as of 5/18 on the April firmware.

              However, now that I'm on the May firmware "Version 2018.5, Build 2039" aka "v2018.05.32", I keep seeing this error:
              Code:
              {
                ...
                "controller": "ConfigListController",
                "url": "/devices",
                "call": "/cgi-bin/dl_cgi?Command=GridProfileGet",
                "data": {
                  "result": "error",
                  "msg": "Could not connect to mime.",
                  "success": false
                },
                "method": "GET"
              }

              Comment


              • Originally posted by RedTechie View Post
                I am presently looking into getting a SunPower Equinox system.

                Can anyone confirm if the above workaround still works? I did see some posts mentioning they changed something that made getting detailed stats impossible.

                Thanks!
                This still worked for me as of 5/18 on the April firmware.

                However, now that I'm on the May firmware "Version 2018.5, Build 2039" aka "v2018.05.32", I keep seeing this error:
                Code:
                {
                  ...
                  "controller": "ConfigListController",
                  "url": "/devices",
                  "call": "/cgi-bin/dl_cgi?Command=GridProfileGet",
                  "data": {
                    "result": "error",
                    "msg": "Could not connect to mime.",
                    "success": false
                  },
                  "method": "GET"
                }

                Comment


                • Originally posted by boing View Post
                  I had drawn conclusions similar to intipower's findings. (I wish I had seen this thread first before having to discover it on my own!) A couple of additional/similar findings are below. I'm wondering if my supervisor is a bit different than yours based on what I'll highlight below.

                  First, this is what mine looks like. Note the two LAN ports on the top right. Is that how all of yours are too?

                  Am I overlooking anything, or does that seem to be the best approach?
                  boing,

                  All the SunPower Supervisor RS-485 info is in the first 5 or so pages, then everything changes to pvs5. Look for robillard astroboy and of course sensij. It is a moving target and SP keeps making changes.
                  19 SPR X21 345 / SMA 6000TL-US22

                  Comment


                  • it is interesting to me that my system keeps chugging along with the script i wrote so long ago. i am not sure why other people are getting firmware updates but i am not; i have not installed any kind of firewall rules for the supervisor so i feel like it should be able to download whatever it wanted to. maybe it is a misconfiguration by the installer? at any rate i'm OK with it!

                    Comment


                    • I am trying to see why these seemingly great solutions are not working for me. Can someone send me a tcpdump statement they used to find the post data? Here is what i have tried and the only thing returned is below.

                      I am fairly certain the IP of my SunPower supervisor is 192.168.1.3; the not dst statements are to eliminate a VERY chatty DNS lookup

                      tcpdump -vvv "src 192.168.1.3 and not dst google-public-dns-b.google.com and not dst resolver2.opendns.com and not dst google-public-dns-a.google.com"
                      tcpdump -s 0 -A 'src 192.168.1.3 and tcp dst port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)'

                      I am only seeing the 100/102 messages - i have been monitoring for an hour during daylight - overcast, but still generating some power.

                      100 SPMS 10 ZT163185000441C1526 20180525004318
                      102 AiF/Gs2xWTn4EpZdZF17

                      Comment


                      • Originally posted by cord_thomas View Post
                        I am only seeing the 100/102 messages - i have been monitoring for an hour during daylight - overcast, but still generating some power.

                        100 SPMS 10 ZT163185000441C1526 20180525004318
                        102 AiF/Gs2xWTn4EpZdZF17
                        Yeah, unfortunately, around June 25th of last year (2017), Sunpower rolled out new firmware that uses a different site, and encrypted data traffic for their data uploads. That apparently applied to all micro-inverter-based installs, so our main house got hit by that, and I have no way to retrieve per-panel data now. On May 11th of this year (2018), this firmware update also hit our rental house (which uses DC from the panels to a single inverter), as well. Now the supervisor still pings the sunpower server using the old protocol, but that's just the 100/102 message (which appears to just be an "I'm alive and OK" message). All other traffic goes to a different server, and is encrypted. Very, very unfortunate. SunPower just does not realize that, as system owners, this is _our_ data, not theirs...

                        Comment


                        • To be clear: you can still use the older method of simply downloading the data from the sunpower site using their web API (well, it's not an API, but it's usable), and I continue to do that to cache the data locally, but for those of us with micro-inverter based systems, that data does not include the very useful per-panel data...

                          Comment


                          • Thank you robillard - that is what i thought I understood from the thread but wanted to be sure. I posted an issue to ehampshire's Github project and he said he's still able to pull data from the supervisor - maybe his firmware hasn't been updated? I hadn't even thought to get the per-panel generation data. I want data locally to blend it with my consumption data and occupancy information to understand how my generation/consumption match up to better inform some automation of how my various appliances operate, car charging time, etc. Also wanted to see how my panels' performance aligned with the solar situation: https://github.com/SunPower/SolarUtils.

                            I am looking for documentation then on their "API" - i found this https://github.com/jeffkowalski/sunpower project which points to some "help" files. What is not clear to me is whether you can get more than just System cumulative and interval energy production (which is all I really need) or the stats on the supervisor and inverter as well - all the data that is presented in the sonpowermonitor.com site.

                            Thanks again

                            Comment


                            • robillard,

                              My system is the same as yours
                              https://www.solarpaneltalk.com/forum...342#post240342
                              No consumption monitoring though, but I'm assuming the same PV supervisor model, as we likely installed around the same time (as joining this forum).

                              My PV Supervisor has two Ethernet jacks and they only use one for the powerline. In my comm box where they mounted only the PV super. they of course had to mount a standard elec. outlet to connect the powerline to, mine happens to be a dual outlet. Is there a reason I can't simply buy my own powerline and use the second PV super. port, (since their powerline is encrypted) and get the data that way?

                              Thanks,
                              LLB
                              19 SPR X21 345 / SMA 6000TL-US22

                              Comment


                              • Originally posted by LLB View Post
                                My PV Supervisor has two Ethernet jacks and they only use one for the powerline. In my comm box where they mounted only the PV super. they of course had to mount a standard elec. outlet to connect the powerline to, mine happens to be a dual outlet. Is there a reason I can't simply buy my own powerline and use the second PV super. port, (since their powerline is encrypted) and get the data that way?
                                The problem is not the powerline. I bypassed the powerline (disconnected it at the other end) and have it provisioned to use the Ethernet port.

                                the problem is that SP changed where and how they transmit the data to their server. They no longer send messages to SunPowers server in the clear. Instead they use an AWS-hosted service to receive the data, and they send the data encrypted to that server.

                                so no, changing from powerline to the hardwired Ethernet will make no difference.

                                Comment

                                Working...
                                X