Announcement

Collapse
No announcement yet.

Mirroring/intercepting SunPower Monitoring Traffic?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • It may be that the built-in cellular has taken over. When I spoke with SP support, they indicated that cellular will function for up to 4 hours (as per my memory). They asked me to leave my powerline disconnected for the duration while they researched wifi status.

    Comment


    • I now blocked 52.7.213.242 (which is the AWS machine I saw traffic going to) as well as 204.194.111.66 which is the collector end point. I am still seeing usage and generation data being reported by the SunPower website. The updates are on the hour now, not every 5 minutes. So, it cannot be using WiFi as I do not see any unrecognized devices. Furthermore, all traffic on the FireWall has been blocked to the above two IP addresses. I am starting to think that there is some sort of a low-band wireless device built into the unit such that it can still deliver stats, but only once per hour.

      Comment


      • Does that mean 4 hours per day or 4 hours total or 4 hours of actual sending of data (which could take quite a bit of time to exhaust as it probably takes a couple of seconds to send the stats for an hour)?

        What phone number did you call to reach SP? I had my system purchased so I think my options are to either chat with them via the web-site or talk with my installer.

        Interesting device... wondering if I should wrap it with tin foil...

        This is the unit I have: http://www.southcoastsolar.com/docum...-Datasheet.PDF. According to the specs it has 3G UMTS.
        Last edited by apara; 07-01-2017, 12:39 PM.

        Comment


        • Well, evidence is mounting that SP stopped sending clear text for you. Can you think of any reason why you might be on the cutting edge with them? Recent purchase maybe? Maybe related to whether or not signed up for consumption monitoring (I did not). Sure wish we could figure out firmware revisions.

          I wonder what will happen if you block just 52.7.213.242, forcing exclusive path through of 204.194.111.66.

          But I'm not sure that blocking specific IP addresses is definitive. Load balancing could leverage other IPs.

          I too purchased panels, but SP was also my dealer/installer. I called the residential number listed at sunpower.com/support. They told me that I could send email, can't remember if it was supposed to go to technicalsupport or customersupport@sunpower.com.

          I took the 4 hours to mean a hard continuous limit, though I can't be certain. I was under the impression that it was a 3G cost issue for them.

          Given the flaky nature of my powerline, I'm surprised that your installer didn't provide a second network option. You might want to start with them. Tell them you've noticed networking issues in your house affecting powerline and that you're concerned about the integrity of your data.

          Comment


          • Our install was done June 6th this year. So a fairly new install. Plugged in by power block back and have been monitoring traffic with WireShark. I did exactly what you suggested. Blocked any AWS servers (found 3 so far) that the unit tries to talk with while leaving 204.194.111.66 open. The only traffic I continue to see via 204.194.111.66 is the 100/102 reporting.

            Looks like the unit does support "Automatic Firmware Upgrades".
            Last edited by apara; 07-01-2017, 03:54 PM.

            Comment


            • I played most of the day today trying to figure out how this is working. I did notice the cellular kick in when the broadband was disconnected. I was able to block with tinfoil around my unit. I also was able to create the firewall rules to block access to port 8883 which it appears to be using now to report via an encrypted method. If I used Block on the firewall, after a few minutes of trying the network unit inside the monitor resets and tries again with a different mac address. When set to Reject, the unit attempted to retry for a long time. All during the testing, the path through 204.194.111.66 was indeed open, however, the unit never returned to it.

              Feels like there was a firmware update such that the unit started to send encrypted data.

              Comment


              • Here are the steps to figure out the Firmware version of the device:

                1) Remove the front panel, it's held in by the 3 screws on the bottom of the unit
                2) Connect an Ethernet cable to the "installer's port" (see image 1)
                3) Connect a device to the other end configured for DHCP
                4) Visit sunpowerconsole.net
                5) Press "Residential Install"
                6) Press Next through to the Firmware Verification step (see image 2)
                7) The screen will show the Firmware currently running


                Attached Files

                Comment


                • Here are a few interesting commands gathered from the session:

                  1) Get version number: http://sunpowerconsole.net/cgi-bin/dl_cgi?Command=Start
                  2) Get communication interfaces: http://sunpowerconsole.net/cgi-bin/d...mmand=Get_Comm
                  3) Get a list of devices: http://sunpowerconsole.net/cgi-bin/d...and=DeviceList

                  Comment


                  • Good detective work, and good information. Thanks for sharing.

                    When I click on "Residential", I get "The PVS indicated there was a problem."

                    Is SP collecting your consumption data along with generation? They absolutely would have to encrypt consumption.

                    I just noticed that my browser opened up another tab at some point while trying to connect to the PVS. It rendered raw json. The only interesting things I see scanning through it is: "MODEL":"PV Supervisor PVS5", "HWVER":"3.3", "SWVER":"5.0.0, Build 313".

                    So you are 12 builds ahead of me.
                    Attached Files
                    Last edited by JJNorcal; 07-02-2017, 12:19 PM.

                    Comment


                    • I cannot take the credit... this was in part discussed earlier in this thread.

                      While connected, you can try: http://sunpowerconsole.net/cgi-bin/d...and=DeviceList to get a list of devices and their status.

                      I got some data for inverter production which matches the old SunPower website. However, I am having trouble comparing totals to what the new site is reporting. My total lifetime energy generated does not quite match what is reported on the site.

                      Need to re-read the thread to see if anyone produced any math for converting the numbers to what Sunpower shows. The individual values match on the dot, but totals are off.

                      Based on the numbers I am seeing for the total lifetime energy, the sum of all my panels comes up with 1192 kWh (DC?), but the Sunpower monitoring site shows that lifetime production is 1130.33 kWh (AC?). I cannot explain the discrepancy between these two numbers and if there is DC->AC conversion going on, not sure how to do that conversion.

                      Has anyone come up with calculations on total consumption/generation that somehow matches what the Sunpower site is reporting? Again, the individual numbers (as I have labeled them) are on the dot. It is the cumulative and lifetime totals which seem to be somewhat off.
                      Last edited by apara; 07-02-2017, 03:42 PM.

                      Comment


                      • While the individual values I am capturing for panels and consumption meter match the old SunPower site, the total consumption and total production do not match the values being shown on the new Sunpower site. The numbers are a quite a magnitude different.

                        For example, for production today, my count based on the sum of the deltas in total production at each 5 minute sampling period for each panel (deltas between ltea_3phsum_kwh for each panel):

                        select
                        sum(l.total_lifetime_energy_delta_kwh)
                        from
                        line l
                        where
                        l.type=130 and
                        l.ts > date(now());

                        Consumption comes out to 6.19 kWh, however, the new SunPower site is showing only 4.3 kWh.

                        Likewise, if I compare net energy usage, the number I get by comparing the start of the day to now is (the difference between the start of the day and now in net_ltea_3phsum_kwh) :

                        SELECT
                        ((select l.total_lifetime_energy_kwh from line l where l.type = 140 and l.ts > date(now()) order by l.ts desc limit 1) -
                        (select l.total_lifetime_energy_kwh from line l where l.type = 140 and l.ts > date(now()) order by l.ts asc limit 1) )

                        Production comes out to 14.61, however, the new SunPower site is showing usage at 15.1.

                        Are there some other calculations I am missing? I believe all these numbers are already reported in AC.




                        Comment


                        • Ack! For some reason I didn't get an email alert there were updates to this thread. I have lost my monitoring via the tcpdump/snooping method as well. My direct queries to http://sunpowerconsole.net/cgi-bin/d...and=DeviceList also changed the data format, so I'm pretty sure our PV supervisors got a firmware update. I've updated my direct query scripts to parse the new data properly, I will give an explanation below. I thought I'd share my PV's version:

                          "supervisor":
                          {
                          "SWVER":"5.0.0, Build 325",
                          "SERIAL":"ZT162585000441C1402",
                          "MODEL":"PVS5"
                          }
                          }

                          So, I'm not sure if they turned on TLS/SSL or why we lost all our data. I think it may have switched over to using the cellular connection - I can confirm my PV supervisor has a SIM card and is able to communicate over the cell network. I no longer see 140 or 130 messages coming across in the clear. But, at least direct query still works so I can get my production values (I still haven't worked out consumption on this). Here's the new format:

                          PV monitor (140 line):
                          {
                          "ISDETAIL":"1",
                          "SERIAL":"PVS5M508095c",
                          "TYPE":"PVS5-METER-C",
                          "STATE":"working",
                          "STATEDESCR":"Working",
                          "MODEL":"PVS5M0400c",
                          "DESCR":"Power Meter PVS5M508095c",
                          "DEVICE_TYPE":"Power Meter",
                          "SWVER":"4",
                          "PORT":"",
                          "DATATIME":"2017,06,30,14,56,51",
                          "ct_scl_fctr":"125",
                          "net_ltea_3phsum_kwh":"-640.39",
                          "p_3phsum_kw":"-1.8784",
                          "q_3phsum_kvar":"0.5875",
                          "s_3phsum_kva":"2.0794",
                          "tot_pf_rto":"-0.9033",
                          "freq_hz":"60.02",
                          "CAL0":"125",
                          "CURTIME":"2017,06,30,14,57,04"
                          },

                          Inverter example (130 line):
                          {
                          "ISDETAIL":"1",
                          "SERIAL":"414051637006465",
                          "TYPE":"SOLARBRIDGE",
                          "STATE":"working",
                          "STATEDESCR":"Working",
                          "MODEL":"AC_Module_Type_C",
                          "DESCR":"Inverter 414051637006465",
                          "DEVICE_TYPE":"Inverter",
                          "SWVER":"951007408",
                          "PORT":"",
                          "MOD_SN":"",
                          "NMPLT_SKU":"",
                          "DATATIME":"2017,06,30,14,56,21",
                          "ltea_3phsum_kwh":"284.4779",
                          "p_3phsum_kw":"0.107",
                          "vln_3phavg_v":"247.3326",
                          "i_3phsum_a":"0.4554",
                          "p_mpptsum_kw":"0.112",
                          "v_mppt1_v":"56.2324",
                          "i_mppt1_a":"2.0239",
                          "t_htsnk_degc":"30.75",
                          "freq_hz":"60.042",
                          "CURTIME":"2017,06,30,14,57,04"
                          },

                          So, because I'm lazy here's the copy/paste from my code showing what I used to be parsing (commented out with the # sign) to what the values now are:

                          #insert_line.append(x["ac_curr"])
                          insert_line.append(x["i_3phsum_a"])

                          #insert_line.append(x["ac_power"])
                          insert_line.append(x["p_3phsum_kw"])

                          #insert_line.append(x["ac_volt"])
                          insert_line.append(x["vln_3phavg_v"])

                          #insert_line.append(x["dc_curr"])
                          insert_line.append(x["i_mppt1_a"])

                          #insert_line.append(x["dc_power"])
                          insert_line.append(x["p_mpptsum_kw"])

                          #insert_line.append(x["dc_volt"])
                          insert_line.append(x["v_mppt1_v"])

                          #insert_line.append(x["energy_total"])
                          insert_line.append(x["ltea_3phsum_kwh"])

                          #insert_line.append(x["heatsink_temp"])
                          insert_line.append(x["t_htsnk_degc"])


                          I've uploaded what I have for parsing to the capture folder of my project here: https://github.com/ehampshire/sunpower/tree/dev/capture . It's called sp_monitor_json2.py and has a conf file (sp_monitor_json.conf). The way I reach my PV monitor is through a Raspberry Pi in my garage hooked into the installer port on my PV supervisor. This Pi then bridges to my wireless network and I put in a static route on whatever machine I want to reach it through, like so:

                          echo "Adding static route to 172.27.153.0 netmask 255.255.255.0 via 192.168.1.163"
                          route add -net 172.27.153.0 netmask 255.255.255.0 gw 192.168.1.163

                          Comment


                          • Hi ehampshire. I stopped getting my notifications about thread updates too.
                            So it looks like the software version of the PV Supervisor went from 5.0.0 build 313 to 5.0.0 build 325.
                            It looks like the update changed the inverter field names and they added two fields (
                            MOD_SN which is maybe the module serial number and NMPLT_SKU which I will guess means "name plate stock keeping unit" but these are probably populated in newer modules.) For my data they don't have anything in them.
                            I'm just dumping the data into a mongo database so it worked out that no data lost since mongo doesn't define a schema to worry about (lucked out on that one.) I'll just have to remap the names for any data presentation layer in my applications.
                            You should switch from SQL type database to mongo since the data is reported in json and mongo stores data as json documents it's a much nicer solution with way less overhead on the DB setup and you won't loose data if they update things and switch field names, etc.


                            Comment


                            • i had missed the updates to the thread as well. interestingly i have had no drama like any of the posters here had recently. i don't know if my supervisor is capable of remote firmware updates or not, or if somehow i'm just further down the list, but i have had 130 messages in the clear continuously all this time. my script continues to snarf the data.

                              i guess i would not be surprised for SP to try to turn on encryption as it's essentially best practice, especially if they are sending consumption data - which can easily tip off whether or not someone is home to a careful observer.

                              one question would be if the firmware updates come from a different IP address (than the monitor), perhaps some of us could block that IP. another is if this method of logging into the supervisor on the other ethernet interface still works even when the snooping method does not (seems likely, but i never investigated it.)

                              finally all of this stupidity could be avoided if SM would just provide an API so that other programs could just ask for the data...

                              Comment


                              • Anyone know when the software on the PV supervisor was updated again. I had not noticed it went from build 325 to "SWVER": "5.0.0, Build 327",

                                Comment

                                Working...
                                X