Announcement

Collapse
No announcement yet.

Mirroring/intercepting SunPower Monitoring Traffic?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Mirroring/intercepting SunPower Monitoring Traffic?

    Anyone have any experience intercepting the network traffic of a SunPower Monitoring system?

    We have a SunPower-based install (12 335-W panels in two strings and an SMA SunnyBoy inverter), with monitoring (including the consumption monitoring kit), but while the monitoring equipment captures all kinds of interesting data from the inverters (temperature, per-string stats, etc.), meter and monitoring system, the only data that gets shown on the dumbed-down SunPower Monitoring site is simple production and consumption (in 5-minute intervals).

    I'd like to "tee-off" the monitoring packets that the SunPower box is sending to the SunPower servers, and mirror that data to a local server (currently a Raspberry Pi), so I can collect the data locally for processing/display offline.

    Seems likely that there would be a way to control where the SunPower monitoring hardware sends the data, and I could either have it send to both the SP monitoring server and my local server, or have it send to my server, and I could forward it from there to the SP monitoring server. Or I suppose I could set myself up as a network proxy, and configure the SP monitoring hardware to use my proxy to communicate with the SP server.

    Is there anyone here who has familiarity with this equipment, and can advise me?

    Thanks in advance, and if this is the wrong forum for a question like this, my apologies (this is my first post)...

  • #2
    You can view the 'data rich' version of their website too. Have you seen it? Sounds like exactly what you want to see.
    I use the simple version for my ph and the fully loaded version on my pc.

    Comment


    • #3
      Originally posted by FishGun View Post
      You can view the 'data rich' version of their website too. Have you seen it? Sounds like exactly what you want to see.
      I use the simple version for my ph and the fully loaded version on my pc.
      Do you have a link? The only website I know about is the standard monitoring website, which just offers basic consumption and production monitoring data...

      Comment


      • #4
        Geez I should. I have it saved and go there pretty much daily...can't seem to find the way in from the standard site anymore. It's been awhile!

        Here's the link sans my 5digit address if that helps:

        https://www.sunpowermonitor.com/part...ddressid=XXXXX

        Comment


        • #5
          Excellent, thanks!

          The site you indicated works with the normal (lame) user account.

          I had been disappointed with https://monitor.us.sunpower.com and the lack of any detailed information.

          The site https://www.sunpowermonitor.com/partner/partner.aspx# takes the same username and password and provides a lot more data, specifically with information on your inverter(s).

          Comment


          • #6
            Thanks for the link to the old advanced view SunPower monitoring site. That's really cool!

            However, there is no way to download the data without interacting with the UI. Again, the goal here is to download the data locally, so I can present my own web page with the data. This Flash-based () website only allows downloading the data that is currently being presented in the graph, and only 4 parameters at a time, and you have to manually click on the export button to get it. There is no URL to curl programmatically to get the data (the flash-based data exchange is tied up with their authentication model).

            So in case there are others lurking, again does anyone have any experience intercepting the network packets from the monitoring "supervisor" unit to the sunpower servers, so I can collect the data in parallel?

            Comment


            • #7
              There are ways to intercept and rebroadcast the TCP/IP packets, but I'm not sure that is really the best approach to achieve your goal. I will be attempting to solve the same problem on a coworker's system in the next couple weeks, and will let you know how it goes. I intend to try polling the inverter directly. Do you have a Webconnect card in your inverter? How is the Sunpower equipment connected to the inverter... via RS-485, or ethernet? What kind of device to you intend to use for your data management... something dedicated like a Raspberry Pi, or maybe Windows software that will run on a server that is always on?
              CS6P-260P/SE3000 - http://tiny.cc/ed5ozx

              Comment


              • #8
                Originally posted by sensij View Post
                There are ways to intercept and rebroadcast the TCP/IP packets, but I'm not sure that is really the best approach to achieve your goal. I will be attempting to solve the same problem on a coworker's system in the next couple weeks, and will let you know how it goes. I intend to try polling the inverter directly. Do you have a Webconnect card in your inverter? How is the Sunpower equipment connected to the inverter... via RS-485, or ethernet? What kind of device to you intend to use for your data management... something dedicated like a Raspberry Pi, or maybe Windows software that will run on a server that is always on?
                I tried to respond earlier this morning, but my message timed out will writing. I'm interested in knowing the answer too. Sensi has you headed down the right path. SMA brands a module called a "webconnect" that provides direct ethernet connection from the inverter to your home network. If you have a direct network connection to your inverter, and find its IP, there's a chance that some creative SMA users/developers have developed software that queries the inverter, stores it in a mysql database, and uploads to PVoutput.

                If this sounds like your setup, PM me, and I'll provide some of the Pi based software I know PVoutput users are using for their SMA setups. I dont want to get in trouble with the moderators by posting it, especially if it turns out it is a dead end.

                If you have the RS-485 setup, then you are probably using another device that has the ethernet connection and performs the upload to Sunpower. Thats likely a non starter with the SMA approach. Perhap GITHUB may provide some results.
                [I][url]http://pvoutput.org/list.jsp?userid=27957[/url][/I]

                Comment


                • #9
                  Originally posted by FishGun View Post
                  Geez I should. I have it saved and go there pretty much daily...can't seem to find the way in from the standard site anymore. It's been awhile!

                  Here's the link sans my 5digit address if that helps:

                  https://www.sunpowermonitor.com/part...ddressid=XXXXX
                  thanks for share

                  Comment


                  • #10
                    I'm pretty sure that the SunPower monitoring setup uses the RS-485 connection method.

                    The SunPower monitoring device basically takes the place of the SMA WebBox.

                    Comment


                    • #11
                      Originally posted by sensij View Post
                      There are ways to intercept and rebroadcast the TCP/IP packets, but I'm not sure that is really the best approach to achieve your goal. I will be attempting to solve the same problem on a coworker's system in the next couple weeks, and will let you know how it goes. I intend to try polling the inverter directly. Do you have a Webconnect card in your inverter? How is the Sunpower equipment connected to the inverter... via RS-485, or ethernet? What kind of device to you intend to use for your data management... something dedicated like a Raspberry Pi, or maybe Windows software that will run on a server that is always on?
                      Thanks, sensij, I will be interested to hear your progress! The inverter is connected to the SunPower "supervisor" directly via RS-485, otherwise (if it were via the Webconnect card) I think it would be more straightforward.

                      I'm using a Raspberry Pi for the data management. I've had it for a while, got on a whim, and didn't have a project for it at the time, so it sat idle for a long while. It's working beautifully at this task now; surprisingly well for a $35 device.

                      Currently, I download the prod/cons data from the SunPower site every 10 minutes, massage it and store it locally. I have a web interface to graph and present the data, which fits my needs. But so far all I have is the prod/cons data, and I know (from the old SunPower monitoring site) that there are a lot more data types available, not only from the inverter. Regardless, the rPi should be sufficient for any processing, data movement/interception or other tasks.

                      I have done some packet captures using the rPi as a man-in-the-middle bridge, and I can see the POSTs that the SunPower supervisor is doing to its posting server, but most of the data is base64 packed-binary data, which without knowing the specifics of the protocol, and without extensive reverse-engineering, will be difficult (at best) to figure out... Was hoping someone else may already have done this. I doubt SunPower would be helpful, and am loathe to contact them, as their "tech support" folks seem to be somewhat clueless (even for simple things like when their website is down, I get answers like "is your computer plugged in?" (*facepalm*) ).

                      Comment


                      • #12
                        Originally posted by logdin View Post
                        If you have the RS-485 setup, then you are probably using another device that has the ethernet connection and performs the upload to Sunpower. Thats likely a non starter with the SMA approach. Perhap GITHUB may provide some results.
                        Thanks, logdin, yes, that's the setup I have.

                        I wouldn't mind getting the WebConnect card, do you know if both the RS-485 and WebConnect option cards can be active at the same time? I'd rather not perturb the existing SunPower monitoring setup, but would love to be able to additionally add in the ability to directly access the SMA inverter via the WebConnect card. Yes, I like to both retain, and consume, my baked goods...

                        Comment


                        • #13
                          Unfortunately, you can only one monitoring card installed in the inverter at a time.

                          Comment


                          • #14
                            Originally posted by quitepossibly View Post
                            Unfortunately, you can only one monitoring card installed in the inverter at a time.
                            Bummer...

                            I think I'm back to my MitM solution then, sniffing the packets coming from the SunPower supervisor, and decoding that. From my experimentation to date, this gets me all the data in realtime _except_ the actual cons/prod energy data (which, erm, is the most important...), and then I can download that about 10-15 minutes after the fact from the SunPower site... I suppose I could try to figure out how the SP supervisor is deriving that data, since it must be deriving it from the data from the inverter and WattNode monitor data, presumably?

                            Or I could figure out the format of the binary base64 payload that the SP supervisor is sending as "system" data...

                            If I get either of those figured out, then I would have no need to poll the SP website... It's the last piece of the puzzle...

                            Comment


                            • #15
                              I think the architecture on all of the Sunpower systems is not identical.

                              The system I'm interested in has the following:
                              RS-485 from SMA inverter to "PV Supervisor"
                              Ethernet from PV Supervisor to Powerline converter
                              AC power from Powerline converter to Ethernet Gateway
                              Ethernet from Ethernet Gateway to modem/router

                              No consumption monitoring has been installed yet, but the system has not yet received PTO and the installer won't fully hook up the monitoring system until after it has been running for a week or so. Perhaps a Wattnode meter with CT's is still coming. If it does, I am guessing it will either hook up to RS-485 port 2 on the PV Supervisor, or it will daisy chain with the RS-485 to the inverter.

                              The RS-485 messaging should be much easier to decode than the TCP/IP, but the hardware to get in the middle of that is more expensive, especially if the Wattnode meter and the Inverter are not on the same RS-485 chain. SMA's RS-485 protocol is available (although, for whatever reason, they choose not to support Sunspec/Modbus in the RS-485 card, only the Webconnect card), as well as Wattnode's.

                              Open questions for me, that some of you might be able to answer:

                              1) Is data also available from the SMA Sunny Portal, or only the Sunpower site?
                              2) Is data from the Sunpower site available from a direct API call, or are you scraping it from a page?

                              Honestly, getting production and consumption from the Sunpower site, even with a 15 min delay, would be an adequate solution that would enough to publish to PVOutput, but if it is only scraped, it isn't likely to be very robust.
                              CS6P-260P/SE3000 - http://tiny.cc/ed5ozx

                              Comment

                              Working...
                              X