RSAC: Hacking a Solar Power Controller—And Pretending to Generate a Gigawatt

At the recent RSAC, there was a great presentation by Waylon Grange on hacking the embedded software in his neighbor's solar power controller. He was stuck at home during lockdown, and his neighbor had solar panels installed. He emphasized that the particular vendor he picked on was not singled out because it was particularly good or bad, but rather it happened to be what his neighbor installed.

Every year at RSAC, I try and find a presentation that looks like it is going to tell an interesting story, albeit with an underlying security message. The last two years were:

• RSAC: Motherhood and Apple Pie...and Breaking into a Prison
• Some Real Russian Hacking

This year, Wayton Grange. He is principal threat researcher at Stage 2 Security. His presentation was titled His Power Level Is Over 9000! Vulnerabilities in Solar Panel Controllers. He used this example as way of both showing his methodology for pen testing an embedded system like this, and also to give details on the vulnerabilities that he discovered.

Open Source Approach

First, he did open-source research and finds what he can on the internet. He discovered from the company's marketing material that there is a large cloud component, there is an iPad app that is used to both install and control the system. The heart of the installation is the Enphase IQ Combiner, the system that monitors power production and consumption. So the first thing he wanted to do was find a copy of the firmware image from the controller. But he couldn't find one.

Hats off to the vendor, protecting the firmware image is important.

He moved on to the apps. There are two, an installer and a manager. These apps consisted of two parts, a top level written in a scripting language, and a lower level written in C and compiled into object code. Waylon said that you see this a lot, where part can easily be reverse engineered, but the stuff that they wanted to protect (in particular, the cybersecurity) is compiled. But today, with reverse assemblers, it is easy to look inside those compiled binaries. He warned that compiling part of the code like that is almost an invitation.

It’s almost like a request to look here for the crypto stuff.

What he discovered is that they computed an MD5 hash of a string that they built starting from the serial number. Then they calculated a six-character CRC of the hash, and that is what they used as the device password.

A lot of vendors “do their own thing” in creating their own crypto that ends up hurting them in the end.

He also finally discovered where the firmware was. It was downloaded from an S3 bucket on AWS. It was open for reading so he could see the images, the install scripts, and more. He pointed out that this used to be more common, but open buckets are now less common, however misconfiguration of the bucket is not unusual.

To decrypt the firmware images, he needed a tool called EECrypt, but he didn't have the keys. Time to start attacking the device itself.

Attacking the Hardware

Waylon went on Ebay and purchased one of the controllers. You can see the board in the picture. The lower left is the high power area (connected to solar panels etc). The top right is where the low-power embedded system is running. The main processor is a Texas Instruments SoC, with an eMMC memory, which presumably held the operating system. There was a custom Enphase energy chip that seemed to be used to calculate power production and consumption. Over to the right, there is a JTAG interface and a UART. There is also a Wi-Fi chip. Plus he could probe traces on the board, of course.

There are multiple avenues to attack, typically during the boot process. He ran through how the system boots. It used to be the case that the UART might well present a root (do anything) console. It's not so common now, but always worth checking. In this case the UART was output only. JTAG is attackable at any time, as long as the processor is running. However, embedded design has improved so sometimes the JTAG has been disabled, as was the case here.

The next thing to try was glitching, causing the memory to malfunction and seeing if the operating system went into some sort of "what do I do now?" mode. But it did not, it panicked (that is the term for Linux/Unix to mean the operating system simply gave up and stopped):

This is the correct response. A win for them.

The boot sequence works like this. The first level bootstrap (loaded from a small flash memory) finds the second level bootstrap called U-boot. U-boot finds the file system, copies the Linux kernel into memory, and then transfers control to the kernel. U-boot is a common and well-known program. By default, it has a three-second window where you can interrupt it and it then gives options (such as loading the operating system over the network). However, that is easy to lock down (set the three-second default to 0, CRC all the environment variables to detect tampering, and so on).

U-boot had been locked down effectively, so Waylon focused on the first-level bootstrap. This is tiny but it is also risky to mess around with since if you break this stage you risk bricking the entire device. But what he was able to do was put a clip on the flash chip and read out the exact contents. He could also alter them, but flash is complicated to later since you can just alter a single byte, you have to erase a page and rewrite the entire page. So he did that. He changed the boot delay to five seconds so he would have the ability to interrupt U-boot.

He discovered that the operating system image was encrypted with AES, but the key was not explicitly in the code. They actually had a routine to generate the key by grabbing blocks and shuffling them.

I believe that developers knew that you don’t embed the key in the binary explicitly, so they implemented this little data shuffler to generate the key. The end result is the same, I still know the key. So I was able to decrypt the firmware image. The fact that they don’t make the image available and they encrypted it meant it took me some time. I'm on slide 18 now, and finally I can start to look for vulnerabilities. Just took an hour or two to find a big one.

The first bug he found ultimately gave him the capability to execute any code on the box. The first thing he wanted to do was change the root password. He did that, but he still could not log in. Something was weird. What was weird was that they had implemented their own authentication and intercept all authentication attempts. The root password is not used. Waylon pointed out how bad this is. The password is hand-crafted so that there is a different password for every system. However, it is created algorithmically from the system serial number. The serial number is on a QR code on the board, inside the service panel, on the base of the system, on the packing box for the system, on the website (unprotected).

With solar, there are financial incentives called SREC for Solar Renewable Energy Credit. These are traceable. If you are able to fool the system into how much solar you are generating then you are basically printing your own money. So, of course, he did.

In this picture, he is suddenly generating 9KW of power (hence the title of his presentation).  But why stop there, he went up to 1.21 gigawatts. Waylon pointed out that he never signed up with his own account so he never got any credits, but the system was clearly reporting it.

You would hope the system would notice that a domestic installation was generating a GW of power. Hackers could potentially generate their own money.

Conclusion

Embedded security is getting better but still has ways to go. In particular, embedded system designers are getting more disciplined about disabling debug access in production product.

Never, never, never roll your own crypto. Just because you can't break it yourself doesn't mean it is secure.

Encrypted and signed images with limited access to the image goes a long way (it took him until slide 20 to just get around this).

Sign up for Sunday Brunch, the weekly Breakfast Bytes email.