Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REQUEST] Utilize new common "Silicon Labs firmware builder" automated build system from Home Assistant / Nabu Casa developers to create reproducible builds of Zigbee and Thread firmware images #36

Open
Hedda opened this issue Feb 10, 2023 · 10 comments

Comments

@Hedda
Copy link
Contributor

Hedda commented Feb 10, 2023

@xsp1989 Please consider making use of the "Silicon Labs firmware builder" which is a new open-source firmware build system that Home Assistant founders have now among other things announced as a common script tool set for building firmware images for Silabs based hardware.

https://www.home-assistant.io/blog/2023/02/08/state-of-matter-and-thread/#building-firmware-for-silicon-labs-chips

(Note that this is only just one section from https://www.home-assistant.io/blog/2023/02/08/state-of-matter-and-thread/ )

In summary, they have open-sourced and released the ""Silicon Labs firmware builder" project which is a collection of automated build scripts to help make it easier to create reproducible firmware builds for Zigbee and Thread based SoCs from Silicon Labs.

Please consider making use of this firmware build system and if possible also contribute any improvements to upstream project:

https://github.com/NabuCasa/silabs-firmware-builder

As I understand it, right now it allows you to build either Zigbee EmberZNet NCP (Zigbee Coordinator on Silicon Labs Network Co-Processor) firmware images, OpenThread RCP (Radio Co-Processor), or RCP Multi-PAN firmware images for OTBR (multi-protocol Zigbee EmberZNet and OpenThead Border Router using Radio Co-Processor) profiled for their own Home Assistant SkyConnect USB stick/dongle and their EFR32MG21-based radio module integrated into the Home Assistant Yellow appliance, however, the idea is that you can build profiles for other Silicon Labs hardware adapters/dongles/sticks/modules as well. It could of course also be extended to not only support different Silicon Labs based hardware adapters/dongles/sticks/modules but also different type of firmware images, (like maybe OpenThead Border Router on NCP).

(Examples of firmware image builds that it can produce can be found here -> https://github.com/NabuCasa/silabs-firmware ).

Note that was just one of their announcements in that blog post; in related news, they also released "SL Web Tools" which enable flashing firmware to Silabs-based adapters directly from a web page as well as a "SiLabs multi-flasher" as a Python script flashing tool:

https://www.home-assistant.io/blog/2023/02/08/state-of-matter-and-thread/#silabs-multi-flasher--sl-web-tools

(Again note that this is only just one section from https://www.home-assistant.io/blog/2023/02/08/state-of-matter-and-thread/ )

https://github.com/NabuCasa/universal-silabs-flasher

https://github.com/NabuCasa/sl-web-tools

https://www.youtube.com/watch?v=-88K23e8XYw&t=2s&ab_channel=HomeAssistant

The "SL Web Tools" project is available as open source. It allows users to manage the firmware on Zigbee/Thread sticks using SiLabs chips directly from their browser, with no terminals or compilation necessary. This allows any creator to offer easy firmware updates to their users and make experimental updates available to test bug fixes quickly."

To see it in action, SL Web Tools for flashing firmware has been added to the Home Assistant SkyConnect documentation.

https://skyconnect.home-assistant.io

SL Web Tools is powered by another new open-source Python package “silabs-universal-flasher” → GitHub - NabuCasa/universal-silabs-flasher: Flashes Silicon Labs radios running EmberZNet or CPC multi-pan firmware

For reference; the firmware build system for Silicon Labs was made by Home Assistant developer/engineer and Nabu Casa employee Stefan Agner (a.k.a. agners, who also developed the EFR32MG21-based Home Assistant SkyConnect USB stick/dongle as well as the EFR32MG21-based radio module integrated into the Home Assistant Yellow appliance for Rasberry Pi 4 Compute Module). And the "universal-silabs-flasher" firmware tool was made by puddly who is also one of the lead developers of zigpy and the radio libraries for zigpy that is used in Home Assistant's ZHA integration (among other projects).

PS: They made updates to their documentation as relevant to Silicon Labs based adapters with correct firmware for Zigbee or Thread:

@Hedda Hedda changed the title [REQUEST] Utilize automated "Silicon Labs firmware builder" build system from Home Assistant / Nabu Casa developers to create reproducible firmware builds for Zigbee and Thread [REQUEST] Utilize new common "Silicon Labs firmware builder" automated build system from Home Assistant / Nabu Casa developers to create reproducible firmware builds for Zigbee and Thread Feb 10, 2023
@Hedda Hedda changed the title [REQUEST] Utilize new common "Silicon Labs firmware builder" automated build system from Home Assistant / Nabu Casa developers to create reproducible firmware builds for Zigbee and Thread [REQUEST] Utilize new common "Silicon Labs firmware builder" automated build system from Home Assistant / Nabu Casa developers to create reproducible builds of Zigbee and Thread firmware images Feb 10, 2023
@Hedda
Copy link
Contributor Author

Hedda commented Mar 14, 2023

FYI, ksjh has begun some work on such automated firmware building using Github Actions and started a related discussion on it:

https://github.com/ksjh/silabs-firmware-builder

ksjh/silabs-firmware-builder#2

Note that most of the automation magic with silabs-firmware-builder looks to happen in Github Actions workflows:

https://github.com/features/actions

https://docs.github.com/en/actions

@Hedda
Copy link
Contributor Author

Hedda commented Apr 18, 2023

@xsp1989 perhaps you have some interest and input or feedback on this discussion(?), see -> ksjh/silabs-firmware-builder#17

Also check out this repo by darkxst for patches:

https://github.com/darkxst/silabs-builder-remote

PS: Note that it is not the first post there itself that is interesting but rather the discussion further down that it lead to.

@Hedda
Copy link
Contributor Author

Hedda commented Jun 19, 2023

FYI, darkxst fork of Nabu Casa's silabs-firmware-builder has added support for multiple hardware manifests to allow override configs for automatic builds to enable automatic building of firmware images for many different Silabs based EFR32 adapters:

https://github.com/darkxst/silabs-firmware-builder/blob/manifestjs/manifests

As such he is the first to have firmware builds for Silicon Labs EmberZNet 7.3 (though untested since they are automatically built):

https://github.com/darkxst/silabs-firmware-builder/tree/main/firmware_builds

PS: He is also offering experimental builds for the same EmberZNet version and hardware but with different baud rate speeds.

@Hedda
Copy link
Contributor Author

Hedda commented Aug 15, 2023

@xsp1989 FYI, darkxst is not only continuing to improve his fork of silabs-firmware-builder and he has now also forked the Silabs Flasher add-on for Home Assistant and is working on making it compatible with many different Silicon Labs based USB adapters, see:

PS: Another tip is to also check out his blog -> https://dialedin.com.au/blog/

@Hedda
Copy link
Contributor Author

Hedda commented Sep 19, 2023

@xsp1989 slightly off-topic but thought that you might be interested to know that darkxst have begun working on an experimental "Silicon Labs Multipan Flasher Add-on" as a fork of the official "Silabs Flasher" addon for Home Assistant to let users easily flash the firmware images from Home Assistant's GUI for built with Silicon Labs firmware builder on the other products that the official Home Assistant SkyConnect dongle, e.i. Zigbee Coordinator adapters like Easyiot zigbee 3.0 USB Dongle model ZB-GW04, ITead’s "Sonoff Zigbee 3.0 USB Dongle Plus V2” (model “ZBDongle-E”), and Elelabs ELU013 USB radio stick:

https://github.com/darkxst/multipan_flasher/tree/main

https://github.com/darkxst/multipan_flasher/blob/main/multipan_flasher/README.md

"This is a fork of the official Silabs Flasher HA Addon, but is shipped with my firmware builds. Once configured, It will automatically update dongle firmware on next reboot of HA, or you can manually run it after installing/updating. Currently supports ZBDongle-E and a few others."

Developers and users that have issues with it can create an issue here:

https://github.com/darkxst/silabs-firmware-builder/issues

@xsp1989
Copy link
Owner

xsp1989 commented Sep 21, 2023

Thank you very much for your suggestions. I think this is a very good project, but there are still some commercial development projects that need support. After I have completed the development of other projects, I will switch to a new development platform. This question Keep it for now, don't close it

@Hedda
Copy link
Contributor Author

Hedda commented Jun 10, 2024

FYI, it looks like puddly has in the past few weeks rewritten and reorganized much of Nabu Casa’s upstream Silabs firmware builder, and even added Z-Wave Controller too, plus archived their repo for binaries:

https://github.com/NabuCasa/silabs-firmware-builder/pulls?q=is%3Apr+is%3Aclosed

https://github.com/NabuCasa/silabs-firmware-builder/commits/main/

Important

New firmwares are released straight from the SiLabs Firmware Builder repository.
This repository contains all past firmwares and will not receive future updates.

https://github.com/NabuCasa/silabs-firmware

So, new official Zigbee and Thread firmware releases for SkyConnect + Yellow and future Silicon Labs firmware will from now be released straight from the SiLabs Firmware Builder repository

As such the "old" silabs-firmware repository is now listed as archived and will not receive more updates but it will remain available as it contains all past firmware versions.

PS: Off-topic however interesting is that the silabs-firmware-builder now also contain Z-Wave Controller firmware for the upcoming official Home Assistant Z-Wave Controller USB adapter hardware that Nabu Casa is working on -> https://www.home-assistant.io/blog/2024/05/08/zwave-is-not-dead

@Hedda
Copy link
Contributor Author

Hedda commented Nov 7, 2024

FYI, many other downstream devs are now collaborating on improving upstream, including added Bluetooth + OpenThread RCP:

PS: Upstream will soon also migrate from Gecko SDK to Simplicity SDK which will bring larger changes to dependecies and builds.

@Hedda
Copy link
Contributor Author

Hedda commented Jan 8, 2025

@xsp1989 FYI, zha/zigpy and Zigbee2MQTT/zigbee-herdsman community developers puddly, darkxst, and Nerivec are now also working together on adding a standardized way to build Zigbee Router firmware with same Nabu Casa’s silabs-firmware-builder:

NabuCasa/silabs-firmware-builder#85

Nerivec Is still experimenting with this so you can test his unofficial builds from his fork:

For reference, they intend to enable use ZAP tooling in combination with ways to manipulate custom configuration of that in silabs-firmware-builder to allow building Zigbee Router and other custom firmware for different Silicon Labs based radio adapters:

Implement a new protocol for communicating with the builtin CLI on Silabs router firmware and integrate through out. This will allow to probe and flash router firmware

Router firmware must implement a custom CLI command bootloader reboot that calls the bootloader_rebootAndInstall api.

This has been tested both standalone and integrated with sl-web-tools.

Probe of CPC 115200 is fixed in NabuCasa/universal-silabs-flasher#86, but also requires router firmware built with SL_CLI_LOCAL_ECHO: 0

PS: "ZAP" in this case refers to the ”ZCL Advanced Platform” (i.e. ZAP) tool/format which is already a tool that is being used by the official Matter project (a.k.a. Project CHIP) for configuring Matter devices in a standardized way:

@Hedda
Copy link
Contributor Author

Hedda commented Jan 20, 2025

@xsp1989 FYI, zha/zigpy and Zigbee2MQTT/zigbee-herdsman community developers puddly, darkxst, and Nerivec are now also working together on adding a standardized way to build Zigbee Router firmware with same Nabu Casa’s silabs-firmware-builder:

FYI, Nerivec is now experimental with adding support for building Zigbee Router firmware with Nabu Casa's silabs-firmware-builder:

He is now posting experimental builds so you can post feedback or bug reports to him as issues there in his repo + discussion there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants