build system: issues with reporting supported boards

[mguetschow]

Description

@Lukas-Luger found some issues with the reported supported boards for an application:

1. make info-build and make info-boards-supported show different results when BOARD is defined in the environment (before make).
2. the output is again different when BOARD is given as a make variable (after make)
3. when set as a make variable, the output takes much longer to appear.

Steps to reproduce the issue

wrong (missing, e.g., nrf52840dk):

$ BOARD=feather-nrf52840-sense make -C tests/drivers/nrf802154 info-build 
...
supported boards:
adafruit-clue adafruit-itsybitsy-nrf52 arduino-nano-33-ble arduino-nano-33-ble-sense feather-nrf52840 feather-nrf52840-sense nrf52840-mdk-dongle nrf52840dongle particle-argon particle-boron particle-xenon
...

correct (afaict):

$  BOARD=feather-nrf52840-sense make -C tests/drivers/nrf802154 info-boards-supported
adafruit-clue adafruit-itsybitsy-nrf52 arduino-nano-33-ble arduino-nano-33-ble-sense e104-bt5011a-tb feather-nrf52840 feather-nrf52840-sense microbit-v2 nrf52840-mdk nrf52840-mdk-dongle nrf52840dk nrf52840dongle particle-argon particle-boron particle-xenon reel waveshare-nrf52840-eval-kit

wrong (should restrict to nrf52 CPU):

$ make -C tests/drivers/nrf802154 info-build BOARD=feather-nrf52840-sense
...
supported boards:
acd52832 adafruit-clue adafruit-grand-central-m4-express adafruit-itsybitsy-m4 adafruit-itsybitsy-nrf52 adafruit-metro-m4-express adafruit-pybadge airfy-beacon alientek-pandora arduino-due arduino-duemilanove arduino-leonardo arduino-mega2560 arduino-mkr1000 arduino-mkrfox1200 arduino-mkrwan1300 arduino-mkrzero arduino-nano arduino-nano-33-ble arduino-nano-33-ble-sense arduino-nano-33-iot arduino-uno arduino-zero atmega1284p atmega256rfr2-xpro atmega328p atmega328p-xplained-mini atmega8 atxmega-a1-xplained atxmega-a1u-xpro atxmega-a3bu-xplained avr-rss2 avsextrem b-l072z-lrwan1 b-l475e-iot01a b-u585i-iot02a bastwan blackpill-stm32f103c8 blackpill-stm32f103cb bluepill-stm32f030c8 bluepill-stm32f103c8 bluepill-stm32f103cb calliope-mini cc1312-launchpad cc1350-launchpad cc1352-launchpad cc1352p-launchpad cc2538dk cc2650-launchpad cc2650stk derfmega128 derfmega256 dwm1001 e104-bt5010a-tb e104-bt5011a-tb e180-zg120b-tb ek-lm4f120xl esp32-ethernet-kit-v1_0 esp32-ethernet-kit-v1_1 esp32-ethernet-kit-v1_2 esp32-heltec-lora32-v2 esp32-mh-et-live-minikit esp32-olimex-evb esp32-ttgo-t-beam esp32-wemos-lolin-d32-pro esp32-wroom-32 esp32-wrover-kit esp32c3-devkit esp32c3-wemos-mini esp32s2-devkit esp32s2-lilygo-ttgo-t8 esp32s2-wemos-mini esp32s3-box esp32s3-devkit esp32s3-pros3 esp32s3-usb-otg esp32s3-wt32-sc01-plus esp8266-esp-12x esp8266-olimex-mod esp8266-sparkfun-thing f4vi1 feather-m0 feather-m0-lora feather-m0-wifi feather-nrf52840 feather-nrf52840-sense firefly frdm-k22f frdm-k64f frdm-kl43z frdm-kw41z gba_cartridge gd32vf103c-start generic-cc2538-cc2592-dk hamilton hifive1 hifive1b hip-badge i-nucleo-lrwan1 ikea-tradfri im880b iotlab-a8-m3 iotlab-m3 limifrog-v1 lobaro-lorabox lora-e5-dev lsn50 maple-mini mbed_lpc1768 mcb2388 mega-xplained microbit microbit-v2 microduino-corerf msb-430 msb-430h msba2 msbiot mulle native native64 nrf51dk nrf51dongle nrf52832-mdk nrf52840-mdk nrf52840-mdk-dongle nrf52840dk nrf52840dongle nrf52dk nrf5340dk-app nrf6310 nrf9160dk nucleo-c031c6 nucleo-f030r8 nucleo-f031k6 nucleo-f042k6 nucleo-f070rb nucleo-f072rb nucleo-f091rc nucleo-f103rb nucleo-f207zg nucleo-f302r8 nucleo-f303k8 nucleo-f303re nucleo-f303ze nucleo-f334r8 nucleo-f401re nucleo-f410rb nucleo-f411re nucleo-f412zg nucleo-f413zh nucleo-f429zi nucleo-f439zi nucleo-f446re nucleo-f446ze nucleo-f722ze nucleo-f746zg nucleo-f767zi nucleo-g070rb nucleo-g071rb nucleo-g431rb nucleo-g474re nucleo-l011k4 nucleo-l031k6 nucleo-l053r8 nucleo-l073rz nucleo-l152re nucleo-l412kb nucleo-l432kc nucleo-l433rc nucleo-l452re nucleo-l476rg nucleo-l496zg nucleo-l4r5zi nucleo-l552ze-q nucleo-u575zi-q nucleo-wl55jc nz32-sc151 olimex-msp430-h1611 olimex-msp430-h2618 olimexino-stm32 omote opencm904 openlabs-kw41z-mini openlabs-kw41z-mini-256kib openmote-b openmote-cc2538 p-l496g-cell02 p-nucleo-wb55 particle-argon particle-boron particle-xenon pba-d-01-kw2x phynode-kw41z pinetime pyboard qn9080dk reel remote-pa remote-reva remote-revb rpi-pico rpi-pico-w ruuvitag samd10-xmini samd20-xpro samd21-xpro same54-xpro saml10-xpro saml11-xpro saml21-xpro samr21-xpro samr30-xpro samr34-xpro seeedstudio-gd32 seeeduino_arch-pro seeeduino_xiao sensebox_samd21 serpente sipeed-longan-nano sipeed-longan-nano-tft slstk3400a slstk3401a slstk3402a slstk3701a sltb001a sltb009a slwstk6000b-slwrb4150a slwstk6000b-slwrb4162a slwstk6220a sodaq-autonomo sodaq-explorer sodaq-one sodaq-sara-aff sodaq-sara-sff spark-core stk3200 stk3600 stk3700 stm32f030f4-demo stm32f0discovery stm32f3discovery stm32f429i-disc1 stm32f429i-disco stm32f469i-disco stm32f4discovery stm32f723e-disco stm32f746g-disco stm32f7508-dk stm32f769i-disco stm32g0316-disco stm32l0538-disco stm32l476g-disco stm32l496g-disco stm32mp157c-dk2 teensy31 telosb thingy52 ublox-c030-u201 udoo usb-kw41z waspmote-pro waveshare-nrf52840-eval-kit weact-f401cc weact-f401ce weact-f411ce weact-g030f6 wemos-zero xg23-pk6068a yarm yunjia-nrf51822 z1 zigduino
...

wrong (should restrict to nrf52 CPU):

$  make -C tests/drivers/nrf802154 info-boards-supported BOARD=feather-nrf52840-sense
acd52832 adafruit-clue adafruit-grand-central-m4-express adafruit-itsybitsy-m4 adafruit-itsybitsy-nrf52 adafruit-metro-m4-express adafruit-pybadge airfy-beacon alientek-pandora arduino-due arduino-duemilanove arduino-leonardo arduino-mega2560 arduino-mkr1000 arduino-mkrfox1200 arduino-mkrwan1300 arduino-mkrzero arduino-nano arduino-nano-33-ble arduino-nano-33-ble-sense arduino-nano-33-iot arduino-uno arduino-zero atmega1284p atmega256rfr2-xpro atmega328p atmega328p-xplained-mini atmega8 atxmega-a1-xplained atxmega-a1u-xpro atxmega-a3bu-xplained avr-rss2 avsextrem b-l072z-lrwan1 b-l475e-iot01a b-u585i-iot02a bastwan blackpill-stm32f103c8 blackpill-stm32f103cb bluepill-stm32f030c8 bluepill-stm32f103c8 bluepill-stm32f103cb calliope-mini cc1312-launchpad cc1350-launchpad cc1352-launchpad cc1352p-launchpad cc2538dk cc2650-launchpad cc2650stk derfmega128 derfmega256 dwm1001 e104-bt5010a-tb e104-bt5011a-tb e180-zg120b-tb ek-lm4f120xl esp32-ethernet-kit-v1_0 esp32-ethernet-kit-v1_1 esp32-ethernet-kit-v1_2 esp32-heltec-lora32-v2 esp32-mh-et-live-minikit esp32-olimex-evb esp32-ttgo-t-beam esp32-wemos-lolin-d32-pro esp32-wroom-32 esp32-wrover-kit esp32c3-devkit esp32c3-wemos-mini esp32s2-devkit esp32s2-lilygo-ttgo-t8 esp32s2-wemos-mini esp32s3-box esp32s3-devkit esp32s3-pros3 esp32s3-usb-otg esp32s3-wt32-sc01-plus esp8266-esp-12x esp8266-olimex-mod esp8266-sparkfun-thing f4vi1 feather-m0 feather-m0-lora feather-m0-wifi feather-nrf52840 feather-nrf52840-sense firefly frdm-k22f frdm-k64f frdm-kl43z frdm-kw41z gba_cartridge gd32vf103c-start generic-cc2538-cc2592-dk hamilton hifive1 hifive1b hip-badge i-nucleo-lrwan1 ikea-tradfri im880b iotlab-a8-m3 iotlab-m3 limifrog-v1 lobaro-lorabox lora-e5-dev lsn50 maple-mini mbed_lpc1768 mcb2388 mega-xplained microbit microbit-v2 microduino-corerf msb-430 msb-430h msba2 msbiot mulle native native64 nrf51dk nrf51dongle nrf52832-mdk nrf52840-mdk nrf52840-mdk-dongle nrf52840dk nrf52840dongle nrf52dk nrf5340dk-app nrf6310 nrf9160dk nucleo-c031c6 nucleo-f030r8 nucleo-f031k6 nucleo-f042k6 nucleo-f070rb nucleo-f072rb nucleo-f091rc nucleo-f103rb nucleo-f207zg nucleo-f302r8 nucleo-f303k8 nucleo-f303re nucleo-f303ze nucleo-f334r8 nucleo-f401re nucleo-f410rb nucleo-f411re nucleo-f412zg nucleo-f413zh nucleo-f429zi nucleo-f439zi nucleo-f446re nucleo-f446ze nucleo-f722ze nucleo-f746zg nucleo-f767zi nucleo-g070rb nucleo-g071rb nucleo-g431rb nucleo-g474re nucleo-l011k4 nucleo-l031k6 nucleo-l053r8 nucleo-l073rz nucleo-l152re nucleo-l412kb nucleo-l432kc nucleo-l433rc nucleo-l452re nucleo-l476rg nucleo-l496zg nucleo-l4r5zi nucleo-l552ze-q nucleo-u575zi-q nucleo-wl55jc nz32-sc151 olimex-msp430-h1611 olimex-msp430-h2618 olimexino-stm32 omote opencm904 openlabs-kw41z-mini openlabs-kw41z-mini-256kib openmote-b openmote-cc2538 p-l496g-cell02 p-nucleo-wb55 particle-argon particle-boron particle-xenon pba-d-01-kw2x phynode-kw41z pinetime pyboard qn9080dk reel remote-pa remote-reva remote-revb rpi-pico rpi-pico-w ruuvitag samd10-xmini samd20-xpro samd21-xpro same54-xpro saml10-xpro saml11-xpro saml21-xpro samr21-xpro samr30-xpro samr34-xpro seeedstudio-gd32 seeeduino_arch-pro seeeduino_xiao sensebox_samd21 serpente sipeed-longan-nano sipeed-longan-nano-tft slstk3400a slstk3401a slstk3402a slstk3701a sltb001a sltb009a slwstk6000b-slwrb4150a slwstk6000b-slwrb4162a slwstk6220a sodaq-autonomo sodaq-explorer sodaq-one sodaq-sara-aff sodaq-sara-sff spark-core stk3200 stk3600 stk3700 stm32f030f4-demo stm32f0discovery stm32f3discovery stm32f429i-disc1 stm32f429i-disco stm32f469i-disco stm32f4discovery stm32f723e-disco stm32f746g-disco stm32f7508-dk stm32f769i-disco stm32g0316-disco stm32l0538-disco stm32l476g-disco stm32l496g-disco stm32mp157c-dk2 teensy31 telosb thingy52 ublox-c030-u201 udoo usb-kw41z waspmote-pro waveshare-nrf52840-eval-kit weact-f401cc weact-f401ce weact-f411ce weact-g030f6 wemos-zero xg23-pk6068a yarm yunjia-nrf51822 z1 zigduino

significant delay difference:

$ time BOARD=feather-nrf52840-sense make -C tests/drivers/nrf802154 info-boards-supported 
...
BOARD=feather-nrf52840-sense make -C tests/drivers/nrf802154   0.53s user 0.25s system 104% cpu 0.744 total
$ time make -C tests/drivers/nrf802154 info-boards-supported BOARD=feather-nrf52840-sense 
...
make -C tests/drivers/nrf802154 info-boards-supported   3.49s user 6.08s system 99% cpu 9.585 total

Expected results

outputs should be correct everywhere and take roughly the same amount of time

Actual results

the opposite

Versions

Operating System Environment
----------------------------
         Operating System: "Debian GNU/Linux" "12 (bookworm)"
                   Kernel: Linux 6.1.0-26-amd64 x86_64 unknown
             System shell: /usr/bin/dash (probably dash)
             make's shell: /usr/bin/dash (probably dash)

...

Installed development tools
---------------------------
                   ccache: ccache version 4.7.5
                    cmake: cmake version 3.25.1
                 cppcheck: missing
                  doxygen: 1.9.4
                      git: git version 2.39.5
                     make: GNU Make 4.3
                  openocd: Open On-Chip Debugger 0.12.0
                   python: missing
                  python2: missing
                  python3: Python 3.11.2
                   flake8: error: /usr/bin/python3: No module named flake8
               coccinelle: spatch version 1.1.1 compiled with OCaml version 4.13.1