ESP32 Infinite Loop with mbedTLS (QEMU-258)

[rabryn]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate
• Read the documentation to confirm the issue is not addressed there and your configuration is set correctly
• Tested with the latest version to ensure the issue hasn't been fixed

How often does this bug occurs?

always

Expected behavior

Using mbedTLS with MBEDTLS_HARDWARE_MPI enabled, I expect to call mbedtls_rsa_pkcs1_verify without experiencing a lockup due to an infinite loop.

Actual behavior (suspected bug)

Instead, the firmware locks up spinning on this infinite loop...

void mpi_hal_enable_hardware_hw_op(void)
{
    mpi_ll_clear_power_control_bit();
    while (mpi_ll_check_memory_init_complete()) {
    }
    // Note: from enabling RSA clock to here takes about 1.3us

#if !CONFIG_IDF_TARGET_ESP32
    mpi_ll_disable_interrupt();
#endif
}

Here's the call stack...

esp_dport_access_reg_read(uint32_t reg) (<snip>/esp-idf/components/soc/esp32/dport_access.c:20)
mpi_ll_check_memory_init_complete() (<snip>/esp-idf/components/hal/esp32/include/hal/mpi_ll.h:91)
mpi_hal_enable_hardware_hw_op() (<snip>/esp-idf/components/hal/mpi_hal.c:19)
esp_mpi_enable_hardware_hw_op() (<snip>/esp-idf/components/mbedtls/port/bignum/bignum_alt.c:24)
mbedtls_mpi_mul_mpi(mbedtls_mpi * Z, const mbedtls_mpi * X, const mbedtls_mpi * Y) (<snip>/esp-idf/components/mbedtls/port/bignum/esp_bignum.c:547)
mbedtls_mpi_mul_int(mbedtls_mpi * X, const mbedtls_mpi * A, mbedtls_mpi_uint b) (<snip>/esp-idf/components/mbedtls/port/bignum/esp_bignum.c:572)
mbedtls_mpi_div_mpi(mbedtls_mpi * Q, mbedtls_mpi * R, const mbedtls_mpi * A, const mbedtls_mpi * B) (<snip>/esp-idf/components/mbedtls/mbedtls/library/bignum.c:1476)
mbedtls_mpi_mod_mpi(mbedtls_mpi * R, const mbedtls_mpi * A, const mbedtls_mpi * B) (<snip>/esp-idf/components/mbedtls/mbedtls/library/bignum.c:1544)
calculate_rinv(mbedtls_mpi * Rinv, const mbedtls_mpi * M, int num_words) (<snip>/esp-idf/components/mbedtls/port/bignum/esp_bignum.c:210)
esp_mpi_exp_mod(mbedtls_mpi * Z, const mbedtls_mpi * X, const mbedtls_mpi * Y, const mbedtls_mpi * M, mbedtls_mpi * _Rinv) (<snip>/esp-idf/components/mbedtls/port/bignum/esp_bignum.c:399)
mbedtls_mpi_exp_mod(mbedtls_mpi * X, const mbedtls_mpi * A, const mbedtls_mpi * E, const mbedtls_mpi * N, mbedtls_mpi * _RR) (<snip>/esp-idf/components/mbedtls/port/bignum/esp_bignum.c:473)
mbedtls_mpi_exp_mod_unsafe(mbedtls_mpi * X, const mbedtls_mpi * A, const mbedtls_mpi * E, const mbedtls_mpi * N, mbedtls_mpi * prec_RR) (<snip>/esp-idf/components/mbedtls/mbedtls/library/bignum.c:1769)
mbedtls_rsa_public(mbedtls_rsa_context * ctx, const unsigned char * input, unsigned char * output) (<snip>/esp-idf/components/mbedtls/mbedtls/library/rsa.c:1263)
mbedtls_rsa_rsassa_pkcs1_v15_verify(mbedtls_rsa_context * ctx, mbedtls_md_type_t md_alg, unsigned int hashlen, const unsigned char * hash, const unsigned char * sig) (<snip>/esp-idf/components/mbedtls/mbedtls/library/rsa.c:2710)
mbedtls_rsa_pkcs1_verify(mbedtls_rsa_context * ctx, mbedtls_md_type_t md_alg, unsigned int hashlen, const unsigned char * hash, const unsigned char * sig) (<snip>/esp-idf/components/mbedtls/mbedtls/library/rsa.c:2755)
...

I managed to work around the issue by disabling MBEDTLS_HARDWARE_MPI.

Error logs or terminal output

Steps to reproduce the behavior

• Enable MBEDTLS_HARDWARE_MPI.
• Call mbedtls_rsa_pkcs1_verify.

Project release version

esp-develop-9.2.2-20250228 with IDF 5.3.2.

System architecture

Intel/AMD 64-bit (modern PC, older Mac)

Operating system

Linux

Operating system version

Ubuntu 22.04.5 LTS

Shell

Bash

Additional context

No response