esp32c3 esp_partition_write() call on encrypted partition results in partially wrong data written to flash (QEMU-246)

[gw8484]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate
• Read the documentation to confirm the issue is not addressed there and your configuration is set correctly
• Tested with the latest version to ensure the issue hasn't been fixed

How often does this bug occurs?

always

Expected behavior

Plaintext data written to encrypted partition using:

esp_partition_erase_range()
esp_partition_write()

should result in properly encrypted data in flash that can be decrypted and read back using:

esp_partition_read()

Actual behavior (suspected bug)

Partially correct data is encrypted and written to flash.

Example plaintext data:

[20150.00h.04m.37s.558] 30 31 32 33 34 35 36 37  38 39 21 40 23 24 25 5E  |  0123456789!@#$%^
[20150.00h.04m.37s.559] 26 2A 61 62 63 64 65 66  67 68 69 6A 6B 6C 6D 6E  |  &*abcdefghijklmn
[20150.00h.04m.37s.560] 6F 70 71 72 73 74 75 76  77 78 79 7A 41 42 43 44  |  opqrstuvwxyzABCD
[20150.00h.04m.37s.561] 45 46 47 48 49 4A 4B 4C  4D 4E 4F 50 51 52 53 54  |  EFGHIJKLMNOPQRST
[20150.00h.04m.37s.562] 55 56 57 58 59 5A 30 31  32 33 34 35 36 37 38 39  |  UVWXYZ0123456789
[20150.00h.04m.37s.563] 21 40 23 24 25 5E 26 2A  61 62 63 64 65 66 67 68  |  !@#$%^&*abcdefgh
[20150.00h.04m.37s.564] 69 6A 6B 6C 6D 6E 6F 70  71 72 73 74 75 76 77 78  |  ijklmnopqrstuvwx
[20150.00h.04m.37s.565] 79 7A 41 42 43 44 45 46  47 48 49 4A 4B 4C 4D 4E  |  yzABCDEFGHIJKLMN
[20150.00h.04m.37s.566] 4F 50 51 52 53 54 55 56  57 58 59 5A 30 31 32 33  |  OPQRSTUVWXYZ0123
[20150.00h.04m.37s.566] 34 35 36 37 38 39 21 40  23 24 25 5E 26 2A 61 62  |  456789!@#$%^&*ab
[20150.00h.04m.37s.569] 63 64 65 66 67 68 69 6A  6B 6C 6D 6E 6F 70 71 72  |  cdefghijklmnopqr
[20150.00h.04m.37s.570] 73 74 75 76 77 78 79 7A  41 42 43 44 45 46 47 48  |  stuvwxyzABCDEFGH
[20150.00h.04m.37s.570] 49 4A 4B 4C 4D 4E 4F 50  51 52 53 54 55 56 57 58  |  IJKLMNOPQRSTUVWX
[20150.00h.04m.37s.571] 59 5A 30 31 32 33 34 35  36 37 38 39 21 40 23 24  |  YZ0123456789!@#$
[20150.00h.04m.37s.572] 25 5E 26 2A 61 62 63 64  65 66 67 68 69 6A 6B 6C  |  %^&*abcdefghijkl
[20150.00h.04m.37s.573] 6D 6E 6F 70 71 72 73 74  75 76 77 78 79 7A 41 42  |  mnopqrstuvwxyzAB

is not correctly encrypted and written to flash. When the above data is decrypted and read back from flash the result is:

[20150.01h.20m.00s.926] 30 31 32 33 34 35 36 37  38 39 21 40 23 24 25 5E  |  0123456789!@#$%^
[20150.01h.20m.00s.926] 26 2A 61 62 63 64 65 66  67 68 69 6A 6B 6C 6D 6E  |  &*abcdefghijklmn
[20150.01h.20m.00s.927] 30 31 32 33 34 35 36 37  38 39 21 40 23 24 25 5E  |  0123456789!@#$%^
[20150.01h.20m.00s.928] 26 2A 61 62 63 64 65 66  67 68 69 6A 6B 6C 6D 6E  |  &*abcdefghijklmn
[20150.01h.20m.00s.928] 55 56 57 58 59 5A 30 31  32 33 34 35 36 37 38 39  |  UVWXYZ0123456789
[20150.01h.20m.00s.929] 21 40 23 24 25 5E 26 2A  61 62 63 64 65 66 67 68  |  !@#$%^&*abcdefgh
[20150.01h.20m.00s.930] 55 56 57 58 59 5A 30 31  32 33 34 35 36 37 38 39  |  UVWXYZ0123456789
[20150.01h.20m.00s.930] 21 40 23 24 25 5E 26 2A  61 62 63 64 65 66 67 68  |  !@#$%^&*abcdefgh
[20150.01h.20m.00s.931] 4F 50 51 52 53 54 55 56  57 58 59 5A 30 31 32 33  |  OPQRSTUVWXYZ0123
[20150.01h.20m.00s.931] 34 35 36 37 38 39 21 40  23 24 25 5E 26 2A 61 62  |  456789!@#$%^&*ab
[20150.01h.20m.00s.932] 4F 50 51 52 53 54 55 56  57 58 59 5A 30 31 32 33  |  OPQRSTUVWXYZ0123
[20150.01h.20m.00s.933] 34 35 36 37 38 39 21 40  23 24 25 5E 26 2A 61 62  |  456789!@#$%^&*ab
[20150.01h.20m.00s.933] 49 4A 4B 4C 4D 4E 4F 50  51 52 53 54 55 56 57 58  |  IJKLMNOPQRSTUVWX
[20150.01h.20m.00s.934] 59 5A 30 31 32 33 34 35  36 37 38 39 21 40 23 24  |  YZ0123456789!@#$
[20150.01h.20m.00s.935] 49 4A 4B 4C 4D 4E 4F 50  51 52 53 54 55 56 57 58  |  IJKLMNOPQRSTUVWX
[20150.01h.20m.00s.935] 59 5A 30 31 32 33 34 35  36 37 38 39 21 40 23 24  |  YZ0123456789!@#$

Error logs or terminal output

Steps to reproduce the behavior

write a block of a known data using the following function sequence to an encrypted partition:

esp_partition_erase_range()
esp_partition_write()
esp_partition_read()

Project release version

9.2.2

System architecture

Intel/AMD 64-bit (modern PC, older Mac)

Operating system

Linux

Operating system version

Ubuntu 22.04

Shell

Bash

Additional context

It looks like the second 32-byte sub-block of every 64-byte block is encrypted and written incorrectly using the plaintext data of the first 32-byte sub-block of every 64-byte block.

The same tests were done on the esp32 qemu of the same version (9.2.2) and works correctly.

[Harshal5]

Hello @gw8484,

Thanks for reporting an issue! I tried recreating the issue but was unable to reproduce the results that you have shared.

Following is my test workflow along with the test example:

    ESP_LOGW("TAG", "Erasing");
    assert(ESP_OK == esp_partition_erase_range(test_part, offset, test_part->erase_size));

    ESP_LOGW("TAG", "Writing %d encrypted bytes to offset 0x%x", length, offset);
    assert(ESP_OK == esp_partition_write(test_part, offset, plaintext, length));

    ESP_LOGW("TAG", "Reading %d bytes from offset 0x%x", length, offset);
    assert(ESP_OK == esp_partition_read(test_part, offset, readback, length));

    ESP_LOGW("TAG", "Expected plaintext:");
    ESP_LOG_BUFFER_HEXDUMP("TAG", plaintext, sizeof(plaintext), ESP_LOG_INFO);

    ESP_LOGW("TAG", "Readback bytes:");
    ESP_LOG_BUFFER_HEXDUMP("TAG", readback, sizeof(plaintext), ESP_LOG_INFO);

    assert(0 == memcmp(plaintext, readback, sizeof(plaintext)));

Test example: hello_world.zip

Logs: qemu.log

Please let me know if there should be any update in the test code to correctly recreate the issue or if possible could you share your minimal test application?

[gw8484]

Thank you for the update. I don't really see meaningful differences in the test code itself. However, I am using Arduino as a component (2.0.14) but it's not obvious how that would matter to qemu. In case it helps, since the initial report I have also ran the test code on a physical esp32c3 board and it works.

Here is the test code I used:

#include <Arduino.h>

#include <esp_flash_encrypt.h>
#include <esp_partition.h>
#include <esp_task_wdt.h>

char writebuf[256];
char readbuf[256];
const char *writePartLabel = "ffs0";
const char *alphanum = "0123456789!@#$%^&*abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";


void dumpHexPretty(const void *data, size_t size, const char *prefix, const char *postfix) {
  char ascii[17];
  size_t i, j;
  ascii[16] = '\0';

  if (prefix != nullptr)
    Serial.print(prefix);

  for (i = 0; i < size; ++i) {
    Serial.printf("%02X ", ((unsigned char *)data)[i]);
    if (((unsigned char *)data)[i] >= ' ' && ((unsigned char *)data)[i] <= '~') {
      ascii[i % 16] = ((unsigned char *)data)[i];
    } else {
      ascii[i % 16] = '.';
    }
    if ((i + 1) % 8 == 0 || i + 1 == size) {
      Serial.printf(" ");
      if ((i + 1) % 16 == 0) {
        Serial.printf("|  %s \n", ascii);
      } else if (i + 1 == size) {
        ascii[(i + 1) % 16] = '\0';
        if ((i + 1) % 16 <= 8) {
          Serial.printf(" ");
        }
        for (j = (i + 1) % 16; j < 16; ++j) {
          Serial.printf("   ");
        }
        Serial.printf("|  %s \n", ascii);
      }
    }
  }

  if (postfix != nullptr)
    Serial.print(postfix);
}



void setup() {

  esp_task_wdt_reset();
  esp_task_wdt_init(30, false);

  Serial.begin(115200);

  vTaskDelay(1000 / portTICK_PERIOD_MS);


  if (esp_flash_encryption_enabled()) {
    Serial.println("Encryption Enabled");
  } else {
    Serial.println("Encryption Not Enabled");
  }

  const esp_partition_t *p = esp_partition_find_first(ESP_PARTITION_TYPE_ANY, ESP_PARTITION_SUBTYPE_ANY, writePartLabel);

  if (p != nullptr) {
    int writeLen = sizeof(writebuf);
    int offset = 0;

    Serial.printf("\nPartition '%s' encrypted=%d\n", p->label, p->encrypted);
    Serial.printf("write buf: addr=0x%08x, len=%d, aligned=%s\n",
                  (uint32_t)writebuf, sizeof(writebuf),
                  (((uint32_t)writebuf) % sizeof(uint32_t) ? "NO" : "YES"));
    Serial.printf("read buf: addr=0x%08x, len=%d, aligned=%s\n",
                  (uint32_t)readbuf, sizeof(readbuf),
                  (((uint32_t)readbuf) % sizeof(uint32_t) ? "NO" : "YES"));


    Serial.printf("\nBefore WRITE, read out of write target partition range (off=%d, len=%d):\n", offset, writeLen);
    esp_partition_read(p, offset, readbuf, writeLen);

    Serial.printf("part_read(%d):\n", writeLen);
    dumpHexPretty(readbuf, writeLen, nullptr, "\n");



    const char *cp = alphanum;
    for (int i = 0; i < writeLen; i++) {
      writebuf[i] = *cp;
      cp++;
      if (*cp == '\0')
        cp = alphanum;
    }

    esp_partition_erase_range(p, offset, SPI_FLASH_SEC_SIZE);

    esp_partition_write(p, offset, writebuf, writeLen);

    Serial.printf("\nAfter WRITE, data buffer written to target partition range (off=%d, len=%d):\n", offset, writeLen);
    Serial.printf("part_write(%d):\n", writeLen);
    dumpHexPretty(writebuf, writeLen, nullptr, "\n");



    Serial.printf("\nAfter WRITE, read out of write target partition range (off=%d, len=%d):\n", offset, writeLen);

    Serial.printf("\nPartition '%s' encrypted=%d\n", p->label, p->encrypted);

    memset(readbuf, 0, writeLen);
    esp_partition_read(p, offset, readbuf, writeLen);

    Serial.printf("part_read(%d):\n", writeLen);
    dumpHexPretty(readbuf, writeLen, nullptr, "\n");


    if(!memcmp(writebuf,readbuf,writeLen))
    {
      Serial.printf("\nVerify Success\n");
    } else {
      Serial.printf("\nVerify Failed\n");
    }

  } else {
    Serial.printf("Unable to find partition '%s'\n", writePartLabel);
  }

}

void loop()
{
  esp_task_wdt_reset();
  vTaskDelay(1000 / portTICK_PERIOD_MS);
}

For reference, I am attaching the qemu log and binary image files that I used with the following qemu command line:

qemu-system-riscv32 -nographic -machine esp32c3 \
                    -icount 3\
                    -drive file=qemu_firmware.bin,if=mtd,format=raw \
                    -drive file=qemu_efuse.bin,if=none,format=raw,id=efuse \
                    -global driver=nvram.esp32c3.efuse,property=drive,value=efuse \
                    -global driver=timer.esp32c3.timg,property=wdt_disable,value=true \
                    -serial /dev/tnt2

qemu_images.zip

qemu.log