How to BLE OTA Functionality in this code ?

[IoTopsDeveloper]

Hey @fbiego I am IoT Developement here your recent commenter on your recent Youtube video. Below here is my code in which I want to add BLE OTA part, and I am confused how to add it with the help of your code.Kindly look into it.

Here is the code

[fbiego]

Do you have a GitHub repository? That would be much easier

[IoTopsDeveloper]

Do you have a GitHub repository? That would be much easier

https://github.com/IoTopsDeveloper/BLEwithWifi

[IoTopsDeveloper]

@fbiego any guidance or anything adding in the Arduino Sketch ?

Or the OTA part in your firmware requires Service UUID with Characterstics UUID for Update and Installation or else Service UUID for Service and both CHARACTERSTIC_UUID_RX and CHARACTERISTIC_UUID_TX are the one which perform operation in the characteristic callback function?

[fbiego]

I'm checking it now

[fbiego]

Check the PR on your repo

[IoTopsDeveloper]

Actually I find the problem in the firmware when I implemented my code with BLE OTA where the firmware update and install function has run in the void loop function while the libraries of Update.h , FS.h FFat.h and SPIFFS.h part run always run the void setup() mode and that's where it crashed when I implemented my code with BLE OTA

[fbiego]

Have you tried out the bare BLE OTA sketch without adding your code?
Does it work?

[IoTopsDeveloper]

It worked but in Serial Monitor and in Application their is error 9 displaying.

Actually my point was that the BLE OTA withmy code crashes get occured due to both Wifi part and Update, FS , FFat and SPIFF, all run in same void loop And adding Wifi functions in void loop itself creates an infinite loop, without working of all Update FS FFat SPIFF of OTA Part, this is where it created problem.

You make very good code and I am really appreciating your efforts you have done with Arduino Sketch and Android Application. Infact your updated application V1.3 worked really well with any BIN file .

It would be very helpful if you provide the Source Code for DIY ESP 32 Clock with BLE OTA at your own choice. Perhaps the last hope may help me to solve my problem.
Or I will try try and try more.

Apologies if any inconvenience caused

[fbiego]

The clock sketch is based on BLE OTA, it has additional functionality for time, LEDs, etc.
The file system used is FFat.

The BLE OTA sketch should be a good start then add your desired functionality

#include <Update.h>
#include "FS.h"
#include "FFat.h"
#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLEServer.h>
#include <BLE2902.h>
#include <ESP32Time.h>
#include <WS2812FX.h>
//#include "ESP_RMT_Driver.h"
//#include "start.h"

#define LED_PIN     23
#define LED_COUNT  28
#define BRIGHTNESS 50
#define OFF_BUTTON  32

#define FORMAT_FFAT false
#define FLASH FFat

#define NORMAL_MODE   0
#define UPDATE_MODE   1
#define OTA_MODE      2


uint8_t major = 1;
uint8_t minor = 6;
uint8_t ver[] = {0xFA, major, minor};  // code version

uint8_t updater[16384]; // >= MainActivity.PART
uint8_t updater2[16384];
static uint32_t color = GREEN;
static uint32_t color2 = BLUE;
static uint32_t clr = 0x000000;

uint8_t digits[11] = {
  0x77,       //  0
  0x44,       //  1
  0x3E,       //  2
  0x6E,       //  3
  0x4D,       //  4
  0x6B,       //  5
  0x7B,       //  6
  0x46,       //  7
  0xFF,       //  8
  0x6F,       //  9
  0x00        //  blank
};


#define SERVICE_UUID              "fb1e4001-54ae-4a28-9f74-dfccb248601d"
#define CHARACTERISTIC_UUID_RX    "fb1e4002-54ae-4a28-9f74-dfccb248601d"
#define CHARACTERISTIC_UUID_TX    "fb1e4003-54ae-4a28-9f74-dfccb248601d"

static BLECharacteristic* pCharacteristicTX;
static BLECharacteristic* pCharacteristicRX;

WS2812FX ws2812fx = WS2812FX(LED_COUNT, LED_PIN, NEO_GRB  + NEO_KHZ800);
ESP32Time rtc;


static bool deviceConnected = false;
static int id = 0;
static bool writeFile = false;
static int writeLen = 0;
static int writeLen2 = 0;
static bool current = true;
static int parts = 0;
static int next = 0;
static int cur = 0;
static int MTU = 0;
static int MODE = NORMAL_MODE;

bool hr24 = true, notify = true;
int sec = 0, bright = 100, otaLed = 21, otaLed2 = 24, led = 0, ldr = 0;
static bool ldrMs = false;

int otaLeds [8] = {21, 22, 23, 24, 25, 26, 27, 24};
int otaSeg [] = {4, 0, 1, 5, 6, 2, 7, 11, 12, 10, 14, 18, 17, 15, 16, 20};

static void rebootEspWithReason(String reason) {
  Serial.println(reason);
  delay(1000);
  ESP.restart();
}

class MyServerCallbacks: public BLEServerCallbacks {
    void onConnect(BLEServer* pServer) {
      deviceConnected = true;

    }
    void onDisconnect(BLEServer* pServer) {
      deviceConnected = false;
      id = 0;
    }
};

class MyCallbacks: public BLECharacteristicCallbacks {

    void onStatus(BLECharacteristic* pCharacteristic, Status s, uint32_t code) {
      Serial.print("Status ");
      Serial.print(s);
      Serial.print(" on characteristic ");
      Serial.print(pCharacteristic->getUUID().toString().c_str());
      Serial.print(" with code ");
      Serial.println(code);
    }

    void onNotify(BLECharacteristic *pCharacteristic) {
      uint8_t* pData;
      std::string value = pCharacteristic->getValue();
      int len = value.length();
      pData = pCharacteristic->getData();
      if (pData != NULL) {
        Serial.print("Notify callback for characteristic ");
        Serial.print(pCharacteristic->getUUID().toString().c_str());
        Serial.print(" of data length ");
        Serial.println(len);
        Serial.print("TX  ");
        for (int i = 0; i < len; i++) {
          Serial.printf("%02X ", pData[i]);
        }
        Serial.println();
      }
    }

    void onWrite(BLECharacteristic *pCharacteristic) {
      uint8_t* pData;
      std::string value = pCharacteristic->getValue();
      int len = value.length();
      pData = pCharacteristic->getData();
      if (pData != NULL) {

        if (pData[0] == 0xAB) {
          switch (pData[4]) {
            case 0x93:
              rtc.setTime(pData[13], pData[12], pData[11], pData[10], pData[9], pData[7] * 256 + pData[8]);
              break;
            case 0x7C:
              hr24 = pData[6] == 0;

              break;

            case 0x01:
              bright = pData[6];
              break;
            case 0xC0:
              if (pData[5] == 0) {
                color = pData[6] * 256 * 256 + pData[7] * 256 + pData[8];
              }
              if (pData[5] == 1) {
                color2 = pData[6] * 256 * 256 + pData[7] * 256 + pData[8];
              }
              break;
          }

        }


      }
      if (pData[0] == 0xFB) {
        int pos = pData[1];
        for (int x = 0; x < len - 2; x++) {
          if (current) {
            updater[(pos * MTU) + x] = pData[x + 2];
          } else {
            updater2[(pos * MTU) + x] = pData[x + 2];
          }
        }

      } else if  (pData[0] == 0xFC) {
        if (current) {
          writeLen = (pData[1] * 256) + pData[2];
        } else {
          writeLen2 = (pData[1] * 256) + pData[2];
        }
        current = !current;
        cur = (pData[3] * 256) + pData[4];

        writeFile = true;
      } else if (pData[0] == 0xFD) {
        if (FLASH.exists("/update.bin")) {
          FLASH.remove("/update.bin");
        }
      } else if  (pData[0] == 0xFE) {
        //rebootEspWithReason("Rebooting to start OTA update");

      } else if  (pData[0] == 0xFF) {
        parts = (pData[1] * 256) + pData[2];
        MTU = (pData[3] * 256) + pData[4];
        MODE = UPDATE_MODE;

      } else if (pData[0] == 0xA0) {
        ldrMs = true;
      } else if (pData[0] == 0x0F) {
        digitalWrite(OFF_BUTTON, HIGH);
      }

    }


};

void writeBinary(fs::FS &fs, const char * path, uint8_t *dat, int len) {

  //Serial.printf("Write binary file %s\r\n", path);

  File file = fs.open(path, FILE_APPEND);

  if (!file) {
    Serial.println("- failed to open file for writing");
    return;
  }
  file.write(dat, len);
  file.close();
}

void initBLE() {
  BLEDevice::init("ESP Clock");
  BLEServer *pServer = BLEDevice::createServer();
  pServer->setCallbacks(new MyServerCallbacks());

  BLEService *pService = pServer->createService(SERVICE_UUID);
  pCharacteristicTX = pService->createCharacteristic(CHARACTERISTIC_UUID_TX, BLECharacteristic::PROPERTY_NOTIFY );
  pCharacteristicRX = pService->createCharacteristic(CHARACTERISTIC_UUID_RX, BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_WRITE_NR);
  pCharacteristicRX->setCallbacks(new MyCallbacks());
  pCharacteristicTX->setCallbacks(new MyCallbacks());
  pCharacteristicTX->addDescriptor(new BLE2902());
  pCharacteristicTX->setNotifyProperty(true);
  pService->start();


  // BLEAdvertising *pAdvertising = pServer->getAdvertising();  // this still is working for backward compatibility
  BLEAdvertising *pAdvertising = BLEDevice::getAdvertising();
  pAdvertising->addServiceUUID(SERVICE_UUID);
  pAdvertising->setScanResponse(true);
  pAdvertising->setMinPreferred(0x06);  // functions that help with iPhone connections issue
  pAdvertising->setMinPreferred(0x12);
  BLEDevice::startAdvertising();
  Serial.println("Characteristic defined! Now you can read it in your phone!");
}


void setup() {
  Serial.begin(115200);
  Serial.println("Starting BLE work!");
  pinMode(OFF_BUTTON, OUTPUT);
  digitalWrite(OFF_BUTTON, LOW);

  if (!FFat.begin()) {
    Serial.println("FFat Mount Failed");
    FFat.format();
    return;
  }

  ws2812fx.init();
  ws2812fx.setBrightness(64);
  ws2812fx.setMode(FX_MODE_STATIC);
  //rmt_tx_int(RMT_CHANNEL_0, ws2812fx.getPin());

  ws2812fx.start();
  //play_tune(oo7);



  initBLE();

}

void loop() {

  ws2812fx.service();

  switch (MODE) {

    case NORMAL_MODE:
      if (rtc.getHour(true) > 21 || rtc.getHour(true) < 7) {
        ws2812fx.setBrightness(0x10);
      } else {
        ws2812fx.setBrightness(bright);
      }
      printLocalTime();

      if (ldrMs) {
        ldr = map (analogRead(35), 0, 4095, 0, 255);
        uint8_t com[] = {0xA0, 0, ldr};
        pCharacteristicTX->setValue(com, 3);
        pCharacteristicTX->notify();
        ldrMs = false;
      }

      break;

    case UPDATE_MODE:

      ws2812fx.setBrightness(150);
      if (cur + 1 == parts) { // received complete file
        uint8_t com[] = {0xF2, (cur + 1) / 256, (cur + 1) % 256};
        pCharacteristicTX->setValue(com, 3);
        pCharacteristicTX->notify();
        delay(50);
        MODE = OTA_MODE;
      }

      if (writeFile) {
        if (!current) {
          writeBinary(FLASH, "/update.bin", updater, writeLen);
        } else {
          writeBinary(FLASH, "/update.bin", updater2, writeLen2);
        }

        writeFile = false;
      } else {
        vTaskDelay(50);
      }
      showNotification(PURPLE, 0x000000, map(cur, 0, parts, 0, 16));   // show ota
      if (sec != rtc.getSecond()) {
        sec = rtc.getSecond();
        updateNeo();
        otaLed2 = otaLeds[led];
        led++;
        if (led > 7) {
          led = 0;
        }
        otaLed = otaLeds[led];

      }

      break;

    case OTA_MODE:
      updateFromFS(FLASH);
      break;

  }

}

void updateNeo(void) {
  uint8_t *pixels = ws2812fx.getPixels();
  // numBytes is one more then the size of the ws2812fx's *pixels array.
  // the extra byte is used by the driver to insert the LED reset pulse at the end.
  uint16_t numBytes = ws2812fx.getNumBytes() + 1;
  //rmt_write_sample(RMT_CHANNEL_0, pixels, numBytes, false); // channel 0
}



void showDigit(int no, uint32_t clr1, uint32_t clr2) {
  int dig = no % 10000;
  int dig1 = dig / 1000;
  if (dig1 == 0) {
    dig1 = 10;
  }
  dig %= 1000;
  int dig2 = dig / 100;
  dig %= 100;
  int dig3 = dig / 10;
  int dig4 = dig % 10;;

  for (int i = 0; i < 7; i++) {
    if ((digits[dig1] >> i) % 2 == 1) {
      ws2812fx.setPixelColor(i, clr1);
    } else {
      ws2812fx.setPixelColor(i, clr);
    }
  }
  for (int i = 0; i < 7; i++) {
    if ((digits[dig2] >> i) % 2 == 1) {
      ws2812fx.setPixelColor(i + 7, clr1);
    } else {
      ws2812fx.setPixelColor(i + 7, clr);
    }
  }
  for (int i = 0; i < 7; i++) {
    if ((digits[dig3] >> i) % 2 == 1) {
      ws2812fx.setPixelColor(i + 14, clr2);
    } else {
      ws2812fx.setPixelColor(i + 14, clr);
    }
  }
  for (int i = 0; i < 7; i++) {
    if ((digits[dig4] >> i) % 2 == 1) {
      ws2812fx.setPixelColor(i + 21, clr2);
    } else {
      ws2812fx.setPixelColor(i + 21, clr);
    }
  }

}

void showNotification(uint32_t  colr, uint32_t clr, int prog) {
  for (int k = 0; k < 16; k++) {
    if (prog >= k) {
      ws2812fx.setPixelColor(otaSeg[k], BLUE);
    } else {
      ws2812fx.setPixelColor(otaSeg[k], colr);
    }

  }

  ws2812fx.setPixelColor(3, clr);
  ws2812fx.setPixelColor(8, clr);
  ws2812fx.setPixelColor(9, clr);
  ws2812fx.setPixelColor(13, clr);
  ws2812fx.setPixelColor(19, clr);

  for (int i = 21; i < 28; i++) {
    if (i == otaLed) {
      ws2812fx.setPixelColor(i, GREEN);
    } else if (i == otaLed2) {
      ws2812fx.setPixelColor(i, 0x006600);
    } else {
      ws2812fx.setPixelColor(i, clr);
    }
  }


}


void printLocalTime() {
  int i = rtc.getHour(hr24) * 100 + rtc.getMinute();
  if (sec != rtc.getSecond()) {
    sec = rtc.getSecond();
    if (rtc.getHour(true) > 21 || rtc.getHour(true) < 7) {
      showDigit(i, 0x221100, 0x220011);
    } else {
      showDigit(i, color, color2);
    }
    updateNeo();
  } else {
    vTaskDelay(10);
  }
  //showDigit(i, 0x222200, 0x220000, 0x000000);
}

void sendOtaResult(String result) {
  pCharacteristicTX->setValue(result.c_str());
  pCharacteristicTX->notify();
  delay(200);
}


void performUpdate(Stream &updateSource, size_t updateSize) {
  char s1 = 0x0F;
  String result = String(s1);
  if (Update.begin(updateSize)) {
    size_t written = Update.writeStream(updateSource);
    if (written == updateSize) {
      Serial.println("Written : " + String(written) + " successfully");
    }
    else {
      Serial.println("Written only : " + String(written) + "/" + String(updateSize) + ". Retry?");
    }
    result += "Written : " + String(written) + "/" + String(updateSize) + " [" + String((written / updateSize) * 100) + "%] \n";
    if (Update.end()) {
      Serial.println("OTA done!");
      result += "OTA Done: ";
      if (Update.isFinished()) {
        Serial.println("Update successfully completed. Rebooting...");
        result += "Success!\n";
      }
      else {
        Serial.println("Update not finished? Something went wrong!");
        result += "Failed!\n";
      }

    }
    else {
      Serial.println("Error Occurred. Error #: " + String(Update.getError()));
      result += "Error #: " + String(Update.getError());
    }
  }
  else
  {
    Serial.println("Not enough space to begin OTA");
    result += "Not enough space for OTA";
  }
  if (deviceConnected) {
    sendOtaResult(result);
    delay(5000);
  }
}

void updateFromFS(fs::FS &fs) {
  File updateBin = fs.open("/update.bin");
  if (updateBin) {
    if (updateBin.isDirectory()) {
      Serial.println("Error, update.bin is not a file");
      updateBin.close();
      return;
    }

    size_t updateSize = updateBin.size();

    if (updateSize > 0) {
      Serial.println("Trying to start update");
      performUpdate(updateBin, updateSize);
    }
    else {
      Serial.println("Error, file is empty");
    }

    updateBin.close();

    // when finished remove the binary from spiffs to indicate end of the process
    Serial.println("Removing update file");
    fs.remove("/update.bin");

    rebootEspWithReason("Rebooting to complete OTA update");
  }
  else {
    Serial.println("Could not load update.bin from spiffs root");
  }
}```

[IoTopsDeveloper]

@fbiego Thank You very much for providing the source code. It definitely help a-lot to understand the problem. But their is one error 9 displaying every time Sketch been uploaded. Is the size of the BIN file is set below 1MB in the sketch or we can upload upto 2-3 MB in the updater section ?

[fbiego]

Which partition scheme are you using?
error 9 is defined as UPDATE_ERROR_ACTIVATE ("Could Not Activate The Firmware") not sure what is causing that

[IoTopsDeveloper]

Customized Partiton as defined in the article Custom Partition Scheme What are the options you used while uploading the firmware in the tools menu for Arduino IDE like Upload Speed Upload Frequency Flash Frequency Flash Mode Flash Size Partition Scheme Core Debug Level.

Is it possible to change the code with working of FFAt SPIFF FS in void setup So that the loop may get free to run any task As per the libraries of these

[fbiego]

Can you define your partitions and sizes?

it is up to you to decide where to run the update from, void setup or void loop. Loop is convenient because the update can be started right after the file is received otherwise you would have to reboot to update it from setup.
I don't see why you should be running other tasks when updating

[IoTopsDeveloper]

it is up to you to decide where to run the update from, void setup or void loop. Loop is convenient because the update can be started right after the file is received otherwise you would have to reboot to update it from setup.
I don't see why you should be running other tasks when updating

I had made different Partition Scheme for the my previous code as mention in this article Custom Partition Scheme . How could I run update section in void loop with BLE ?

If I can run two seperate tasks in parallel using FreeRTOS i.e Wifi Running with MQTT and BLE Firmware Update or Disconnect Wifi when OTA Update starts ?

[IoTopsDeveloper]

Which type of Partition Scheme you used while uploading the Firmware sketch via Arduino IDE at which Baud Rate , Flash Size and Core Debug Level ?

[KordelFranceTech]

Which type of Partition Scheme you used while uploading the Firmware sketch via Arduino IDE at which Baud Rate , Flash Size and Core Debug Level ?

Yes, can you indicate which partition scheme, flash size, baud rate, etc. you are using to build your "firmware.bin" file? I am having issues with this as well. Your "firmware.bin" file transfers perfectly fine, but I cannot get any other lightweight .bin file to transfer over.

[fbiego]

I have tested on two boards using Arduino IDE.
The settings are defaults including upload speed which is 921600. The only changes are as below;

Board 1

• Board: DOIT ESP32 DEVKIT V1
  The partition type will be SPIFFS, so in the code,
  #define USE_SPIFFS //comment to use FFat

Board 2

• Board: ESP32 Dev Module
• Flash Size: 16MB (128Mb)
• Partiton Scheme: 16M Flash(3MB APP/9MB FATFS)
  The partition type will be FAT
  //#define USE_SPIFFS //comment to use FFat