ESP32 & ESP-IDF Insights Series by Naveen PS

#1 Which SoC to Use for your project

ESP32 S2,S3,C3,C5,C6,H2 – Confused? Which SoC to Use for your project? Click the link to my blog below.


https://www.psnaveen.com/post/which-esp32-soc-to-buy-esp32-s2-s3-c3-c5-c6-h2


#2 STOP using Arduino IDE for ESP32!



Beginners working on ESP32 get too comfortable with Arduino IDE. I say you should jump to ESP-IDF as soon as possible. When I was a beginner, I was clueless with ESP IDF, especially with setting it up properly. But now it’s almost a 1 Click Setup with VS Code Extension & the official guides are excellent. So, shift to ESP IDF ASAP – Thank me later.


Neil Kolban (The author of the most comprehensive ESP32 guide) explains his point of view regarding Arduino APIs vs ESP IDF APIs, or as he calls it a "red vs blue" discussion.


Let me quote him:

“The benefit of ESP-IDF (over Arduino APIs) is that the ESP-IDF is native to the ESP32 (by definition). It is the API (short of register poking) that will give you the highest fidelity against the ESP32 device itself. There are capabilities available in ESP-IDF that aren’t (yet?) present in mapped Arduino APIs. They could quite easily be mapped by a programmer but that then implies one already has an ESP-IDF skill level.” - Kolban


Do read his full reply in this thread: https://lnkd.in/gj8pq_vP (1st Answer in the thread)


#3 Why ESP-IDF on VS Code is better?



Working on ESP IDF has never been this easier. All possible due to this VS Code ESP IDF Extension. Whether you are using a Linux, Windows or Mac System, the steps are the same to setup on VS Code. Direct ESP IDF installation is totally a different story.


People often right away jump to project building after installing in VS Code. But I recommend to go visit this extensive VS Code Centric Tutorial on Working with ESP IDF. This will definitely give you the knowledge to utilize the full capabilities of the extension. https://lnkd.in/g2JC4WQE



#4 Which ESP-IDF Version to work with?




Which ESP-IDF Version should you choose from all these? I have had instances where the right version solved many of the Errors and Warnings. As a beginner with ESP-IDF, choosing the right version can be critical. You may even stop using IDF just because some error caused by not using the right version.


For Just Prototyping or to play with new ESP-IDF Features, install the “master” version.


For Production Purposes use the current stable version, install any of “v4.X.X” versions, preferably the latest. But for some reason if it throws compilation errors or warnings, install the older stable versions.


But say, if a required feature is not yet available in stable release, it is recommended to use a pre-release version or a release branch. These are ones with “release/vX.X” .


Along with this its good to know the Version Scheming too:


Major Release example : v4.0

Minor Release example : v4.1

Bugfix Release example : v4.1.1



#5 Decoding the ESP IDF Project Directory Tree


As a beginner with ESP-IDF, when you load an example code to start learning, you will be exposed to a folder structure, which include much more than just the main source code file. When I started off, I didn’t care much about it. But knowing the Project Directory Tree is pretty important. Let’s see it in detail:


1. The Top Level CMakeLists.txt is used by CMake to learn how to build the project. It also defines project name and defines the project, while setting project wide CMake Variables that helps in building the project


2. The “sdkconfig” is the configuration file created or updated when “idf.py menuconfig” runs, and holds configuration for all of the components in the project. When you change the Processor Frequency or Add WiFi SSID or Password, it gets added/updated in this file.


3. The optional “components” directory contains components & libraries that are part of the project. It is useful for easy structuring, making easily portable reusable code or to include 3rd part libraries to ESP-IDF.


4. The “main” directory is of course the main folder that contains the source code for the project itself. It is considered as a “special” component folder by the ESP IDF


5. The “build” directory is where the build output is created. Sometimes Even before you hit build, this directory will be created. This happens when use configure the sdkconfig. CMake at this point, generates interim build files.


6. The Component directories each will also contain a component CMakeLists.txt file. It defined variables that control the build process of the component, and its integration into main source file.


7. Each component may also include a Kconfig file defining the component configuration options that can be set via menuconfig




#6 How do OTA Updates work in ESP32?

Over the Air Updates are essential for any IoT Devices. Other than adding features, the main concern it addresses is Security. Do you want to know how the OTA Mechanism works in ESP32 & also how it will handle a failed OTA Update? Check out this video from Makerdemy. This is a video from a course that I co-authored.



#7 Cloud OTA Updates in ESP32




Want to enable Cloud Over The Air Updates for your Prototype ESP32 Project?

Want to do this in super quick easy steps, and using minimum lines of code in your ESP IDF Project? – Mongoose OS is a good option!


Mongoose OS’s mDash IoT Cloud Device Management Platform and its libraries are top notch.


Check out Mongoose OS Guide On OTA Updates:

https://lnkd.in/g3XQFA_q


mDash Library: https://lnkd.in/gnagh_PX



#8 Instant Power Consumption Reduction in ESP32



Want to Reduce the Power Consumption of your Battery Powered ESP32 by 30% easily?


The Quick Solution is to do "Clock Reduction". 90% of battery powered projects on ESP32 will work almost the same with 80 MHz Clock Frequency instead of 240 MHz.


If you still need the full performance in short bursts, you may want to try out Dynamic Scaling of Clock Frequency.


You may also want to check out this video: https://lnkd.in/gpvpGTDr


For both cases, the page below is a good read:

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/power_management.html


#9 WiFi Manager & AutoConnect in ESP32



When we buy a new smart bulb, the first thing we do is input the WiFi name and password to which it must connect to. Want to add such a feature to your ESP32 Project? Furthermore, do you want the configuration local server to load automatically when you connect to, like those in Airport WiFi?


This WiFiManager Library from git user “tzapu” is what you need.


Git: https://lnkd.in/gXTUEWYk


#10 FreeRTOS Dual Core Performance in ESP32

We all know the ESP32 has 2 cores, but do you know most probably you are only using 1 core. ESP32 unlike Desktops doesn’t smartly use both cores unless explicitly programmed. You are leaving a lot of performance on table. FreeRTOS implementation on ESP32 is your friend to effectively utilize both the Cores.


Want to see the performance of ESP32 with and without FreeRTOS, and compared to Arduino UNO and ESP8266? Check out this Makerdemy Video on the same. This is a video from a course that I co-authored.


#11 Espressif's Libraries & Frameworks


Everyone knows the hardware Espressif makes. But You need to check out the Official Libraries and Frameworks too. They pretty much have a library stack for any leading fields now –Cloud, computer vision, AIoT, Matter, Speech Recognition and even Drones. We will cover them.


1. ESP-ADF : It is comprehensive framework for audio applications like Music Players, Recorders, Audio Processing, Bluetooth Speakers, Internet Radio

2. ESP-CSI : Sensorless Human Occupancy & Activity Detection using WiFi

3. ESP-DSP: The library provides algorithms optimized specifically for digital signal processing applications

4. ESP-WIFI-MESH: It is a development framework that uses WiFi to enable Mesh Capabilities in an IoT Network.

5. ESP-WHO: An Image Processing Development Platform that enables Human Face Dection, Recognition, Gesture Recognition and much more.

6. ESP-DL : Its a deep-learning library for AIoT applications which provides APIs for Neural Network Inference, Image Processing, Math Operations and some Deep Learning Models. It is used in ESP-WHO also

7. ESP-RainMaker: It is the most feature rich & easy to use platform for ESP32 Rapid Prototyping as it handles everything like OTA, Fleet Management, Apps, Voice Assistant Support, Easy Provisioning etc


There are so many more like, ESP-Skainet, ESP-Hosted, ESP-HMI, ESP-AVS, ESP-AFE, ESP-DRONE, ESP-NOW, ESP-AT, ESP-USB-BRIDGE to name a few.


You get the gist right? Want to learn more? Go here first: https://lnkd.in/g5hTBm7G


Want to see all the repos of ESP? Go here: https://lnkd.in/gYwpCNjc


#12 - ESP32 is actually a Triple Core Processor


It is common knowledge that an ESP32 has 2 Processors, but did you know that the Actual SoC package has 1 more Processor? Code for it is written in Assembly! What is its purpose? It allows the ESP32 to essentially Sleep Walk.


The ESP32 has an “Ultra Low Power (ULP) coprocessor”. While the Main Dual Core processors are in Deep Sleep, this Processor can be designated to perform for example:

1. ADC Measurements

2. I2C Sensor Measurements

3. Very Basic Programs


The Datasheet claims that in Deep Sleep Mode, while ULP is being used, the ESP32 only consumes 150uA, but the surrounding circuity determines this. The Chip may still be using the rated ultra low power, but the surrounding circuity can increase the current consumption even by an Order of Magnitude.


This Analysis by Jeremy Cook is a great read on this matter: https://lnkd.in/gYx7SBhS


#13 Ultra Low Power Coprocessor in ESP32


In my last post, we did an overview of the ULP Coprocessor on the ESP32. Now let’s dive a little deeper. First Let’s address the Assembly Language.


The ULP Registers: 4x 16 bit Registers + 1x 8 bit Register

1. R0, R1, R2, R3

2. STAGE_CNT – Stage Count, used to implement loops


The ULP Instruction Set: 26 Total Instructions. Doing an overview of this will make you understand the capabilities of the ULP. Will Cover basics:

1. Getting Value into registers: MOV (move) , ST (store), LD (load)

2. Arithmetic: ADD, SUB

3. Logical: AND, OR, Bitwise: LSH, RSH

4. Jump Commands: JUMP, EQ, OV(Overflow), JUMPS(relative jump)

5. STAGE Commands: STAGE_RST, STAGE_INC, STAGE_DEC

6. Program Control: HALT, WAKE, WAIT imm(wait for number of cycles)

7. Peripheral Control: TSENS(read from build in temp sensor), ADC, I2C_RD, I2C_WR


All Instructions are 32 bit. The instruction syntax is case insensitive. Other the peripheral support it can also access peripheral registers in RTC_CNTL, RTC_IO, and SENS, and thus access the 8k bytes of RTC_SLOW_MEM memory region. This means you can even buffer some set of sensor data. One the ESP32 Main course boot back up it can access all those in one go and possibly send to the cloud, and then go back to sleep.


For more detailed documentation, visit here: https://lnkd.in/g_CMfN7j


You can do ULP Programming in Arduino IDE, ESP-IDF, and Platform IO


#14 ESP32 supports 3 ways to store data. SPIFFS, FAT & NVS – Which should you use?


ESP-IDF on ESP32 supports 3 ways to store data. SPIFFS, FAT & NVS – Which should you use for your project?


NVS – Non Volatile Storage, which can store key value that supports different data types like integers 8-64 bits, strings, blobs. It supports Flash Encryption. As there is a 4000 byte limit on strings, 508,000 byte limit on blobs, I would recommend to use it only for storing factory settings and anything else which is reasonably small like storing device attributes like serial, hardware info, certificates, keys securely. It does not fail without power as it is “non volatile”.


SPIFFS & FATFS are file systems supported by ESP-IDF.


SPIFFS is used for storing large files. It is reliable as it does wear levelling. But unfortunately, it doesn’t support flash encryption. Fails without power, but fails safely


FATFS can also store large files, and unlike SPIFFS its support Flash Encryption. But it is not natively Flash-friendly, nor reliable. FAT FS natively doesn’t do wear levelling well, but ESP adds this. Like SPIFFS not only does it fail without power, it doesn’t do this safely, meaning chances of corruption are high.


Now that you understand the pro and cons of each, I hope you can choose correctly.


#15 Use SPIFFS in ESP32 to store small Files


ESP32 can store small files using SPIFFS (SPI Flash File Storage). You can think about SPIFFS as a very small SD Card. SPIFFS library allows access to flash memory as if it were a normal file system like that of a computer. How is it useful?


Let’s say you are creating an ESP32 Webserver, you would have to write or hardcode the HTML and CSS code into the source code itself. Using SPIFFS, allows you to actually store the HTML and CSS files as separate files that will be used by the source code.


I will list some good Resources to learn more about SPIFFS below:


1.https://lnkd.in/gw_ZcE-x

2.https://lnkd.in/gBHjr3uG


#16 Cloud Framework Agents for ESP32


Working with ESP-IDF for ESP32, means that you have full access to Cloud Framework Agents & Examples to essentially connect to any and all major cloud services. Let’s look at the rich set of offerings from Espressif.