Author: Luka Mustafa, CEO
At IRNAS automated testing is a the heart of product development, aligned with our development principles. Low power operation is presently critical with small Bluetooth devices and must as such be carefully managed throughout the development and manufacturing process. IRNAS integrates best in class components when designing novel products. Industrial and consumer solutions often consist of Nordic Semiconductor nRF52810 packaged in InSightSIP ISP1507-AL for very low power operation. Likewise, low-power but with LoRaWAN connectivity, Murata Type ABZ modules are used in a number of production projects at the moment, for example, SmartParks Lion tracker prior to moving to the latest generation LR1110 LoRaWAN transceivers as described in a recent blog update.
Making sure each revision of the firmware is fully operational as well as at the same time preparing the automated manufacturing test process, a cost-efficient and flexible setup is defined. Advanced features of Otii Arc power supply and measurement unit are used to create a real-time interface between the device power consumption and Python-based test framework. Custom designed peripheral solutions enable additional interfacing to the devices under test for maximum efficiency.
We are fortunate to be able to base this blog around the open-source project for SmartParks – Lion tracker OpenCollar.io where we are able to disclose the source code of these tools and offer you the best insight into our process.
Benefits
Fully automated test system benefits are best recognized in catching potential device problems during development as well as ensuring every device manufactured is flawless at the exit of the production line. Particularly for smaller batches of 100-1000 devices, the cost of development and test system setup has traditionally proven to be a limiting factor. Not having them in place introduces a high risk that we are not willing to afford. We have invested a lot of thought into our design process to minimize this extra cost. This way even the early trials are spared experiencing bugs in operation and can focus on other aspects that product owners should pay attention to.
Moreover, automated testing allows us to break the system down into small manageable modules that we can then optimize individually. This way we can maximize devices’ performance as well as battery lifetime, as power consumption can be tuned for each module or state.
WEBINAR: Hands-on experience with LR1110 LoRaWAN transceiver with GNSS and WiFi functionality
Semtech LR1110 LoRaWAN transceiver with GNSS and WiFi scanning evaluation has shown promising results. To best share our hands-on experience at the first instance we are organizing this session to explore the capabilities of the solution and share our initial experience.
More Details
The test bench
The test bench is set up to enable the following key features:
- Measure power consumption per firmware function (or a set of them), for example measuring the analog channel, LoRaWAN transmission, BLE transmission, LED blink and others
- Automatically provision device firmware and parameters with a set-up that fully disconnects the programmer to avoid any current leakage
- Actuate and detect device inputs and outputs, for example, detect an LED blink, create an analog pulse on the input etc.
- Connect via LoRaWAN or BLE and validate correct operation and settings.
Hardware setup in one particular example consists of:
- Otii Arc power supply and measurement unit
- Tag Connect Otii ST-Link
- IRNAS otii power consumption analysis python code available open-source
The software
The key to a fast test procedure is the use of Otii Enterprise (or Otii Automation Toolbox), which provides a TCP-server for simple Otii control using a JSON-based API. Main program is a Python script which uses the unittest module and a custom wrapper around Otii TCP Client for Python. Basic features include firmware upgrade and power analysis. Depending on the project these are extended with BLE connectivity and I/O capabilities to interface with the device.
Test configuration
Test configuration is specified in JSON which makes it most universal and available for the user to readily changed. This is best observed in the Lion tracker use-case as shown below, which is configured to operate as a state-machine solution. With an UART print of state transitions, one can easily mark the time when these transitions happen, as well as the power consumption in-between. Then by defining the boundaries for these transitions, we can evaluate the correct operation.
"from"is the message where measuring begins, matches text"to"is the message where measuring ends, matches text"avg_limit_low"is the lowest amount of energy (in μWh) that can be used up in the specified duration (on average)."avg_limit_high"is the highest amount of energy (in μWh) that can be used up in the specified duration (on average)."timeout"is the highest amount of time (in ms) allowed between messages. If timeout is0, the limit is infinite.
On the firmware side, the following states are defined and printed in the enumerated format as transitions: fsm("from">"to") on which the evaluation is specified.
enum state_e{
INIT,
LORAWAN_INIT,
LORAWAN_JOIN_START,
LORAWAN_JOIN,
GENERAL_INIT,
IDLE,
SETTINGS_SEND,
STATUS_SEND,
RF_SEND,
GPS_START,
GPS_READ,
GPS_SEND,
GPS_LOG_SEND,
LORAWAN_TRANSMIT,
HIBERNATION
};Each of these transitions is evaluated for power consumption. The sample test config below shows that INIT state between two cycles of main code in this state should not use more than 0.1 uWh. The timeout is not defined as that duration may vary. Another more interesting test is of the GPS subsystem as part of the GENERAL_INIT procedure, such that the correct operation and power consumption of GPS, step-down converter and switching circuit is tested at the same time. An ERROR can be reported via UART directly and that test will fail, while at the same time this happens if the energy consumed is outside the boundaries or the duration is longer than the timeout.
...
{
"from": "fsm(0",
"to": "fsm(0",
"avg_limit_low": 0,
"avg_limit_high": 0.1,
"timeout": 0
},
{
"from": "fsm(3",
"to": "fsm(4",
"avg_limit_low": 100,
"avg_limit_high": 170,
"timeout": 8
},
...Test results
The test progress can be monitored using Otii Enterprise (or Otii Automation Toolbox) software. We can observe power consumption as well as state parameters throughout the test procedure.
The test output is a report of power consumption for all tested states in the sequence. The output below shows a fully functional device with power consumption within limits in all states.
Upon testing a batch of devices, one should expect most devices will pass all the tests. The ones that don’t must, of course, provide all debugging information so the problem can be resolved. Using this method we have been very successful to immediately recognize even the unexpected issues such as:
- GPS IC not soldered correctly
- Short-circuit in step-down converter
- LoRaWAN RF problem as the transmission failed
As part of the development process, the same automated test is run and power consumption evaluated for each state separately. From one revision to the next we can then reliably determine if any bugs have been introduced that affect the overall power consumption as well as how well are we managing to improve the solution. This method and tools have enabled us to improve the robustness and therefore the efficiency of our development process as it gives us a chance to validate all modules and functionality in a quick and easy way at all times. We are excited to further develop our automated testing methods with tools like Otii.
