|
| 1 | +# ESP-Modem Testing |
| 2 | + |
| 3 | +This folder contains automated and manual tests for esp-modem component. Beside these tests, some jobs are executed in CI to exercise standard examples (please refer to the CI definition and CI related sdkconfigs in examples). |
| 4 | + |
| 5 | +List of test projects: |
| 6 | + |
| 7 | +* `host_test` -- esp_modem is build on host (linux), modem's terminal in mocked using Loobpack class which creates simple responders to AT and CMUX mode. This test is executed in CI. |
| 8 | +* `target` -- test executed on target with no modem device, just a pppd running on the test runner. This test is executed in CI. |
| 9 | +* `target_ota` -- Manual test which perform OTA over PPP. |
| 10 | +* `target_iperf` -- Manual test to measure data throughput via PPP. |
| 11 | + |
| 12 | +## Manual testing |
| 13 | + |
| 14 | +Prior to every `esp_modem` release, these manual tests must be executed and pass |
| 15 | +(IDF-9074 to move the manual tests to CI) |
| 16 | + |
| 17 | +### `target_ota` |
| 18 | + |
| 19 | +Make sure that the UART ISR is not in IRAM, so the error messages are expected in the log, but the ESP32 should recover and continue with downloading the image. |
| 20 | + |
| 21 | +Perform the test for these devices |
| 22 | +* SIM7600 (CMUX mode) |
| 23 | +* BG96 (CMUX mode) |
| 24 | +* SIM7000 (PPP mode) |
| 25 | +* A7672 (CMUX mode -- the only device with 2 byte CMUX payload), so the test is expected to fail more often if (`CONFIG_ESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD=y` && `CONFIG_ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED=n` && dte_buffer < device max payload) |
| 26 | +* NetworkDCE -- no modem device, pppd (PPP mode) |
| 27 | + |
| 28 | +Perform the test with these configurations: |
| 29 | +* CONFIG_TEST_USE_VFS_TERM (y/n) |
| 30 | +* CONFIG_ESP_HTTP_CLIENT_ENABLE_CUSTOM_TRANSPORT (y/n) |
| 31 | +* CONFIG_ESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD (y/n) |
| 32 | +* CONFIG_ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED (y/n) |
| 33 | + |
| 34 | +**Criteria for passing the test** |
| 35 | + |
| 36 | +The test should complete the download with maximum 1 retry (50% of OTA failure) |
| 37 | +In case of CMUX mode, we're trying to exit CMUX at the end of the test. This step might also fail for some devices, as the CMUX-exit is not supported on certain devices (when the final error message appears that the device failed to exit CMUX, we just verify the new image by reseting the ESP32) |
| 38 | + |
| 39 | +### `target_iperf` |
| 40 | + |
| 41 | +Run these 4 `iperf` configurations (either manually or using `pytest`): |
| 42 | +* tcp_tx_throughput |
| 43 | +* tcp_rx_throughput |
| 44 | +* udp_tx_throughput |
| 45 | +* udp_rx_throughput |
| 46 | + |
| 47 | +And verify in all four cases the value is about 0.70 Mbps |
0 commit comments