Hello,
We're planning to make some changes to the way you can run Mbed TLS unit tests on embedded platforms. We want to make it easier to run the tests on platforms that have limited space for code and don't have a filesystem to read data from.
At the moment, there are two ways to build the test suites. If you run 'make test', this builds one a host executable using host_test.function for each test_suite_*.function and one “compiled” data file for each test_suite_*.data, and the executable reads the compiled data file at runtime. It's also possible to build an executable with target_test.function, which includes calls to the Mbed-tools Greentea library to communicate over the serial port, and you can use the script mbedtls_test.py to run the tests with Greentea. Neither method works directly for embedded platforms other than Mbed OS.
If you've run the Mbed TLS unit tests without reading the .datax file from the filesystem, and without using Mbed OS and Greentea, how did you do it? If you haven't run the unit tests but would like to do so, what changes would you like to see what Mbed TLS provides?
I have a few more specific questions:
• Does Mbed TLS need to include some abstraction for serial communication? If not, what would the interface of the test functions be? (Tests can currently take inputs that are zero-terminated text string, binary string with length, or int, and we'd like to add the possibility to add other integer types such as uint64_t, but nothing more complex. The outcome of a test is pass/fail/skip, and if fail/skip location information about the failure.)
• Is it useful to have a way to build the test data into the executable, so that the only interaction that the executable needs is to write out the test results? It's more convenient but doesn't work everywhere (the biggest test suite has >200kB of test data).
• Should test executables be self-contained in their reporting (including error messages)? Or should they minimize code size, and then you'd need an extra host-side script to make sense of results?
• The current .function-to-.c transformation script (generate_test_code.py) generates a single C file with all of the test code (but not the test data). Is this a useful requirement?
We welcome any thoughts you may have on the topic.
-- Gilles Peskine Mbed TLS developer
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.