Test Framework For Testing Embedded Systems In Python

I would like to test the features of an embedded device. To simplify I can say it is an humanoid robot remotely controlled by a PC through a C/C++ API. I am very interested to use

Solution 1:

Long time since this question was asked.

Embedded Systems present special characterists to implement acceptance testing automation (one of the most important is that, most likely, the "Device Under Test" is not the same device as the one executing the test cases; hence same kind of interaction interface is required). This is not "excatly" the case when doing test automation of a Web Page or a PC Application or even when running unit testing of an embedded software (which can also be executed outside of the device). Based on this assumption, I think a framework which is developed for doing unit testing is not the best tool to develop an Emedded System Test Bench for performing acceptance tests.

At the moment we are facing a similar situation trying to choose a development environment to implement automation testing for an embedded device. We are looking into:

• Robot Framework, which is a generic acceptance test automation framework based on keyword-driven testing approach.

• FitNesse (http://www.fitnesse.org)

• Pycopia

There are also other tools that don`t use Python. For example the ones described in this thread (MxVDev)

Solution 2:

I think the Robot Framework is the right tool for you. You can split your tests in test-suites and if one test fails the next test will run.

Solution 3:

We use htf for hardware in the loop tests. It offers many features to automate tests for embedded systems, a nice report, a dashboard, fixtures, data driven testing, tagging, Docker integration and it is professionally developed.

There is also a demo with some videos to get a quick overview.

It also supports manual tests with an interactive mode.

Binary data abstraction can be realized with oser.

htf can be used for free.

Solution 4:

Again, long time since this was asked, but I figured I could contribute.

We're currently building a complete test solution exactly aimed at testing embedded devices for verification & validation purposes. Our flagship implementation is based on Google's OpenHTF: https://github.com/google/openhtf

Here's the hello world example:

import openhtf as htf

from openhtf.output.callbacks import json_factory

from openhtf.plugs import user_input

@htf.measures(htf.Measurement('hello_world_measurement'))
def hello_world(test):
  """A hello world test phase."""
  test.logger.info('Hello World!')
  test.measurements.hello_world_measurement = 'Hello Again!'


if __name__ == '__main__':
  test = htf.Test(hello_world)
  test.add_output_callbacks(
      json_factory.OutputToJSON('./{dut_id}.hello_world.json', indent=2))

  test.execute(test_start=user_input.prompt_for_test_start())

You can extend OpenHTF with different modules:

• "Plugs", which are interfaces with external equipment or device under test. I.e. a COM Port plug.
• "Callbacks", which are custom export interfaces.

OpenHTF comes with its own GUI and definitely gives a huge headstart to development of either production testing testbenches or design validation/verification testing, as is the case for this question.

I'd be glad to help anyone in need of guidance.

Solution 5:

i'm starting to look at openhtf as well. I have two scenarios:

1. Testing should quit upon first failure. is this possible?
2. Testing should continue w/ the next test upon failure. is this possible? Also does someone have a nuts-to-bolt usage example for openhtf posted on a share site such as git-hub.