Integration Testing Challenges for the IoT

Integration testing applications for the Internet of Things face many challenges. A big one is that the standards and protocols for building these connected devices are in the early phases, which can limit the ability to use tools like service virtualization to find defects earlier in the development process.

To address this gap, the Internet Protocol for Smart Objects Alliance (IPSO) is working on helping to improve the underlying infrastructure for connecting smart objects via Internet Protocols. We reached Christian Légaré, CTO, Micrium Inc. and Vice President, IPSO Alliance to get some answers on what this means for the enterprise. Where is IoT entering the enterprise?

Christian Légaré: Although historically many new technologies have entered the enterprise at the product design level, that isn’t the trend we’re seeing with the IoT. Over the past few years, becoming part of the IoT has become a C-level decision. Often, CMOs or CIOs make this decision, which is based on the following considerations:

1)      Can it increase product or service features;

2)      Can it increase customer loyalty by providing a better customer experience, or better customer service response to product failures; and

3)      Does it offer opportunities for cost reduction (through for example better knowledge of the production process, better maintenance programs, etc.).

What is the IPSO alliance doing to improve IoT integration?

For the IoT to be successful, IT and embedded systems need to work together. IT is the realm of web languages like Java, Python, Ruby and the like. Servers are primarily running Linux and every component talks to other components via APIs. However, the embedded systems that will run the majority of the devices in IoT are deeply embedded, use C as their main language, and run with or without an operating system. In the IoT, these universes collide, and must learn to work together, so integration becomes critical.

The IPSO Alliance is working to define the software architecture, at the IP networking level, to help bridge the gap between embedded systems and IT. Standard protocols exist today and this is what the IPSO Alliance wants to demonstrate with its Reference Architecture Committee (RAC). IPSO is working to provide:

  • A full stack device reference architecture, including  security, management, provisioning, and smart object definitions for semantic interoperability with an initial focus on a RESTful implementation;
  • An initial architecture white paper, providing the contexts of IPSO devices and smart objects;
  • Other architectures towards different communications mechanisms (for example: publish/subscribe) plus interoperability, mapping and application protocols in general;
  • The application of these reference architectures will vary depending on the type of devices (wireless sensor node, consumer device, gateway, etc.). The hardware resources available and its cost dictates how much of the reference architecture can be implemented directly in the end device or has to be centralized in a gateway.

What kinds of shifts are required in the way people implement IoT applications?

When implementing IoT applications, designers always have to keep in mind the data that will be generated by the devices and how they can store, analyze, use (and use in conjunction with other data sources), as well as extract directions and conclusions about their products, customers and business in general. This is requiring a shift wherein the product and system designs need to be done by collaborative IT/embedded teams to effectively address the full range of design considerations.

What are some of the best practices for addressing this shift in an efficient and effective manner?

IoT implementation is a multidisciplinary activity, so expertise will be needed from the following teams from the inception of the project: IT, infrastructure, database, web applications, network management, electronic hardware, embedded software. Though the IoT may be in its infancy, there are many experts and resources available. One example is the IPSO Alliance, which is documenting how to use the existing IP protocols to meet the requirement of Smart Objects (i.e. the devices) and IP networking. Companies will benefit from taking advantage of this type of existing resources as they enter the new world of the IoT.

How does the rise of IoT applications change the nature of software testing in an agile software environment?

The existing software development tools are adequate, however, vertical markets will need to address questions regarding security and privacy. For example, will we have to establish certification processes like we do for industrial (IEC), medical (FDA or IEC), consumer (UL or IEC) and avionics (FAA).

What is the current state of tools and practices for simulating the pieces of software and devices required for integration testing and performance testing new IoT related code?

Right now there are a variety of approaches and little consensus across the industry. This is leading some industry groups, like the Industrial Internet Consortium (IIC), to define interoperability testing.

The IPSO Alliance is also planning an interoperability day for SmartObjects in its May 2015 member meeting in Stockholm. These types of industry discussions are needed to develop some level of standardization across the IoT regarding integration and performance testing for IoT-related code.