ud3tn/doc/testing.md

µD3TN Testing Documentation

µD3TN relies on intensive testing to ensure system stability as well as project quality. The following information should be taken into account by all project contributors.

There are currently three ways to check for code quality issues and verify the proper function of µD3TN:

• Static analysis
• Unit tests
• Integration tests

The tests can be either executed manually, or automatically via a Continuous Integration server. This document summarizes which tests are available and how to execute them.

For patches or pull requests to be accepted, all of the below-listed tests have to return a successful result on all platforms.

Static Analysis

Compiler Warnings

Most of the available compiler warnings are turned on automatically in the Makefile. This usually gives already an indication of some obvious problems in new code. To check that there are no problems compiling µD3TN for various platforms, the following commands should be run:

make clean && make posix type=debug werror=yes
make clean && make posix type=release werror=yes
make clean && make posix TOOLCHAIN=clang werror=yes

Linter (clang-tidy)

Clang Tidy is an extensible linter that can be used to check for some typical programming errors. It can be executed for µD3TN as follows:

make clang-tidy-posix

Stylecheck

The checkpatch.pl utility available in the Linux kernel source tree is used for µD3TN to enforce compatibility with the project's coding style which is the one used for the Linux kernel. The coding style compliance check can be executed by running:

make check-style

No errors or warnings must be shown.

Unit Tests

For unit testing, the lightweight Unity test framework is used. It provides a simple API and everything necessary to check assertions and generate a test report.

The tests are located in test/unit and, including the Unity test framework, are compiled into the µD3TN test binary (via make unittest-posix).

POSIX

To run all unit tests against the POSIX port of µD3TN, a make command is provided:

make run-unittest-posix

This will automatically build µD3TN plus the tests and execute them. An output similar to the following should be displayed:

[...]
......................................

-----------------------
44 Tests 0 Failures 0 Ignored
OK
[Mon Mar 26 14:26:00 2018]: Unittests finished without errors! SUCCESS! (-1) [components/test/src/main.c:72]

Integration Tests

There are several integration test scenarios which check µD3TN's behavior. For the integration tests to work, an instance of µD3TN first has to be started and the Python venv has to be activated.

Example for POSIX:

Start µD3TN in a dedicated terminal:

make posix
build/posix/ud3tn -R -r

!!! note

µD3TN needs to be started with the flags `-R`, `--allow-remote-config` and `-r`, `--status-reports` as both features are needed by the integration tests.

Start the integration tests in a separate terminal:

source .venv/bin/activate
make integration-test

!!! note

The integration tests can also be run against µD3TN combined with the compatible external BDM. This is achieved by specifying the `-d, --external-dispatch` command line argument when invoking µD3TN and starting the BDM service additionally via `aap2-bdm-ud3tn-routing --insecure-config` (inside the Python virtual environment). The `--insecure-config` flag is necessary as some integration tests inject configuration commands as if they were received from an external source.

ION Interoperability Tests

µD3TN ships with simple interoperability test scenarios that validate BPv6 and BPv7 interoperability with JPL's ION and are described in the following subsections. At the end of this section, some hints on doing manual interoperability tests are provided.

!!! note "Interoperability tests with other implementations"

µD3TN also ships with interoperability test scripts interacting with [HDTN](https://gitlab.com/d3tn/ud3tn/-/blob/master/test/run_hdtn_test.sh) and [DTN7-rs](https://gitlab.com/d3tn/ud3tn/-/tree/master/test/dtn7_interoperability), plus corresponding [Docker images](https://gitlab.com/d3tn/ud3tn-docker-images/container_registry). The usage of these scripts and images is equivalent to the ION tests described below.

Prerequisites

All tests expect an environment with ION and µD3TN (incl. Python dependencies) installed. There are two options to set up all requirements:

1. Use the Docker image:

   • Run a container built from the registry.gitlab.com/d3tn/ud3tn-docker-images/ion-interop:<version> image, whereas <version> is the ION version you would like to test against. The µD3TN installation/source should be mounted inside the container and the µD3TN binary should be built with the settings you would like to test.

     docker run -it -v "<ud3tn-path>:/ud3tn" registry.gitlab.com/d3tn/ud3tn-docker-images/ion-interop:<version> bash
     

   • Activate the Python virtual environment and prepare the µD3TN modules:

     source /ud3tn_venv/bin/activate
     pip install -e "/ud3tn/pyd3tn"
     pip install -e "/ud3tn/python-ud3tn-utils"
     

   • You can accelerate these two steps by using the script we maintain for our CI toolchain. Replace <version> by the ION version and <build-args> by additional arguments to be passed to µD3TN's make command (e.g. for enabling sanitizers or other compile-time preferences; may be left out). The script builds µD3TN and sets up the Python dependencies. It assumes that the µD3TN source is mounted under /ud3tn and one of our Docker images is used.

     cd <ud3tn-path>
     docker run -it -v "$(pwd):/ud3tn" registry.gitlab.com/d3tn/ud3tn-docker-images/ion-interop:<version> \
         bash -c '/ud3tn/test/ion_interoperability/prepare_for_test.sh /ud3tn /ud3tn_build <build-args> && cd /ud3tn_build && source /ud3tn_venv/bin/activate && bash'
     

   • Now you can cd /ud3tn and run the test scripts from there.

2. Set up the environment manually.

   • Install ION according to the instructions provided with it.
   • Obtain the version of µD3TN you would like to test (or mount it as a volume if you are using a container).
   • Install µD3TN's dependencies and build µD3TN as documented in README.md and the Quick Start Guide.
   • Install the Python dependencies as documented in the Quick Start Guide.

Minimal Forwarding Test

This test, provided in test/ion_interoperability/minimal_forwarding_test, checks the following aspects:

• forwarding bundles from µD3TN to another instance of µD3TN via an ION instance using TCPCL
• receiving bundles sent by ION (via bpsource) in µD3TN (via aap2-receive)
• receiving bundles sent by µD3TN (via aap2-send) in ION (via bprecvfile)

The script run.sh expects two arguments:

/path/to/run.sh <bp-version> <scheme>

• <bp-version>: the Bundle Protocol version to be used, i.e., 6 or 7
• <scheme>: the EID scheme to be used, i.e., dtn or ipn

When invoked, the script will start ION (via ionstart -I) and two µD3TN instances, configure µD3TN to connect to ION via TCPCL, and try to send three bundles to test the three aspects mentioned above. The ION configuration (rc) files can be found in the same directory as run.sh, for the dtn and ipn EID schemes, respectively. On completion, ION will be shut down using ionstop and the µD3TN instances will receive a SIGTERM.

MTCP Minimal Forwarding Test

This test is equivalent to the "Minimal Forwarding Test", but uses the MTCP (ION: STCP) convergence layer. It is located in test/ion_interoperability/mtcp_minimal_forwarding_test.

Note: To be compatible with ION's STCP (Simple TCP) convergence layer, which implements an earlier version of the MTCP draft, pass the following argument to make when building µD3TN: CFLAGS=-DCLA_MTCP_ION_STCP_COMPATIBILITY=1 (or set this in config.mk)

BIBE Interoperability Test

A second test script checks whether forwarding of a bundle encapsulated using BIBE works via ION, to validate µD3TN's BIBE implementation.

This test always uses BPv7 and the dtn scheme, thus, no command line arguments are necessary. The used ION configuration (ionstart.rc) can be found in the test directory.

Manual Interoperability Testing

Beside using the prepared test scripts, manual interoperability testing is possible. Developers are advised to check out the "minimal forwarding test" shell script (run.sh) to get an idea which commands have to be executed.

In summary:

1. Prepare the environment as described above.

2. Start ION and configure it as needed:

   • If ION should receive bundles from µD3TN, there should be at least one induct configured that µD3TN can connect to.
   • If ION should send bundles to µD3TN, you should configure a) an outduct and b) an egress plan. Check out the configuration of the minimal forwarding test to get an idea of this.
   • Note that the only Convergence Layer Adapter currently supported in both µD3TN and ION is TCPCLv3 (µD3TN option -c tcpclv3:<host>,<port>).

3. Start µD3TN instances as needed. If you want to test with multiple instances, don't forget to specify different sockets/ports for AAP (via -s or -a/-p) and the CLAs (via -c). Refer to the README.md concerning the command line syntax. You can also find some pointers on running multiple instances in the Quick Start Guide.

4. Configure the µD3TN instance(s) as needed: Especially, add all outgoing contacts to other µD3TN or ION instances using aap2-config, e.g.:

   aap2-config --tcp <aap2-host> <aap2-port> \
       --schedule <start> <duration> <rate> \
       <eid> tcpclv3:<host>:<port>
   

5. Start the receiving end(s). If you want to receive bundles sent to a µD3TN instance, attach an aap2-receive sink, e.g. via:

   aap2-receive --tcp <aap2-host> <aap2-port> \
       -a <demux|service-number> -v
   

   See the help output of aap2-receive (via -h) for additional options to verify received payload data, etc.

   If you want to receive data in ION, you can start an instance of bprecvfile or bpsink (see ION's documentation).

6. Inject bundles / run your tests. For sending a bundle through µD3TN, you can use aap2-send, e.g. via:

   aap2-send --tcp <aap2-host> <aap2-port> \
       --agentid <demux|service-number> <dest-eid> "<payload-data>" -v
   

   If you want to send a bundle through ION, you may use bpsource (see ION's documentation).