Commit graph

757 commits

Author SHA1 Message Date
Felix Walter
9fc5f753b4 hal_task: Remove custom priorities and stack size
We implemented this to mirror the behavior on STM32 with FreeRTOS. We do
not need it anymore on POSIX systems, so better go with the defaults of
the OS.

Closes: #162

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-12-20 10:32:22 +01:00
Felix Walter
60645f1286
Merge branch 'hotfix/164-dtn7-socket-conflict'
See !141
2023-12-18 15:42:10 +01:00
Felix Walter
865151474f
Merge branch 'hotfix/157-valgrind-build-test'
See !139
2023-12-18 15:41:52 +01:00
Felix Walter
9ac47887ca
Merge branch 'feature/ci-add-libud3tn-test'
See !140
2023-12-18 15:40:56 +01:00
Oleksandr Nazymko
a8fb34530b CI: Add test for libud3tn.so
This adds test for shared library of uD3TN to the CI. The test builds and starts uD3TN from library and runs integration test.

Signed-off-by: Oleksandr Nazymko <oleksandr.nazymko@d3tn.com>
2023-12-18 12:54:53 +01:00
Felix Walter
3c4a69e3f7 test: DTN7: Specify AAP 2.0 socket explicitly
As we do not allow reuse of the same socket file by concurrently running
uD3TN instances, uD3TN may crash on startup if we do not use different
socket filenames.

See: https://gitlab.com/d3tn/ud3tn/-/jobs/5740147850#L232
See: #156
Closes: #164
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-12-14 11:15:53 +01:00
Felix Walter
9e8e0d3d1f CI: Remove job "build-valgrind-memory-check"
Everything that this job does is covered by the more extensive
integration test job `integration-test-valgrind-memory-check`.
There also is a potential race condition which makes this test flaky,
see #157.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-12-05 10:12:26 +01:00
Felix Walter
1c54cf18a4
Merge branch 'feature/add-dtn7-interoperability-test'
See !133
2023-12-05 10:00:53 +01:00
Felix Walter
f0be8b5d14 Change used image in CI "dtn7-interoperability-test"
See suggestion in !133: This uses the correct Docker image which
contains the fix from DTN7's `master` branch.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-12-05 10:00:00 +01:00
Felix Walter
cc296ca258
Merge branch 'feature/logging-improvements'
See !135
2023-12-01 07:43:54 +01:00
Felix Walter
cfca8700a1
Merge branch 'fix/nix-develop-build'
See !138
2023-12-01 07:42:52 +01:00
Felix Walter
7d16dbbb2c
Merge branch 'hotfix/158-nix-runner-broken'
See !137
2023-12-01 07:42:08 +01:00
Felix Walter
857c412469
Merge branch 'hotfix/156-hdtn-test'
See !136
2023-12-01 07:41:27 +01:00
Fiona Fuchs
c1c7f3f4b8 Adjust exit_handler in bash script of DTN7 interoperability test
This excludes the case when one of the process IDs is zero, otherwise `kill` would unnecessarily send `SIGTERM`.

Signed-off-by: Fiona Fuchs <fiona.fuchs@d3tn.com>
2023-11-30 11:46:35 +01:00
Felix Walter
685f0631bd CLA: Abort on failure to accept connection
This failure case effectively prevents the CLA from accepting further
connections and, thus, should be handled explicitly. See discussion in
!135 - we propose to restart uD3TN via a service manager or fault
handler in this case.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-30 10:59:52 +01:00
Fiona Fuchs
df1133cdb1 Change to downloading DTN7 from GitHub instead of crates.io
This changes the download source of DTN7 from the version on crates.io (version 0.19.0) to the one on the GitHub repository of DTN7 (commit 732d1a0). This was due to a bug on the initial version, that threw an error (InvalidUrlFormat) in an irregular manner. This bug has been fixed with the most recent commit.

Signed-off-by: Fiona Fuchs <fiona.fuchs@d3tn.com>
2023-11-30 10:19:50 +01:00
Maximilian Nitsch
f98dc4c673 nix: Disable compiler/linker hardening features for the devShell environment
With enabled hardening, nix adds `-D_FORTIFY_SOURCE=2` to the compiler
options, which only works in combination with the optimization level
`-O2`.[^1] For debug builds, optimization is disabled in the Makefile,
which leads to the following warning:

`warning _fortify_source requires compiling with optimization (-o)`

To remove this warning, hardening is deactivated for the devShell
environment.

[^1]: https://nixos.org/manual/nixpkgs/stable/#sec-hardening-in-nixpkgs

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2023-11-29 17:05:56 +01:00
Fiona Fuchs
4fa4a7cf0e Add changes in Bash Script
This modifies the `exit_handler`, so that all the logs will still be printed if any `wait` statement returns an error code. In addition, redundant checks at the end of setting up each instance are removed, as they did not work properly.

Signed-off-by: Fiona Fuchs <fiona.fuchs@d3tn.com>
2023-11-29 11:15:27 +01:00
Fiona Fuchs
3616774d43 Add uD3TN - DTN7-rs interoperability test
This adds
- a Dockerfile, which contains all the necessary dependencies for the interoperability test,
- a bash script, that performs bundle forwarding between uD3TN and DTN7-rs,
- a config file for the bundle forwarding scenarios in the script,
- and a CI job, which runs the uD3TN - DTN7-rs interoperability test.

Signed-off-by: Fiona Fuchs <fiona.fuchs@d3tn.com>
2023-11-28 15:43:04 +01:00
Felix Walter
5dd53b37c8 Ensure proper termination on fatal errors in release builds
Previously we were using ASSERT(0|false), which only works in debug
builds. In some cases, this is intended, but we cannot reliably continue
operation in others. We now use abort() where we have to terminate the
program abnormally. In cases where only debug builds should terminate, a
comment is added.

This is done along with the logging updates as the code is typically
related and would cause conflicts otherwise.

Closes: #163
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-24 15:52:00 +01:00
Felix Walter
4ec1e4642a bundle_processor: Improve debug output concerning reassembly
Fixes: #48
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-24 15:39:54 +01:00
Felix Walter
6525a453ba Adjust all logging calls to include a log level
Rules that have been applied:

- ERROR: Anything that is or may be problematic for the program to
  continue running.
- WARN: Anything abnormal but definitely not critical for program
  continuation, especially issues encountered when interacting with
  clients (e.g. AAP).
- INFO: All state changes not corresponding to the "normal" processing of
  individual data or messages/bundles, as well as further informative and
  helpful messages. Includea errors that occur during normal data
  processing.
- DEBUG: Messages to trace what happens with individual bundles and
  other fine-grained information about internal processes.

At the moment, `LOG_ERRNO` always emits messages with the "ERROR" level.
We might want to change this in the future to better differentiate WARN
and ERROR in this regard also.

Closes: #140
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-24 14:50:25 +01:00
Felix Walter
49e039a89a CI: Use Docker runner for NixOS job
This uses the `nixos/nix` Docker image on a standard Docker CI runner to
execute the Nix build job. We already tested and reviewed this in !134 -
it does not greatly increase the pipeline runtime. Another advantage is
that it validates the build without an active Nix cache.

In addition to the configuration in !134 it pins the NixOS image version
to prevent issues based on a different version in this regard (e.g.,
differences in experimental command or submodule support).

Note that currently newer images than `nixos/nix:2.18.1` do not work
because of issues with submodule handling. This is tracked in #160.

Fixes: #158
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-23 13:32:23 +01:00
Felix Walter
d39876f557
Merge branch 'fix/nix-flake-aap2-protobuf'
See: !134
2023-11-21 11:53:45 +01:00
Felix Walter
96743e4058 test: HDTN interop: Specify AAP2 socket explicitly
Otherwise we run into a conflict with the default socket and one
instance may not start properly.

Fixes: #156

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-21 11:51:51 +01:00
Maximilian Nitsch
a84501cf03 ud3tn-utils: Set the minimum required Python version to >=3.7
protobuf==4.21.12 requires python>=3.7[^1]. To resolve the version
conflict between μD3TN and the protobuf package, the required minimum
Python version is set to >=3.7.

[^1]: https://pypi.org/project/protobuf/4.21.12/

Closes: #153
Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2023-11-16 17:02:33 +01:00
Maximilian Nitsch
8f55444280 nix+ud3tn-utils: Use protobuf==4.21.12
Since protobuf==4.24.4 is not (currently) part of nixpkgs, but
protobuf==4.21.12, the pinned version is changed to reduce the effort
for packaging version 4.24.4 itself.

The protobuf Python language bindings have been regenerated to
incorporate the changes from the modified version.

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2023-11-16 17:02:33 +01:00
Maximilian Nitsch
a6f0bcb136 CI: Add nix build job
Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2023-11-16 17:02:20 +01:00
Felix Walter
7b7ff98a2d cmdline, config.mk.example: Make it clear that log level 4 is debug-only
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-16 14:20:04 +01:00
Felix Walter
8dbbb9d692 hal_io: Add LOG(F)_INFO to explicitly log with "INFO" level
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-16 14:20:04 +01:00
Felix Walter
1b1285c126 cmdline: Set default log level for release builds to 2 (WARNING)
This reduced unnecessary clutter by default if we do not want to debug
uD3TN. Should also slightly increase performance by issuing less
`printf` statements.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-16 14:20:04 +01:00
Felix Walter
afc53e0658 Add missing includes to hal_time.h
These were shadowed by `hal_io.h` transitively including it.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-16 14:20:04 +01:00
Felix Walter
f02fdc042d hal_io: Protect logging via mutex
This resolves the race condition documented in #139.
The log macros now each call only a single function which locks
a binary semaphore during execution.

Fixes: #139

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-16 14:20:04 +01:00
Maximilian Nitsch
3ee60453ca nix: Add protobuf for AAP2
To build μD3TN with AAP2, the protobuf language bindings in the
`generated/` directory must be added as an input source.

In addition, `python3Packages.protobuf` and `python3Packages.setuptools`
must be added as a dependency for `python-ud3tn-utils`.

To interactively generating nanopb protobuf C-language bindings via
`make aap2-proto-headers` additional packages are added to the devShells
environment.

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2023-11-16 14:12:26 +01:00
Felix Walter
43c67f1e68
Merge branch 'feature/aap20'
See !126
2023-11-14 16:51:10 +01:00
Felix Walter
b26939ce83 HDTN test: Clean up and remove race conditions
- `bpgen` sometimes sends a different amount of bundles, supposedly because
  of a race condition. We already test the transmission of bundles this part
  was testing with the following test.
- We wait for graceful termination of uD3TN because of the leak
  sanitizer that runs on termination. We do not need this for HDTN and
  other utilities. We should also make it part of the normal testing
  routine, not of the exit function, which might fail and not print the
  logs if waiting for uD3TN returns non-zero.
- Some delays were unnecessary.
- A delay is introduced before `bpsendfile` to ensure it is only run
  after `aap_receive` has connected to uD3TN.
- We now ensure the uD3TN 2 log is always shown.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-13 11:45:38 +01:00
Felix Walter
dfbec5d175 tools/analysis: Pin Python package versions
See also: #145

This ensures that the analysis jobs will not break in the future.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-09 14:33:48 +01:00
Felix Walter
2dd83bb7be CI: Fix path to coverage report
The report is created in `/ud3tn_build`, not in the directory
created by GitLab runner. However, it has to be uploaded from there.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-09 14:33:48 +01:00
Felix Walter
d797c816c9 CI: Do not install Python modules twice
This is already done by `prepare_for_test.sh`.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-09 14:33:48 +01:00
Felix Walter
f31f7f590a CI: Ignore gcov parse errors in coverage test
In some cases the coverage test might produce invalid coverage files due
to a problem that the GCC instrumentation has with detached threads,
which we are using by default.

See docs: https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fprofile-update

See also issue: #147

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-09 14:33:48 +01:00
Felix Walter
485a06b381 AAP, AAP2: Move common socket handling parts to dedicated module
There is some common functionality that should not be copied.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-09 14:33:48 +01:00
Felix Walter
82d90da82b python-ud3tn-utils: Add new modules to setup.py
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-09 14:33:48 +01:00
Felix Walter
e11b7826f7 python-ud3tn-utils: Document public interfaces of AAP2Client
This adds proper docstrings including documenting return values and
exceptions for the `aap2_client` module.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
5e6cfb5f7e tools/aap2: Move keepalive argparse argument to common helpers
We now perform the range check via argparse and provide a function to
add the parser argument like with other common parser arguments.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
22c6511a03 Pin version of protobuf Python dependency
See: #145, !126

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
aef2b668c6 style: Fix style of typedef usage in aap2_client.c
This adds the AAP 2.0 typedefs to the checkpatch list, so it will not
complain about style when declaring a variable or function argument as
`<typedef> *const <name>`.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
594eca6c8d Apply first suggestions from !126
- remove comments on "callback-style" serialization (not needed)
- define magic number in Python AAP 2.0 client via descriptive var
- make `secret` in Python AAP 2.0 client a private variable (used
  nowhere else and should not be a class property)
- add missing Python docstrings
- add some comments explaining behavior
- language enhancements and fixes in docs and comments

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
704045ec8b doc: aap20: Fix path to figure
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
7362c0dafb aap2: Temporarily remove FIB and dispatch functionality
There might be incompatible changes in those messages before their
public release. As we only support basic send/receive functionality over
AAPv2, we do not need to include the definitions of other messages.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
4ddca7c225 CI: Add a test for AAPv2 tooling
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00