Commit graph

1215 commits

Author SHA1 Message Date
Maximilian Nitsch
a837c7d722 docs: Add manpage to mkdocs
Uses pandoc to convert the manpage into a markdown document and add it
to the mkdocs documentation.

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2025-06-13 17:55:24 +02:00
Felix Walter
42d312d958 Merge branch '15-207-improve-contact-plan-handling-in-bdm' into 'master'
Improve contact plan handling in DFCF BDM and tools

Closes #207, #15, and #238

See merge request d3tn/ud3tn!199
2025-06-06 10:11:05 +02:00
Felix Walter
04993e1eee Merge branch 'feature/241-status-report-support' into 'master'
Implement status report support in AAP 2.0

Closes #241

See merge request d3tn/ud3tn!215
2025-06-06 10:10:48 +02:00
Felix Walter
20eff9058b test/integration: Pin same version of jsonschema as in utils
Otherwise, we may get dependency conflicts, depending on the order in
which we install the modules and requirements file with pip.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-06 09:46:32 +02:00
Felix Walter
578b02a528 test: Clean up timeouts applied in ION-BIBE interop test
We should not need to wait between setting up individual contacts, as
the commands are even sent to different BPA instances. We only need to
wait after configuring the contacts and before sending the bundle, so
that the contact has definitely started.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-06 09:14:57 +02:00
Felix Walter
bc2fc915f2 test: Delay sending bundles in interoperability tests
We must ensure that the receiving end can start before injecting
bundles, otherwise this becomes a race condition. We saw this issue in
the ION-BIBE interoperability test.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-06 09:14:57 +02:00
Felix Walter
a8a6f8c845 python: Apply review comments from !199
- Use format strings
- Use enum field instead of magic number
- Cast to enum type
- Add type annotations to parse functions
- Improve docstrings mentioning Set types where function accepts Iterable

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-06 09:14:53 +02:00
Felix Walter
8b3d84e6f4 doc/testing: Document that now we need -r in integration tests
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-06 09:01:41 +02:00
Felix Walter
f657efd1fa test/integration: Use parentheses instead of backslash for continuation
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-06 09:01:41 +02:00
Felix Walter
3421db9cbd CI: Add linebreaks to CI configuration to improve readability
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-06 09:01:41 +02:00
Felix Walter
4cf56aac6c CI: Enable status reports in all integration tests
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:14:03 +02:00
Felix Walter
2c4a8b9f71 test/integration: Add a test for bundle status reports
As discussed in #215.

Note: The basis of this code has been AI-generated using Claude Sonnet 4,
but it was reviewed line-by-line and heavily refactored.

Prompts:

- "Create a new integration test in `test/integration` that tests the
  newly-introduced status report feature. The test should send a bundle with
  a set report-to EID, which may be the (local) sender. The created status
  report should then be received and checked that it contains the
  correct data."
- There is an issue in `_wait_for_status_report`: `sub_client.receive_msg`
  can block indefinitely and there is no possibility to specify a timeout.
  However, you could make the test async which allows aborting the receive
  function. There is an async AAP2 client and an example for a test based
  on it in `test_aap2_async_client.py`.
- I updated the test file. Adapt it so it passes the flake8 check which
  gives the following errors: (pasted line-length issues)
- Extend the check of the received status report in the test by parsing it
  as valid BPv7 (RFC 9171) status report. An implementation that can be
  used for serializing status reports can be found in lines 525-566 of
  `pyd3tn/bundle7.py`. The status report is in CBOR format. We use the
  `cbor2` library in the project. The BPv7 spec can be found here:
  https://www.rfc-editor.org/rfc/rfc9171.html#name-bundle-status-reports

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:14:03 +02:00
Felix Walter
cc701a1e52 pyd3tn: Implement BPv7 status report parsing
This adds a `from_cbor` method to the `BundleStatusReport` class so
binary status reports can be parsed.

Note: Some of this code has been AI-generated using Claude Sonnet 4, but
it was reviewed line-by-line and then heavily refactored.

Prompt: "Write a `from_cbor` function that can be added to the
`BundleStatusReport` class in `bundle7.py` so we get suitable status
report parsing logic there."

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:14:03 +02:00
Felix Walter
2b98459fd6 pyd3tn: Make BundleStatusReport independent of a bundle instance
The constructor for BundleStatusReport always required a bundle, so that
it was not possible to create a standalone status report instance. A
`from_bundle` method is added that provides the old behavior.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:14:03 +02:00
Felix Walter
b63d36edc4 bundle_processor: Allow local destination as report-to EID
There are many use cases where status reports generated locally should
be delivered locally.

Note: As the bundles now need to be passed to `bundle_dispatch`, it would
have been needed to remove the `const` qualifier of that method, as it
mutates the BP state (for checking for duplicates and adding bundle fragments
to the reassembly list). To prevent this from being necessary, an alternative
function is introduced that does not perform such a check and does not
support fragments.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:14:02 +02:00
Felix Walter
973f35d132 Implement status report support in AAP 2.0
This enables AAP 2.0 clients to receive status reports with a new ADU
flag (as we already deliver BIBE bundles). Moreover, it allows clients
to set a report-to EID and sets all of the status report flags on newly
created bundles in case a report-to EID is provided.

Closes: #241

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:13:17 +02:00
Felix Walter
dac4d4b0b8 CI: Bump interop testing images to include Jansson dependency
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
d9fc8ee5d4 CI: Simplify integration test specification
We do not need a hidden job due to the new default image setting.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
ad11fc266a CI: Set our compiled image as default
This way we do not need to specify it everywhere. Due to the new Jansson
(JSON library) dependency, we need it also for the simple builds, as
buildpack-deps only provides sqlite beside basic build tools.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
87a458c3fd CI: Use JSON config in integration test
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
53188b458a stylecheck: Ignore SPLIT_STRING
The JSON parser unit test heavily uses multiline strings so we can read
the embedded JSON properly. It seems there is no better way to make
checkpatch happy here.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
030b0bf75a routing/compat: Implement JSON configuration parser
This adds the JSON parser logic as an alternative to the custom legacy
format and also adds necessary tests.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
ed20c501cf test: Update Dockerfiles and introduce jansson dependency
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
644d4fd162 dockerfiles: Update version and add missing dependencies
- add sqlite and jansson dependencies, which were missing
- use newest supported Python release

Closes: #238

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
aebe697809 config.mk.example: Explain that the DISABLE_* flags apply when set
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
868bac67b6 router_agent, mk: Add stub of JSON configuration feature
This adds a function to decode JSON configuration via Jansson to the
router agent. The JSON functionality and, thus, the dependency on
Jansson, can be disabled during the build process by setting the
`DISABLE_JSON` compile-time flag (see `config.mk.example` for details).

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
229f884a57 python-ud3tn-utils: Introduce JSON configuration format
This introduces a JSON contact configuration format for the
deterministic first-contact forwarding (DFCF) implementation as a modern
alternative to our homebrewn configuration messages. For now, it is only
supported in the external DFCF BDM, but integration into the "router
agent" is planned.

Example JSON configuration string (from our tests):

```
{

    "command": "ADD",
    "node_id": "dtn://ud3tn2.dtn/",
    "cla_addr": "mtcp:127.0.0.1:4223",
    "reachable_eids": [
        "ipn:1.0"
    ],
    "contact_list": [
        {
            "start": 1401519306972,
            "end": 1401519316972,
            "data_rate": 2400,
            "reachable_eids": [
                "dtn://66553/",
                "dtn://89326/"
            ]
        },
        {
            "start": 1401519506972,
            "end": 1401519516972,
            "data_rate": 1200,
            "reachable_eids": [
                "dtn://12349/",
                "dtn://89326/"
            ]
        }
    ]
}
```

Closes: #15

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
1cc9560852 BDM: Be more permissive about the allowed parts of a config message
We implicitly expected all incoming config message data structures to
contain a list of contacts as well as a list of reachable EIDs in both the
config message and each contact in the contact list itself. This may not
always be the case if the config command does not contain such fields.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
fb2d0b021c cla, mk: Fix disabling SQLite functionality
Previously it was not possible to disable SQLite because of a linker
error. Also, `sqlite3.h` was always required to be present. This adds
an example to `config.mk.example` and fixes both issues.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
4d39b9ccfa test/integration: Minor comment adjustments
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
9446988ed3 test/integration: Add contact configuration and query test
This adds a test for the new JSON-based query feature of the BDM. The
test is only enabled in the debug integration test for the external
dispatcher and does not run by default, as it does not specify an AAP
2.0 secret and also does not work with the built-in DFCF forwarding
module.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
89957b9e81 python-ud3tn-utils: BDM: Fix re-scheduling upon contact removal
Previously, we were relying on a race condition for re-scheduling: if a
suitable contact is ongoing, even if it is in the process of being
dropped, it will be considered for re-scheduling a bundle.

This explicitly ignores the contact being dropped.

Due to the increased complexity, `_calc_bundle_schedule` is refactored
slightly.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
e6380b3c8a python-ud3tn-utils: BDM: Fix updating existing contacts
When we update an existing contact, e.g, to change the set of reachable
EIDs, we delete the previous entry, cancel the task, and add a new
entry. For this to work, we must wait for the task to terminate after
cancellation.

Additionally, we relax the check for which contacts to update to include
contacts that compare equal, which might be the case if only the
"reachable EIDs" list is different.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
6aa7fdcb33 python-ud3tn-utils: Do not add contacts that are ongoing
This would result in conflicts. We previously used an assertion to
ensure it, however, this could be triggered by external config commands
and crashed the BDM.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
1e7413297b python-ud3tn-utils: Fix BDM not dropping contact if connection fails
If the link cannot be established, we need a timeout to remove the
contact from the active contacts set.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
fe56b0846d python-ud3tn-utils, BDM: Allow querying contact plan
This re-impplements the QUERY command through our configuration
interface so we can get the contact plan back out of the BDM.
aap2_config.py is extended such that it allows for querying and deleting
contacts.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
348c56ea2e python-ud3tn-utils: Move config parser from BDM to library
As this can also be used for the built-in routing agent, we provide the
functionality in the shared Python module.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
d28545ff5e python-ud3tn-utils: Remove deprecated ManagementCommand
This had been removed from µD3TN for a long time.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 17:03:39 +02:00
Felix Walter
91b89facc0 Merge branch '244-strict-bpv7-validation' into 'master'
Improve validation of bundles before further processing

Closes #243 and #244

See merge request d3tn/ud3tn!219
2025-06-05 17:01:50 +02:00
Felix Walter
4e6acaf6f2 Merge branch '246-update-external-dependencies' into 'master'
Update dependencies in external/

Closes #246

See merge request d3tn/ud3tn!218
2025-06-05 17:01:36 +02:00
Felix Walter
f5c5a3168c Merge branch 'chore/nixos-25.05' into 'master'
Bump nixos-25.05

See merge request d3tn/ud3tn!217
2025-06-05 13:54:33 +02:00
Felix Walter
4a2f10aecb bundle: Remove BUNDLE_FLAG_REPORT_STATUS_TIME from SR flags
It only controls how status reports should look like, not which status
reports are requested. We only want to check for the latter. ION appears
to always set this flag.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 13:14:27 +02:00
Felix Walter
37ec79e249 bundle: Implement dedicated bundle validation before further processing
This adds a `bundle_is_valid` function checking for further MUST
constraints defined by the spec., which further processing inside uD3TN
may depend on.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 11:05:47 +02:00
Maximilian Nitsch
94218d6116 python-ud3tn-utils: Re-generate protobuf files
Regenerates the Python protobuf bindings with the updated `protoc`.

```sh
make aap2-proto-headers
make storage-agent-proto-headers
```

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2025-06-05 10:58:12 +02:00
Felix Walter
f94bcddc39 bundle7: Fix include guard C&P issue
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 10:43:26 +02:00
Felix Walter
a727b308cc bundle7/parser: Enforce version field to be set to 7
See: #243
Closes: #243

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 10:42:16 +02:00
Felix Walter
8bfc7575ba Bump tinycbor dependency to v0.6.1
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 08:55:07 +02:00
Felix Walter
a17892e900 Re-generate nanopb files
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 08:53:15 +02:00
Felix Walter
5ca8341ea6 Bump nanopb dependency to 0.4.9.1
Fixes: #246

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2025-06-05 08:52:46 +02:00
Maximilian Nitsch
87c25c3e8a treewide: Bump python>=3.9
As Python 3.8 is end-of-life and the latest python-protobuf package
requires at least Python 3.9, the minimum Python version is set to 3.9.

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2025-06-03 17:12:08 +02:00