mirror of
https://gitlab.com/d3tn/ud3tn.git
synced 2026-08-13 12:33:27 +02:00
9d07eaa9ae
This was mostly implemented using Claude Code, with careful manual review and some manual steps. Prompts & manual steps: - > There has been an update to the ipn EID scheme in the following RFC: https://www.rfc-editor.org/info/rfc9758 -- Add support for parsing and serializing this to the bundle7 component. Both the old and the new format should be supported. - > Adapt the `validate_ipn_eid` function in `components/ud3tn/eid.c` and all uses of it. Merge the new "extended" code into the function. Backwards compatibility outside of the project code tree is not necessary. Remove the redundant implementations in `components/bundle7/eid.c` and `components/bundle7/bundle7.c` and instead make use of the updated common function. - > The Python implementations in `pyd3tn/pyd3tn/eid.py` and potentially `python-ud3tn-utils/ud3tn_utils` also need adjustment to add support. Check where extensions are needed and implement them. - > Extend the integration tests: test the 3-element format in a suitable place in `test/integration` and use it at least once in one of the end-to-end tests defined for the CI in `.gitlab-ci.yml` so both the implementation in the Python tools as well as the one in the C code are used indirectly. - the integration tests were then reverted manually as it tried to modify the fragmentation test - > Please check again the use of the `-a` / `--agentid` parameter in python tools such as `aap2-ping`. It only specifies the agent id, which is the service number in case of the ipn scheme. - > Adapt `perform_basic_test` in `test/integration/test_bundle_send_receive` so that it tests a list of different EIDs with different schemes, including an ipn-scheme EID with the new format. The latter is only supported by BPv7, so a parameter should be added to the function specifying this. You may want to analyze `test/integration/helpers.py` and `python-ud3tn-utils/ud3tn_utils/config.py` to get an understanding of the functions and constants used in the test file. The "contact" that is configured contains a list of "reachable EIDs" which may be extended for the purpose of testing multiple destination EIDs so it is not necessary to configure multiple contacts. - (^C when asked to execute integration test) > As no uD3TN instance is running this will fail; also, oftentimes the log output of uD3TN is required to be able to determine the cause of the test failure as pytest will only spit out something like "pyd3tn.helpers.CommunicationError: select operation ran into timeout" in case nothing is received. You may want to document that tests should be run manually and ask for the results in the future. I ran the test for you - the result was that `test_send_receive_tcpcl_bundle6` and `test_send_receive_tcpcl_bundle7` did not receive the bundle back. uD3TN logged for the former "[DEBUG] Router: Contact payload capacity (0 bytes) too low for re-scheduling bundle 0x7f7364001330 [0, 41] of size 107 bytes [components/routing/compat/router.c:448]" (after a DELETE command was processed) and for the latter "[DEBUG] Router: Could not determine a node over which the destination "dtn://receiver.dtn/" for re-scheduling bundle 0x7f7364001190 [0, 45] is reachable [components/routing/compat/router.c:421]" (2 times, after a DELETE command was processed). - > The test `test_send_receive_tcpcl_bundle7` is still failing: the bundle to the ipn EID with allocator number is enqueued but not sent. In the course of adjusting the test, simplify it such that a single sending contact and a single receiving contact, each with a static EID (maybe different schemes), are used. Testing the different EID schemes is mostly important for routing and bundle destination EID processing, so just specify them in the "reachable EIDs" and in the bundle destination. - manually extended `aap2-test` in CI config - manually cleaned up `test/integration/test_bundle_send_receive.py` - > `python-ud3tn-utils/ud3tn_utils/aap2/aap2_client.py` needs to be adapted so that it understands 3-element node IDs coming from uD3TN. Search for "prefix". - > I cannot register via AAP2 when uD3TN runs with an ip EID with allocator. The logged error message is: "[WARNING] AgentManager: Tried to register a sink with an invalid ipn service number! [components/ud3tn/agent_manager.c:172]". It turns out that the AAP agent uses `get_agent_id_ptr` in `components/ud3tn/eid.c`, which has not been adapted. - manually adjust setting `local_eid_prefix` in `bundle_processor_task`; use `get_agent_id_ptr` there and drop `const` in this function to also make it usable for non-`const` use cases - manual testing - > The echo agent still uses the wrong source EID when responding: [log snippet] - > With the extended implementation, we face the issue that the kind of representation (i.e. 2-element or 3-element) is always kept as it was and we would need to take special considerations when forwarding bundles, e.g., when matching a bundle coming from a BPv7 implementation imcompatible with the 3-element ipn EID (thus using the 2-element representation for the destination field) to an entry in the FIB, which may use the 3-element representation. I propose to always use the 3-element string representation internally when the allocator number is non-zero and use the 2-element string representation when it is zero. BPv6 bundles should always use the 2-element representation upon serialization and for BPv7 bundles there can be a compile-time switch (added to the header and `config.example.mk`) to enforce that the 3-element representation is always converted to a 2-element representation in case we are dealing with an incompatible node further down the path. See section 6.2. ipn EID CBOR Decoding of RFC 9758 for decoding details. - > There is an issue with the implementation: The standard does not allow node numbers (!= FQNN) to be larger than 2^32-1, so they can be clearly distinguished from FQNN. When receiving bundles with a 2-element representation but the node number exceeding this value, we must assume that it has an allocator number != 0 and decode it this way. We also should adjust our validation functions and tests such that 3-element representations with either the node or the allocator number larger than 2^32-1 are rejected. In the 2-element representation, the node number, being always equal to the FQNN, can have a range until 2^64-1. - > The issue is that the adapted `bundle7_eid_sizeof` (`bundle7.c`) function calls `validate_ipn_eid` (`eid.c`), which yields an allocator number of `UINT64_MAX` if the 2-element format has a node number > 2^32-1. The node number is kept as `UINT64_MAX` and `bundle7_eid_sizeof` then uses the 3-element branch because `allocator != 0` and calculates this wrong value. What about making the validation function automatically perform normalization iff `allocator_out != NULL`? - > Check the RFC again. The normalization you propose is wrong. - > The allocator number is in the 32 high bits, the node number is in the 32 lower bits of the FQNN. Here is example code from the CBOR decoding section in https://www.rfc-editor.org/rfc/rfc9758.html: if enc_eid.len() == 2 { ipn_eid.allocator_identifier := enc_eid[0] >> 32; ipn_eid.node_number := enc_eid[0] & (2^(32-1)); ipn_eid.service_number := enc_eid[1]; } - > Check the changes in `git diff` made to files in `pyd3tn` and `python-ud3tn-utils`. think hard if something should be adapted, e.g., regarding normalization or validation. Recommend changes, if any. - Then, Claude completely turned up to 100 and added a whole lot of Python code for normalization and so on... - > We should keep the Python implementation simple. It should not do any normalization or conversion, but just support 2-element and 3-element EIDs. Remove the normalization and simplify the code you added. - > I saw you removed some test cases in `pyd3tn/pyd3tn/eid.py` (which is valid) and then performed some manual tests, e.g., with large numbers. You can add sensible additional test cases in `pyd3tn/pyd3tn/eid.py`. - > Add a `pytest`-compatible `test_` function to `bundle7.py` that automates the manual test you just did. - > Now we take another look at the C code. For deterministic processing, I propose to only perform normalization once: when a bundle enters the system. This can be integrated into `eid_parse_ipn` before creating the string representations. - > Remove the use of `_GNU_SOURCE` in `components/ud3tn/eid.c`. - minor manual refactoring of `eid.c` - > Add a test for the `normalize_ipn_eid` function to the unit tests in `test_eid.c`. - > There is a second way in which EIDs can enter uD3TN: when bundles are created locally. Adjust `bundle6_create_local` and `bundle7_create_local` such that these functions normalize EIDs before putting them into the bundle data structure. You may add a generic normalization function that supports the dtn scheme (NOP) and the ipn scheme (calling the ipn normalization) to `eid.c`. - > There is one last place we need to adjust: when uD3TN starts, it configures a local node ID (that can be specified on the command line). This also needs to be normalized. Do it in `components/ud3tn/init.c`, see lines 55-58. - > Your `BPV7_FORCE_IPN_2_ELEMENT` is implemented incorrectly in `bundle7/bundle7.c` and `bundle7/eid.c`: in `bundle7_eid_sizeof`, the serialized size should be calculated, which needs to take into account whether `BPV7_FORCE_IPN_2_ELEMENT` is set. In `eid.c` in `serialize_ipn`, a non-zero allocator is just discarded if this feature is enabled. - > The BPv6 (bundle6) implementation also needs adjustments. In the parser, `bundle6_read_eid` (for non-CBHE) and `bundle6_create_eid_cbhe` (for CBHE) need to perform normalization. In the serializer, through `bundle6_calculate_dict`, `analyze_eid` (in `bundle6/bundle6.c` is called transitively and there de-normalization (always reverting to the 2-element format) needs to be performed if necessary (BPv6 does not support the new RFC). - > Put your test in the unit tests and execute it using the normal unit testing workflow. - > The function `bundle6_get_dict_length` is not necessary anymore and does not do the right thing with the recent changes. We can remove it. - > When I run `make clean && make run-unittest-posix -j4 CFLAGS="-DBPV7_FORCE_IPN_2_ELEMENT=1"`, I get the following error: test/unit/test_bundle7Serializer.c:518:TEST(bundle7Serializer, dtn_ipn):FAIL: Expected 22 Was 21 - It tried to re-write everything which I aborted and checked the test with `gdb`, after which I fixed the test manually. - > `BPV7_FORCE_IPN_2_ELEMENT` is always "defined" because you define it in `bundle.h` to `0` if it is not defined. Thus, you use the wrong constant to test against in the test. The code is correct. I fixed it by using `#if` instead of `#ifdef`. Add some further test cases to the test function for the bundle7 serializer so we also test the 3-element format now. Closes: #226 |
||
|---|---|---|
| .. | ||
| ud3tn_utils | ||
| .gitignore | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| requirements.txt | ||
python-uD3TN-utils
The Python package uD3TN-utils is a utility library to simplify the interaction with the µD3TN daemon within python applications.
The included AAP2Client enables user-friendly communication with the µD3TN
daemon via local or remote sockets using the Application Agent Protocol 2 (AAP 2.0).
Besides sending and receiving bundles, it is also possible to change the
configuration of the µD3TN daemon via AAP messages.
Installation
From source:
git clone https://gitlab.com/d3tn/ud3tn
pip install [-e] ud3tn/python-ud3tn-utils
From PyPi directly:
pip install ud3tn-utils
Development
For examples on the usage of this library, check out the contained CLI tools
in the aap/bin (for the old AAP v1 protocol) and aap2/bin (for AAP 2.0)
subdirectories.
python-uD3TN-utils is maintained as part of the µD3TN project and follows its development processes. Please see the µD3TN repository and the µD3TN web documentation for further information.