Commit graph

57 commits

Author SHA1 Message Date
Felix Walter
7b959e4cb8 aap2, FIB: Implement direct dispatch
This adds a "flags" field to the FIB link, allowing for configuring a
link for "direct dispatch", i.e., to make it usable without first
contacting a BDM. It is important to make this optional as otherwise
contact-based routing and QoS could not be implemented properly via a
BDM.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-06-21 12:39:59 +02:00
Felix Walter
b07407112a sqlite: Remove destination EID from compund bundle ID
It is not part of the unique bundle identifier as per RFC 9171.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-06-21 10:36:30 +02:00
Felix Walter
25a45d6b98 ud3tn_utils: Add missing AAP2 async types to global API
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-06-21 10:36:30 +02:00
Felix Walter
9c14798997 python-ud3tn-utils: Add asyncio AAP 2.0 client
This adds an asynchronous implementation of the AAP 2.0 client classes
so they can be used with asyncio.

Note that only some functions are concurrency-safe:
If we access the AAP2AsyncClient from multiple places concurrently, the
calls to `send()` and `receive_response()` may overlap. When it is not
intended to use multiple connections, an `asyncio.Lock` together with
functions to send and receive the response directly in sequence now ensure
that this cannot happen. The new `_rpc` functions are provided for that
purpose. Which functions are concurrency-safe is also indicated in the
docstrings.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-06-21 10:36:30 +02:00
Felix Walter
11ad47be81 python-ud3tn-utils: AAP2Client: Always raise error if connection breaks
Previously, in some cases, we might otherwise return None which can
result in misleading error messages. This makes the connection breakage
explicit also in `_receive_all`.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-06-21 10:36:30 +02:00
Felix Walter
b7a69d0715 python-ud3tn-utils: AAP2Client: Fix varint decoding
This allows us to receive delimited messages longer than 127 bytes.
Previously we only used the last-received byte as length even though we
correctly decoded the full varint.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-06-21 10:36:30 +02:00
Felix Walter
6d2f204af5 Replace integrated next-hop forwarding with AAP 2.0
This adds functionality to dispatch bundles and control the FIB to AAP
2.0 and the bundle processor, replacing the old "router" completely.
AAP 2.0 is extended slightly to enable fragmentation.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-06-17 13:33:32 +02:00
Felix Walter
58e6e39944 aap2: Integrate Protobuf definitions for link and dispatch control
This adds the data structures necessary for controlling the set of CLA
links and dispatching bundles from AAP 2.0 Clients.
See also the new documentation in doc/aap20.md.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-06-17 13:33:31 +02:00
Maximilian Nitsch
b7ce5ef89b SQLiteCLA+Agent: Extend StorageCall Protobuf definition
- Introduces a CompoundBundleId to filter exactly one bundle
- Filter bundles either by their ID or selected metadata
- Transfer the entire StorageCall object to the CLA via the agent queue

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2024-06-03 13:41:58 +02:00
Maximilian Nitsch
bfec915a77 ud3tn-utils: Add SQLiteAgent Protobuf bindings
Adds protobuf Python language bindings and a build target to create them
to the python-ud3tn-utils package.

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2024-06-03 13:41:58 +02:00
Felix Walter
d6ddf0dacd python: AAP2Client: Simplify send() by using _VarintBytes
See discussion in !160

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-04-17 09:58:36 +02:00
Felix Walter
fc4187f2c3 python: AAP2Client: Send all bytes of delimited msg at once
This makes AAP2Client.send() write everything to the socket at once to
better leverage OS optimizations (e.g. if Nagle is disabled we would
send 2 packets otherwise). Also it prevents our tests for the connection
closing behavior of AAP 2.0 for failing randomly because of a race
between the two send() operations and cloding the AAP socket from uD3TN
side.

See: #188
Fixes: #188

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2024-04-14 09:31:27 +02:00
Felix Walter
00be534830 python: Bump version to v0.13.0
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-12-22 11:36:43 +01:00
Maximilian Nitsch
5c8cfedf33 python: Update pinned dependencies
- pyd3tn: Fix pinned cbor version
- python-ud3tn-utils: Update protobuf to version 2.24.4
- tools: Update flake8 to version 6.1.0

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2023-12-22 11:22:31 +01:00
Maximilian Nitsch
f7856db8fb python: Update minimum required python version
Updates the minimum required Python version to 3.8, as 3.7 is EOL.

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2023-12-22 11:22:31 +01:00
Felix Walter
311249c0db python-ud3tn-utils: AAP2: Ignore OS errors when shutting down socket
On some platforms, notably macOS, the `shutdown()` methos will return an
error when the socket is not connected anymore. This raises an exception
if we use an `AAP2Client` in a `with` block and the connection is
terminated before the end of the block, e.g., because the peer closed
it. This commit changes the behavior such that any `OSError` raised by
the `shutdown` method is ignored (it is just there to cleanly signal
termination to the other end anyway and if this is not possible it is
unnecessary).

Fixes: #174
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-12-20 19:13:31 +01:00
Maximilian Nitsch
a84501cf03 ud3tn-utils: Set the minimum required Python version to >=3.7
protobuf==4.21.12 requires python>=3.7[^1]. To resolve the version
conflict between μD3TN and the protobuf package, the required minimum
Python version is set to >=3.7.

[^1]: https://pypi.org/project/protobuf/4.21.12/

Closes: #153
Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2023-11-16 17:02:33 +01:00
Maximilian Nitsch
8f55444280 nix+ud3tn-utils: Use protobuf==4.21.12
Since protobuf==4.24.4 is not (currently) part of nixpkgs, but
protobuf==4.21.12, the pinned version is changed to reduce the effort
for packaging version 4.24.4 itself.

The protobuf Python language bindings have been regenerated to
incorporate the changes from the modified version.

Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
2023-11-16 17:02:33 +01:00
Felix Walter
82d90da82b python-ud3tn-utils: Add new modules to setup.py
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-09 14:33:48 +01:00
Felix Walter
e11b7826f7 python-ud3tn-utils: Document public interfaces of AAP2Client
This adds proper docstrings including documenting return values and
exceptions for the `aap2_client` module.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
22c6511a03 Pin version of protobuf Python dependency
See: #145, !126

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
594eca6c8d Apply first suggestions from !126
- remove comments on "callback-style" serialization (not needed)
- define magic number in Python AAP 2.0 client via descriptive var
- make `secret` in Python AAP 2.0 client a private variable (used
  nowhere else and should not be a class property)
- add missing Python docstrings
- add some comments explaining behavior
- language enhancements and fixes in docs and comments

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
7362c0dafb aap2: Temporarily remove FIB and dispatch functionality
There might be incompatible changes in those messages before their
public release. As we only support basic send/receive functionality over
AAPv2, we do not need to include the definitions of other messages.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:53 +01:00
Felix Walter
3b35ecb253 python-ud3tn-utils: Raise AAP2-specific exceptions in AAP2 client
This defines exception classes for specific types of errors that may
occur within the AAPv2 client, so we can drop the assert statements and
the user of the client can handle errors appropriately.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:52 +01:00
Felix Walter
036545f0d9 aap2: Implement keepalive for receiving bundles in Python tooling
This adds the option to enable keepalive messages for AAPv2 in the
aap_receive and aap_ping tooling, so that the AAPv2 connections stay
alive until bundles are received.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:52 +01:00
Felix Walter
536aee55fd aap2: Check secret for new agent registrations
If there is already an agent registered for the other direction of
communication, registration will fail as long as the provided secrets do
not match.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:52 +01:00
Felix Walter
a0b2fe8909 aap2: Receive response from bundle subscribers
We specify AAP 2.0 to be an RPC protocol whereas the direction of calls
can be switched during configuration. Each call needs to be answered.
This change adds the answer for bundle reception by a client.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:52 +01:00
Felix Walter
4853036d5c aap2: Add lifetime to Bundle message
This is an important field for routing/forwarding decisions.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:52 +01:00
Felix Walter
a007f89a68 aap2: Remove FIBInfo message
This is considered a future extension as it is a performance
optimization (overhead reduction).

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:52 +01:00
Felix Walter
4c45c53069 python-ud3tn-utils: Allow passing AAP 2.0 auth type
This allows for passing the requested AAP 2.0 authorization in the
`configure` method.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:52 +01:00
Felix Walter
785507a583 aap2: Allow providing more information to the dispatcher
This adds a DispatchRequest message containing a DispatchReason in
extension to the bundle headers. By that we can pass more information to
the BDM about why the dispatch request was triggered and have it
extensible in the future.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:52 +01:00
Felix Walter
a81f481e22 python-ud3tn-utils, tools: Add basic AAP2 tooling (send, receive, ping)
This adds adapted versions of `aap_send.py`, `aap_receive.py`, and
`aap_ping.py` for AAP2 and implements the required client classes in
the `ud3tn_utils` module.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-11-08 15:11:52 +01:00
Felix Walter
3d2d50a88f python-ud3tn-utils: Serialize bundle ID instead len() of it
The integer value `bundle_id` does not have a "length" - this seems like
a C&P error.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-06-08 12:56:44 +02:00
Felix Walter
7276aa7de9 python: Bump version to v0.12.0
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-05-23 10:59:20 +02:00
Felix Walter
769b1b0abe python-ud3tn-utils: Document functions handling the bundle ID
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-05-15 11:40:47 +02:00
Felix Walter
ce7900c93f
Merge branch 'feature/60-return-creation-timestamp'
See !90
2023-05-12 08:28:14 +02:00
Felix Walter
b07a653389 AAP: Return bundle creation timestamp and sequence number in bundle ID
See the added description in the AAP documentation - this adds an
internal format to the 64 bit integer value returned as "bundle ID" in
the `SENDCONFIRM` AAP message to uniquely identify the created bundle in
conformance to RFC 9171.

Closes: #60

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-05-11 17:03:29 +02:00
Felix Walter
67fe84a395 node: Identify unit for bitrate in field name
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-05-11 16:59:13 +02:00
Felix Walter
43ae0e59ef test: Add Dockerfiles and scripts for ION interoperability testing
This provides two new Dockerfiles for testing, one just containing ION
and one with an additional Python3 virtual environment for
interoperability testing.

Additionally, two scripts are provided:
* build_docker_images.sh: build images that can be pushed to the
  registry and then used by the CI toolchain
* prepare_for_test.sh: a script to be run inside a newly created Docker
  container based on the provided `ion-interop` image, to build uD3TN
  and add the Python modules from the uD3TN source tree efficiently

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2023-05-11 16:40:53 +02:00
Felix Walter
22e0ede032 ud3tn_utils: Set log level in AAP client to DEBUG
The AAP client class prints some log messages, but we do not want them
to be printed by default if we turn on INFO logging. This uses
`logger.debug` instead.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2022-11-11 10:12:47 +01:00
Felix Walter
4002eed29f
Merge branch 'hotfix/93-ipn-eid-printing'
See !66
2022-10-11 18:05:16 +02:00
Felix Walter
d9f67e83fc python_ud3tn_utils: Drop leading zero from ipn service number
Do not simply concatenate the node EID and agent ID in case of `ipn`
EIDs.

Fixes: #93
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2022-09-19 08:07:42 +02:00
Felix Walter
092e7a7b2f python_ud3tn_utils: Limit auto-generated ipn service nbr to INT32_MAX
See #92 - ION only supports 32 bit service numbers.

Fixes: #92
Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2022-09-19 07:52:21 +02:00
Felix Walter
d41bab7f77 tools: Use getaddrinfo() for establishing TCP connections
This allows us to support IPv6 properly. A slight re-structuring was
necessary as we have to attempt multiple connections with different
sockets one after another. Thus, the socket instance is created in
connect() instead of __init__ and we use abc.ABC to prevent
instantiating the AAPClient without overriding the connect() method.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2022-05-02 10:19:31 +02:00
Felix Walter
83b1b07186 python: Bump version to 0.11.0 2022-03-04 21:10:44 +01:00
Georg Alexander Murzik
298431c68f Insert SPDX license expression comments into our .c, .h, .py, and .sh code
ud3tn is available under multiple licenses and we want to reflect this in
our source code. But which license information should appear first and how
can we manage this efficiently in the future? The Linux Kernel uses SPDX
expressions instead of boilerplate sections. This seems to be a great
approach, so we do the same here.

Signed-off-by: Georg Alexander Murzik <georg.murzik@d3tn.com>
2022-02-20 16:24:38 +01:00
Felix Walter
37e39104aa Fix EID validation for dtn and ipn according to BPv7 draft 31
-> https://datatracker.ietf.org/doc/html/draft-ietf-dtn-bpbis-31#section-4.2.5

The previous validation methods were not sufficient to check validity of
dtn as well as ipn EIDs. This adds further functions to easily detect
the EID scheme and validate dtn plus ipn EIDs. Tests for the most common
cases where validity checks should be successful and where they might
fail are added as well.

Due to this change the default EID of uD3TN gets a slash appended, as
dtn EIDs always have to contain the name delimiter '/'.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2022-01-26 21:05:41 +01:00
Felix Walter
449d0bc17c Add proper support for the ipn EID scheme
Previously uD3TN implicitly assumed string EID in many places such as
the agent manager. This adds proper handling for ipn-scheme EIDs,
using numerical service numbers as agent IDs in case uD3TN is started
with an ipn-scheme EID. If the ipn scheme is used, uD3TN always has to
be configured with the service number 0. The config and management agent
IDs can be configured in config.h and are 9000 and 9001, respectively,
to keep the lower range of service numbers free. It is intentional that
agents can register with a service number of zero - uD3TN does not
provide specific functionality under this service number.

Signed-off-by: Felix Walter <felix.walter@d3tn.com>
2022-01-26 21:05:41 +01:00
Tobias Nöthlich
10028924d6 Refactor Python components
This commit fixes typos and indentation issues and simplifies parts
of the bundle7.py, aap_message.py and aap_receive.py files.

Signed-off-by: Tobias Nöthlich <tobias.noethlich@d3tn.com>
2022-01-26 21:04:02 +01:00
Tobias Nöthlich
60813aa49e Fix python-stylecheck warnings and errors
This commit remedies code smells introduced to pyd3tn and the python
tools while implementing the BIBE CLA.

Signed-off-by: Tobias Nöthlich <tobias.noethlich@d3tn.com>
2021-11-29 13:29:31 +01:00