Communication with sat-rs based software

Communication is a vital topic for remote system which are usually not (directly) connected to the internet and only have 1-2 communication links during nominal operation. However, most of these systems have internet access during development cycle. There are various standards provided by CCSDS and ECSS which can be useful to determine how to communicate with the satellite and the primary On-Board Software.

Most communication with space systems is usually packet based. For example, the CCSDS space packet standard only specifies a 6 byte header with at least 1 byte payload. The sat-rs library provides some support for the CCSDS space packet protocol.

1. UDP TMTC Server. UDP is already packet based which makes it an excellent fit for exchanging space packets.
2. TCP TMTC Server Components. TCP is a stream based protocol, so the library provides building blocks to parse telemetry from an arbitrary bytestream. Two concrete implementations are provided:
   • TCP spacepackets server to parse tightly packed CCSDS Spacepackets.
   • TCP COBS server to parse generic frames wrapped with the COBS protocol.

Working with telemetry and telecommands (TMTC)

The commands sent to a space system are commonly called telecommands (TC) while the data received from it are called telemetry (TM). One way to model the packet handling is to introduce the concept of a TC source and a TM sink can be applied to most satellites. The TM sink is the one entity where all generated telemetry arrives in real-time. The most important task of the TM sink usually is to send all arriving telemetry to the ground segment of a satellite mission immediately.

Another important task might be to store all arriving telemetry persistently. This is especially important for space systems which do not have permanent contact like low-earth-orbit (LEO) satellites.

The diagram below shows one concrete example of how this could look like.

flowchart LR
    Dev[Device Handlers] --> Sink[TM Sink]
    Sub[Subsystem Handlers] --> Sink
    Sink --> Ground[Ground Link]
    Sink --> Store[Persistent Storage]
    Sink --> Udp[UDP Server]
    Sink --> Tcp[TCP Server]

The most important task of a TC source is to deliver the telecommands to the correct recipients. For component oriented software using message passing, this usually includes demultiplexing to determine where a command needs to be sent.

The diagram below shows one concrete example of how this could look like.

flowchart LR
    Udp[UDP Server] --> Source[TC Source]
    Tcp[TCP Server] --> Source
    Radio[Radio Handler] --> Source
    Source --> Dev[Device Handlers]
    Source --> Sub[Subsystem Handlers]
    Source --> File[File Service Handler]

Using a generic concept of a TC source and a TM sink as part of the software design simplifies the flexibility of the TMTC infrastructure: Newly added TM generators and TC receiver only have to forward their generated or received packets to those handler objects.

Packet format

We talked about some basic support for the CCSDS space packet protocol. This is a really simple protocol which just specifies a header that every exchanged TMTC packet has:

This is a protocol which already provides us with some useful fields:

• ID field provided by the Application Process Identifier (APID). This can also be useful for packet multiplexing
• Basic sequence counter which can be used to determine missed packets

However, how does the actual payload that we want to send to or from the satellite actually look like? We recommend a payload format which is created with the excellent serde library. The TMTC modelling chapter provides more information.

Low-level protocols and the bridge to the communcation subsystem

Many satellite systems usually use the lower levels of the OSI layer in addition to the application layer. This oftentimes requires special hardware like dedicated FPGAs to handle forward error correction fast enough. sat-rs might provide components to handle standard like the Unified Space Data Link Standard (USLP) in software but most of the time the handling of communication is performed through custom software and hardware. Still, connecting this custom software and hardware to sat-rs can mostly be done by using the concept of TC sources and TM sinks mentioned previously.

TMTC modelling using Rust

Before we talk about how to model telecommand and telemetry data using Rust, we are going to present some basic concepts and useful libraries first.

Serialization

Serialization and deserialization is the process of converting (Rust) data structures into some format which can be stored or transmitted. We can use this system for generating the payload of our telecommand and telemetry packets. This allows us to model our payloads with Rust data structures, fits perfectly into the data-driven approach that Rust programs tend to favor and allows us to use the excellent type system.

The Rust ecosystem provides the serde library for this task. The library makes it trivial to add serialization support to custom datastructures by providing a derive macro. In almost all cases, you can just add this derive macro to a data structure to make it serializable with any serde compatible serializer.

There are various serializers available which are well suited to the requirements of space systems.

• Generally, we try to minimize the payload size to save data bandwidth.
• The data does not necessarily have to be human-readable

We recommend the postcard serializer, which fulfills these requirements and also works well for embedded systems.

Modelling telecommands and telemetry

Using a serializer library like serde allows us to do some interesting things. For example, let’s assume you have a Camera object in software that you want to send some commands to. This object should have the following capability:

• Process a ping command
• Capture an image
• Send back configuration data

You can now model a request to your Camera object using the following data structure

#![allow(unused)]
fn main() {
#[derive(Debug, serde::Serialize, serde::Deserialize)]
pub enum CameraRequest {
    Ping,
    CaptureImage,
    RequestConfig,
}
}

This data structure models all the requests that the Camera provides. On the telemetry side, you would have a similar object

#![allow(unused)]
fn main() {
#[derive(Debug, serde::Serialize, serde::Deserialize)]
pub enum CameraResponse {
    Ok,
    Config(ConfigStructure)
}
}

where ConfigStructure would be some other wrapped configuration structure, and the Ok response would be the reply for successful execution for all other commands which do not have additional telemetry information.

Rust makes it trivial to move components into a new shared library. You can now put these data structures in a shared types or data library which can be re-used by both a ground system library and by the on-board software.

On the ground system, you could use a function like postcard::to_allocvec to generate the byte representation of a CameraRequest, which is then sent as the payload inside a CCSDS space packet. On the on-board software side, you can use postcard::from_bytes to deserialize the CameraRequest from the raw payload bytes. In both cases, you do not need to hand-write the serialization and de-serialization code anymore. The only trade-off is that you need a Rust conversion layer if you want to create your telecommands in another language like Python.

Using Rust structures like this also has other advantages. Once you have the CameraRequest structure, you can match on it to cover all commands that the device handler needs to cover. If you add a new field, you have to handle the new field variant as well and you can not forget to handle a variant.

One trade-off to keep in mind is that a Rust enum will always have the size of its largest variant in memory. If you need to send large payload to and from the on-board software, you can also add this data as a secondary data blob behind the primary serde payload, and still send something like small metadata as part of the payload. postcard can tell you the size of the deserialized payload which helps with determining the size of any additional payload data.

We recommend this approach for all TMTC definitions where you control all sides of the communication.

Working with Constrained Systems

Software for space systems oftentimes has different requirements than the software for host systems or servers. Currently, most space systems are considered embedded systems.

For these systems, the computation power and the available memory are important resources which are also constrained. This might make completeley heap based memory management schemes which are oftentimes used on host and server based systems unfeasable. Still, completely forbidding heap allocations might make software development unnecessarilly difficult, especially in a time where the OBSW might be running on Linux based systems with hundreds of MBs of RAM.

A useful pattern commonly used in space systems is to limit heap allocations to program initialization time and avoid frequent run-time allocations. This prevents issues like running out of memory (something even Rust can not protect from) or heap fragmentation on systems without a MMU.

Using an embedded allocator

The embedded-alloc library provides a global allocator based on statically sized memory blocks. It also exposes an API which allows run-time tracking of the memory usage.

Using pre-allocated pool structures

A candidate for heap allocations is the TMTC and handling. TC, TMs and IPC data are all candidates where the data size might vary greatly. The regular solution for host systems might be to send around this data as a Vec<u8> until it is dropped. sat-rs provides another solution to avoid run-time allocations by offering pre-allocated static pools. These pools are split into subpools where each subpool can have different page sizes. For example, a very small telecommand (TC) pool might look like this:

The core of the pool abstractions is the PoolProvider trait. This trait specifies the general API a pool structure should have without making assumption of how the data is stored.

This trait is implemented by a static memory pool implementation. The code to generate this static pool would look like this:

use satrs::pool::{StaticMemoryPool, StaticPoolConfig};

let tc_pool = StaticMemoryPool::new(StaticPoolConfig::new(vec![
    (6, 16),
    (4, 32),
    (2, 64),
    (1, 128)
]));

It should be noted that the buckets only show the maximum size of data being stored inside them. The store will keep a separate structure to track the actual size of the data being stored. A TC entry inside this pool has a store address which can then be sent around without having to dynamically allocate memory. The same principle can also be applied to the telemetry (TM) and inter-process communication (IPC) data.

You can read

• StaticPoolConfig API
• StaticMemoryPool API

for more details.

In the future, optimized pool structures which use standard containers or are Sync by default might be added as well.

Using special crates to prevent smaller allocations

Another common way to use the heap on host systems is using containers like String and Vec<u8> to work with data where the size is not known beforehand. The most common solution for embedded systems is to determine the maximum expected size and then use a pre-allocated u8 buffer and a size variable. Alternatively, you can use the following crates for more convenience or a smart behaviour which at the very least reduces heap allocations:

1. smallvec.
2. arrayvec which also contains an ArrayString helper type.
3. tinyvec.

Using a fixed amount of threads

On host systems, it is a common practice to dynamically spawn new threads to handle workloads. On space systems this is generally considered an anti-pattern as this is considered undeterministic and might lead to similar issues like when dynamically using the heap. For example, spawning a new thread might use up the remaining heap of a system, leading to undeterministic errors.

The most common way to avoid this is to simply spawn all required threads at program initialization time. If a thread is done with its task, it can go back to sleeping regularly, only occasionally checking for new jobs. If a system still needs to handle bursty concurrent loads, another possible way commonly used for host systems as well would be to use a threadpool, for example by using the threadpool crate.

Working with Actions

Space systems generally need to be commanded regularly. This can include commands periodically required to ensure a healthy system, or commands to reach the mission goals.

These commands can be modelled using the concept of Actions. If you have not read the TMTC modelling chapter yet, it is recommended to read it first.

For a low number of actions, it is recommended to add the actions as enum variants of your Request type. For a higher number of actions, you can create a dedicated ActionRequest structure.

Modes

Modes are an extremely useful concept to model complex systems. They allow simplified system reasoning for both system operators and OBSW developers. They also provide a way to alter the behaviour of a component and also provide observability of a system. A few examples of how to model the mode of different components within a space system with modes will be given.

Pyhsical device component with modes

The following simple mode scheme with the following three mode

• OFF
• ON
• NORMAL

can be applied to a large number of simpler device controllers of a remote system, for example sensors.

1. OFF means that a device is physically switched off, and the corresponding software component does not poll the device regularly.
2. ON means that a device is pyhsically switched on, but the device is not polled perically.
3. NORMAL means that a device is powered on and polled periodically.

If a devices is OFF, the device handler will deny commands which include physical communication with the connected devices. In NORMAL mode, it will autonomously perform periodic polling of a connected physical device in addition to handling remote commands by the operator. Using these three basic modes, there are two important transitions which need to be taken care of for the majority of devices:

1. OFF to ON or NORMAL: The device first needs to be powered on. After that, the device initial startup configuration must be performed.
2. NORMAL or ON to OFF: Any important shutdown configuration or handling must be performed before powering off the device.

Controller components with modes

Controller components are not modelling physical devices, but a mode scheme is still the best way to model most of these components.

For example, a hypothetical attitude controller might have the following modes:

• SAFE
• TARGET IDLE
• TARGET POINTING GROUND
• TARGET POINTING NADIR

We can also introduce the concept of submodes: The SAFE mode can for example have a DEFAULT submode and a DETUMBLE submode.

Achieving system observability with modes

If a system component has a mode in some shape or form, this mode should be observable. This means that the operator can also retrieve the mode for a particular component. This is especially important if these components can change their mode autonomously.

If a component is able to change its mode autonomously, this is also something which is relevant information for the operator or for other software components. This means that a component should also be able to announce its mode.

This concept becomes especially important when applying the mode concept on the whole system level. This will also be explained in detail in a dedicated chapter, but the basic idea is to model the whole system as a tree where each node has a mode. A new capability is added now: A component can announce its mode recursively. This means that the component will announce its own mode first before announcing the mode of all its children. Using a scheme like this, the mode of the whole system can be retrieved using only one command. The same concept can also be used for commanding the whole system, which will be explained in more detail in the dedicated systems modelling chapter.

In summary, a component which has modes has to expose the following 4 capabilities:

1. Set a mode
2. Read the mode
3. Announce the mode
4. Announce the mode recursively

Health

Health is an important concept for systems and components which might fail. Oftentimes, the health is tied to the mode of a system component in some shape or form, and determines whether a system component is usable. Health is also an extremely useful concept to simplify the Fault Detection, Isolation and Recovery (FDIR) concept of a system.

The following health states are based on the ones used inside the FSFW and are enough to model most use-cases:

• HEALTHY
• FAULTY
• NEEDS RECOVERY
• EXTERNAL CONTROL

1. HEALTHY means that a component is working nominally, and can perform its task without any issues.
2. FAULTY means that a component does not work properly. This might also impact other system components, so the passivation and isolation of that component is desirable for FDIR purposes.
3. NEEDS RECOVERY is used to attempt a recovery of a component. For example, a simple sensor could be power-cycled if there were multiple communication issues in the last time.
4. EXTERNAL CONTROL is used to isolate an individual component from the rest of the system. For example, on operator might be interested in testing a component in isolation, and the interference of the system is not desired. In that case, the EXTERNAL CONTROL health state might be used to prevent mode commands from the system while allowing external mode commands.

Housekeeping Data

If you have not read the TMTC modelling chapter yet, it is recommended to do that first.

Remote systems like satellites and rovers oftentimes generate data autonomously and periodically. An example for this could be temperature or attitude data. Data like this is commonly referred to as housekeeping data, and is usually one of the most important and most resource heavy data sources received from a satellite.

First, we are going to list some assumption and requirements about Housekeeping (HK) data:

1. HK data is generated periodically by various system components throughout the systems.
2. An autonomous and periodic sampling of that HK data to be stored and sent to Ground is generally required. A minimum interface consists of requesting a one-shot sample of HK, enabling and disabling the periodic autonomous generation of samples and modifying the collection interval of the periodic autonomous generation.
3. HK data often needs to be shared to other software components. For example, a thermal controller wants to read the data samples of all sensor components.

Modelling our data

Generally, it makes sense to model the data with Rust data structures for various reasons. For example, the sensor data received from a 3-axis magnetometer might me modelled like this:

#![allow(unused)]
fn main() {
#[derive(Debug, Copy, Clone, serde::Serialize, serde::Deserialize)]
pub struct MgmData {
    pub x: i16,
    pub y: i16,
    pub z: i16,
}
}

You can then re-use this data structure for various purposes. Also note the serde implementations, which are useful for generating the housekeeping data sent to ground.

We can model the housekeeping requests for a handler with a single data set like this:

#![allow(unused)]
fn main() {
#[derive(Debug, Copy, Clone, serde::Serialize, serde::Deserialize)]
pub enum HkRequest {
    OneShot,
    EnablePeriodic,
    DisablePeriodic,
    ModifyInterval(core::time::Duration)

}
}

which might then be a part of a top level request type, e.g.

#![allow(unused)]
fn main() {
#[derive(Debug, Copy, Clone, serde::Serialize, serde::Deserialize)]
pub enum Request {
    Ping,
    Hk(HkRequest)
}
}

A corresponding Response type might just include a HK data variant:

#![allow(unused)]
fn main() {
#[derive(Debug, Copy, Clone, serde::Serialize, serde::Deserialize)]
pub enum Response {
    Ok,
    Hk(MgmData)
}
}

If the software object managed multiple data sets, you could model it like this:

#![allow(unused)]
fn main() {
/// Example set ID.
#[derive(Debug, Copy, Clone, serde::Serialize, serde::Deserialize)]
pub enum SetId {
    Data,
    Config
}

#[derive(Debug, Copy, Clone, serde::Serialize, serde::Deserialize)]
pub enum Request {
    Ping,
    Hk {
        set_id: SetId,
        request: HkRequest
    }
}
}

Sometimes, you need to share the generated data as well. Furthermore, it might make sense to decouple the HK generation from the data acquisition and only return the latest snapshot of the data. In this case, you can put the MgmData inside an appropriate lock structure for your platform/runtime to share it safely with other software components. For example, in a std system, you might simply use an Arc<Mutex<MgmData>> or a Arc<RwLock<MgmData>> for this.

Now, you can update that shared data structure when acquiring new data, and other software objects or the HK generation routine can safely read from it.

Helper components

You need some application logic to track whether periodic data generation is enabled, what the current generation interval is and whether a HK set needs to be generated if the interval period has elapsed.

sat-rs provides some simple helper components for this inside the hk module. The module documentation contains more information.

Events

Events are an important mechanism used for remote systems to monitor unexpected or expected anomalies and events occuring on these systems.

System View

This chapter gives a system level view of how a typical flight software built with sat-rs, spacepackets and cfdp is layered. It complements the previous chapters, which focus on individual components, by showing how those components fit together and where the line between application and platform is usually drawn.

Generic layering

Flight software built with sat-rs is generally structured into three layers.

• Application: The mission specific logic. This is the code a developer writes for a particular mission. It covers mission logic, TMTC handling, event handling, FDIR and command scheduling. sat-rs provides re-usable building blocks for all of these, but the concrete wiring and mission behaviour lives here.
• System / platform: The set of services the application is built on. This covers concepts like logging, serialization, IPC, task and memory management, hardware access, filesystem access and time. Most of these components are provided by external libraries and APIs.
• Hardware: The physical target the software runs on.

The application layer stays largely the same across missions and targets. The system / platform layer is where the target environment determines which concrete crates and mechanisms are used.

Embedded Linux

On an embedded Linux target, the platform layer is provided by the Rust standard library and a small set of additional crates.

The application layer uses sat-rs together with spacepackets for CCSDS/ECSS packet handling and cfdp for file transfer. The platform layer relies on std for tasks, IPC, memory, time and filesystem access, serde and postcard for serialization and log/fern for logging. Hardware access typically goes through Linux mechanisms like uio.

Embedded async targets (Embassy / RTIC)

On smaller microcontrollers without an operating system, the platform layer looks quite different, even though the application layer stays the same.

Here the platform layer is built around an async-centric executor, either Embassy or RTICv2. alloc-based crates like heapless and embedded-alloc replace std collections and allocation, defmt replaces log for logging and hardware access goes through a board support package (BSP), a hardware abstraction layer (HAL) and a peripheral access crate (PAC) instead of the OS.

Library Design

Satellites and space systems in general are complex systems with a wide range of requirements for both the hardware and the software. Consequently, the general design of the library is centered around many light-weight components which try to impose as few restrictions as possible on how to solve certain problems.

There are still a lot of common patterns and architectures across these systems where guidance of how to solve a problem and a common structure would still be extremely useful to avoid pitfalls which were already solved and to avoid boilerplate code. This library tries to provide this structure and guidance the following way:

1. Providing this book which explains the architecture and design patterns in respect to common issues and requirements of space systems.
2. Providing an example application. Space systems still commonly have large monolithic primary On-Board Softwares, so the choice was made to provide one example software which contains the various features provided by sat-rs.
3. Providing a good test suite. This includes both unittests and integration tests. The integration tests can also serve as smaller usage examples than the large satrs-example application.

This library has special support for standards used in the space industry. This especially includes standards provided by Consultative Committee for Space Data Systems (CCSDS) and European Cooperation for Space Standardization (ECSS). It does not enforce using any of those standards, but it is always recommended to use some sort of standard for interoperability.

A lot of the modules and design considerations are based on the Flight Software Framework (FSFW). The FSFW has its own documentation, which will be referred to when applicable. The FSFW was developed over a period of 10 years for the Flying Laptop Project by the University of Stuttgart with Airbus Defence and Space GmbH. It has flight heritage through the 2 mssions FLP and EIVE. Therefore, a lot of the design concepts were ported more or less unchanged to the sat-rs library. FLP is a medium-size small satellite with a higher budget and longer development time than EIVE, which allowed to build a highly reliable system while EIVE is a smaller 6U+ cubesat which had a shorter development cycle and was built using cheaper COTS components. This library also tries to accumulate the knowledge of developing the OBSW and operating the satellite for both these different systems and provide a solution for a wider range of small satellite systems.

sat-rs can be seen as a modern port of the FSFW which uses common principles of software engineering to provide a reliable and robust basis for space On-Board Software. The choice of using the Rust programming language was made for the following reasons:

1. Rust has safety guarantees which are a perfect fit for space systems which generally have high robustness and reliablity guarantees.
2. Rust is suitable for embedded systems. It can also be run on smaller embedded systems like the STM32 which have also become common in the space sector. All space systems are embedded systems, which makes using large languages like Python challenging even for OBCs with more performance.
3. Rust has support for linking C APIs through its excellent FFI support. This is especially important because many vendor provided libaries are still C based.
4. Modern tooling like a package managers and various development helper, which can further reduce development cycles for space systems. cargo provides tools like auto-formatters and linters which can immediately ensure a high software quality throughout each development cycle.
5. A large ecosystem with excellent libraries which also leverages the excellent tooling provided previously. Integrating these libraries is a lot easier compared to languages like C/C++ where there is still no standardized way to use packages.

sat-rs Example Application

The sat-rs library includes a monolithic example application which can be found inside the satrs-example subdirectory of the repository. The primary purpose of this example application is to show how the various components of the sat-rs framework could be used as part of a larger on-board software application.

Structure of the example project

The example project contains components which could also be expected to be part of a production On-Board Software. A structural diagram of the example application is given to provide a brief high-level view of the components used inside the example application:

flowchart TD
    subgraph TMTC[TMTC Infrastructure]
        subgraph TMTCRow1[ ]
            direction LR
            Udp[UDP Server]
            Tcp[TCP Server]
        end
        subgraph TMTCRow2[ ]
            direction LR
            Source[TC Source]
            Sink[TM Sink]
        end
    end

    subgraph AOCS[AOCS Stack]
        subgraph AOCSRow1[ ]
            direction LR
            Mgm0[MGM 0 Handler]
            Mgm1[MGM 1 Handler]
            Assy[MGM Assembly]
        end
        subgraph AOCSRow2[ ]
            direction LR
            AcsCtrl[ACS Controller]
            Mgt[MGT Handler]
            AcsSub[ACS Subsystem]
        end
    end

    subgraph EPS[EPS Stack]
        Pcdu[PCDU Handler]
    end

    subgraph Core[Core]
        direction LR
        Ctrl[Core Controller]
        Evt[Event Manager]
    end

    Sim[Sim Client]:::optional

    TMTC ~~~ EPS
    AOCS ~~~ Core
    Core ~~~ Sim

    classDef optional stroke-dasharray: 5 5;
    classDef invisible fill:none,stroke:none;
    class TMTCRow1,TMTCRow2,AOCSRow1,AOCSRow2 invisible;

The dotted lines are used to denote optional components. In this case, the simulation client is optional because a dummy interface can be used instead to run the example without the simulator. Some additional explanation is provided for the various components.

TCP/IP server components

The example includes a UDP and TCP server to receive telecommands and poll telemetry from. This might be an optional component for an OBSW which is only used during the development phase on ground. The UDP server is strongly based on the UDP TC server. This server component is wrapped by a TMTC server which handles all telemetry to the last connected client.

The TCP server is based on the TCP Spacepacket Server class. It parses space packets by using the CCSDS space packet ID as the packet start delimiter. All available telemetry will be sent back to a client after having read all telecommands from the client.

TMTC Infrastructure

The most important components of the TMTC infrastructure include the following components:

• A TC source component which demultiplexes and routes telecommands based on parameters like packet APID and a target ID which is part of the packet payload.
• A TM sink sink component which is the target of all sent telemetry and sends it to downlink handlers like the UDP and TCP server.

You can read the Communications chapter for more background information on the chosen TMTC infrastructure approach.

Dataflow

TMTC component group

This group is the primary interface for clients to communicate with the on-board software using the combination of CCSDS space packets and serde serialized payloads. In the future, this might be extended with the CCSDS File Delivery Protocol.

A client can connect to the UDP or TCP server to send telecommands to the on-board software. These servers forward all telecommands to a centralized TC source component, which demultiplexes them and routes each one to its target component.

All telemetry generated by the on-board software is sent to a centralized TM sink. The core controller also forwards events to the event manager, which converts them into telemetry and sends it to the TM sink as well. The TM sink performs a demultiplexing step to forward all telemetry to the relevant recipients, which in the example case are the last connected UDP client and any connected TCP client.

Application Group

The application group contain some components you might also find in a real satellite software. This includes an AOCS stack with various device handlers and system level objects.

Shared components and functional interfaces