🪴 Zero's Garden

BPF development with Aya on aarch64

4 min read

Prerequisites

Rust toolchain

Install with rustup.

cargo-generate

Install openssl headers first.

Build bpf-linker with Nix’s LLVM

bpf-linker requires LLVM. It’s a good idea to use Nix, but it does not provide the prefix needed if installed with nix-env -i.

error: No suitable version of LLVM was found system-wide or pointed
              to by LLVM_SYS_180_PREFIX.
       
              Consider using `llvmenv` to compile an appropriate copy of LLVM, and
              refer to the llvm-sys documentation for more information.
       
              llvm-sys: https://crates.io/crates/llvm-sys
              llvmenv: https://crates.io/crates/llvmenv

It’s recommended to use nix-shell for one-shot builds like this.

nix-shell -p llvm_18 -p libxml2 --run "cargo install bpf-linker --no-default-features"

Note that we used LLVM 18 here. The LLVM version used to build bpf-linker must match the LLVM version from the rust nightly toolchain defined in {{project-name}}-ebpf/rust-toolchain.toml, which is used to build the eBPF program in your project. For LLVM 18, that means we should specify nightly-2024-07-31 or earlier in the toolchain config file.

Albeit not mentioned in bpf-linker’s README, libxml2 is also required.

Map Types

  • BPF_MAP_TYPE_LRU_HASH provides general purpose hash map storage, and will automatically evict the least recently used entries when the hash table reaches capacity.

Pinning

From Map::pin():

When a map is pinned it will remain loaded until the corresponding file is deleted.

Log

Supported display hints: https://github.com/aya-rs/aya/blob/5397c1ca4b77cd27082e96aab9ab931631df7fa8/aya-log-parser/src/lib.rs#L55-L65

Reading Kconfig

Kernel configs are defined in Kconfig.* files. For example, HZ is defined in https://github.com/torvalds/linux/blob/b831f83e40a24f07c8dcba5be408d93beedc820f/kernel/Kconfig.hz.

There is a PR that implements the same extern data support as libbpf, but only a sample C BPF program is provided. https://github.com/aya-rs/aya/pull/1017

Programs

sock_ops

NOTE

Until https://github.com/aya-rs/aya/issues/987 is implemented, killing the program does not unload the cgroup attachment automatically. However, a new attachment would replace the existing one.

If the program return code is not 1, __cgroup_bpf_run_filter_sock_ops considers it a failure and returns -EPERM.

/**
 * __cgroup_bpf_run_filter_sock_ops() - Run a program on a sock
 * @sk: socket to get cgroup from
 * @sock_ops: bpf_sock_ops_kern struct to pass to program. Contains
 * sk with connection information (IP addresses, etc.) May not contain
 * cgroup info if it is a req sock.
 * @type: The type of program to be exectuted
 *
 * socket passed is expected to be of type INET or INET6.
 *
 * The program type passed in via @type must be suitable for sock_ops
 * filtering. No further check is performed to assert that.
 *
 * This function will return %-EPERM if any if an attached program was found
 * and if it returned != 1 during execution. In all other cases, 0 is returned.
 */
int __cgroup_bpf_run_filter_sock_ops(struct sock *sk,
				     struct bpf_sock_ops_kern *sock_ops,
				     enum cgroup_bpf_attach_type atype)
{
	struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
	int ret;
 
	ret = BPF_PROG_RUN_ARRAY_CG(cgrp->bpf.effective[atype], sock_ops,
				    bpf_prog_run);
	return ret == 1 ? 0 : -EPERM;
}
EXPORT_SYMBOL(__cgroup_bpf_run_filter_sock_ops);

__cgroup_bpf_run_filter_sock_ops is lightly wrapped by the BPF_CGROUP_RUN_PROG_SOCK_OPS macro, and its return value is processed by tcp_call_bpf().

/* Call BPF_SOCK_OPS program that returns an int. If the return value
 * is < 0, then the BPF op failed (for example if the loaded BPF
 * program does not support the chosen operation or there is no BPF
 * program loaded).
 */
#ifdef CONFIG_BPF
static inline int tcp_call_bpf(struct sock *sk, int op, u32 nargs, u32 *args)
{
	struct bpf_sock_ops_kern sock_ops;
	int ret;
 
	memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
	if (sk_fullsock(sk)) {
		sock_ops.is_fullsock = 1;
		sock_owned_by_me(sk);
	}
 
	sock_ops.sk = sk;
	sock_ops.op = op;
	if (nargs > 0)
		memcpy(sock_ops.args, args, nargs * sizeof(*args));
 
	ret = BPF_CGROUP_RUN_PROG_SOCK_OPS(&sock_ops);
	if (ret == 0)
		ret = sock_ops.reply;
	else
		ret = -1;
	return ret;
}

-EPERM is not 0, so the returned value is -1 and considered a failure.

In case of success, sock_ops.reply is returned to the caller, which is defined in linux/bpf.h as below.

/* User bpf_sock_ops struct to access socket values and specify request ops
 * and their replies.
 * Some of this fields are in network (bigendian) byte order and may need
 * to be converted before use (bpf_ntohl() defined in samples/bpf/bpf_endian.h).
 * New fields can only be added at the end of this structure
 */
struct bpf_sock_ops {
	__u32 op;
	union {
		__u32 args[4];		/* Optionally passed to bpf program */
		__u32 reply;		/* Returned by bpf program	    */
		__u32 replylong[4];	/* Optionally returned by bpf prog  */
	};

Contributing to Aya

If a PR touches public API, run the following command to update xtask/public-api/*.txt files and commit them.

cargo +nightly xtask public-api --bless

aya-ebpf.txt can be regenerated on macOS, but others may require Linux.

error[E0432]: unresolved imports `libc::SYS_bpf`, `libc::SYS_perf_event_open`
  --> aya/src/sys/mod.rs:19:19
   |
19 | use libc::{pid_t, SYS_bpf, SYS_perf_event_open};
   |                   ^^^^^^^  ^^^^^^^^^^^^^^^^^^^ no `SYS_perf_event_open` in the root
   |                   |
   |                   no `SYS_bpf` in the root

References

Graph View