I am making a thing to help me with another thing. This is not procrastination, I rather call it tools programming. I am making a tool to help me deal with streams of logs.

The title of this post is a reference to the Ray Tracing in One Weekend series.

# Idea

The gist of it is pretty simple: display tokio-rs/tracing logs in a gui, in a way that doesn't pollute my stdout output. I could redirect the output to a file, but this is slightly sub-optimal. I like the idea of automatically connecting to my program output whenever it runs, and I guess I could make a cursed script using tail, but I could also do something fun instead.

So, a slightly more elaborate sum of the problem space is:

Requirements:

• Display tracing logs in real time while my program is running.
• Be able to easily filter and search them.
• Quickly jump to the code in my existing editor session (nvim in this case).

Constraints:

• Quick to implement.
• Fun.
• Snappy.
• Improve my life.

# Architecture

So GUI in Rust, quick, snappy, lots of logs. Only one answer, GTK (obviously not), egui . egui ("e-gooey", but I always say "e-gee-u-i") is an immediate mode GUI framework akin to Dear ImGui (I believe? I've never actually used it, so feel free to correct me on that).

# Egui and Tokio

These immediate mode GUIs always want to run in the main thread. This means, update() is called once per frame (so 60 times a second at least), which in turn, means, it should be fast and non-blocking. I've been doing my fair share of lock free, real time programming lately, more on that later. But effectively this means that the "backend" should be decoupled from the GUI code (frontend, from now onwards).

For my use case, a regular OS thread would be more than sufficient, but once again, I chose fun. I have been working (and learning) quite a lot about tokio, just by virtue of having an excuse to use it. So naturally, I wanted to use the niceties of asynchronous programming.

The way to do this in with egui, is to run a tokio runtime alongside egui. I use what I called an EguiTokioBridge to handle initialization of both contexts.

#[derive(Debug, Clone)]
pub struct TokioEguiBridge {
    /// This field starts as `None` and is set when the Egui context is registered.
    /// A notification is sent when the context is available.
    egui_ctx: Arc<std::sync::OnceLock<egui::Context>>,
    egui_ctx_available: Arc<OneshotNotify>,

    /// This field starts as `None` and is set when the Tokio runtime is initialized.
    /// A notification is sent when the runtime is available.
    tokio_rt: Arc<std::sync::OnceLock<tokio::runtime::Handle>>,
    tokio_rt_available: Arc<OneshotNotify>,

    cancel: CancellationToken,
}

Both egui and tokio have a handle to these struct, upon start, they each initialize their context (for tokio, that would be it's runtime handle, for egui, that would be it's egui::Context). OneShotNotify is my own construct as well, it's simply a wrapper around tokio::sync::Notify and std::sync::Condvar. This allows me to interface with it from sync and async context. Triggering a OneShotNofity is blocking only on the first call.

pub struct OneshotNotify {
    flag: AtomicBool,
    async_notify: Notify,
    sync_notify: (Mutex<bool>, Condvar),
}

Finally, I have functions to handle initialization.

impl TokioEguiBridge {
    pub fn register_egui_context(&self, ctx: egui::Context) {
        self.egui_ctx.set(ctx).expect("Egui context already set");
        self.egui_ctx_available.notify();
    }

    pub async fn wait_egui_ctx(&self) -> egui::Context {
        self.egui_ctx_available.wait().await;
        self.egui_ctx.get().expect("Egui context not set").clone()
    }

    pub fn wait_egui_ctx_blocking(&self) -> egui::Context {
        self.egui_ctx_available.wait_blocking();
        self.egui_ctx.get().expect("Egui context not set").clone()
    }

    // Same for tokio's handle
}

Having tokio in the backend now makes it easier to handle incoming TCP connections, UI events and more concurrently. And thanks to the TokioEguiBridge I can also:

• Lay out text, with egui::LayoutJob from the backend
• Spawn tokio tasks from the frontend (*but not block on them)

# Real time safety

The other aspect I would like to mention is the real-time nature of immediate mode GUIs. "Time waits for no one" said someone in the 15th century, when we did not have computers, and also The Rolling Stones, when we barely had them. The fundamental implication of real-time programming is that it cannot stop. This is unfortunate because Linux is not conceived as a real-time operating system, though ( PREEMPT_RT exists ).

What does blocking mean? - everything and nothing. An expensive computation can be thought as "blocking" if it takes too long to complete. And depending on how strict we need to be (for example, real time audio is a lot less forgiving), waiting for the kernel to respond might be fine. In any case, we generally are talking about:

• Locks
• Syscalls

And it gets worse, it is not always intuitive:

For example, one would think that calling Mutex::try_lock would not block, because if the mutex is unavailable, it will return an error immediately. However, once the MutexGuard falls out of scope, Drop is called (RAII), and a call to pthread_mutex_unlock is made. This is a syscall, and so it gives control back to the kernel, for an undefined amount of time. See: Using locks in real time audio processing safely for a more detailed explanation.

On the other hand, Instant::now() can be a non-blocking call, even if in theory it would issue a time syscall, modern kernels may actually map this function (under certain conditions) to user space via vDSO .

# Processing logs

Basically, I need to process logs in the backend, and then display them in the UI without blocking it. Since I cannot use any kind of lock (*maybe I could use a Futex with try_lock), I decided to use a lock-free triple buffer, in the same way that game engines do.

The backend is in charge of filtering the logs that will be displayed in the UI. I use a tokio::sync::mpsc::UnboundedSender to send UI events to the backend (i.e. when the user changes the display settings).

The UI needs the whole log history, and it needs to be snappy, so I decided against implementing a virtual scrolling mechanism, where the visible logs are requested to the backend on demand. However, it is still important to not deep clone the data every time I publish an updated buffer, I use reference counting. And so my data structure looks something like this:

type Logs = VecDeque<Arc<Log>>;

There are obviously optimizations that can be made here, in order to speed up search and filtering, but for now this works well enough.

# Closing

There is more stuff. Turns out Neovim can be controlled from a socket, so I use that to jump to code when the user clicks a log path.

I have a custom tracing Layer to send events over TCP.

I have utilities to save and load tracing data from disk.

You are welcome to look at the code on Github .

# Future

The project is not finished, but it has reached a state where it is ready for my personal use. There are a number of features that I'd like to implement in the future, here are some ideas:

• Opentelemetry: Instead of using a custom log format, I could use (or add the option to enable) OpenTelemetry .
• Log compression: both for storage, and for network transfer.
• Vim bindings
• Tabs for keeping track of multiple log streams.