The C++ Alliance

Richard’s October Update

2021-10-10T00:00:00+00:00

Aims and Objectives

This blog is presented in two sections.

The first is a general discussion about completion tokens.

The second is a practical demonstration of a production-grade completion token which adds significant utility to any asynchronous operation that supports the new cancellation feature that arrived in Asio 1.19 (Boost 1.77).

This blog ships with an accompanying github repository in case you want to play with examples. The repository is here.

Asio and the Power of Completion Tokens

Asio (available standalone and in Boost) defines a pattern for writing asynchronous operations. There have been a few examples in my blogs of custom composed operations written in terms of several asynchronous operations made available by the library.

Another often overlooked feature of Asio is the ability to define a customisation point which defines the “behaviour during initiation and the result type” of the asynchronous initiating function.

But what does that mean?

Well, consider the following code:

/* void */ asio::async_read(sock, buffer, "\r\n", [](error_code ec, std::size_t n) { /* handler */ });

This is a verbatim (you could say old-style) asynchronous initiating function which will read from the socket into the buffer until either:

The buffer is full, or
the sequence \r\n is found in the input stream, or
There is some other IO error.

Whichever happens, the lambda is called in the context of the associated executor of the socket.

(Let’s call this “the operation”)

The operation is started immediately and the lambda will be invoked at some point in the future once the operation is complete. The initiating function returns void.

Now consider:

auto n = co_await asio::async_read(sock, "\r\n", asio::use_awaitable);

This code is using the same initiating function to invoke initiate the same asynchronous operation. However, this time instead of providing a Completion Handler we have provided a Completion Token.

The only difference in the two invocations is the presence of the token. The actual asynchronous operation is the same in both cases.

However, now invocation of the operation has been modified such that:

The initiating function returns an asio::awaitable<std::size_t> which can be co_awaited.
The initiating function has been transformed into a C++20 coroutine.
The operation will not commence until the returned awaitable has been co_awaited.

We can say that the completion token has implemented a customisation point at both the initiation step and the completion step.

(For great notes on completion step I would recommend reading one of the many excellent papers, follow-ups or videos), published by Chris Kohlhoff - the author of Asio.

Here is another interesting example:

using asio::experimental::deferred;
using asio::use_awaitable;

auto my_op = asio::async_read(sock, "\r\n", deferred);
...
auto n = co_await my_op(use_awaitable);

In this case, the async_read initiating function has been invoked with the deferred completion token. This token has two side effects:

The asynchronous operation is not actually initiated, and
It changes the return type to be an invocable object which when called will behave as if you called the initiating function.

The returned invocable object is a unary function object whose argument is a completion token, which means that the operation can be further customised at the point of use. You can think of it as an asynchronous packaged tasks awaiting one last customisation before use.

Note that as long as the packaged asynchronous operation is started with reference arguments or lightweight copyable arguments, it can be re-used and copied. All arguments of Asio and Beast initiating functions conform to this principle. The original design decision of passing buffers and return values by reference to asynchronous operations was to ensure that when operations are composed, they do not allocate memory - the caller can specify the memory management strategy. It so happens that this design decision, taken something like 16 years ago, has enabled efficient composition of completion tokens.

Finally, on the subject of deferred, deferring a deferred initiating function yields the same deferred initiating function. I guess one way to think about completion tokens is that they are transforms or higher order functions for manipulating the initiation and result types of asynchronous operations.

example:

asio::awaitable<void>
reader(asio::ip::tcp::socket sock)
{
  using asio::experimental::deferred;
  using asio::use_awaitable;
  using asio::experimental::as_tuple;

  // An easy but not efficient read buffer
  std::string buf;

  // created the deferred operation object
  auto deferred_read = async_read_until(
      sock,
      asio::dynamic_buffer(buf),
      "\n",
      deferred);

  // deferring a deferred operation is a no-op
  auto deferred_read2 = deferred_read(deferred);

  // tokens are objects which can be composed and stored for later
  // The as_tuple token causes the result type to be reported as a
  // tuple where the first element is the error type. This prevents
  // the coroutine from throwing an exception.
  const auto my_token = as_tuple(use_awaitable);

  bool selector = false;
  for(;;)
  {
    // use each deferred operation alternately
    auto [ec, n] = co_await [&] {
      selector = !selector;
      if (!selector)
        return deferred_read(my_token);
      else
        return deferred_read2(my_token);
    }();
    if (ec)
    {
      std::cout << "reader finished: " << ec.message() << "\n";
      break;
    }
    auto view = std::string_view(buf.data(), n - 1);
    std::cout << "reader: " << view << "\n";
    buf.erase(0, n);
  }
}

A table of completion tokens packaged with Asio is presented here:

token	Initiation Policy	Completion Behaviour/Result Type	Notes
detached	Initiate immediately	Ignore all errors and results	When used with `co_spawn`, causes the spawned asynchronous chain of coroutines to have behaviour analogous to a detached thread.
experimental::deferred	Do not initiate	Return a function object which when invoked with a completion token, behaves as if the original initiating function was called with that same token	Analogous to an asynchronous packaged task.
use_future	Initiate immediately	Return a std::future which will yield the completion arguments
use_awaitable	Initiate when awaited	Return an awaitable object yield the completion arguments when `co_await`ed
yield_context	Initiate immediately	Yield the current stackful coroutine. Once the operation is complete, resume and return the handler arguments
as_tuple(token)	Initiate as indicated by the supplied `token`	Modify the completion arguments to be a single tuple of all arguments passed to the completion handler. For example, `void(error_code, size_t)` becomes `void(tuple<error_code, size_t>)`. In practical terms this token ensures that partial success can be communicated through `use_future`, `use_awaitable` and `yield`	Very useful when used with `yield`, `use_future` or `use_awaitable` if we’d like to handle the error without exception handling or when a partial success must be detected. For example, the error_code might contain `eof` but `size` might indicate that 20 bytes were read into the buffer prior to detecting the end of stream condition.
redirect_error(token, &ec)	Initiate as indicated by the supplied `token`	For operations whose first completion handler argument is an `error_code`, modify the completion handler argument list to remove this argument. For example, `void(error_code, size_t)` becomes `void(size_t)`. The error code is redirected to object referenced by `ec`	Similar to the above use, but allows overwriting the same `error_code` object which can be more elegant in a coroutine containing multiple calls to different initiating functions.
experimental::as_single(token)	Initiate as indicated by the supplied `token`	Similar to `as_tuple` except in the case where the only argument to the completion handler is an error. In this case, the completion handler arguments are unaltered.	Experience of use suggests to me that this token is less useful than `redirect_error` and `as_tuple`.
experimental::append(token, values…)	Initiate as indicated by the supplied `token`	When the completion handler is invoked, the `values...` arguments are appended to the argument list.	Provides a way to attaching more information to a completion handler invocation. examples
experimental::prepend(token, values…)	Initiate as indicated by the supplied `token`	When the completion handler is invoked, the `values...` arguments are prepended to the argument list.	Provides a way to attaching more information to a completion handler invocation. examples

A Custom Completion Token

All very interesting and useful, no doubt. But what if we wanted to do something more clever.

The other day I was speaking to Chris about timed cancellation. Now there are ways of doing timed cancellation that in Chris’ view are correct and maximally performant (covered in this video). However many users don’t need maximum performance. What they often want is maximum teachability or maintainability.

So I posed the question: “Imagine I wanted a function which took any existing Asio composed asynchronous operation and produced a new composed operation which represented that same operation with a timeout. How would I do that?”

For example, imagine we had a deferred read operation:

    auto read_op = async_read(stream, buffer, deferred);

Which we can invoke in a coroutine like so:

    co_await read_op(use_awaitable);

imagine we could write:

    co_await with_timeout(read_op, 5s, use_awaitable);

or to write it out in full:

    co_await with_timeout(
        async_read(stream, buffer, deferred),
        5s,
        use_awaitable);

The answer that came back was to me quite surprising: “It starts with a completion token”.

Which means that the way to achieve this is to write the with_timeout function in terms of a composition of completion tokens:

template <typename Op, typename CompletionToken>
auto with_timeout(Op op, std::chrono::milliseconds timeout, CompletionToken&& token)
{
    return std::move(op)(timed(timeout, std::forward<CompletionToken>(token)));
}

In the above code, timed is a function that returns a parameterised completion token. It will look something like this:

template <typename CompletionToken>
timed_token<CompletionToken> 
timed(std::chrono::milliseconds timeout, CompletionToken&& token)
{
    return timed_token<CompletionToken>{ timeout, token };
}

The actual token type would look like this:

template <typename CompletionToken>
struct timed_token
{
    std::chrono::milliseconds timeout;
    CompletionToken& token;
};

So far, so simple. But how will this work?

Well, remember that a completion token controls the injection of logic around an asynchronous operation. So somehow by writing the token, we will get access to the packaged operation prior to it being initiated and we get access to the following building blocks of the async operation provided by Asio’s initiation pattern:

The initiation - this is a function object that will actually initiate the packaged asynchronous operation, and
The initiation arguments - the arguments that were supplied to the initial initiation function. In our example above, these would be stream and buffer

Note that the initiation is an object that describes how to launch the underlying asynchronous operation, plus associated data such as the associated executor, associated allocator and associated cancellation slot.

In Asio, the customisation point for initiating an asynchronous operation with a given completion token is the template class async_result.

Here is the specialisation:

// Specialise the async_result primary template for our timed_token
template <typename InnerCompletionToken, typename... Signatures>
struct asio::async_result<
      timed_token<InnerCompletionToken>,  // specialised on our token type 
      Signatures...>
{
    // The implementation will call initiate on our template class with the
    // following arguments:
    template <typename Initiation, typename... InitArgs>
    static auto initiate(
        Initiation&& init, // This is the object that we invoke in order to
                           // actually start the packaged async operation
        timed_token<InnerCompletionToken> t, // This is the completion token that
                                             // was passed as the last argument to the
                                             // initiating function
        InitArgs&&... init_args)      // any more arguments that were passed to
                                      // the initiating function
    {
        // we will customise the initiation through our token by invoking
        // async_initiate with our own custom function object called
        // timed_initiation. We will pass it the arguments that were passed to
        // timed(). We will also forward the initiation created by the underlying
        // completion token plus all arguments destined for the underlying
        // initiation.
        return asio::async_initiate<InnerCompletionToken, Signatures...>(
                timed_initiation<Signatures...>{},
                  t.token,   // the underlying token
                  t.timeout, // our timeout argument
                std::forward<Initiation>(init),  // the underlying operation's initiation
                std::forward<InitArgs>(init_args)... // that initiation's arguments
        );
    }
};

It’s a bit of a wall of text, but most of that is due to my comments and C++’s template syntax. In a nutshell, what this class is doing is implementing the function which wraps the initiation of the underlying operation (i.e the async_read) in an outer custom initiation which is going to add a timeout feature.

All that remains is to define and implement timed_initiation<>, which is nothing more than a function object. We could have written it inline as a lambda, but for clarity it has been broken out into a separate template.

async_initate looks complicated but in actual fact is doing a simple transformation:

Given:

tok is a CompletionToken
Signatures... is a type pack of function signatures that are required to be supported by a CompletionHandler built from tok.
initiation is a function object
args... is a set of arbitrary arguments

async_initiate is a helper function which calls async_result<>::initiate(). Calling this will first transform tok into a CompletionHandler which we will call handler. Then it will simply call initiation(handler, args...). i.e. it will invoke the initiation with the correct completion handler and any other arguments we happen to give it.

// Note: this is merely a function object - a lambda.
template <typename... Signatures>
struct timed_initiation
{
    template <
        typename CompletionHandler,
        typename Initiation,
        typename... InitArgs>
    void operator()(
      CompletionHandler handler,         // the generated completion handler
      std::chrono::milliseconds timeout, // the timeout specified in our completion token
      Initiation&& initiation,           // the embedded operation's initiation (e.g. async_read)
      InitArgs&&... init_args)           // the arguments passed to the embedded initiation (e.g. the async_read's buffer argument etc)
    {
        using asio::experimental::make_parallel_group;

        // locate the correct executor associated with the underling operation
        // first try the associated executor of the handler. If that doesn't
        // exist, take the associated executor of the underlying async operation's handler
        // If that doesn't exist, use the default executor (system executor currently)
        auto ex = asio::get_associated_executor(handler,
                                                asio::get_associated_executor(initiation));

        // build a timer object and own it via a shared_ptr. This is because its
        // lifetime is shared between two asynchronous chains. Use the handler's
        // allocator in order to take advantage of the Asio recycling allocator.
        auto alloc = asio::get_associated_allocator(handler);
        auto timer = std::allocate_shared<asio::steady_timer>(alloc, ex, timeout);

        // launch a parallel group of asynchronous operations - one for the timer
        // wait and one for the underlying asynchronous operation (i.e. async_read)
        make_parallel_group(
                // item 0 in the group is the timer wait
                asio::bind_executor(ex,
                                    [&](auto&& token)
                                    {
                                        return timer->async_wait(std::forward<decltype(token)>(token));
                                    }),
                // item 1 in the group is the underlying async operation
                asio::bind_executor(ex,
                                    [&](auto&& token)
                                    {
                                        // Finally, initiate the underlying operation
                                        // passing its original arguments
                                        return asio::async_initiate<decltype(token), Signatures...>(
                                                std::forward<Initiation>(initiation), token,
                                                std::forward<InitArgs>(init_args)...);
                                    })
        ).async_wait(
                // Wait for the first item in the group to complete. Upon completion
                // of the first, cancel the others.
                asio::experimental::wait_for_one(),

                // The completion handler for the group
                [handler = std::move(handler), timer](
                    // an array of indexes indicating in which order the group's
                    // operations completed, whether successfully or not
                    std::array<std::size_t, 2>,

                    // The arguments are the result of concatenating
                    // the completion handler arguments of all operations in the
                    // group, in retained order:
                    // first the steady_timer::async_wait
                    std::error_code,

                    // then the underlying operation e.g. async_read(...)
                    auto... underlying_op_results // e.g. error_code, size_t
                    ) mutable
                {
                    // release all memory prior to invoking the final handler
                    timer.reset();
                    // finally, invoke the handler with the results of the
                    // underlying operation
                    std::move(handler)(std::move(underlying_op_results)...);
                });
    }
};

Now that the token and its specialisation of async_result is complete, we can trivially write a timed read from console that won’t throw as a coroutine in one line:

    // using the completion token direct
    auto [ec1, n1] = co_await async_read_until(in, dynamic_buffer(line), '\n',
                                              as_tuple(timed(5s, use_awaitable)));

    // using the function form
    auto [ec2, n2] = co_await with_timeout(
        async_read_until(in, asio::dynamic_buffer(line), '\n', deferred),
        5s,
        as_tuple(use_awaitable));

The full code for this example is here. Note that this example is written in terms of a posix console stream. To demonstrate on Windows, you would need to replace the posix::stream_descriptor in(co_await this_coro::executor, ::dup(STDIN_FILENO)); with a stream type compatible with Windows, such as a socket or named pipe… or even adapt the example to use a Beast http::async_read - and presto! You have a ready-made HTTP server which applies a timeout to reading messages.

Update 2021-10-11: I have since modified the example so that on windows a local tcp socket pair is created and a coroutine is spawned to handle the input side of things. The demo now compiles and runs with MSVC2019.

A Note on Performance

It is important that I point out that this example token has been written with ease of use as the primary motivating factor. There is a pessimisation in its design in that use of the token allocates a new timer for every asynchronous operation where the timeout is being applied. This of course becomes completely un-necessary if we redesign the token so that we pass a reference to an existing timer to its construction function.

The call-site would then look more like this:

    auto timer = asio::steady_timer(co_await this_coro::executor, 5s);
    auto [ec1, n1] = co_await async_read_until(in, dynamic_buffer(line), '\n',
                                              as_tuple(use_deadline(timer, use_awaitable)));

Writing it this way would actually result in a simpler initiation and would ensure that the general Asio principle of giving the caller control over object lifetimes and allocation strategies is maintained.

Another way to avoid repeated allocations of the timer while retaining the “easy mode” interface is to make use of Asio’s execution context service facility. In this way timers would be cached in the service object associated with the associated executor’s execution_context.

Asio was originally designed for highly scalable and latency-sensitive applications such as used in the finance, automotive and defence industries. Out of the box it provides the basic building blocks with which to assemble performance and memory-critical applications. However as it has become more widely adopted there is a growing demand for “easy mode” interfaces for people who just want to get things done.

This message has not gone unheard. I would expect a number of interesting new features to be added to the library in short order.

Thanks for reading.

Richard Hodges
for C++ Alliance
hodges.r@gmail.com

Richard’s May 2021 Update

2021-05-30T00:00:00+00:00

The Month in Review

It’s been a month of minor maintenance fixes, and a fair amount of support requests via the C++ Alliance Slack workspace.

Property_tree

On the maintenance front, there are a number of historic pull requests in the property_tree repo which need working through. Some of these take some unravelling and a degree of care, as I am still new to this venerable library and I have the impression that it is used fairly ubiquitously in many code bases of varying pedigree.

Currently I have no way of reaching out to users (not knowing exactly who they are) so the only way to know whether a change is going to break someone’s build is to release it, by which time it is too late.

I think the answer here is to start building out more test cases. Property_tree only recently gained CI, and so far I have not gotten around to adding test coverage. No doubt I’ll get to this in due course.

Beast

There are a lot of eager developers out there keen to use Beast and Asio, which is encouraging. The less encouraging thing is the amount of time I find myself spending giving ad-hoc support to people who have hit the Asio mental brick wall (which I remember when learning this fantastic library all too well).

I have written blogs in this series before covering some of the topics I think are important and developers often misunderstand, but there is more to do.

With this in mind, an idea has been germinating over the past few months, which finally started to develop into a new library this month. I’ll come back to this later.

Asio

A few months ago I attended a WG21 meeting where a formal means of providing cancellation to asynchronous operations was proposed. A few people at that meeting, myself included, were concerned that the proposal in its current form would constrain the development style of asynchronous programs, making the fundamental objects a little more complex than they often need to be.

I have recognised that Asio needs a formal task cancellation mechanism for some time, this being the basis of the async cancellation_token mentioned in a previous blog.

I have been able to get some of Chris Kohlhoff’s valuable time to discuss this to see whether there is a way to get effortless cancellation into Asio without impacting performance or compiled size when cancellation is not required.

Chris, as he is wont to do, made the rather brilliant connection that in Asio, a 1-shot cancellation token can be associated with each asynchronous completion handler, with the default token type being a zero-cost abstraction of a null cancellation token - i.e. one that will never invoke the stop callback.

The general idea being that if you want an operation to be cancellable, you would invoke it like this:

// This is the signal object that you would use to 
// cancel any operation that depends on one of its slots
asio::cancellation_signal sig; 

// some IO Object
timer t(ioc, chronons::seconds(5));

// perform an asynchronous operation bound to the cancellation signal
t.async_wait(
    bind_cancellation_slot(sig.slot(),
      [](system::error_code ec)
      {
        // if the signal is invoked, the timer's asynchronous operation will notice
        // and the operation will complete with ec equal to asio::errors::operation_aborted
      });
      
// signal the cancellation
sig.emit():

The interesting thing about this is that the cancellation slot is associated with the asynchronous operation’s handler. This is not only useful for a library-provided asynchronous operation such as a timer wait. Because of the existence of a function called get_associated_cancellation_slot(handler), the current slot is available in any context where user code has access to the current asynchronous completion handler.

One such place is in a user-defined composed operation, and therefore by extension, a c++ coroutine running in the context of an Asio executor.

This now becomes possible:


asio::awaitable<void>
my_coro(some_async_op& op)
{
    // The cancellation state allowed us to detect whether cancellation has been requested
    // It also allows us to register our own cancellation slot
    auto cs = asio::this_coro::cancellation_state;
    
    // Create a new slot from the cancellation state and register a callback which will 
    // invoke our own custom cancel signal on the some_async_op
    // note: A
    auto slot = cs.slot();
    slot.emplace([&]{ op.cancel(); });

    // continue to wait on the some_async_op
    co_await op.wait_to_finish();
}

This coroutine could be invoked in a couple of ways:


// In this case the cancellation state is a no-op cancellation. 
// the code at note A above will do nothing. This coroutine is not cancellable.
co_await asio::co_spawn(exec, 
                        my_coro(op), 
                        asio::use_awaitable);

// In this case, the coroutine has become cancellable because the code at note A will actually
// create a functioning slot and register the lambda.
// The coroutine is cancellable through the cancellation signal sig.
asio::cancellation_signal sig;
co_await asio::co_spawn(exec, 
                        my_coro(op), 
                        asio::bind_cancellation_signal(
                            asio::use_awaitable, sig));

This code is experimental at the moment, but is available
on the generic-associators branch of Boost.Asio.

The Project du Jour

Coming back to the “Asio is hard at the beginning” meme, I was speaking to my son recently. He works with a number of languages, including Python, Go and C++.

During a conversation about these he mentioned that Go was a very uninspiring language (to him) but it was very easy to get fairly complex asynchronous programs functioning reliably in a short amount of time.

I asked him what the single most effective feature of the language was, to which he replied, “channels”.

For anyone who does not already know, a golang channel is simply a multi-producer, multi-consumer ring buffer with an asynchronous interface.

It has the following behaviour:

Producer coroutines will suspend when providing values to the channel if the ring buffer is full and there is no consumer pending a consume operation.
Consumer coroutines will suspend when consuming if the ring buffer is empty and there is no pending producer operation in progress.
The ring buffer capacity is specified upon construction, and may be zero. Producers and consumers of a zero-sized channel will only make progress if there is a corresponding pair of producer and consumer pending at the same time. In this way, the channel also acts as a coroutine synchronisation primitive.
Finally, the channel may be closed. A closed channel will allow a consumer to consume remaining values in the ring buffer, but it will not allow a producer to provide more values, whether into the ring buffer or directly to a pending consume operation. A consume operation against an empty, closed channel will yield a default-constructed object plus a boolean false indicating that there are no more values to consume.

There are some other nice features in Go, such as the select keyword which interact with channels in a pleasing way, but for now I’ll focus on how we might implement the channel in asynchronous C++.

The rationale here being:

Channels make writing complex asynchronous interactions simple.
Make simple things simple is the mantra to which I subscribe.
Perhaps C++ enthusiasts would benefit from an implementation of channels.
Given the flexibility of C++, we might be able to do a better job than Go, at least in terms of giving the programmer some choice over implementation tradeoffs.
Maybe a little library offering this functionality in a simple, reusable way would be a useful addition to Boost.

I put some feelers out in the CppLang slack. So far the response to the idea has been only positive. So I decided to make a start.

TLDR - you can monitor how far I am getting by looking at the Github repository.

Approach

I wanted the channels library to be built on top of Asio. The reason for this is that I happen to think that the Asio executor model is very elegant, and allows the programmer to transpose the same fundamental idea onto a number of different concurrency strategies. For example, thread pools, IO loop, threads and futures, and so on.

Asio’s completion tokens allow the adaptation of asynchronous initiating functions to any or all of these strategies and I wanted to make sure that the library will provide this functionality.

Furthermore, asynchronous programs become complex quickly. Asio is a natural fit for IO, but does not provide the primitives that programmers often find they need to create rich programs.

It is my hope that this channels library provides people with a useful tool to make high performance, highly concurrent programs easier to write in C++.

Design Decisions

I have elected to write library in two sections. The first will contain the basic objects to handle the concurrent communication and asynchronous completions. These objects will not be thread-safe, just like any other object in Asio.

The second will be a thread-safe interface written in terms of the first. The truth is that Asio objects do not need to be thread-safe if programmers use the correct discipline vis-a-vis strands and ensuring that work is dispatched to the correct strand. Another truth is that many programmers just want things to be easy. So why not provide an easy-mode interface too?

Comparison

OK, so let’s take a simple Go program and see how we could express that in terms of Asio and C++ coroutines. Now I’m no expert, so I’m sure there are many ways to improve this program. It’s about the third Go program I’ve ever written. Please by all means let me know.

package main

import (
	"fmt"
	"sync"
)

func produce(wg *sync.WaitGroup, c chan<- string) {
	defer wg.Done()
	c <- "The"
	c <- "cat"
	c <- "sat"
	c <- "on"
	c <- "the"
	c <- "mat"
	close(c)
}

func consume(wg *sync.WaitGroup, name string, c <-chan string) {
	defer wg.Done()
	for {
		s, more := <-c
		if more {
			fmt.Println(name, ":", s)
		} else {
			fmt.Println(name, ": Channel closed", name)
			break
		}
	}
}

// Main function
func main() {
	var wg sync.WaitGroup
	wg.Add(4)
	c := make(chan string)
	go consume(&wg, "a", c)
	go consume(&wg, "b", c)
	go consume(&wg, "c", c)
	go produce(&wg, c)
	wg.Wait()
}

And this is how I would envision it would look in the first cut of the C++ version:

auto
produce(channels::channel< std::string > &c) 
    -> asio::awaitable< void >
{
    constexpr auto wait = asio::use_awaitable;
    co_await c.async_send("The", wait);
    co_await c.async_send("cat", wait);
    co_await c.async_send("sat", wait);
    co_await c.async_send("on", wait);
    co_await c.async_send("the", wait);
    co_await c.async_send("mat", wait);
    c.close();
}

auto
consume(std::string_view name, channels::channel< std::string > &c)
    -> asio::awaitable< void >
{
    auto ec  = channels::error_code();
    auto tok = asio::redirect_error(asio::use_awaitable, ec);
    for (;;)
    {
        auto s = co_await c.async_consume(tok);
        if (ec)
        {
            std::cout << name << " : " << ec.message() << "\n";
            break;
        }
        else
            std::cout << name << " : " << s << "\n";
    }
}

int
main()
{
    auto ioc = asio::io_context();
    auto c   = channels::channel< std::string >(ioc.get_executor());

    asio::co_spawn(ioc, consume("a", c), asio::detached);
    asio::co_spawn(ioc, consume("b", c), asio::detached);
    asio::co_spawn(ioc, consume("c", c), asio::detached);
    asio::co_spawn(ioc, produce(c), asio::detached);

    ioc.run();
}

One example of the output of the Go program (the order is actually nondeterministic) is:

a : The
a : cat
b : sat
b : mat
b : Channel closed b
a : on
a : Channel closed a
c : the
c : Channel closed c

while the output of the C++ program is a more deterministic:

a : The
b : cat
c : sat
a : on
b : the
c : mat
a : Channel is closed
b : Channel is closed
c : Channel is closed

I’m not an expert in Go by any means but I imagine the nondeterminism in the Go program is in part due to the fact that the goroutine implementation is allowed to take shortcuts to consume data synchronously if it’s available. The Asio model requires that each completion handler is invoked as-if by a call to post(handler). In this program, these posts are being made to a single-threaded io_context and so are being executed sequentially, preserving the order of invocation during execution.

If this program were multi-threaded, it might be a different story. But this will have to wait until the basic single-threaded implementation is complete.

Implementation Details

The implementation of the channel is actually fairly straightforward. The asynchronous initiation interfaces are standard asio, e.g.:

template < class ValueType, class Executor >
template < BOOST_ASIO_COMPLETION_TOKEN_FOR(void(error_code)) SendHandler >
BOOST_ASIO_INITFN_RESULT_TYPE(SendHandler, void(error_code))
channel< ValueType, Executor >::async_send(value_type    value,
                                           SendHandler &&token)
{
    if (!impl_) [[unlikely]]
        BOOST_THROW_EXCEPTION(std::logic_error("channel is null"));

    return asio::async_initiate< SendHandler, void(error_code) >(
        [value = std::move(value), this](auto &&handler) {
            auto send_op = detail::create_channel_send_op(
                std::move(value),
                this->impl_->get_executor(),
                std::forward< decltype(handler) >(handler));
            impl_->notify_send(send_op);
        },
        token);
}

The macros are supplied by Asio and simply ensure that the most up-to-date compiler facilities are used to ensure that the completion token/handler has the correct signature. BOOST_ASIO_INITFN_RESULT_TYPE deduces the return type of the selected specialisation of async_initiate. It is what ensures that async_send returns an awaitable when the completion token is of type asio::use_awaitable, or a std::future if we were to pass in asio::use_future.

The actual work of the send is performed in the implementation class:

    void
    notify_send(detail::channel_send_op_concept< ValueType > *send_op)
    {
        // behaviour of send depends on the state of the implementation.
        // There are two states, running and closed. We will be in the closed
        // state if someone has called `close` on the channel.
        // Note that even if the channel is closed, consumers may still consume
        // values stored in the circular buffer. However, new values may not
        // be send into the channel.
        switch (state_)
        {
        case state_running:
            [[likely]] if (consumers_.empty())
            {
                // In the case that there is no consumer already waiting,
                // then behaviour depends on whether there is space in the
                // circular buffer. If so, we store the value in the send_op
                // there and allow the send_op to complete.
                // Otherwise, we store the send_op in the queue of pending
                // send operations for later processing when there is space in
                // the circular buffer or a pending consume is available.
                if (free())
                    push(send_op->consume());
                else
                    senders_.push(send_op);
            }
            else
            {
                // A consumer is waiting, so we can unblock the consumer
                // by passing it the value in the send_op, causing both
                // send and consume to complete.
                auto my_receiver = std::move(consumers_.front());
                consumers_.pop();
                my_receiver->notify_value(send_op->consume());
            }
            break;
        case state_closed:
            // If the channel is closed, then all send operations result in
            // an error
            [[unlikely]] send_op->notify_error(errors::channel_closed);
            break;
        }
    }

An interesting feature of the send operation class is that when it is instructed to complete, it must:

Move the value out of itself,
Move the completion handler out of itself,
Destroy itself, returning memory back to the allocator.
Post the completion handler to the correct executor.
Return the value.

The order is important. Later on we will be adding Asio allocator awareness. In order to maximise efficiency, Asio asynchronous operations must free their memory back to the allocator before completing. This is so that during the execution of the completion handler, the same memory that was just freed into asio’s special purpose allocators will be allocated and used to compose the next completion handler. This memory will be at the head of the allocator’s list of free blocks (and therefore found first) and it will be in cached memory, having just been touched.

template < class ValueType, class Executor, class Handler >
auto
basic_channel_send_op< ValueType, Executor, Handler >::consume() -> ValueType
{
    // move the result value to the local scope
    auto result  = std::move(this->value_);
    
    // move the handler to local scope and transform it to be associated with
    // the correct executor.
    auto handler = ::boost::asio::bind_executor(
        std::move(exec_),
        [handler = std::move(handler_)]() mutable { handler(error_code()); });
    
    // then destroy this object (equivalent to delete this)
    destroy();
    
    // post the modified handler to its associated executor
    asio::post(std::move(handler));
    
    // return the value from the local scope to the caller (but note that NRVO
    // will guarantee that there is not actually a second move)
    return result;
}

That’s all for now. I’ll add extra blog entries as and when I make any significant progress to the library.

In the meantime, I’m always happy to receive queries by email or as issues in the github repo.

Thanks for reading.

Richard Hodges
for C++ Alliance
hodges.r@gmail.com

Richard’s April Update

2021-04-30T00:00:00+00:00

A new responsibility

Last month I took over the maintenance of the Boost Property Tree library.

This library is in ubiquitous use but has been without a full time maintainer for some time. There was a backlog of commits on develop that had not made it to master for the boost releases. This is changing in upcoming 1.76 release. Property Tree PRs and issues will get more attention going forward. In addition, Property Tree has gained CI courtesy of GitHub Actions. The test status can be seen here

Beast CI

Beast has acquired two new continuous integration pipelines. Drone and Github Actions. Drone is the canonical source of truth for the Linux/Windows badge on the readme page while GitHub Actions provides a more comprehensive matrix of targets, C++ standards and compiler versions which I use during the development cycle.

Boost Version 1.76

The 1.76 release of boost is imminent. As far as Beast and Property Tree is concerned, this is a maintenance release.

Richard’s February/March Update

2021-03-30T00:00:00+00:00

Boost.PropertyTree is back in the saddle!

Last month I took over the maintenance of the Boost Property Tree library.

Beast CI

Boost Version 1.76

The 1.76 release of boost is imminent. As far as Beast and Property Tree is concerned, this is a maintenance release.

Other Activities

Much of the past two months has been devoted to resolving user queries, and examining suggestions for changes to Beast and Property Tree.

Changes to Property tree are not taken likely. It is used in a great deal of legacy code in the wild and there are few tests. It would be a shame to break existing code noisily, or worse, silently. For better or worse, the behaviour is probably not going to change very much going forward until there are more tests in place.

Accepting changes to beast’s behaviour or interface is something we consider very carefully for a different reason. Beast is already a complex library. Like Asio upon which it is built, it is already extremely configurable, with a myriad (actually potentially an unbounded set) of completion tokens, completion handlers and buffer types.

Something I am often asked is whether we would consider a “multi-send” interface. For example:

std::vector<std::string> messages;
ws.write(buffer, messages);

Arguably there are some efficiencies available here, since we could potentially build all the resulting websocket frames in one pass, and send as one IO operation.

There are some issues however. One is buffer space. What do we do if the buffer runs out of space while be are half way through building the frames? Fail the operation, return a partial failure? Block? Also, what do we do if the ensuing IO operation(s) partially fail(s)? Asio currently has no protocol to report a partial failure. We would have to create such a protocol, test it and then teach it.

In reality people’s requirements are often different. Some people may require confirmation of each frame within a time period, some will want all-or-nothing delivery and some are happy with partial delivery.

One approach is to start providing options for such things as timeouts, partial completion, etc. But the feeling here is that this starts to dictate to users of the library how they write code, which we feel is outside the remit of Beast.

Perhaps there is a case for a function that separates out the building of a websocket frame from the sending of it. I’ll give it some thought.

Emil’s January Update

2021-02-20T00:00:00+00:00

History

This is my first entry on the C++ Alliance web site. I am happy to have been invited to join the organization.

I first talked to Vinnie on Slack, and was surprised to find that he is practically a neighbor. So we got in touch, had breakfast together, talked about C++ and life, the universe and everything. We became friends, he took care of our cats when we flew to Europe (pre-Covid), and I brought back some Rakia (next time I’ll bring him some of my father’s home-made stuff!)

(Boost) URL

My immediate focus after joining the C++ Alliance is to work on what will hopefully become Boost.URL. C++ has long been missing a solid standard-conforming library for working with URLs, we often get contacted by impatient users asking about the state of things.

Building on the foundation already in place and requiring only C++11, the library is being developed to the highest standards of portability, safety and efficiency. We’re also careful to keep the design simple, with a lot of attention paid to minimizing physical coupling so that compilation is as fast as possible.

About

I started coding in middle school on the Bulgarian-made Apple ][ clone Правец-82 (named after – you guessed it – the birth place of the leader of the Bulgarian Communist Party at the time). This has resulted in a lot of useless information being engraved on my brain, I still remember some of the opcodes of the 6502 CPU. :)

After graduating the Sofia University I moved to L.A. to work in the video game industry where I’ve spent most of my professional career, starting with the original Playstation all the way to modern consoles and handhelds. For the last few years I’ve shifted to projects that utilize my background in realtime graphics engines in areas that are not directly connected to gaming.

Pretty much all of my projects have been written in C++. This includes several Boost libraries, the latest one being LEAF, a Lightweight Error Augmentation Framework for C++11. I’ll probably post about error handling at some later time, as this is one of my key areas of interest.

Thanks for reading.

Emil Dotchevski

Dmitry’s January Update

2021-02-15T00:00:00+00:00

Dmitry’s January Update

In January 2021 I started working on improving Boost.JSON for The C++ Alliance. The time during this first month was mostly spent on getting familiar with the project, particularly with its list of issues on GitHub.

It turns out, half of the open issues are related to documentation. For example, the section about conversions might need a rewrite, and related entries in the reference need to provide more information. There should be more examples for parsing customizations. There also needs to be a separate section dedicated to library’s named requirements. There was also a bug in coversion customization logic that was fixed by me this month.

The next two large blocks are issues related to optimization opportunities and dealing with floating point numbers (some issues are present in both groups). The next group is issues related to build and continuous integration. A couple of build bugs were fixed in January.

The final group consists of feature requests, mostly for convenient ways to access items inside json::value. And this month I started implementing one such feature – Json.Pointer support. The work is still in the early stages, though.

Richard’s January Update

2021-01-31T00:00:00+00:00

A year in the C++ Alliance

January marks one year since I joined the C++ Alliance and started maintaining the Boost.Beast library.

It’s a pleasure to work with interesting and fun people who are passionate about writing C++, and in particular good C++.

During the past year I have spent some time attending ISO WG21 meetings online as an alternate representative of the C++ Alliance. Prior to joining the organisation, during my life as a developer I always felt that the standards committee did the developer community a disservice. Without knowing much about the inner workings, it seemed to me that the committee lived in an Ivory Tower. So my intention was to see if there was a way to bring something useful to the table as a keen a prolific writer of C++ in the financial sector. In particular I had a personal interest in the standardisation of asynchronous networking, forked from the wonderful Asio library.

I ended the year feeling no less jaded with the entire standards process, concluding that there is not much I can do to help.

I feel it’s important to say that the committee is attended by very bright, passionate people who clearly enjoy the C++ language as much as I do.

What I think does not work, at least from the point of view of delivering useful progress, is the process of committee itself. During my commercial life there has been one fundamental truth, which is that things go well when there is focus of attention and the taking of personal responsibility. It seems to me that committees in general undermine these important fundamentals. The upshot of this is that in my mind, C++ developers are not going to get the tools they need, in the timescales they need, if they wait for the slow grind of WG21’s wheels of stone.

It is to me noteworthy that many of the libraries I actually use (fmt, spdlog, boost, jwt, openssl, and so on) have been in some way standardised or are in the process of being standardised, but always in a lesser form than the original, created by a small, passionate team of individuals who enjoyed autonomy and freedom of design.

Even now, if a feature is available in the standard and as a 3rd party library, I will almost always choose the third party version. It will generally have more features and a design undamaged by a process that externalises costs.

Which brings me back to my old C++ mantra, proven true for me over the past 15 years or so, Boost is the Standard. Having said this, I must in fairness mention the wonderful fmtlib and spdlog libraries, and the gargantuan Qt, without which a back-end developer like me would never be able to get anything to display on a screen in a cross- platform manner.

In the end I find myself in the same place I was a year ago: My view is that the only thing C++ needs is a credible dependency management tool. Given that, the developer community will produce and distribute all needed libraries, and the most popular will naturally become the standard ones.

The Year Ahead

Therefore, it is my intention this year to do what I can to bring more utility to the Boost ecosystem, where one person can make a useful impact on the lives of developers, and taking personal responsibility for libraries is the norm.

The Big Three

It is my view that there are a number of areas where common use cases have not been well served in Boost. These are of course JSON, Databases and HTTP clients.

JSON

At the end of 2020, Vinnie and the team finally brought a very credible JSON library to Boost, which I have used to write some cryptocurrency exchange connectors. On the whole, it’s proven to be a very pleasant and intuitive API. In particular the methods to simultaneously query the presence of a value and return a pointer on success with if_contains, if_string etc. have seen a lot of use and result in code that is readable and neat.

Currently, Boost.JSON favours performance over accuracy with respect to parsing floating point numbers (sometimes a number is 1 ULP different to that which you’d expect from std::strtod). I had a little look to see if there was a way to address this easily. It transpired not. Parsing decimal representations of floating point numbers into a binary representation that will round-trip correctly is a hard problem, and beyond my level of mathematical skill. There is currently work underway to address this in the JSON repo.

There is also work in progress to provide JSON-pointer lookups. This will be a welcome addition as it will mean I can throw away my jury-rigged “path” code and use something robust.

MySQL

A very common database in use in the web world is of course MySQL. I have always found the official connectors somewhat uninteresting, particularly as there is no asynchronous connector compatible with Asio.

That was until a certain Rubén Pérez decided to write an asio-compatible mysql connector from scratch, using none of the code in the Oracle connector. Rubén has started the arduous journey of getting his library ready for Boost review.

I have been asked to be the review manager for this work, something I am happy to do as, whether or not the admission is ultimately accepted, I think the general principle of producing simple, self-contained libraries that meet a common requirement is to be encouraged.

If this library is successful, I would hope that others will rise to the challenge and provide native connectors for other common database systems.

HTTP Client

I mentioned in an earlier blog that an HTTP Client with a similar interface to Python’s Requests library woudl be worked on. As it happens, other priorities took over last year. This year I will be focussing efforts on getting this library in shape for a proof of concept.

I mean Big Four

Redis is ubiquitous in server environments. A quick search for Redis C++ clients turned up this little gem. I’d like to find time to give this a try at some point.

Drone CI

2021-01-15T00:00:00+00:00

Overview

A message currently appears (mid-January 2021) at the top of the travis-ci.org website.

“Please be aware travis-ci.org will be shutting down in several weeks, with all accounts migrating to travis-ci.com. Please stay tuned here for more information.”

The transition has not been a smooth one, with long, disruptive delays occurring on existing builds, and lack of clear communication from the company. Many were unaware of the impending change. Some informative posts about the topic are Travis CI’s new pricing plan threw a wrench in my open source works and Extremely poor official communication of the .org shutdown.

The C++ Alliance has decided to implement an in-house CI solution and make the new service available for Boost libraries also.

Selection Process

The first step was choosing which software to use. There are truly a surprising number of alternatives. An extensive review was conducted, including many Continuous Integration services from awesome-ci which lists more than 50. Coincidentally, Rene Rivera had recently done an analysis as well for ci_playground, and his example config files eventually became the basis for our new config files.

The top choices:
Appveyor
Azure Pipelines
BuildBot
CircleCI
CirrusCI
Drone
Github Actions
Semaphore
Shippable
TeamCity

From this list, Appveyor and Drone seemed the most promising to start with. Both allow for 100% self-hosting.

Appveyor

The appeal of Appveyor is that the config files are basic yaml, and everything runs in a Docker container. It sounds perfect. However, after experimentation there were a few issues.

Appveyor was originally designed for Microsoft Windows, with .NET and Powershell being key ingredients. While it can run on Linux, it’s not a native Linux application. Most of the CI testing done by Boost and CPPAlliance runs on Linux.
Specifically, the Docker experience on Windows is not nearly as smooth as Linux. I encountered numerous complexities when setting up Appveyor Windows Docker containers.
Bugs in their app.

Due to a combination of those reasons, Appveyor was not the best choice for this project.

Drone

Within the first day or two experimenting with Drone, it became clear that this was an excellent CI framework:

Simple, usable UI
Easy installation
Linux and Docker native
Small number of processes to run
Integrates with PostgreSQL, MySQL, Amazon S3
Autoscale the agents
Badges on github repos

The main drawback with Drone is the absence of “matrix” builds. Their alternative for matrices is jsonnet or starlark, which are flexible scripting languages. I was apprehensive about this point, thinking that the end-user would prefer simple yaml files - exactly like Travis. And, in fact, that is probably the case. Basic yaml files are easier to understand. However, on balance, this was the only noticable problem with Drone, and everything else seemed to be in order. The resulting starlark files do have a matrix-like configuration where each job can be customized.

To discuss Starlark for a moment - it’s a subset of python, and therefore easy to learn. Why would I use Starlark instead of just Python? “Sandboxing. The primary reason this was written is for the “hermetic execution” feature of Starlark. Python is notoriously difficult to sandbox and there didn’t appear to be any sandboxing solutions that could run within Python to run Python or Python-like code. While Starlark isn’t exactly Python it is very very close to it. You can think of this as a secure way to run very simplistic Python functions. Note that this library itself doesn’t really provide any security guarantees and your program may crash while using it (PRs welcome). Starlark itself is providing the security guarantees.”

Running a drone server

Create a script startdrone.sh with these contents:

#!/bin/bash
  
docker run \
  --volume=/var/lib/drone:/data \
  --env=DRONE_GITHUB_CLIENT_ID= \
  --env=DRONE_GITHUB_CLIENT_SECRET= \
  --env=DRONE_RPC_SECRET= \
  --env=DRONE_TLS_AUTOCERT= \
  --env=DRONE_SERVER_HOST= \
  --env=DRONE_SERVER_PROTO= \
  --env=DRONE_CONVERT_PLUGIN_ENDPOINT= \
  --env=DRONE_CONVERT_PLUGIN_SECRET= \
  --env=DRONE_HTTP_SSL_REDIRECT= \
  --env=DRONE_HTTP_SSL_TEMPORARY_REDIRECT= \
  --env=DRONE_S3_BUCKET= \
  --env=DRONE_LOGS_PRETTY= \
  --env=AWS_ACCESS_KEY_ID= \
  --env=AWS_SECRET_ACCESS_KEY= \
  --env=AWS_DEFAULT_REGION= \
  --env=AWS_REGION= \
  --env=DRONE_DATABASE_DRIVER= \
  --env=DRONE_DATABASE_DATASOURCE= \
  --env=DRONE_USER_CREATE= \
  --env=DRONE_REPOSITORY_FILTER= \
  --env=DRONE_GITHUB_SCOPE= \
  --publish=80:80 \
  --publish=443:443 \
  --restart=always \
  --detach=true \
  --name=drone \
  drone/drone:1

Fill in the variables. (Many of those are secure keys which shouldn’t be published on a public webpage.)
Then, run the script.

./startdrone.sh

Drone is up and running.

Next, the starlark plugin. Edit startstarlark.sh:

#!/bin/bash
  
docker run -d \
  --volume=/var/lib/starlark:/data \
  --env= \
  --publish= \
  --env=DRONE_DEBUG= \
  --env=DRONE_SECRET= \
  --restart=always \
  --name=starlark drone/drone-convert-starlark

and run it:

./startstarlark.sh

Starlark is up and running. Finally, the autoscaler.

#!/bin/bash
  
docker run -d \
  -v /var/lib/autoscaler:/data \
  -e DRONE_POOL_MIN= \
  -e DRONE_POOL_MAX= \
  -e DRONE_SERVER_PROTO= \
  -e DRONE_SERVER_HOST= \
  -e DRONE_SERVER_TOKEN= \
  -e DRONE_AGENT_TOKEN= \
  -e DRONE_AMAZON_REGION= \
  -e DRONE_AMAZON_SUBNET_ID= \
  -e DRONE_AMAZON_SECURITY_GROUP= \
  -e AWS_ACCESS_KEY_ID= \
  -e AWS_SECRET_ACCESS_KEY= \
  -e DRONE_CAPACITY_BUFFER= \
  -e DRONE_REAPER_INTERVAL= \
  -e DRONE_REAPER_ENABLED= \
  -e DRONE_ENABLE_REAPER= \
  -e DRONE_AMAZON_INSTANCE= \
  -e DRONE_AMAZON_VOLUME_TYPE= \
  -e DRONE_AMAZON_VOLUME_IOPS= \
  -p  \
  --restart=always \
  --name=autoscaler \
  drone/autoscaler

Start the autoscaler.

./startautoscaler.sh

Windows autoscaler is still experimental. For now, both Windows and Mac servers have been installed manually and will be scaled individually. Because they are less common operating systems, with most boost builds running in Linux, the CPU load on these other machines is not as significant.

Configs

The real complexities appear when composing the config files for each repository. After manually porting .travis.yml for https://github.com/boostorg/beast and https://github.com/boostorg/json, the next step was creating a Python script which automates the entire process.

Drone Converter

A copy of the script can be viewed at https://github.com/CPPAlliance/droneconverter-demo

The idea is to be able to go into any directory with a .travis.yml file, and migrate to Drone by executing a single command:

cd boostorg/accumulators
droneconverter

The converter ingests a source file, parses it with PyYAML, and dumps the output in Jinja2 templates. The method to write the script was by beginning with any library, such as boostorg/array, and just get that one working. Then, move on to others, boostorg/assign, boostorg/bind, etc. Each library contains a mix of travis jobs which are both similar and different to the previously translated libraries. Thus, each new travis file presents a new puzzle to solve, but hopefully in a generalized way that will also work for all repositories.

Versions of clang ranging from 3.3 to 11 are targeted in the tests. While later releases such as clang-6 or clang-7 usually build right away without errors, the earlier versions in the 3.x series were failing to build for a variety of reasons. First of all, those versions are not available on Ubuntu 16.04 and later, which means being stuck on Ubuntu 14.04, preventing an across-the-board upgrade to a newer Ubuntu. Then, if a standard “base” clang is simultaneously installed, such as clang-7 or 9, this seems to cause other dependent packages and libraries to be installed, which conflict with clang-3. The solution was to figure out what travis does, and copy it. Travis images have downloaded and installed clang-7 into a completely separate directory, not using the ordinary system packages. Then the extra directory /usr/local/clang-7.0.0/bin has been added to the $PATH.

Some .travis.yml files have a “matrix” section. Others have “jobs”. Some .travis files place all the logic in one main “install” script. Others refer to a variety of script sections including “install”, “script”, “before_install”, “before_script”, “after_success”, which may or may not be shared between the 20 jobs contained in the file. Some job in travis were moving (with the mv command) their source directory, which is baffling and not usually permitted in Jenkins or Drone. This must be converted to a copy command instead. “travis_wait” and “travis_retry” must be deleted, they are travis-specific. Many travis variables such as TRAVIS_OS_NAME or TRAVIS_BRANCH need to be set and/or replaced in the resulting scripts. Environment variables in the travis file might be presented as a list, or a string, without quotes, or with single quotes, or double quotes, or single quotes embedded in double quotes, or double quotes embedded in single quotes, or missing entirely and included as a global env section at the top of the file. The CXX variable, which determines the compiler used by boost build, isn’t always apparent and could be derived from a variety of sources, including the COMPILER variable or the list of packages. The list of these types of conversion fixes goes on and on, you can see them in the droneconverter program.

Apple Macs Developer Tools depends on Xcode, with an array of possible Xcode version numbers from 6 to 12, and counting. Catalina only runs 10 and above. High Sierra will run 7-9. None of these will operate without accepting the license agreement, however the license agreement might reset if switching to a newer version of Xcode. The presence of “Command Line Tools” seems to break some builds during testing, however everything works if “Command Line Tools” is missing and the build directly accesses Xcode in the /Applications/Xcode-11.7.app/Contents/Developer directory. On the other hand, the package manager “brew” needs “Command Line Tools” (on High Sierra) or it can’t install packages. A solution which appears to be sufficiently effective is to “mv CommandLineTools CommandLineTools.bck”, or the reverse, when needed.

Drone will not start on a Mac during reboot unless the Drone user has a window console sessions running too. So, Apple is not an ideal command-line-only remote server environment.

Drone Deployments

Pull requests with the new drone configs were rolled out to all those boost repositories with 100% travis build success rate and 100% drone build success rate, which accounts for about half of boost libraries. The other half of boost libraries are either missing a .travis.yml file, or they have a couple of travis/drone jobs which are failing. These should be addressed individually, even if only to post details about it in the pull requests. Ideally, attention should be focused on these failing tests, one by one, until the jobs attain a 100% build success rate and the badge displays green instead of red. The long tail distribution of edge-cases require individual attention and rewritten tests.

Github Actions

The droneconverter script is being enhanced to also generate Github Actions config files. A first draft, tested on a few boost libraries, is building Linux jobs successfully. Ongoing.

Richard’s New Year Update - Reusable HTTP Connections

2021-01-01T00:00:00+00:00

Reusable HTTP(S) Connections

Something I am often asked by users of Boost Beast is how to code a client which effectively re-uses a pool of HTTP connections, in the same way a web browser does.

The premise is straightforward - if our client is going to be making multiple calls to a web server (or several of them) then it makes sense that once a connection has been used for one request, it is returned to a connection pool so that a subsequent request can make use of it.

It also makes sense to have a limit on the number of concurrent connections that can be open against any one host. Otherwise, if the client needs to make multiple requests at the same time, it will end up creating new connections in parallel and lose the efficiency of re-using an existing connection.

From these requirements, we can start to think about a design.

Firstly, we can imagine a connection cache, with connections kept in a map keyed on host + scheme + port (we can’t re-use an HTTP port for an HTTPS request!).

When a request needs a connection, it will either create a new one (connection limit per host not met) or will wait for an existing connection to become available (which implies a condition variable).

Once a request has a connection to use, it will send the HTTP request and wait for the response.

At this stage, there is a possibility that the active connection which has been allocated could have been idle since the last time it was used. In TCP there is no way to atomically check whether the remote host has closed the connection (or died). The only way to know is to actually read from the socket with a timeout. If the remote host has shutdown the socket, we will be notified as soon as the RST frame arrives at our host. If the remote host has stopped working or the network is bad, we’ll be notified by the timeout.

Thus if our read operation results in an error and we have inherited the connection from the cache, we ought to re-try by reopening the connection to the remote host and repeating the write/read operations. However, if there is an error reported on the subsequent attempt, then we can conclude that this is a legitimate error to be reported back to the caller.

In simplified pesudocode, the operation might look something like this (assuming we report transport errors as exceptions):

response 
read_write(connection& conn)
{
  response resp;
  
  auto retry = false;
   
  if(conn.is_initialised())
    retry = true;
  else
    conn.connect(...);
  
  for(;;)
  {
    conn.write(request);
    auto err = conn.read(resp);
    if (err)
    {
      if(!std::exchange(retry, false))
        throw system_error(err);
      request.clear();
      conn.disconnect();
    }
    else
      break;
  }

  return resp;
}

Structuring the Code

General Principles

In my previous blog I mentioned my preference for writing the implementation of a class in such a way that it does not need to take care of its own lifetime or mutual exclusion. These concerns are deferred to a wrapper or handle. Methods on the handle class take care of marshalling the call to the correct executor (or thread) and preserving the implementation’s lifetime. The implementation need only concern itself with the logic of handling the request.

Here’s an example about which I’ll go into more detail later:

net::awaitable< response_type >
connection_cache::call(beast::http::verb   method,
                       const std::string & url,
                       std::string         data,
                       beast::http::fields headers,
                       request_options     options)
{
    // DRY - define an operation that performs the inner call.
    auto op = [&] {
        return impl_->call(method,
                           url,
                           std::move(data),
                           std::move(headers),
                           std::move(options));
    };

    // deduce the current executor
    auto my_executor = co_await net::this_coro::executor;

    // either call directly or via a spawned coroutine
    co_return impl_->get_executor() != my_executor
        ? co_await op()
        : co_await net::co_spawn(impl_->get_executor(), op, net::use_awaitable);
}

Coroutines

In this implementation I will be providing a C++20 coroutine interface over Asio executors once again. I am using coroutines because they are easier to write when compared to Asio’s composed operations, but are fundamentally the same thing in a prettier package.

Mutual Exclusion

For mutual exclusion I will be embedded an asio strand into each active object. The advantage of doing so means that no thread of execution is ever blocked which means we can limit the number of threads in the program to the number of free CPUs giving us maximum throughput of work. In reality of course, one thread is more than enough computing power for almost all asynchronous programs. It’s therefore better to think in terms of one executor per program component, with the implicit guarantee that a given executor will only perform work on one thread at a time. Thinking this way allows us to write code in a way that is agnostic of whether the final program is compiled to be single or multi-threaded.

But What About Single-threaded Programs?

In order that I don’t need to rewrite code when should I decide to make a single-threaded program multi-threaded or vice versa, I have a couple of little utility functions and types defined in config.hpp

Specifically:

namespace net
{
using namespace asio;

using io_executor = io_context::executor_type;

#ifdef MULTI_THREADED

using io_strand = strand< io_executor >;

inline io_strand
new_strand(io_executor const &src);

inline io_strand
new_strand(io_strand const &src);

#else

using io_strand = io_context::executor_type;

inline io_strand
new_strand(io_executor const &src);

#endif

inline io_executor
to_io_executor(any_io_executor const &src);
}   // namespace net

Any object type in the program which would require its own strand in a multi-threaded program will simply use the type io_strand whether the program is compiled for single-threaded operation or not. Any code that would notionally need to construct a new strand simply calls new_strand(e) where e is either a strand or a naked executor.

Any code that needs access to the notional underlying executor would call to_io_executor(e).

Determinism

Since we’re using asio’s executors for scheduling, it means that we can use asio’s timer as a deterministic, ordered asynchronous condition variable, which means that requests waiting on the connection pool will be offered free connections in the order that they were requested. This guarantee is implicit in the way that the timers’ cancel_one() method is specified.

As we’ll see later, asio’s timers also make it trivial to implement an asynchronous semaphore. In this case we use one to ensure that requests are handled concurrently but no more than some upper limit at any one time.

Interface

I’m going to create a high-level concept called a connection_cache. The interface will be something like this:

struct connection_cache
{
  using response_ptr = std::unique_ptr< response >;
  
  net::awaitable< response_ptr >
  rest_call(
    verb method, 
    std::string const& url, 
    std::optional<std::string> data = std::nullopt,
    headers hdrs = {},
    options opts = {});
};

There are a few things to note here.

The return type of the coroutine is a unique_ptr to a response. A natural question might be whether the response should simply be returned by value. However, in practice I have found that there are a number of practical reasons why it’s often better to return the response as a pointer. Firstly it allows conversion to a shared_ptr<const response> in environments where the response might be passed through a directed acyclic graph. Secondly, would allow a further enhancement in that having finished with the response, the client could post it back to the cache, meaning that it could be cached for re-use.
The only two required arguments are the method and url. All others can be defaulted.
An optional string may be passed which contains the payload of a POST request. This is passed by value because, as we’ll see later,the implementation will want to move this into the request object prior to initiating communications. I have chosen a string type for two reasons
- text in the form of JSON or XML is the most common form of messaging in REST calls.
- strings can contain binary data with no loss of efficiency.
hdrs is simply a list of headers which should be set in the request. Again, these are passed by value as they will be moved into the request object.
The last parameter of the call is an as-yet undefined type called options. This will allow us to add niceties like timeout arguments, a stop_token, redirect policies, a reference to a cookie store and so on.

When called, rest_call will attempt to reuse an existing connection. If a connection is not available, it will create a new one if we are under the connection threshold for the deduced host and if not, it will wait until a connection is available.

Furthermore, the number of concurrent requests will be throttled to some upper limit.

Transport failures will be reported as an exception (of type system_error) and a successful response (even if a 400 or 500) will be returned in the response_ptr. That is to say, as long as the transport layer works out, the code will take the non-exceptional path.

Implementation details

URL Parsing

Among the things I am often asked about in the Beast slack channel and in the Issue Tracker is why there is no URL support in Beast. The is that Beast is a reasonably low level library that concerns itself with the HTTP (and WebSocket) protocols, plus as much buffer and stream management as is necessary to implement HTTP over Asio. The concept of a URL the subject of its own RFCs and is a higher level concern. The C++ Alliance is working on Boost.URL but it is not ready for publishing yet. In the meantime, I found a nifty regex on the internet that more-or-less suffices for our needs:

    std::tuple< connection_key, std::string >
    parse_url(std::string const &url)
    {
        static const auto url_regex = std::regex(
            R"regex((http|https)://([^/ :]+):?([^/ ]*)((/?[^ #?]*)\x3f?([^ #]*)#?([^ ]*)))regex",
            std::regex_constants::icase | std::regex_constants::optimize);
        auto match = std::smatch();
        if (not std::regex_match(url, match, url_regex))
            throw system_error(net::error::invalid_argument, "invalid url");

        auto &protocol = match[1];
        auto &host     = match[2];
        auto &port_ind = match[3];
        auto &target   = match[4];
        /*
        auto &path     = match[5];
        auto &query    = match[6];
        auto &fragment = match[7];
        */
        return std::make_tuple(
            connection_key { .hostname = host.str(),
                             .port     = deduce_port(protocol, port_ind),
                             .scheme   = deduce_scheme(protocol, port_ind) },
            target.str());
    }

Exceptions in Asynchronous Code Considered Harmful

I hate to admit this, because I am a huge fan of propagating errors as exceptions. This is because the combination of RAII and exception behaviour handling makes error handling very slick in C++. However, coroutines have two rather unpleasant limitations:

You can’t call a coroutine in a destructor.
You can’t call a coroutine in an exception handling block.

There are workarounds. Consider:

my_component::~my_component()
{
  // destructor
  net::co_spawn(get_executor(), 
  [impl = impl_]()->net::awaitable<void>
  {
    co_await impl->shutdown();
    // destructor of *impl happens here
  }, net::detached);
}

This is the destructor of a wrapper object that contains a shared_ptr to its implementation, impl_. In this case we can detect the destruction of my_component and use this to spawn a new coroutine of indeterminate lifetime that takes care of shutting down the actual implementation and then destroying it.

This solves the problem of RAII but it mandates that we must author objects that will be used in coroutines as a handle-body pair.

We can similarly get around the “no coroutine calls in exception handlers” limitation if we’re prepared to stomach code like this:

net::awaitable<void> 
my_coro()
{
  std::function<net::awaitable(void)> on_error;
  try {
    co_await something();
  }
  catch(...)
  {
    // set up error_handler
    on_error = [ep = std::current_exception] {
      return handler_error_coro(ep);
    };
  }
  // perhaps handle the error here
  if (on_error)
      co_await on_error();
}

I think you’ll agree that this is a revolting solution to an unforgivable omission in the language. Not only is it untidy, confusing, difficult to teach and error-prone, it also turns exception handling into same fiasco that is enforced checking of return codes.

To add insult to injury, the error handling code in this function takes up 5 times as many lines as the logic!

Therefore my recommendation is that in asynchronous coroutine code, it’s better to avoid exceptions and have coroutines either return a tuple of (error_code, possible_value) or a variant containing error-or-value.

For example, here is some code from the connection_impl in my example project:

net::awaitable< std::tuple< error_code, response_type > >
connection_impl::rest_call(request_class const &  request,
                           request_options const &options)
{
    auto response = std::make_unique< response_class >();

    auto ec = stream_.is_open()
                  ? co_await rest_call(request, *response, options)
                  : net::error::basic_errors::not_connected;

    if (ec && ec != net::error::operation_aborted)
    {
        ec = co_await connect(options.stop);
        if (!ec)
            ec = co_await rest_call(request, *response, options);
    }

    if (ec || response->need_eof())
        stream_.close();

    co_return std::make_tuple(ec, std::move(response));
}

Ensuring that a Coroutine Executes on the Correct Executor

In playing with asio coroutines I stumbled upon something that has become an idiom.

Consider the situation where a connection class is implemented in terms of a handle and body. The body contains its own executor. In a multi-threaded build, this executor will be a strand while in a single threaded-build we would want it to be simply an io_context::executor_type since there will be no need for any of the thread guards implicit in a strand.

Now consider that the implementation has a member coroutine called (say) call. There are two scenarios in which this member will be called. The first is where the caller is executing in the same executor that is associated with the implementation, the second is where the caller is in its own different executor. In the latter case, we must post or spawn the execution of the coroutine onto the implementation’s executor in order to ensure that it runs in the correct sequence with respect to other coroutines initiated against it.

The idiom that occurred to me originally was to recursively spawn a coroutine to ensure the call happened on the correct executor:

net::awaitable< response_type >
connection_cache::call(beast::http::verb   method,
                       const std::string & url,
                       std::string         data,
                       beast::http::fields headers,
                       request_options     options)
{
    auto my_executor = co_await net::this_coro::executor;

    if (impl_->get_executor() == my_executor)
    {
        co_return co_await impl_->call(method,
                                       url,
                                       std::move(data),
                                       std::move(headers),
                                       std::move(options));
    }
    else
    {
        // spawn a coroutine which recurses on the correct executor.
        // wait for this coroutine to finish
        co_return co_await net::co_spawn(
            impl_->get_executor(),
            [&]() -> net::awaitable< response_type > {
                return call(method,
                            url,
                            std::move(data),
                            std::move(headers),
                            std::move(options));
            },
            net::use_awaitable);
    }
}

However, this does have the drawback that a code analyser might see the possibility of infinite recursion.

After discussing this with Chris, Asio’s author, a better solution was found:

net::awaitable< response_type >
connection_cache::call(beast::http::verb   method,
                       const std::string & url,
                       std::string         data,
                       beast::http::fields headers,
                       request_options     options)
{
    // DRY - define an operation that performs the inner call.
    auto op = [&] {
        return impl_->call(method,
                           url,
                           std::move(data),
                           std::move(headers),
                           std::move(options));
    };

    // deduce the current executor
    auto my_executor = co_await net::this_coro::executor;

    // either call directly or via a spawned coroutine
    co_return impl_->get_executor() != my_executor
        ? co_await op()
        : co_await net::co_spawn(impl_->get_executor(), op, net::use_awaitable);
}

There are a few things to note:

All lifetimes are correct even though the op takes arguments by reference. This is because whichever path we take, our outer coroutine will suspend on the call to op.
Note that in the slow path, op is captured by value in the call to co_spawn. Had we written: : co_await net::co_spawn(impl_->get_executor(), op(), net::use_awaitable); then op would have been called during the setup of the call to co_spawn, which would result in impl_->call(...) being called on the wrong executor/thread.

Talk is Cheap

TL;DR. Enough talk, where’s the code?

It’s on github at https://github.com/madmongo1/blog-new-year-2021.

Final Notes.

Before signing off I just wanted to cover a few of the features I have completed in this example and a few that I have not.

What’s Done

There is a limit on the number of concurrent connections to a single host. Host here is defined as a tuple of the transport scheme (i.e. http or https), port and hostname. This is currently defaulted to two, but would be trivial to change.

There is a limit on the number of concurrent requests across all hosts. This defaults to 1000. Simultaneous requests numbering more than this will be processed through what is an asynchronous semaphore, implemented like this:

  while (request_count_ >= max_concurrent_requests_)
  {
      error_code ec;
      co_await concurrent_requests_available_.async_wait(
          net::redirect_error(net::use_awaitable, ec));
  }

  ++request_count_;

  ...
  ... request takes place here
  ...
  
  if (--request_count_ < max_concurrent_requests_)
  {
      concurrent_requests_available_.cancel_one();
  }

What’s Not Done

HTTP 300 Redirect handling. I consider this to be a higher level concern than connection caching.
LRU Connection recycling. At the moment, the program will allow the number of connections to grow without limit if an unlimited number of different hosts are contacted. In a production system we would need to add more active state to each connection and have some logic to destroy old unused connections to make way for new ones.
The stop_token is not executor-aware. I have left the stop_token in for exposition but it should not be activated by a different executor to the one where the connection to it has been created at present. If you’re interested to see how this will look when complete, please submit an issue against the github repo and I’ll update the code and add a demonstration of it.
A Cookie Jar and HTTP session management. Again, these are higher level concerns. The next layer up would take care of these plus redirect handling.
The CMakeLists project in the repo has been tested with GCC 10.2 and Clang-11 on Fedora Linux. Microsoft developers may need to make the odd tweak to get things working. I’m more than happy to accept PRs.
Setting up of the ssl::context to check peer certificates. Doing this properly is complex enough to warrant another blog in its own right.

Thanks for reading. I hope you’ve found blog useful. Please by all means get in touch by:

raising an issue
Contacting me in the #beast channel of CppLang Slack
email hodges.r@gmail.com

Richard’s November/December Update

2020-12-22T00:00:00+00:00

A Coroutine Websocket Using Boost Beast

This month I thought I would present a little idea that I had a few months ago.

Boost.Beast is a very comprehensive and competent websocket implementation, but it is not what you might call “straightforward” to use unless you are already wise in the ways of Asio.

Beast’s documentation and design makes no apology for this. There is a disclaimer in the documentation:

To use Beast effectively, a prior understanding of Networking is required.

This is worth taking seriously (and if you are not fully aware of how Asio works with respect to the posting of completion handlers onto the associated executor, this page is worth studying).

The Interface

I wanted to model my websocket object’s interface roughly on the javascript websocket connection interface. There will be a few differences of course, because the javascript version uses callbacks (or continuations) and I will be using C++ coroutines that execute on an Asio executor. In underlying implementation, these concepts are not actually that far apart, since Asio awaitables are actually implemented in terms of the normal asio completion token/handler interaction.

Furthermore, I want my WebSocket’s connection phase to be cancellable.

My websocket interface will look something like this:

namespace websocket
{
    struct message 
    {
        bool is_binary() const;
        bool is_text() const;
        std::string_view operator*() const;
    }; 
    
    struct event 
    {
        bool is_error() const;
        bool is_message() const;
        error_code const& error() const; 
        message const& message() const;
    };
    
    struct connection
    {
        /// Schedule a frame to be sent at some point in the near future
        void 
        send(std::string_view data, bool as_text = true);

        /// Suspend and wait until either there is a message available or an error        
        net::awaitable<event> 
        consume();
        
        /// Close the websocket and suspend until it is closed.
        net::awaitable<void> 
        close(beast::websocket::close_reason = /* a sensible default */); 

        /// Send the close frame to the server but don't hang around to wait
        /// for the confirmation.
        void 
        drop(beast::websocket::close_reason = /* a sensible default */);

        /// If consume() exits with an error of beast::websocket::error::closed then this
        /// will return the reason for the closure as sent by the server.
        /// Otherwise undefined.        
        beast::websocket::close_reason
        reason() const;
    };
    
    net::awaitable<connection> 
    connect(std::string url, 
            connect_options options /* = some default */);
}

The idea here is to keep the interface as lightweight and as simple as possible. The websocket connection will run on the executor of the coroutine that created it. Any commands sent to the websocket will be executor safe. That is, internally their work will be dispatched to the websocket connection’s executor. The exception to this will be the close_reason method, which must only be called once the connection’s consume coroutine has returned an error event. It is a guarantee that once consume returns an event that is an error, it will never return anything else, and no other method on the connection will mutate its internal state. In this condition, it is legal to call the reason method.

A typical use would look like this:

    // default options
    auto ws = co_await websocket::connect("wss://echo.websocket.org"); 

    for(;;)
    {
        auto event = co_await ws.consume();
        if (event.is_error())
            break;
        else
            on_message(std::move(event.message()));
    }

The above code example does not provide any means to write to the websocket. But it would be trivial to either spawn another coroutine to handle the writer, or call a function in order to signal some orthogonal process that the websocket was ready.

    // default options
    auto ws = co_await websocket::connect("wss://echo.websocket.org"); 
    
    on_connected(ws); // The websocket object should be shared-copyable

    for(;;)
    {
        auto event = co_await ws.consume();
        if (event.is_error()) {
            on_close();
            break;
        }
        else
            on_message(std::move(event.message()));
    }

Another way to visualise a websocket is exactly as javascript’s websocket connection does, using callbacks or continuations in order to notify user code that the websocket has received data or closed. It would be trivial to wrap our coroutine version in order to provide this functionality. We would need to spawn a coroutine in order to run the consume() loop and then somehow signal it to stop if the websocket was disposed of.

User code might then start to look something like this:

    websocket::connect("wss://echo.websocket.org", options)
        .on_connect([](websocket::connection ws)
        {
            run_my_loop(ws);
        });
        
void run_my_loop(websocket::connection ws)
{
    bool closed = false;
    ws.on_close([&]{ closed = true; });
    ws.on_error([&]{ closed = true; });
    ws.on_message([&](websocket::message msg){ process_message(msg); });

    // some time later
    ws.send("Hello, World!");
}

With this style of interface we would need some means of passing the executor on which the continuations would be invoked. A reasonable place to do this might be the options parameter.

In the JavaScript style interface, it would be important to be able to detect when the websocket has gone out of scope and ensure that it closes correctly, otherwise we’ll have a rogue resource out there with no handle by which we can close it. This argues that the actual websocket::connection class should be a handle to an internal implementation and that the destructor of the handle should ensure that the implementation is signalled so that it can drop the connection and shutdown cleanly. Under the covers, we’re implementing this websocket in Boost.Beast. As with all Asio objects, there could be (probably will be) asynchronous operations in progress at the time the websocket handle goes out of scope.

Thinking this through, it means that:

The implementation is going to live longer than the lifetime of the last copy of the handle owning the implementation.
There needs to be some mechanism to cancel the underlying implementation’s operations.

Coroutines can be visualised as threads of execution. In the world of threads (e.g. std::thread) we have primitives such as std::stop_token and std::condition_variable. The C++ Standard Library does not yet have these primitives for coroutines. And if it did it would be questionable whether they would be suitable for networking code where coroutines are actually built on top of Asio composed operations. Does Asio itself provide anything we can use?

Asio’s Hidden Asynchronous condition_variable

The answer is surprisingly, yes. But not in the form I was expecting when I asked Chris Kohlhoff (Asio’s author and maintainer) about it. It turns out that asio’s timer models an asynchronous version of a condition variable perfectly. Consider:

Given:

auto t = net::steady_timer(co_await net::this_coro::executor);
t.expires_at(std::chrono::stready_clock::time_point::max());

Then we can write:

template<class Pred>
net::awaitable<void>
wait(net::steady_timer& t, Pred predicate)
{
    error_code ec;
    while(!ec && !predicate())
    {
        co_await t.async_wait(net::redirect_error(net::use_awaitable, ec));
        if (ec == net::error::operation_aborted)
            // timer cancelled
            continue;
        else
            throw std::logic_error("unexpected error");
    } 
}

void
notify(net::steady_timer& t)
{
    // assuming we are executing on the same executor as the wait()
    t.cancel();
}

Which gives us a simple asynchronous condition_variable (this one does not implement timeouts, but it would be trivial to extend this code to accommodate them).

Asynchronous Stop Token

The std::stop_token is a welcome addition to the standard, but it is a little heavyweight for asynchronous code that runs in an executor, which is already thread-safe by design. A simple in-executor stop source can be implemented something like this:

namespace async {
namespace detail {
struct shared_state {
    void stop()
    {
        if (!std::exchange(stopped_, true))
        {
            auto sigs = std::move(signals_);
            signals_.clear();
            for(auto& s : sigs)
                s();
        }
    }
    
    std::list<std::function<void()>> signals_;
    bool stopped_ =  false;
};
}
struct stop_source {
    void stop() {
        impl_->stop();
    }
    std::shared_ptr<shared_state> impl_;
}

struct stop_connection
{
};

struct stop_token
{
    stop_token(stop_source& source)
    : impl_(source.impl_) {
    }
    
    bool 
    stopped() const { return !impl_ || impl_->stopped_; }
    
    stop_connection 
    connect(std::function<void()> slot);
     
    std::shared_ptr<shared_state> impl_;
}
}

The use case would look something like this:


net::awaitable<void>
something_with_websocket(async::stop_token token)
{
    // passing the stop token allows the connect call to abort early
    // if the owner of the stop_source wants to end the use of the
    // websocket before it is connected
    auto ws = websocket::connect("wss://echo.websocket.org", 
        websocket::connect_options { .stop_token = token });

    // connect a slot to the stop token which drops the connection        
    auto stopconn = token.connect([&] { ws.drop(); };
    
    for(;;} {
        auto event = co_await ws.consume();
        // ...etc
    }
    
}

Now, armed with both a stop_token and a condition_variable, we gain a great deal of flexibility with programs running on an Asio-style executor.

So let’s build a little chat app to talk the the echo bot.

Coding style when using Asio coroutines.

I mentioned earlier that I like to decompose objects with complex lifetimes into an impl and handle. My personal programming style for naming the components is as follows:

The implementation

This is the class that implements the complex functionality that we want. I generally give this class an _impl suffix and apply the following guidelines:

The impl does not control its own lifetime.
Calls to the impl are expected to already be executing on the correct thread or strand, and in the case of multi-threaded code, are expected to have already taken a lock on any mutex.

This is a personal preference which I find tends to lower the complexity of the object, since the interface functions do not have to manage more than one concern, and deadlocks etc are not possible.

The lifetime

When holding an object in shared_ptr, we get a chance to intercept the destruction of the last handle. At this point we do not have to destroy the implementation, but can allow it to shut down gracefully before destruction. In order to do this, particularly with an object that is referenced by internal coroutines, I have found that it’s useful to separate the public lifetime of the object, and its internal lifetime, which may be longer than the public one.

A convenient, if not especially efficient way to do this is to hold two shared_ptr’s in the handle. One being a shared_ptr which has a custom destructor - the lifetime ptr, and one being a normal shared_ptr to the implementation which can be copied in order to extend its private lifetime while it shuts down. It is the responsibility of the custom deleter to signal the implementation that it should start shutting down.

In this case, the websocket connection’s public handle may look something like this:

namespace websocket {

struct connection_lifetime
{
    connection_lifetime(std::shared_ptr<connection_impl>&& adopted)
    : impl_(std::move(adopted))
    , lifetime_(new_lifetime(impl_))
    {
    }
    
    static std::shared_ptr<void>
    new_lifetime(std::shared_ptr<connection_impl> const& impl)
    {
        static int useful_address;
        auto deleter = [impl](int*) noexcept
        {
            net::co_spawn(impl->get_executor(), 
                [impl]() -> net::awaitable<void>
                { 
                    co_await impl->stop();
                }, net::detached);
        };
        
        return std::shared_ptr<void>(&useful_address, deleter);
    }
    
    std::shared_ptr<connection_impl> impl_;
    std::shared_ptr<void> lifetime_;
};

struct connection
{
    connection(connection_lifetime l);
};
}

The interesting part here is in the function new_lifetime. There are a few things going on here. First, we are capturing the internal lifetime of our connection_impl and storing it in the deleter of the lifetime pointer. This of course means that the private implementation will live at least as long as the public lifetime. Secondly, the deleter does not actually delete anything. It merely captures a copy of the impl pointer and runs a coroutine on the impl to completion before releasing the impl pointer. The idea is that this coroutine will not complete until all internal coroutines within the implementation have completed. The provides the fortunate side effect that operations running inside the impl do not have to capture the impl’s lifetime via shared_from_this. It turns out that this aids composability, since subordinate coroutines within the implementation can be written as free functions, and ported to other implementations that may not involve a shared_ptr. It also means that the impl itself can be composed, since it has no restrictions on lifetime semantics. i.e. If I wanted to implement a JSON-RPC connection by deriving from the websocket::connection_impl, I do not have to be concerned about translating shared_ptrs internally in the derived class.

Once it’s all put together

Finally, having created all the primitives (which I really should start collating into a library), we can test our little websocket chat client, which becomes a very simple program:

Here’s main:

int
main()
{
    net::io_context ioctx;

    net::co_spawn(
        ioctx.get_executor(), [] { return chat(); }, net::detached);

    ioctx.run();
}

And here’s the chat() coroutine:

net::awaitable< void >
chat()
{
    // connect the websocket
    auto ws = co_await websocket::connect("wss://echo.websocket.org");

    // spawn the coroutine to read console input and send it to the websocket
    auto stop_children = async::stop_source();
    net::co_spawn(
        co_await net::this_coro::executor,
        [stop = async::stop_token(stop_children), ws]() mutable {
            return do_console(std::move(stop), std::move(ws));
        },
        net::detached);

    // read events from the websocket connection.
    for (;;)
    {
        auto event = co_await ws.consume();
        if (event.is_error())
        {
            if (event.error() == beast::websocket::error::closed)
                std::cerr << "peer closed connection: " << ws.reason()
                          << std::endl;
            else
                std::cerr << "connection error: " << event.error() << std::endl;
            break;
        }
        else
        {
            std::cout << "message received: " << event.message() << std::endl;
        }
    }
    
    // at this point, the stop_source goes out of scope, 
    // which will cause the console coroutine to exit.
}

And finally, the do_console() coroutine. Note that I have used asio’s posix interface to collect console input. In order to run compile in a WIN32 environment, we’d need to do something different (suggestions welcome via PR!).

net::awaitable< void >
do_console(async::stop_token stop, websocket::connection ws)
try
{
    auto console = asio::posix::stream_descriptor(
        co_await net::this_coro::executor, ::dup(STDIN_FILENO));
    auto stopconn = stop.connect([&] { console.cancel(); });

    std::string console_chars;
    while (!stop.stopped())
    {
        auto line_len =
            co_await net::async_read_until(console,
                                           net::dynamic_buffer(console_chars),
                                           '\n',
                                           net::use_awaitable);
        auto line = console_chars.substr(0, line_len - 1);
        console_chars.erase(0, line_len);
        std::cout << "you typed this: " << line << std::endl;
        ws.send(line);
    }
}
catch(...) {
  // error handling here
}

If you’d like to look into the complete code, submit a PR or offer some (probably well-deserved) criticism, you will find the code repository here.