Anonymous functions, or lambdas, were introduced in C++11 as a convenient, lightweight syntax for creating one-off functions. What excited me most about this development was that I could now compile a functional language into C++ by constructing a more faithful embedding of higher-order functions, without requiring me to deal with issues like closures and lifting. It wasn’t perfect though, and there were a number of warts that made lambdas unusable for anything other than the simplest cases.

In this post I’m going to motivate C++14’s initialized lambda capture as a solution to one of the numerous problems hitting one of my most common use-cases: nested lambdas.

Take #1: Pass by Value, Capture by Value

Consider a simple haskell-esque function:

add :: Int -> Int -> Int
add = \x -> \y -> x + y

This translates into a correspondingly simple C++11 lambda as follows:

auto add = [] (int x) {
    return [x] (int y) {
        return x + y;
    }
}

On the face of it, this doesn’t look so bad – an outer lambda which takes an argument and returns an inner lambda which takes its own argument and while capturing the outer argument. This function can be called as add(5)(2), in the usual curried form.

On closer inspection, things can get less than pretty. Specifically, see if you can spot the number of times the value x is actually copied before the addition is performed.

Since add takes x by value, that’s potentially where the first copy happens. x is then copied again, when it is captured in the inner lambda. That’s two copies, when at worst we should only have one (the outer one), and at best none at all.

There are several methods to eliminate some or all of these copies, methods that are available in C++11. They all have some drawbacks, making you forfeit either convenience, semantics or safety. Let’s take a look at a few of these.

Take #2: Pass by Value, Capture by Reference

The obvious way to eliminate the capture copy is to capture by reference, instead of by value. After all, that’s what it’s there for. This leads to issues however, because we’re allowing a reference to a stack-allocated value escape its scope. Here’s the same example, capturing x by reference:

auto add = [] (int x) {
    return [&x] (int y) {
        return x + y;
    }
}

While x is no longer copied at the capture point, this opens us up to a whole host of unforeseen behaviors. Since x is stack allocated, it technically doesn’t exist after add returns. This means that any piece of code that can conceivably call the inner lambda is already dealing with a reference to a memory location that doesn’t necessarily contain the value it did when the lambda was created. At best you’ll get the wrong answer immediately; at worst you’ll get it sometime later where you can no longer correlate the error and the cause.

Take #3: Pass by Reference, Capture by Reference

What if x were guaranteed to exist after add returned? This would solve the inner lambda’s problems, but the only way to accomplish this to force add to accept its own argument by reference.

auto add = [] (int& x) {
    return [&x] (int y) {
        return x + y;
    }
}

Combined with reference capture, this eliminates both copies. This is not however without its own problems. The semantic issue from the previous solution still exists, since we are still capturing in the inner lambda by reference.

A more obvious problem is that we can no longer call add with r-value arguments, such as temporaries and literals. For example, we can’t call add(5)(2), as 5 isn’t an l-value, and cannot therefore be passed by reference. We’re forced to do one of two things: declare the argument ahead of time (int x = 5; add(x)(2)), or use constant references.

Take #4: Pass by Constant Reference, Capture by Reference

Accepting an argument by constant reference allows us to pass in both l-values and r-values – the compiler extends the lifetime of an r-value by just long enough that things work out. Our add function now looks like:

auto add = [] (const int& x) {
    return [&x] (int y) {
        return x + y;
    }
}

Which is great – no copies anywhere, and we can call the function however we’d like. Semantic issues still abound, but apart from the fact that the program might not do what we want it to do, we’re fine.

But we’re not done yet.

Suppose we wanted to change the definition of add just a little bit – increment x before adding it. This is a contrived example, but it’s not that difficult to imagine a function where you’d like to modify your arguments.

auto increment_add = [] (const int& x) {
    ++x;
    return [&x] (int y) {
        return x + y;
    }
}

But this can’t work! We’ve already declared x to be passed in by constant reference, which means we can’t change it. What we really want is for x to be passed in by value. At this point, we realize that we’re precisely back at square one, and throw up our hands.

At least that’s how it was before initialized lambda capture in C++14.

Take #5: Pass by Value, Capture by Move

Initialized lambda capture gives us access to the [id = expression] syntax in the capture specifier list, allowing us to initialize the captured variables however we want.

The utility in our recurrent example is in the realization that x isn’t used in the outer function after the return statement (for obvious reasons), so it can technically be moved into the returned lambda.

auto add = [] (int x) {
    ++x;
    return [x = std::move(x)] (int y) {
        return x + y;
    }
}

By moving x into the lambda, we prevent the capture copy, while permitting x to be modified in the outer function (and indeed, in the inner function as well). Additionally, by accepting x by value in the outer function, we permit the compiler to infer moves there as well. This will result in exactly as many copies as are necessary in order to get the desired behaviour.

This technique exhibits varying degrees of success for different data-types; primitive types such as int and float arguably don’t benefit that much, since a move is about as expensive as a copy. More complex types with heap-allocated resources such as std::vector might take arbitrarily long to copy, making a move quite appealing.

Lastly, for types which can only be moved and not copied – such as std::unique_ptr, this is the only way to capture them.

To kick-start the new site, I’ve resurrected one of the more popular previous entries, albeit touched up, and at a new location. I apologize for all the broken links.

This article was originally published on November 10, 2009.

Terminal capabilities have some of the sparsest documentation I have ever had the misfortune of having to dig through. Popular terminal applications such as vim, emacs, mutt, etc. seem to make use of every last bit of a terminal’s flexibility, but exactly how they do that always appears to be a mystery.

Among these capabilities are those to display a 256 color pallette.

Most terminals can handle at least 16 colors (8, and a bold version of each), and the ANSI codes necessary to invoke them are already a mess. Getting a terminal to display 256 colors requires character gymnastics that are altogether incomprehensible.

After hours of poring over non-existent documentation stretching back to a time when color actually meant ink, I think I’ve identified a pattern. In this entry, I’ll be giving you a run-down on what a 256 color escape seqence looks like, and how you can use them¹.

Terminal Basics

The primary function of a terminal is to display text. Terminals of yore were serial terminals – text came in as a stream, and the terminal had one pass to do whatever it needed to do. While this limitation probably derived from a lack of resources to store more than a small buffer, most of the terminals we use today are still built in this way, so that’s what we’re stuck with.

Terminals use a pattern of delimiters to mark-up regions of text with attributes, the same way an HTML document uses start and end tags to mark-up text in a document. However terminals usually can’t express attributes as rich as HTML, nor can they afford to include a sufficiently powerful parser to do so. The delimiters for terminal attributes are therefore extremely simple, and are known as escape-codes.

Every escape code is a sequence of characters which tell the terminal to do something special. The most common escape code is one you might be familiar with: \n. This tells the terminal to print a newline character instead of the literal characters \ followed by n.

For more complex text attributes such as foreground color, bold-ness, etc, the terminal keeps a small flag to remember which state it’s in. When it sees the turn-on-bold escape code, it sets that flag, and continues to display bold text until it’s given an escape code that tells it otherwise.

For the standard 8-color palette, these codes are fairly well documented. However, finding the codes for the 256 color palette, and furthermore the syntax in which they may be used, has driven me almost to distraction. It would seem that although they’ve managed to increase the number of colors, they haven’t added any more documentation in that direction.

Special Effects

There are a variety of effects that you can apply to text, but not all of them are supported on every terminal/emulator. The following are the most common:

• Reset (Clear all text attributes, including color, effects, etc.)
• Bold
• Italic
• Underline
• Blink
• Reverse (Swap foreground and background)

Colors are themselves special effects, but they need a different code, depending on whether they are to be applied on the foreground or the background.

Anatomy of a Color Code

The general structure of a 256 color code is:

code       :: ^[[(value)m
value      :: (attributes);(foreground);(background)
attributes :: attribute;attributes
attribute  :: 00|01|03|04|05|07|22|23|24|25|27
foreground :: 38;05;color
background :: 48;05;color
color      :: 000-255

A few observations about the above:

• The parentheses aren’t really necessary.
• Any leading zeroes can be skipped.
• That is a literal m in the code.
• The ^[[ is an escape key (U+001B) followed by a literal [.

The attribute codes are one of the following:

The “No-” variations are codes to turn off the particular effect. They’re enormously useful if you have a lot of effects, you only want to turn off one and you don’t want to use reset followed by re-enabling everything else.

That’s pretty much all you need to start making use of them yourself. However, examples are always nice, so:

# Print text normally, in color 214, which happens to be a nice orange.
$> echo "^[[38;05;214mHello, World"

# Make the same text bold.
$> echo "^[[(01);(38;05;214)mHello, World"

# Print underlined and italicized text, with normal foreground, and blue
# background.
$> echo "^[[(03;04);(48;05;20)mHello, World"

# Bold, blinking purple text.
$> echo "^[[(01;05);(38;05;93)mHello, World"

# Simple purple text on yellow background.
$> echo "^[[(38;05;93);(48;05;226)mHello, World"

A Note on Prompt Usage

If you want to use these colors and effects in your prompt, you can do so, but you will need to wrap each color code in a %{%} construct. This tells the shell that everything in between should not actually be printed, and terminal space should not be reserved for those characters.

Spectrum, a ZSH Abstraction

I’ve abstracted these codes and put them together in a zsh function which I call spectrum. The code for spectrum can be found as part of my ~/etc, on github.

To use, first either source or autoload-and-run the spectrum script. It will set up three mappings, FX, FG and BG. FX is a mapping from effect names to their corresponding codes, and FG and BG are mappings from color number to codes. All numbers are 0-padded to 3 digits, so if you want light blue, use $FG[000]. The source is quite readable, take a look. Note that you can’t run the script, as this would execute it in a different shell, and the changes wouldn’t stick.

The above examples then, could be replicated as follows:

# Provided spectrum is in your $fpath,
$> autoload spectrum && spectrum

# Print text normally, in color 214, which happens to be a nice orange.
$> echo "$FG[214]Hello, World"

# Make the same text bold.
$> echo "$FX[bold]$FG[214]Hello, World"

# Print underlined and italicized text, with normal foreground, and blue
# background.
$> echo "$FX[italic]$FX[underline]$BG[020]Hello, World"

# Bold, blinking purple text.
$> echo "$FX[bold]$FX[blink]$FG[093]Hello, World"

# Simple purple text on yellow background.
$> echo "$FG[093]$BG[226]Hello, World"