ʞ / essays /

everything you ever wanted to know about terminals

so here's a short tutorial on ansi escape codes and terminal control, because you philistines won't stop using ncurses and oh my god WHY ARE WE STILL USING NCURSES IT IS THE TWENTY FIRST FUCKING CENTURY

the way terminal emulators handle fancy things like color and cursor shape aren't some mysterious opaque black box you can only access through a library. accessing most of these capabilities is actually extremely simple; they can even be hardcoded into a text file and displayed by cat or less. or even curl! the way you do this is with something called ANSI escape sequences.

almost all UI changes in a terminal are accomplished through in-band signalling. these signals are triggered with the ASCII/UTF-8 character ‹ESC› (0x1B or 27). it's the same ‹ESC› character that you send to the terminal when you press the Escape key on your keyboard or a key sequence involving the Alt key. (typing ‹A-c› for instance sends the characters ‹ESC› and ‹c› in very rapid succession; this is why you'll notice a delay in some terminal programs after you press the escape key — it's waiting to try and determine whether the user hit Escape or an alt-key chord.)

the simplest thing we can do with these escapes is to make the text bold (or "bright"). we accomplish this by sending the terminal the ‹ESC› character followed by [1m. [ is a character indicating to the terminal what kind of escape we're sending, 1 indicates bold/bright mode, and m is the control character for formatting escapes.

all text sent after this escape sequence will be bold until we explicitly turn it off again (even if your program terminates). there are two ways we can turn off bright mode: by clearing formatting entirely, using the m formatting command with no arguments or the argument 0, or more specifically clearing the bold bit with the 21m command. (you'll notice that you can usually turn off modes by prefixing the same number with 2.)

in a C program, this might look like the following:

the \x1b escape here is a C string escape that inserts hex character 0x1B (‹ESC›) into the string. it's kind of ugly and unreadable if you're not used to reading source with explicit escapes in it. you can make it a lot less horrible with a handful of defines, tho:

the beauty of this approach is that all the proper sequences are generated at compile time, meaning the compiler turns all that into a single string interpolated with the raw escapes. it offers much more readability for the coder at zero cost to the end user.

but hang on, where's that semicolon coming from? it turns out, ansi escape codes let you specify multiple formats per sequence. you can separate each command with a ;. this would allow us to write formatting commands like fmt(underline with bright with no italic), which translates into \x1b[4;1;23m at compile time.

of course, being able to style text isn't nearly good enough. we also need to be able to color it. there are two components to a color command: what we're trying to change the color of, and what color we want to change it to. both the foreground and background can be given colors separately - despite what ncurses wants you to believe, you do not have to define """color pairs""" with each foreground-background pair you're going to use. this is a ridiculous archaism that nobody in the 21st fucking century should be limited by.

to target the foreground, we send the character 3 for normal colors or 9 for bright colors; to target the background, we send 4 for normal or 10 for bright. this is then followed by a color code selecting one of the traditional 8 terminal colors.

note that the "bright" here is both the same thing and something different from the "bright" mode we mentioned earlier. while turning on the "bright" mode will automatically shift text it applies to the bright variant of its color if it is set to one of the traditional 8 colors, setting a "bright color" with 9 or 10 will not automatically make the text bold.

when we invoke fmt(fg red with br_bg magenta), this is translated by the compiler into the command string \x1b[31;105m. note that we're using fmt(plain) (\x1b[m) to clear the coloring here; this is because if you try to reset colors with, for instance, fmt(fg black with bg white), you'll be overriding the preferences of users who have their terminal color schemes set to anything but that exact pair. additionally, if the user happens to have a terminal with a transparent background, a set background color will create ugly colored blocks around text instead of letting whatever's behind the window display correctly.

now, while it is more polite to use the "8+8" colors because they're a color palette the end-user can easily configure (she might prefer more pastel colors than the default harsh pure-color versions, or change the saturation and lightness to better fit with her terminal background), if you're doing anything remotely interesting UI-wise you're going to run up against that limit very quickly. while you can get a bit more mileage by mixing colors with styling commands, if you want to give any configurability to the user in terms of color schemes (as you rightly should), you'll want access to a much broader palette of colors.

to pick from a 256-color palette, we use a slightly different sort of escape: \x1b[38;5;(color)m to set the foreground and \x1b[48;5;(color)m to set the background, where (color) is the palette index we want to address. these escapes are even more unwieldy than the 8+8 color selectors, so it's even more important to have good abstraction.

here, the stanza fmt(wfg(248) with wbg(232)) translates into \x1b[38;5;248;48;5;232m. we're hard-coding the numbers here for simplicity but as a rule of thumb, any time you're using 8-bit colors in a terminal, you should always make them configurable by the user.

the opaque-seeming indexes are actually very systematic, and you can calculate which index to use for a particular color with the formula 16 + 36r + 6g + b, where r, g, and b are integers ranging between 0 and 5. indices 232 through 255 are a grayscale ramp from dark (232) to light (255).

of course, this is still pretty restrictive. 8-bit color may have been enough for '90s CD-ROM games on Windows, but it's long past it's expiration date. using true color is much more flexible. we can do this through the escape sequence \x1b[38;2;(r);(g);(b)m where each component is an integer between 0 and 255.

sadly, true color isn't supported on many terminals, urxvt tragically included. for this reason, your program should never rely on it, and abstract these settings away to be configured by the user. defaulting to 8-bit color is a good choice, as every reasonable modern terminal has supported it for a long time now.

but, for users of XTerm, kitty, Konsole, and libVTE-based terminal emulators (such as gnome-terminal, mate-terminal, and termite), it's polite to have a 24-bit color mode in place. for example:

this sort of infrastructure gives you an enormously flexible formatting system that degrades gracefully without tying you to massive, archaic libraries or contaminating the global namespace with hundreds of idiot functions and macros (which is which of course being entirely indistinguishable).

but what if you want more than formatting? what if you want an actual TUI?

depending on the sort of TUI you want, you could actually get away with plain old ASCII. if you're just trying to draw a progress bar, for instance, you can (and should) use the ASCII control character ‹CR›, "carriage return" (in C, \r):

of course, if we want to be really fancy, we can adorn the progress bar with ANSI colors using the escape sequences described above. this will be left as an exercise to the reader.

this is sufficient for basic applications, but eventually, we'll get to the point where we actually need to address and paint individual cells of the terminal. or, what if we wanted to size the progress bar dynamically with the size of the terminal window? it's time to break out ANSI escape sequences again.

the first thing you should always do when writing a TUI application is to send the TI or smcup escape. this notifies the terminal to switch to TUI mode (the "alternate buffer"), protecting the existing buffer so that it won't be overwritten and users can return to it when your application closes.

in ANSI, we achieve this with the sequence ‹ESC›[?1049h (or, as a C string, "\x1b[?1049h").

(n.b. there's another escape, "\x1b47h", with deceptively similar effects as 1049, but it's behavior is subtly broken on some terminals (such as xterm) and it outright doesn't work on others (such as kitty). "\x1b[?1049h" has the correct effects everywhere that the alternate buffer is supported tho.)

once you've switched to the alternate buffer, the first thing you'll want to do is clear the screen and home the cursor, to clean up any debris previous applications might have left behind. for this, we use the sequence ‹ESC›[2J, which clears the screen and nukes scrollback. (we can't use the terminal reset sequence, ‹ESC›c, because it affects not just the active buffer, but the entire terminal session, and will wreck everything that's currently displayed!)

likewise, just before exit, you need to send the TE or rmcup escape. this notifies the terminal to switch back to the previous mode. this sequence, as a C string, is "\x1b[?1049l". to be polite, before you send this escape, you should clean up after yourself, clearing scrollback as before.

(h and l in these escapes seem to stand for "high" and "low," meaning essentially "on" and "off" by reference to hardware IO lines, where high current usually corresponds to a 1 bit and low to a 0 bit. in the hardware terminals of the past eon, it's possible program-configurable modes such as this were implemented with discrete IO lines set to a particular level; it's also possible the ANSI escape code designers just reached for a handy metaphor in an age where booleans weren't yet in vogue. if anyone happens to find out the actual story behind this, please do let me know)

once we're in the alternate buffer, we can safely start throwing around escape sequences to move the cursor to arbitrary positions. however, before we do this, we need to know how big the terminal actually is so we can lay out the UI appropriately.

it's good form to have a function called resize() or similar that you run on program start and later when the terminal window is resized. while there is a horrible way to do this with ANSI escapes, it's better to just bite the bullet and learn how to use ioctls and termios.

termios is a POSIX interface that lets you discover and set properties of the current terminal. it's kind of an unholy mess, but fortunately, we only need to use a very small corner of it to get the information we need.

we start off by importing the <sys/ioctl.h> header. this gives us the functions and structures we need to set ioctls. termios returns the size of the window in a structure called struct winsize. (far more rational than anything you'd find in ncurses, no?) this struct is populated using the function call ioctl(1, TIOCGWINSZ, &ws) where ws is the name of our struct (and 1 is the file descriptor for standard output). terminal width and height can then be accessed in the fields ws_col (for width) and ws_row (for height).

of course, we need to keep these values up to date when the terminal size changes. this is why resize() needs to be its own function - it needs to be called whenever our program is sent the SIGWINCH signal. SIGWINCH is automatically sent to child processes by the controlling terminal emulator whenever its window is reshaped.

a full example of these concepts in action:

throughout all of this, you may have noticed one thing: despite our attempts to create a clean, slick TUI, the cursor itself remains stubbornly onscreen. don't worry, tho; we can fix this.

the escape sequence to show and hide the cursor works much like the one to switch to and from the alternate buffer, except it has the number 25 instead of 1049. we can therefore hide the cursor by printing the string "\x1b[?25l" and show it again with the string "\x1b[?25h".

it's important to track how you're changing the behavior of the terminal, though, and restore it on program exit. otherwise, the user will have to reset the terminal herself, which many don't even know how to do (for the record, it's $ reset or $ echo -ne "\ec"). since you won't necessarily have control over how your program exits, it's important to set an exit handler using the atexit(3) and signal(3) functions. this way, even if the process is terminated with SIGTERM or SIGINT, it will still restore the terminal to its original state.

(it won't do jack shit in case of a SIGKILL, of course, but at that point it's the user's responsibility anyway.)

here's an example of appropriate terminal cleanup:

note the strategic placement of the atexit and signal functions. depending on where the program is in its execution when it receives SIGTERM, the cleanup function may be called before anything following it. if these traps were placed at the top of the program, they might be called before the alternate buffer was even opened, wiping out the ordinary buffer and anything the user had there. this is very impolite: we want to make sure that havoc is minimized.

of course, there is still a very small problem: if by some miracle the program is killed after entering the alternate buffer but before the cleanup function is registered, the user could be left stuck in the alternate buffer. to fix this, we would have to register the cleanup function before anything else, and start off the cleanup function by giving the instruction to enter the alternate buffer. this is a NOP is we're already there; if we're not, it protects the user's terminal from the deleterious effects of the following code.

now we've set the stage for our slick ncurses-free TUI, we just need to figure out how to put things on it.

we can move the cursor to an arbitrary position with ‹ESC›[(r);(c)H. (r) here is the row we want to move to (the first row being 1), and (c) is the target column (also 1-indexed).

there's a number of other control sequences that move the cursor by relative steps, but as a rule, you should always use absolute addressing, as using relative addressing can lead to cumulative errors - and if your program doesn't know the location of the cursor at all times, something is very wrong.

if you actually try this, though, you'll quickly notice a new problem. anything the user types will still appear onscreen, all over your beautiful TUI, whether or not you want it to. this also moves the cursor as a side effect. this is chaos you don't want in a program, so we need to put an end to it. however, there's no standardized escape code to accomplish this.

in other words, we need to use termios. the ugly side of termios.

termios, unlike libraries you might be used to, doesn't just have functions you can call to set properties. instead, we need to download the entire termios struct for the current terminal into a variable of type struct termios, make our modifications, and then upload it back to the terminal.

to do this, we need to define two of those structs: one to hold our modifications, and one to hold the original state of the program so it can be restored by our cleanup function at exit. to download the struct, we use the function tcgetattr(3). this function takes two arguments: a file descriptor representing the current terminal (always 1, for stdout), and a pointer to a struct termios to write into. as soon as we've populated our struct, before we've made any modifications, we need to copy the unmodified struct into our global-scope "backup" struct.

after that, we can turn off echo. local echo is one of a number of flags encoded in the bitfield c_lflag, identified by the constant ECHO. to disable it, we first invert ECHO with the ~ bitwise NOT operator, and then bitwise-AND the field by the resulting value.

once we've made our modifications, we can send them back up with the function tcsetattr(3). this one takes three arguments. the first is the file descriptor to modify, as usual. the second is a constant controlling when these modifications actually take place - for our purposes, this is always TCSANOW. finally, we give it a pointer to the struct we've modified.

having turned off local echo, we now need to handle it (and line-editing) by hand, printing and deleting characters as the user types them. the problem is, the terminal won't actually tell us the user has typed anything until she hits ‹ENTER›, making line-editing (or even just seeing what she's typing) impossible.

the reason this happens is something called canonical mode. canonical mode is the normal mode of operation for dumb terminals and terminal emulators. while in canonical mode, terminals will exhibit traditional Unix-y behaviors, like allowing you to type anything at any time, even if nothing is reading from stdin, and only sending text line-by-line as ‹ENTER› is keyed. remember, unlike DOS, UNIX uses a file/stream metaphor to interact with the terminal: it's just another file, so you can type things in at any time (and they'll be there as soon as a program decides to read from it again).

this doesn't suit our purposes at all, tho. we need DOS-like control over the UI. to achieve this, we need to turn off canonical mode. this is controlled by the ICANON flag, and with it off, we'll be able to read characters keystroke by keystoke.

this is the final piece we strictly need to write a TUI. however, for extra credit:

if you're a vim user, you may have noticed that the cursor changes shape depending on what mode you're in (i-beam for insert, block for normal, or underline for replace). we can do this as well, with the DECSCUSR escape sequences.

these sequences start off as usual, with ‹ESC›[. a numeric character then follows, indicating which cursor we want to employ. we then finish the sequence with the command  q, a literal space followed by the letter q.

the values we can use are 0 or 1 for a blinking block cursor, 2 for a steady block cursor, 3 for a blinking underline cursor, 4 for a steady underline cursor, 5 for a blinking i-beam cursor, and 6 for a steady i-beam.

now let's put it all together:

that's it for the tutorial. i hope you learned something and will reconsider using fucking ncurses next time because jesus fucking christ.

yes, ncurses supplies features like window-drawing and region invalidation (to avoid terminal flicker actually justine tunney has pointed out that this isn’t an issue with modern terminal emulators; you can generally just redraw the whole ui every frame so region invalidation is no longer quite as useful) that are much harder to implement yourself. no, you shouldn't have to implement it yourself. there should be a modern library to replace curses using the capabilities outlined here. but i swear to god developers have so completely forgotten how terminals work that i might be one of a handful of people left on earth who actually has the knowledge to, so they all just layer their bullshit on top of ncurses (which should never have survived the '90s) instead and it's ｍａｄｄｅｎｉｎｇ．

my hope is that this tutorial will curtail some of the more egregiously trivial uses of ncurses and provide others with the knowledge needed to implement a 21st-century terminal UI library. because i sure as fuck don't have the energy to.

also, i have effectively zero pull in the tech community and am also kind of a controversial character who is liable to give projects a bad reputation, which i don't normally care about but this one is important. point is, nothing i wrote would ever gain any traction; any project designed to supplant ncurses needs to come from someone who's actually known to and respected by the FOSS community. and a maintainer who isn't a cripple.

you can find a fuller list of ANSI escapes at wikipedia.

oh, and before anyone starts up:

being compatible only with ANSI-capable terminals is a feature, not a bug, go the fuck away. terminfo is a fucking joke. nobody needs to target obscure dumb terminals (or smart terminals, for that matter) from 1983 anymore.

all sample code in this document is the sole property of the author and is released exclusively under the GNU AGPLv3.