A thought on process substitution

During all the years I've been using Linux and scripting, I've put effort on avoiding using process substitution in bash. When I saw stuff like <(some command) online I'd just empirically test if it worked for my case, and it it did, I'd just accept it and continue with what I was doing.

For me, this was some sort of sorcery, the lost brother of piping, |. I knew process substitution had a solid logic behind it but I was not mentally prepared doing some research on it. Just like not-AI-developers do with AI models these days... Yet, the last time I came across to this bash extension, I somehow managed to convince myself to spend time understanding what is it and how to use it.

Piping vs Process Substitution

One of the first things that we usually learn about bash scripting is piping. A Unix pipe connects STDOUT file descriptor of the process on the left, to the STDIN of the one on the right. As simple as that. When piping, processes are executed sequentially and the output is sent to the next process no matter if the first failed or not. This can be chained as much as we want. For example:

piping.sh bash

grep "ERROR" my.log | cut -d' ' -f3 | sort -u

Here, we first filter all the instances of ERROR in my very creative-named my.log file. The standard output of this file is collected by the next process as standard input, extracting just the third column, and finally that output is again passed as input to be sorted and deduped.

On the other hand, process substituion is a more complex mechanism. Process substitution comes in two different forms: <(command) annd >(command). For both cases, a FIFO file is created under $TMPDIR or uses a file descriptor /dev/fd/*, depending on the OS. The substitution syntax is replaced by the FIFO file or file descriptor, and the command inside it is run in the background. For example, this line:

diff.sh bash

diff <(sort file1) <(sort file2)

Is equivalent to:

diff.sh bash

mkfifo /var/tmp/fifo1
mkfifo /var/tmp/fifo2
sort file1 > /var/tmp/fifo1
sort file2 > /var/tmp/fifo2
diff /var/tmp/fifo1 /var/tmp/fifo2
rm /var/tmp/fifo1 /var/tmp/fifo2

In and out process substitution

As it has been explained before, process substitution comes in two forms, input process substitution <(code) and output process substitution >(code).

Input process substitution acts as a file to be used as input for the principal command. For example:

input.sh bash

diff <(sort file1) <(sort file2)

Output process substitution acts as a destination file to be used as output of the principal command. For example:

output.sh bash

ps aux | tee >(grep firefox > firefox.log) >(grep chrome > chrome.log) > all_processes.log

This one gets the full process list and duplicates that stream in tree ways.

firefox.log via grep filter.
chrome.log via grep filter.
Plain copy to all_processes.log.

Three log files are created, the command is run just once and no temporal files are needed.

As you might expect, we can combine the two in one command line in our scripts. For example:

inNout.sh bash

diff <(grep "ERROR" access1.log) <(grep "ERROR" access2.log) | tee >(wc -l > diff_count.txt)

Two <() substitutions filter each log to error lines only, diff's output goes to tee, which prints it to the terminal as usual and sends a copy into a >() substitution that counts the lines and saves the count to diff_count.txt.

A glance to process tree

It's tempting to assume process substitution and piping look identical in a process tree, since both feel like they're just "connecting streams." But testing it for real tells a different story: the shape of the tree depends on whether bash needs to keep multiple standalone commands alive, or can collapse everything into a single command's own family tree. Piping always does the former. Process substitution, when it's attached to a single command, does the latter.

pipingTree.sh bash

grep "ERROR" access.log | cut -d' ' -f1 | sort -u &
pstree -p $$

result bash

sh(138550)─┬─cut(138889)
           ├─grep(138888)
           └─sort(138890)

Every stage of a pipe is a genuinely separate command that needs its own process. Bash forks grep, cut, and sort all at once, wires each one's stdout to the next one's stdin via a pipe, and lets them run concurrently as flat siblings under the shell. No single process is "in charge" of the others, they just happen to be connected

psubTree.sh bash

diff <(sort file1_big.txt) <(sort file2_big.txt) &
pstree -p $$

result2 bash

bash(144501)───diff(144544)─┬─sort(144545)─┬─{sort}(144560)
                            │              ├─{sort}(144561)
                            │              └─{sort}(144562)
                            └─sort(144546)─┬─{sort}(144556)
                                           ├─{sort}(144557)
                                           └─{sort}(144558)

Here there is only ever one command in the foreground: diff. Before bash execs into it, the forked child process first sets up both process substitutions itself, forking a sort for each one and handing diff two /dev/fd/N paths as ordinary arguments. Because those sort processes are forked before the exec into diff happens, they stay attached as diff's children even afterward. exec swaps the running program but not the process's place in the tree. The result is a nested tree, not a flat one: diff is a parent, not a sibling.

The entries in curly braces, like {sort}(144560), aren't child processes at all. They're threads. GNU sort parallelizes its merge sort across multiple threads on large inputs, and pstree represents each thread of a process this way. So sort(144545) is a single process with three extra worker threads.

The real distinction, then, isn't "pipes are flat, substitution is nested." It's this: piping always needs multiple independent commands running side by side, so the tree stays flat by necessity. Process substitution just supplies a command with fake file paths. If that command is standalone, the substituted processes end up nested underneath it as an artifact of how exec works; if the substitution sits inside a larger pipeline instead, the shape changes again. The tree isn't telling you "pipe vs. substitution", it's telling you how many independent commands bash had to keep alive, and who forked whom before the final exec.

Final thoughts

Getting to untangle the mistery of process substitution has really improved my scripting skills, I hope this post made you appreciate it more and are willing to use it in your own implementations.

Happy scripting! <(-^,^-)=b

Sources

wooledge.org - Process Substitution