Iterate a list of items and do something: three shell idioms

Three shell idioms do the same job: take a list, run one command per item. The choice between them comes down to the input. Each section below covers one idiom, the input shape that fits it, and the cost when the shape is wrong.

Let's say you have the file players.txt that contains:

You could do:

This runs in the current shell, so any variables you set inside it are still set afterwards. The -r flag tells read to treat backslashes as literal characters, not as escape sequences. The example uses just one field per line, so the loop body sees the whole line as name.

This wins when: the data is line-shaped (one record per line) and the loop is short.

Costs: if you write cat file | while read instead of while read < file, the loop runs in a subshell and any variable writes inside the loop are gone when it exits. The redirect sidesteps the question.

Gotcha: cat file.txt | while read and while read < file.txt differ. The pipe forks; the redirect does not.

-I {} says: one invocation per input line, substitute {} with the line inside the command. The redirection > lives outside that scope, so cat {} > /tmp/all-files-combined.txt runs once per file, all writing to the same /tmp/all-files-combined.txt.

The same idiom with an interactive pick in the middle:

The fzf tool shows the same list and lets the reader pick one line; xargs then writes only that file's contents to /tmp/picked.combined.txt. The composition is one pipe insertion.

This wins when: you want to run one command per line, with the line as a single argument, and the work itself is a one-shot command.

Costs: each iteration forks a subprocess. xargs splits on whitespace by default, which mangles any item that has a space in it — -I {} is the safe pick (one item per invocation, and no splitting).

find does the filtering (-name, -type, -mtime) and -exec runs the action. The {} token is the matched path. The \; ends the command, one process per file. The above prints the line count of every .log.gz older than 7 days in /var/log. Nothing is removed; this is the pre-flight check.

Here is the same command, batched:

The + ends the command differently: paths are appended to a single invocation, so wc -l is invoked once with every path as an argument. The work and the filtering are identical; the difference is how many processes the loop spawns.

In the real-life cleanup use case, you would replace wc -l by the rm command.

This wins when: the items are paths from a filtered tree, and the filtering is the main part of the work; you want to do X with everything matching this name/age/permission.

Costs: the subprocess-per-file form (\;) is slow on large trees. Use +.

Three shapes, three idioms. Pick by input shape.

If your data fits one of these shapes, the rest of the script is the pipe. If it doesn't — embedded quotes, CSV with commas in fields, JSON, HTML — that is its own problem, with its own tools (awk -F, csvkit, jq, a real HTML parser). Worth its own issue.