HERE Documents – Preventing Weird Behavior

     $ cat donors
     #
     # simple lookup of our generous donors
     #
     grep $1 <<EOF
     # name amt
     pete $100
     joe  $200
     sam  $ 25
     bill $  9
     EOF
     $

But when you tried running it you got weird output:

     $ ./donors bill
     pete bill00
     bill $  9
     $ ./donors pete
     pete pete00

Solution

Turn off the shell scripting features inside the here-document by escaping any or all of the characters in the ending marker:

     # solution
     grep $1 <<\EOF
     pete $100
     joe  $200
     sam  $ 25
     bill $  9
     EOF

Discussion

It’s a very subtle difference, but the <<EOF is replaced with <<\EOF, or <<‘EOF’ or even <<E\OF—they all work. It’s not the most elegant syntax, but it’s enough to tell bash that you want to treat the “here” data differently.

Normally (i.e., unless we use this escaping syntax), says the bash manpage, “…all lines of the here-document are subjected to parameter expansion, command substi- tution, and arithmetic expansion.”

So what’s happening in our original donor script is that the amounts are being inter- preted as shell variables. For example, $100 is being seen as the shell variable $1 fol- lowed by two zeros. That’s what gives us pete00 when we search for “pete” and bill00 when we search for “bill.”

When we escape some or all of the characters of the EOF, bash knows not to do the expansions, and the behavior is the expected behavior:

     $ ./donors pete
     pete $100
     $

Of course you may want the shell expansion on your data—it can be useful in the correct circumstances, but isn’t what we want here. We’ve found it to be a useful practice to always escape the marker as in <<‘EOF’ or <<\EOF to avoid unexpected results, unless you know that you really want the expansion to be done on your data.