Yuhang He's Blog

Some birds are not meant to be caged, their feathers are just too bright.

sed is a powerful shell processing text file. It is the abbreviation for stream editor, and is often used with combination with awk, grep shell command. Its powerful potential is just beyond your imagination, but, in most cases, we just depend on it to process several common tasks, like substitution, insertion append, deletion, etc. We would briefly talk about all of them in this article.

sed for substitution

Too often sed is applied for substitution. It always goes like sed 's/match_text/to_subs_text/g' file_name, quite like vim editor. sed scans the file line by line and processes each line separately. g indicates the substitute all the matched pattern. Similarly, g can be changed as 1 or 2, which means only substituting the first or the second matched pattern. Of course, sed can substitute specified lines. One example is: sed '3,5s/match_text/to_subs_text/g'. It means only substituting line 3 to line 5. & represents the matched pattern. Remember several important regular expressions that are commonly used in sed:

1. $ indicates the end of a line, ^ indicates the beginning of a line. 2. \< indicates a word’s prefix, \> indicates the suffix of a word. For example: /< h represents all words with prefix h. 3. . indicates any single character. * indicates a character with 0 or N times occurrence. 4. [] indicates the character set. For example: [a-c] means any character equals to a, b or c. [^a] means any character that is not equal to a. Note that [0-9]* would match 0 or N characters that are 0-9. If you want to find a digit string, you’d better use [0-9][0-9]*. If you have multiple substitution tasks at the same time, you can definitely use ; to concatenate multiple commands together. For example: sed '1,5s/hello/world/g ; 6,$s/world/hello/g'. Another appropriate way is to use sed -e to concatenate all commands: sed -e '1,5s/hello/world/g' -e '6,$s/world/hello/g' sed substitution can also be applied to addition, deletion, etc. Two examples are: cat file_name | sed 's/$/ 3/g' adding each line with 3.
cat file_name | sed 's/hello//g' deleting all the hello word.

sed for appending, deletion, insertion

a is to append a line right after the specified line. It works as sed '3 a hello world' file_name, which means appending hello world after the third line. A more complex task works as sed 'nohello/a hello world' file_name, which means only appending after lines where the specified words are matched.
i means insertion. It works similarly with appending but inserts before the specified lines.
c means substitute the whole line. It also works similarly with appending. d means deleting the whole lines. It also work similarly with appending. Several parameters need to pay attention:

1. -n work silently.
2. -i directly output the result to the input file. It is somewhat dangerous due to the fact that it may change to input file. sed -i 'xxx' is not directly allowed on Mac OS. You should create a temporary file to save the original result. Like, sed -i 'file.bak' 's/xxxx'.

Another useful tool of sed is to reverse file with Hold Space: sed '1!G;h;\$!d' input_file