Shell for Developers (2)

Contents

• (hard)coreutils

• Finding Nemo things

• Differences

• SSH - my remote friend

• See Part 1 for a tour de force through Bash, VIm, and screen

(hard)coreutils

Write programs that do one thing and do it well.

UNIX philosophy

• GNU core utilities provide useful tools adhering to this principle.

(hard)coreutils - opposites

• true - do nothing, successfully
• Maybe they should call it joschi ;)
• false - do nothing, unsuccessfully
• Related: yes - output a string repeatedly until killed
• cat - concatenate files and print on the standard output
• tac - concatenate and print files in reverse
• split - split a file into pieces

(hard)coreutils - opposites (2)

• sort - sort lines of text files
  • Different modes can be very useful
  • du -s * | sort -n - get numerically sorted sizes of files/directories
  • du -hs * | sort -h - get sorted human-friendly sizes of files/directories
• shuf - generate random permutations
  • Also: sort -R

(hard)coreutils - opposites (3)

• head - output the first part of files
  • -n [-]K - print the first K lines instead of the first 10; with the leading `-', print all but the last K lines of each file
• tail - output the last part of files
  • tail -F useful for following rotated log files (if you don't like less)
  • -n [+]K - output the last K lines, instead of the last 10; or use -n +K to output lines starting with the Kth

(hard)coreutils - assorted

• uniq - report or omit repeated lines
  • Example: sort $FILE | uniq -c
• cut - remove sections from each line of files
  • Example: cut -f 1 $FILE
• seq - print a sequence of numbers
  • Example: for i in $(seq 1 10); echo "Hello $i"; done
• test or [ - check file types and compare values
  • Example: test -x /usr/bin/vim
• nl - number lines of files

(hard)coreutils - Audience participation \o/

• Print a list of five random directories/files in $HOME (no subdirectories) sorted by size.
• du -sh ~/* | shuf | head -n5 | sort -h
• Print a list of five random directories/files in $HOME (no subdirectories) sorted by size without size field.
• du -sh ~/* | shuf | head -n5 | sort -h | cut -f 2
• Print all but five lines of a file in reverse order with line numbers (of result).
• tac $FILE | head -n -5 | nl
• tail -n +6 $FILE | tac | nl

Finding Nemo things

• Finding files or directories: locate and find
• Finding something in files: grep

Finding Nemo things - locate

• locate - find files by name
  • Relies on static database generated by updatedb (usually in a cronjob)
  • Fast but results might be stale/incorrect
  • Limited query capabilities, but supports regular expressions (through parameter --regex) at least

Finding Nemo things - find

• Swiss Army Knife of file search
• Rich set of parameters, possible to formulate almost any query
• Slower than locate (naturally) since it iterates through the file system
• Often used in conjunction with xargs for running commands on result

Finding Nemo things - find (examples)

• Remove all empty directories in $HOME
• find $HOME -type d -empty -delete
• Find all executable files of user "synyx"
• find -type f -perm /a+x -exec file '{}' \;
• Find all files and directories (but not special files) of user "synyx" and put them in a TAR archive
• find -mindepth 1 \( -type f -or -type d \) -user synyx -print0 | xargs -0 tar -cf synyx.tar
• Build all Maven projects in trunk subdirectories (but not branches or tags)
• find -type f -name pom.xml -path '*/trunk/*' -and ! -path '*/target/*' -exec mvn -f '{}' clean package \;

Finding Nemo things - grep

• grep - print lines matching a pattern
• egrep, fgrep, and rgrep are shortcuts for grep -E, grep -F, and grep -r

Finding Nemo things - grep

• tbd :/

Differences

• diff - compare files line by line
• sdiff - side-by-side merge of file differences
• diff3 - compare three files line by line
• colordiff - a tool to colorize diff output
• cmp - compare two files byte by byte
• comm - compare two sorted files line by line

Differences - examples

• Generate unified diff of two files ignoring white space changes
• diff -u -b $FILE1 $FILE2
• Show differences of two web sites without downloading them
• diff <(wget -q -O - URL1) <(wget -q -O - URL2)
• Show lines unique to two files, omit common lines
• comm -3 $FILE1 $FILE2

SSH - my remote friend

• tbd :/

Thanks for listening!

Any questions?

Back to Overview