Filter & Pipes Examples

In-/Output Redirection

|  : Pipe operator: Connect STDOUT of a command with STDIN 
     of the next command

>  : Redirect Standard Output (into file)

<  : Redirect Standard Input (from file)

2> : Redirect Standard Error (into file)

>> : Redirect Standard Output (append into file)

• attribute names in file ../data/BCHI-dataset_2019-03-04.txt
  head -1 ../data/BCHI-dataset_2019-03-04.txt | tr '|' '\n'
  [Output]

• return the 50 most frequent words from ../data/moby-dick.txt (case insensitive)
  grep -Eo '[A-Za-z]+' ../data/moby-dick.txt | tr 'A-Z' 'a-z' | sort | uniq -c | sort -nr | head -50
  [Output]

• number of different words in ../data/moby-dick.txt
  grep -Eo '[A-Za-z]+' ../data/moby-dick.txt | tr 'A-Z' 'a-z' | sort | uniq | wc -l
  [Output]

• the number of cities per country (countrycode)
  cut -d, -f2 ../data/city.sample.csv | sort | uniq -c | sort -nr
  [Output]

• the number of cities per country (countryname)
  cut -d, -f2 ../data/city.sample.csv | sort | uniq -c |sed 's/^ *//;s/ /,/' | join -t, -12 -22 - ../data/country.csv -o 1.1,2.1| sort -nr | head -n10
  [Output]

• in which news articles, are "Agassi" and "Federer" mentioned?
  grep Federer ../data/bbcsport/*/*.txt -l -m1 | xargs grep Agassi -l
  [Output]

• build an inverted Index over the files in directory ../data/bbcsport/tennis
  rm -f invIndex/* && grep -Eo '[A-Za-z]+' ../data/bbcsport/football/*.txt | tr 'A-Z' 'a-z' | sort -t: -k2 | uniq | awk -F: '{ print $1 >> "invIndex/"$2}'
  [Output]