Text Processing

11/14/2007

Text Processing NET212

What are some text editors?

Text mode text editors • • • •

vi emacs pico nano …

1

11/14/2007

How do we use vi?

vi basics Command mode Move around, edit text Press esc to get to it

Insert mode Work with text as you normally would Press ‘I’ to get to it (from command mode)

Ex mode File operations and more Press ‘:’ to get to it (from command mode)

What should a word processor do?

2

11/14/2007

Common word processing • • • •

Create, edit files Edit, insert, and delete text Copy, move, and paste text Search for text

vi command mode Quickly moving around Forward one screen Ctrl + F Back one screen Ctrl + B Start of current line 0 End of current line $ Beginning of document [[ End of document ]]

vi command mode Deleting Text and undoing changes Current word dw Entire line dd Clear a line D n number of lines (next n lines) ndd Undo last change u

3

11/14/2007

vi command mode Searching (from current line) To search forward for some text /text_searching_for Repeat last search n Repeat last search in reverse N

vi command mode Copying and pasting text Copy number of words nyw Copy one line Y Copy n lines nY Copy everything the current line and all below yG Paste something p

vi command mode vi uses a temporary buffer for work You can paste text you deleted by using ‘p’ p in command mode. This is because deleted text is in the buffer.

4

11/14/2007

vi Ex mode Basic file operations Write a file :w Save as :w file_name Close a file :q Close a file ignoring changes :q!

vi Ex mode Reading/opening files Load a file into the editor :e file file_name name Read a file into the current line :r file_name

vi Ex mode Search and replace text Search for all occurrences and replace them :1,$s/text_to_replace/new_text/g With confirmation :1,$s/text_to_replace/new_text/gc

5

11/14/2007

Can we process text outside of editors?

Useful text processing programs grep search file for pattern awk pattern scanning and processing language sed stream editor cut take certain fields out of a file paste merge lines of text less view pages of text sort arrange a file in order wc count the number of items in a file diff compare files head/tail look at the top or bottom of a file

grep Search a file for some pattern grep pattern file_name file_name2 … grep “dog” myfile ps -aux | grep mingetty Options -A num number of lines after match -B num number of lines before match

6

11/14/2007

cut Extract fields of text from a file cut -option1 -option2 … file1 file2… cut -f 1 -d ‘:’ /etc/passwd (get login names) root Student … date | cut -f 5 -d ‘ ‘ | cut -c 1-2 (get hour) 18

cut options -f Fields to extract -d Delimiter to use (do this with -f) -c c Characters to get Default delimiter is tab

paste Combine files into one output and each file is in its own column (separated by a tab) paste file1 file2 ….

7

11/14/2007

paste example File1: Tom Suzy File2: Sawyer Storm paste file1 file2 Tom Sawyer Suzy Storm

sort sort options -n Sort numerically -r Sort the result in reverse Default is to sort in alphabetical order sort option1 option2 … file1 file2 … sort /etc/passwd sort -r /etc/passwd cut -f 3 -d ‘:’ /etc/passwd | sort -n (sort the uids)

wc By default it will count the number lines, words and characters in a file. wc options -c characters -l lines -w words wc -l /etc/passwd ls | wc -w

8

11/14/2007

diff Compare two files Will say nothing if they are the same, otherwise it will tell you the lines that are different different. diff file1 file2 cp /etc/passwd ~/passwd.bak diff /etc/passwd ~/passwd.bak Echo $? 0

sed • A powerful stream editor. Basically, it will • •

try to match a pattern and perform a simple operation to it. The result of the operation is printed to the console sed is commonly used as a filter like cut (when piping input to it)

sed example Substitute text to those areas matching the pattern sed -e ‘s/pattern_here/new_text/g’ file1 file2 … sed -e ‘s/:/ /g’ /etc/passwd date | sed -e ‘s/ /_/g’

9