About Regular Expressions

A regular expression is a pattern that describes a set of strings. Regular expressions are constructed analogously to arithmetic expressions, by using various operators to combine smaller expressions.

Grep understands two different versions of regular expression syntax: “basic” and “extended.” In GNU grep, there is no difference in available functionality using either syntax. In other implementations, basic regular expressions are less powerful. The following description applies to extended regular expressions, which are used in egrep; differences for basic regular expressions are summarized afterwards.

The fundamental building blocks are the regular expressions that match a single character. Most characters, including all letters and digits, are regular expressions that match themselves. Any metacharacter with special meaning may be quoted by preceding it with a backslash.

A bracket expression is a list of characters enclosed by

[ and ]. It matches any single character in that list; if the first character of the list is the caret ^ then it matches any character not in the list. For example, the regular expression [0123456789] matches any single digit.

Within a bracket expression, a range expression consists of two characters separated by a hyphen (“-“). It matches any single character that sorts between the two characters, inclusive, using the locale’s collating sequence and character set. For example, in the default C locale, [a-d] is equivalent to [abcd]. Many locales sort characters in dictionary order, and in these locales [a-d] is typically not equivalent to [abcd]; it might be equivalent to [aBbCcDd], for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the LC_ALL environment variable to the value C.

Finally, certain named classes of characters are predefined within bracket expressions, as follows. Their names are self explanatory, and they are [:alnum:], [:alpha:], [:cntrl:], [:digit:], [:graph:], [:lower:], [:print:], [:punct:], [:space:], [:upper:], and [:xdigit:]. For example, [[:alnum:]] means [0-9A-Za-z], except the latter form depends upon the C locale and the ASCII character encoding, whereas the former is independent of locale and character set. (Note that the brackets in these class names are part of the symbolic names, and must be included in addition to the brackets delimiting the bracket list.) Most metacharacters lose their special meaning inside lists. To include a literal ] place it first in the list. Similarly, to include a literal ^ place it anywhere but first. Finally, to include a literal place it last.

The period . matches any single character. The symbol \w is a synonym for [[:alnum:]] and \W is a synonym for [^[:alnum]].

The caret ^ and the dollar sign (“$“) are metacharacters that respectively match the empty string at the beginning and end of a line. The symbols \< and \> respectively match the empty string at the beginning and end of a word. The symbol \b matches the empty string at the edge of a word, and \B matches the empty string provided it’s not at the edge of a word.

A regular expression may be followed by one of several repetition operators:

? The preceding item is optional and matched at most once.
* The preceding item will be matched zero or more times.
+ The preceding item will be matched one or more times.
{n} The preceding item is matched exactly n times.
{n,} The preceding item is matched n or more times.
{n,m} The preceding item is matched at least n times, but not more than m times.

Two regular expressions may be concatenated; the resulting regular expression matches any string formed by concatenating two substrings that respectively match the concatenated subexpressions.

Two regular expressions may be joined by the infix operator |; the resulting regular expression matches any string matching either subexpression.

Repetition takes precedence over concatenation, which in turn takes precedence over alternation. A whole subexpression may be enclosed in parentheses to override these precedence rules.

The backreference \n, where n is a single digit, matches the substring previously matched by the nth parenthesized subexpression of the regular expression.

In basic regular expressions the metacharacters ?, +, {, |, (, and ) lose their special meaning; instead use the backslashed versions \?, \+, \{, \|, \(, and \).

Traditional egrep did not support the { metacharacter, and some egrep implementations support \{ instead, so portable scripts should avoid { in egrep patterns and should use [{] to match a literal {.

GNU egrep attempts to support traditional usage by assuming that { is not special if it would be the start of an invalid interval specification. For example, the shell command egrep ‘{1’ searches for the two-character string {1 instead of reporting a syntax error in the regular expression. POSIX.2 allows this behavior as an extension, but portable scripts should avoid it.

Environment Variables

Grep‘s behavior is affected by the following environment variables:

A locale LC_foo is specified by examining the three environment variables LC_ALL, LC_foo, LANG, in that order. The first of these variables that is set specifies the locale. For example, if LC_ALL is not set, but LC_MESSAGES is set to pt_BR, then Brazilian Portuguese is used for the LC_MESSAGES locale. The C locale is used if none of these environment variables are set, or if the locale catalog is not installed, or if grep was not compiled with national language support (NLS).

GREP_OPTIONS

This variable specifies default options to be placed in front of any explicit options. For example, if GREP_OPTIONS is ‘–binary-files=without-match –directories=skip‘, grep behaves as if the two options –binary-files=without-match and –directories=skip had been specified before any explicit options. Option specifications are separated by whitespace. A backslash escapes the next character, so it can be used to specify an option containing whitespace or a backslash.

GREP_COLOR

Specifies the marker for highlighting.

LC_ALL, LC_COLLATE, LANG

These variables specify the LC_COLLATE locale, which determines the collating sequence used to interpret range expressions like [a-z].

LC_ALL, LC_CTYPE, LANG

These variables specify the LC_CTYPE locale, which determines the type of characters, e.g., which characters are whitespace.

LC_ALL, LC_MESSAGES, LANG

These variables specify the LC_MESSAGES locale, which determines the language that grep uses for messages. The default C locale uses American English messages.

POSIXLY_CORRECT

If set, grep behaves as POSIX.2 requires; otherwise, grep behaves more like other GNU programs. POSIX.2 requires that options that follow file names must be treated as file names; by default, such options are permuted to the front of the operand list and are treated as options. Also, POSIX.2 requires that unrecognized options be diagnosed as “illegal”, but since they are not really against the law the default is to diagnose them as “invalid”. POSIXLY_CORRECT also disables _N_GNU_nonoption_argv_flags_, described below.

_N_GNU_nonoption_argv_flags_

(Here N is grep’s numeric process ID.) If the ith character of this environment variable’s value is 1, do not consider the ith operand of grep to be an option, even if it appears to be one. A shell can put this variable in the environment for each command it runs, specifying which operands are the results of file name wildcard expansion and therefore should not be treated as options. This behavior is available only with the GNU C library, and only when POSIXLY_CORRECT is not set.

Examples

egrep "support|help|windows" myfile.txt

Search for patterns of support help and windows in the file myfile.txt.

egrep '^[a-zA-Z]+$' myfile.txt

Match any lines in myfile.txt which begin a line with an alphabetic word which also ends the line.

egrep -c '^begin|end$' myfile.txt

Count the number of lines in myfile.txt which begin with the word ‘begin‘ or end with the word ‘end‘.

Wild-card are shell feature which make command line more powerful than GUI file manager. They are also known as globbing patterns. They are used by various command-line utilities to work with multiple files. Various wild-cards are:1. * : none or any number of characters.

$ ls file*                       list all files which starts with file

$ touch file1 file2 file3
$ ls file*
file1 file2 file3

*touch is use to create empty file/s.

$ ls *file*                     list all files which contains file in their name

$ touch  1_file_my  2_file_your  3_file_others
$ ls *file*
1_file_my   2_file_your   3_file_others  file1  file2  file3

$ ls *.lst                       list all files which end with .lst

$ ls *.lst
abc.lst   def.lst    pqr.lst    sandeep2.lst    sandeep.lst     xyz.lst

* echo *  list all files in current directory.

2. ?  : Match single character only.

$ ls file?                      list all files which has single character after  file

$ touch file1 file2 file3

$ ls file?
file1 file2 file3

$ ls ?file?         list all files which has single character before and after file in their name

$ touch  1file1  2file2  3file3
$ ls ?file?
1file1  2file2  3file3

$ ls ??    List all those files and the content of the directory which has two character long name.

$ mkdir cs
$ cd cs
$ pwd
/home/sandeep/cs


$ touch file1 file2 file3
$ cd
$ pwd
/home/sandeep


$ touch f1 f2 f3
$ ls ??
f1  f2  f3
cs:
file1  file2  file3
js:
index.html  jquery


* There are two things that * and ? can’t match . First they don’t match a file-name beginning with a dot(.) , but they can match any number of dots between the words. Second these character does not match the / in a path name.

$ touch .file1 .file2 .file3
$ ls *file
ls: cannot access *file: No such file or directory
$ ls .file*
.file1 .file2 .file3$ touch file1.1 file2.2 file3.3
$ ls file*
file1 file1.1 file2 file2.2 file3 file3.3

3. [abc] : Match a single character – either a , b or c.

$ touch file1 file2 file3 file4
$ ls file[1234] file1 file2 file3 file4

4. [a-c] : Match a single character that is within ASCII range of the characters a and c.

$ touch file1 file2 file3 file4
$ ls file[1-4] file1 file2 file3 file4

5. [!abc] :  Match a single character that is not an a , b or c.

$ touch file1 file2 file3 file4
$ ls file[!12] file3 file4

6. [!a-c] :   Match a single character that is not within ASCII range of the characters a and c.

$ touch file1 file2 file3 file4 file5 file6 file7
$ ls file[!1-5] file6 file7

7. {pat1,pat2} : Match totally dissimilar patterns.

$ touch  file1.lst  file1.txt   file1.doc
$ ls *.{lst,doc,txt}
file1.doc  file1.txt   file1.lst

– See more at: http://technicalworldforyou.blogspot.ro/2012/01/matching-pattern-with-wild-cards-in.html#sthash.OxBVUrcP.dpuf