grep Pattern Search

What it is

grep is a command-line utility for searching plain-text data sets for lines that match a regular expression.

Installation

Linux

grep is typically pre-installed on most Linux distributions. If not, you can install it using your distribution’s package manager:

Debian/Ubuntu:

sudo apt update
sudo apt install grep

Fedora/CentOS/RHEL:

sudo dnf install grep
# or
sudo yum install grep

macOS

grep is pre-installed on macOS.

Windows

grep is not natively available on Windows. You can obtain it through:

• Git for Windows: Includes a Git Bash terminal with grep and other common Unix utilities. Download from git-scm.com.
• Windows Subsystem for Linux (WSL): Install a Linux distribution from the Microsoft Store and use grep within that environment.

Core Concepts

Patterns

grep searches for patterns, which can be:

• Fixed strings: Literal text to match.
• Basic Regular Expressions (BRE): The default, a subset of regular expression syntax.
• Extended Regular Expressions (ERE): A more powerful set of regular expression syntax, enabled with -E or --extended-regexp.

Input Sources

grep can search:

• Standard input (stdin): Data piped from another command.
• Files: One or more specified files.
• Recursively: Through directories and their subdirectories.

Commands / Usage

Basic Searching

Searching for a fixed string in a file:

grep "error" /var/log/syslog

Find lines containing the exact string "error" in /var/log/syslog.

Searching for a pattern in multiple files:

grep "warning" /var/log/*.log

Find lines containing "warning" in all files ending with .log in /var/log/.

Searching recursively through a directory:

grep -r "TODO" ./src/

Find lines containing "TODO" in all files within the src directory and its subdirectories.

Case-insensitive search:

grep -i "debug" application.log

Find lines containing "debug", "Debug", "DEBUG", etc., in application.log.

Show lines that do not match:

grep -v "info" server.log

Display all lines in server.log that do not contain the string "info".

Count the number of matching lines:

grep -c "success" results.txt

Output the total count of lines containing "success" in results.txt.

List only the names of files that contain a match:

grep -l "critical" /etc/nginx/*.conf

Print the names of configuration files in /etc/nginx/ that contain the word "critical".

List only the names of files that do not contain a match:

grep -L "deprecated" *.py

Print the names of Python files in the current directory that do not contain the string "deprecated".

Highlight matching text:

grep --color=auto "exception" app.log

Display lines containing "exception" from app.log, with the matching text highlighted in color.

Regular Expression Usage

Using basic regular expressions (default):

grep "^root" /etc/passwd

Find lines in /etc/passwd that start with "root". (^ anchors to the beginning of the line).

Using extended regular expressions:

grep -E "([0-9]{1,3}\.){3}[0-9]{1,3}" ip_addresses.txt

Find lines matching a basic IPv4 address format in ip_addresses.txt.

Matching lines that contain either pattern:

grep -E "error|failure" system.log

Find lines containing "error" OR "failure" in system.log.

Matching lines that start with a specific character and contain digits:

grep "^[a-zA-Z][0-9]" data.txt

Find lines in data.txt that begin with any letter followed immediately by any digit.

Matching lines that end with a specific word:

grep "completed$" task_log.txt

Find lines in task_log.txt that end with the word "completed". ($ anchors to the end of the line).

Matching lines with a specific number of occurrences:

grep -o -E "a{2}" text.txt

Extract all occurrences of exactly two consecutive 'a’s from text.txt. (-o prints only the matched part).

Controlling Output

Show lines before the match:

grep -B 3 "segmentation fault" core_dump.log

Display the matching line and the 3 lines preceding it from core_dump.log.

Show lines after the match:

grep -A 2 "connection refused" network.log

Display the matching line and the 2 lines following it from network.log.

Show lines before and after the match:

grep -C 5 "timeout" api_calls.log

Display the matching line and 5 lines before and 5 lines after it from api_calls.log.

Suppress line numbers:

grep -n "pattern" file.txt

Show matching lines with their line numbers. (Default behavior is to show line numbers if searching multiple files, but -n forces it).

Show line numbers:

grep -n "pattern" file.txt

Show matching lines with their line numbers.

Print only the matched part of a line:

grep -o "email: [^ ]+" contacts.txt

Extract only the email addresses (starting with "email: ") from contacts.txt.

Search for whole words only:

grep -w "run" script.sh

Find lines containing the exact word "run", not as part of other words like "running" or "rerun".

File and Directory Handling

Search in specific files:

grep "config" /etc/default/grub /boot/grub/grub.cfg

Search for "config" in both /etc/default/grub and /boot/grub/grub.cfg.

Follow symbolic links when searching recursively:

grep -r -L /path/to/dir

Recursively search through /path/to/dir, following symbolic links.

Exclude directories from recursive search:

grep -r --exclude-dir={.git,node_modules} "function" .

Recursively search the current directory (.) for "function", skipping .git and node_modules directories.

Exclude files from search:

grep -r --exclude="*.min.js" "api_key" src/

Recursively search files in src/ for "api_key", but skip any file ending in .min.js.

Include only specific file types in recursive search:

grep -r --include="*.py" "import" .

Recursively search only Python files (.py) in the current directory (.) for the word "import".

Advanced Options

Use a file containing search patterns:

grep -f patterns.txt data.txt

Search data.txt for lines matching any of the patterns listed in patterns.txt. Each pattern should be on a new line.

Interpret patterns as fixed strings, not regex:

grep -F "[special_char]" config.ini

Search for the literal string [special_char] in config.ini, treating [ and ] as literal characters, not regex metacharacters.

Compute and print checksums:

grep -c "pattern" file1.txt file2.txt

Calculate and print the MD5 checksum for each file. (Note: -c here is for checksums, not count. Use --help for disambiguation.)

Process binary files:

grep -a "string" binary_file

Treat a binary file as text and search for "string". This may output garbage or non-printable characters.

Suppress error messages:

grep "pattern" non_existent_file 2>/dev/null

Attempt to search non_existent_file for "pattern" but hide any "No such file or directory" errors.

Common Patterns

Find all IP addresses in a log file:

grep -E -o "([0-9]{1,3}\.){3}[0-9]{1,3}" access.log

Extracts all sequences that look like IPv4 addresses from access.log.

Find lines containing a specific word, ignoring case, and show context:

grep -i -C 2 "error" /var/log/messages

Searches /var/log/messages for "error" (case-insensitive) and shows the matching line plus 2 lines before and 2 lines after.

Find lines that don’t contain a specific pattern and save to a new file:

grep -v "DEBUG" app.log > production.log

Copies lines from app.log that do not contain "DEBUG" into production.log.

Search for a pattern across all files in the current directory and subdirectories:

grep -R "database_connection" .

Searches recursively from the current directory (.) for the string "database_connection".

Find lines containing a specific string, but exclude results from certain directories:

grep -r --exclude-dir={node_modules,vendor} "api_endpoint" .

Recursively searches the current directory for "api_endpoint", ignoring files within node_modules and vendor directories.

Find lines containing either "warning" or "critical", highlighting the match:

grep -E --color=auto "warning|critical" system.log

Finds lines with either "warning" or "critical" in system.log and highlights the matched word.

Count occurrences of a specific pattern across multiple files:

grep -c "TODO" *.py *.js

Counts the total number of lines containing "TODO" in all .py and .js files in the current directory.

Gotchas

• Metacharacters: By default, grep interprets characters like ., *, ^, $, [, ], \ as special regular expression characters. To search for these literally, you must escape them with a backslash (\) or use the -F (fixed string) option.
  • Example: grep "file\.txt" searches for "file.txt", not "file" followed by any character then "txt".
• Quoting: Always quote your patterns, especially if they contain spaces or shell metacharacters, to prevent the shell from interpreting them. Single quotes (') are generally safer than double quotes (") as they prevent all shell expansion.
  • Example: grep 'Hello world' is safer than grep Hello world.
• Binary Files: grep may behave unexpectedly or print garbage when searching binary files. Use -a or --text to treat them as text, but be aware of the output. Use -I (capital i) to ignore binary files.
• Permissions: grep will report "Permission denied" errors if it cannot read a file or directory. Redirecting stderr (2>/dev/null) can suppress these, but it’s often better to address the underlying permission issues.
• -r vs. -R: -r (recursive) and -R (recursive, follow symlinks) often behave identically on modern systems, but -R explicitly guarantees following symbolic links.
• Exit Status: grep returns an exit status of 0 if a match is found, 1 if no match is found, and 2 if an error occurred (e.g., file not found). This is useful in scripts.
  • Example: grep "pattern" file.txt && echo "Found" will print "Found" only if grep succeeds.