Regular Expression for beginner

In Automation, the most frequently data type we need to handle is String. And Regular expressions are extremely useful and powerful.

Let’s summary what Regular Expression can help us:

1. extracting information from any text by searching or replacing for one or more matches of a specific search pattern
2. String validation. eg: checking it is a valid email or phone number

What is Regular expression ?

It simply is a sequence of characters that forms a search pattern. A regular expression can be a single character, or a more complicated pattern. It can be used to perform all types of text search and text replace operations.

When to use Regular Expression

For simple extracting info like get exactly 4 last characters of a string,

eg: “XXXX XXXX XXXX 4242” I can use built-in methods of String object such as subString(i,j), relace(“a”, “b”), split(delimiter)…

But normally we have to deal with more complex scenarios than above, extracting info from a dynamic String structure, validating string format, Regular Expression is the answer.

eg: extracting number amout in “Total Price: (40.00) $” where currency symbol may be €, £, ¥ …

How to use Regular Expression

To do the matching job with a search pattern, we will need a regular expression engine. For example, if you are using Java, you need to import java util regex.

import java.util.regex.Matcher;
import java.util.regex.Pattern;
Pattern p = Pattern.compile("(\w+)://([\w.]+)");
Matcher m = p.matcher("https://www.google.com");
// use if condition if there is only 1 match, while for multi matches.
while (m.find()) {  
    String schema = m.group(1); // https
    String site= m.group(2); // www.google.com
}

in Python:

import re
result = re.match(pattern, string, flags=0)
result = re.search(pattern, string, flags=0)
results = re.findall(pattern, string, flags=0) # multi matches

Now, let’s learn how to write a regular expression pattern from basic to advance:

First, below is basic string matching:

The word that I want to match is 'welcome'. And some other stuff...

regex pattern : welcome -> matched string: welcome

OR Operator — use character | or [ ]

cat mat and bat are the things that I love.

regex: (c|m|b)at  or [cmb]at -> matched string: cat, mat, bat

Ranging and Bracket Expressions — [a-z, etc….]

Usage:

[a-z] -> Match all characters from a to z (small letters)
[A-Z] -> Match all characters from A-Z (capital letters)
[0-9] -> Match all numbers from 0-9
[r-y] -> Match any small letter between r and y
[D-M] -> Match any capital letter between D and M
[2-7] -> Match any number between 2 and 7
Combining expressions:
[a-zA-Z] -> Match all letters (any small or capital letter)
[a-zA-Z0-9] -> Match all letters and numbers (any small or capital letter or number)

You can also match any character EXCEPT for a single or range of characters.

[^a-z] -> Match anything that is not between a to z(small letters) = No small letters
[^A-Z] -> Match anything that is not between A to Z (capital letters) = No capital letters
[^0-9] -> Match anything that is not between 0-9 = No numbers
[^e-v] -> Match anything that is not between e and v 
Combining expressions:
[^a-zA-Z] -> No alphabets = Match only numbers
[^a-z0-9] -> Match anything except small letters and numbers = Match only capital letters and other characters.

eg: The name of this river is: Ganga. 

regex pattern: [A-Z]  -> matched string: T, G

Quantifiers

Quantifiers are regular expressions metacharacters which can be used to specify how many instances of groups, characters, bracket expressions, character ranges, etc. must be present in the input string.

yes* -> Match the string which has 'ye' followed by zero or more 's'
yes+ -> Matches the string which has 'ye' followed by one or more 's'
yes? -> Matches the string which has 'yes' or 'ye'. The character 's' is optional.
yes{3} -> Matches the string which has 'ye' followed by exactly 3 's'
yes{2,} -> Matches the string which has 'ye' followed by 2 or more 's'
yes{2,4} -> Matches the string which has 'ye' followed by 2 to 4 's'
(yes)* -> Matches the string which has zero or more 'yes'
(ye)+s -> Matches the string which has one or more 'ye' and at end 's'
*? -> Matches between zero and unlimited times, as few times as possible, expanding as needed

example:

website: www.google.com website is nice
www.facebook.com is a nice website
link: www.google.co.in temporary testing

regex pattern:  www.[a-z]+\.(\.?[a-z]+)+  
matched strings: www.google.com, www.facebook.com, www.google.co.in

Character Classes

\d -> Matches any number.
\D is the opposite of what \d does. It matches everything else, except for numbers. It will match alphabets, symbols, etc.
\w -> Matches any alphabet or number. Same behaviour as: [a-zA-Z0-0] bracket expression. \w+ -> match only word
\s -> Matches any whitespace character. This also includes matching \r:Carriage Return, \n:new line(line feed), \t:(horizontal tab) characters.
. -> Matches anything. Any alphabet, number, character... anything.

Anchors ^ and $ (Start and end)

.*Successful!$ 

will not match "Morning's Task: Successful! Still won't match"

Grouping and Capturing

Grouping and capturing are one of the most important concepts of regular expressions. They are one of the most essential tools to structure the pattern matching result and make groups of the pattern matched. 

Groups are generally captured and non-captures. Captured groups are saved in the result of a regular expression search result array. But, a non-captured group isn’t. Captured groups are automatically named numerically according to the group number. We can also manually name them by using a specific syntax.

https://www.google.co.in is a nice website

regex pattern: (\w+)://([\w.]+) 

Match: https://www.google.co.in
 Group 1: https
 Group 2: www.google.co.in

Nested Groups numbering:

Number is: (200)192 168 1234
regex pattern :     \((\d+)\)(\d{3} \d{3} \d{4})
Match 1: (200)192 168 1234
Group 1: 200
Group 2: 192 168 1234

Non-capturing groups:

Programmer234
regex pattern:   ([a-zA-Z]+)(?:\d+)

As we used ‘?:’ in the group, this group is a non-capturing group and will not be captured as a separate group. This regular expression will match a simple username with some alphabets in the beginning and a number at the end. Only the alphabets part will be captured and the number won’t be captured.

Match 1: Hacker123
Group 1: Hacker

Named Capturing Groups(explicitly named groups)

Syntax: 
(?<group_name>regular_expression)
OR
(?'named_group'regular_expression)

eg: "His Birth date: 10-12-1990"

(?<date>\d{2})-(?<month>\d{2})-(?<year>\d{4})

Match : 10-12-1990
Group "date": 10
Group "month": 12
Group "year": 1990

Word boundaries (\b)

Word boundaries are the boundaries between a word and a non-word character. In other words, it is a boundary between \w(any alphabet or number) and its opposite \W(any non-alphabet or non-number). Word boundaries can be used to match a complete word or to match everything within 2 boundaries(non-word characters).

Capital Letter word.
This is another Capital Letter Word.

regex: \b[A-Z]\w*\b    --> only match word with Capital letter

Match 1: Capital
Match 2: Letter
Match 3: This
Match 4: Capital
Match 5: Letter
Match 6: Word

Source Links:

https://blog.usejournal.com/regular-expressions-a-complete-beginners-tutorial-c7327b9fd8eb
https://regexr.com/
https://regex101.com/