Regular Expressions

Written by Ferhawn Shaheen (10/2/2020)

Introduction

Regular expressions, or regex, are a way of easily searching for specific strings within text. This is very important when one is working with larger and more complex programs. It consists of several characters that can be used in tandem to enter a specific query. This lesson will cover the basics of regex.

Definition and a Basic Example

Regexing specifically uses patterns with characters and sequences that can yield a specific result throughout the text. The following is a basic example.

import re

# Check if the string starts with "The" and ends with "Spain":

txt = "The rain in Spain"
match = re.search("^The.*Spain$", txt)

if match:
  print("YES! We have a match!")
else:
  print("No match.")

The re module must be imported to use regular expressions generally. A string is then defined followed by the most important statement, which uses re.search() to specify what is going to be searched for. The first argument in the function is the actual regex string. A simple if-else statement ensues, which works with the results of the re.search() function.

The regex string ^The.*Spain$ uses metacharacters, which are defined further in the following section.

• The ^ means that the start of txt must match the regex, and not somewhere in the middle. E.g., the string "This is The Spain" won't be a match.

• $ ensures that the end of txt must match the regex, and not somewhere else. E.g., the string"The king of Spain is a man" won't match.

• The . and * are two different metacharacters. . means that there can be any single character after the characters T, h, and e. The * means "zero-or-more" occurrences of any characters. When combined with together in the form .*, this means that there can be any number of any characters.

  • E.g., "TheSpain", "TheASpain", "TheAAASpain", and "TheABCSpain" all match.

Metacharacters

Metacharacters hold a special meaning when using them. Here is a list of the ones that are used with regex.

Character	Description	Example
[]	A set of characters	"[a-m]"
\	Signals a special sequence (can also be used to escape special characters. Will be covered below)	"\d"
.	Any character (except newline character)	"he..o"
^	Starts with	"^hello"
$	Ends with	"world$"
*	Zero or more occurrences	"aix*"
+	One or more occurrences	"aix+"
{}	Exactly the specified number of occurrences	"al{2}"
|	Either or	"falls|stays"
()	Capture and group	"^The(.*)Spain$", captures and returns everything between "The" and "Spain". Use the result of re.findall() to iterate through all of the matches. E.g., ["", "A", "AAA", "ABC"].

Special Sequences

Special sequences are used in tandem with the \ metacharacter. They, like the metacharacters, have specific meanings.

Set	Description	Example
\A	Returns a match if the specified characters are at the beginning of the string	"\AThe"
\b	Returns a match where the specified characters are at the beginning or at the end of a word (the "r" in the beginning is making sure that the string is being treated as a "raw string")	r"\bain" r"ain\b"
\B	Returns a match where the specified characters are present, but NOT at the beginning (or at the end) of a word (the "r" in the beginning is making sure that the string is being treated as a "raw string")	r"\Bain" r"ain\B"
\d	Returns a match where the string contains digits (numbers from 0-9)	"\d"
\D	Returns a match where the string DOES NOT contain digits	"\D"
\s	Returns a match where the string contains a white space character	"\s"
\S	Returns a match where the string DOES NOT contain a white space character	"\S"
\w	Returns a match where the string contains any word characters (characters from a to Z, digits from 0-9, and the underscore _ character)	"\w"
\W	Returns a match where the string DOES NOT contain any word characters	"\W"
\Z	Returns a match if the specified characters are at the end of the string	"Spain\Z"

Sets

Sets are a set of characters inside brackets that also denote a specific meaning.

Set	Description
[arn]	Returns a match where one of the specified characters (a, r, or n) are present
[a-n]	Returns a match for any lower case character, alphabetically between a and n
[^arn]	Returns a match for any character EXCEPT a, r, and n
[0123]	Returns a match where any of the specified digits (0, 1, 2, or 3) are present
[0-9]	Returns a match for any digit between 0 and 9
[0-5][0-9]	Returns a match for any two-digit numbers from 00 and 59
[a-zA-Z]	Returns a match for any character alphabetically between a and z, lower case OR upper case

Functions to Use

findall()

The findall() function returns a list of all the items that match the given specifications.

import re

txt = "The rain in Spain"
match = re.findall("ai", txt)
print(match)  # Returns ["ai", "ai"]

Here is another example. Note that an empty list is returned if a desired item does not exist.

import re

txt = "The rain in Spain"
match = re.findall("Portugal", txt)
print(match)  # Returns []

sub()

The sub() function replaces the matches with a text of choice.

import re

txt = "The rain in Spain"
match = re.sub("\s", "9", txt)
print(match)  # Returns "The9rain9in9Spain"

It is worth noting that count controls the number of substitutions.

import re

txt = "The rain in Spain"
match = re.sub("\s", "9", txt, 2)
print(match)  # Returns "The9rain9in Spain"

Important Note

This lesson contains a lot of material to work with. It is highly advised some time is spent to apply and learn what to do with all of the metacharacters, sequences, and sets. Of course, there is more to look into if one does online research.

This online regex tester is also very helpful when learning regex: https://regex101.com/

Conclusion

This lesson covered the basics of regex. It can be very helpful and useful throughout Python. There is certainly a lot of material to cover and learn, so doing more research and having some time to practice is very helpful.