The one with Email Verifier in Python.

Backstory

It was the year 2019 when I was trying to solve a problem where most of the emails getting sent from service were getting bounced as the email addresses provided were incorrect. I decided to come up with a solution that will verify the email addresses before the actual emails were sent to the users. A few approaches were discussed, and some of them which made the cut are:-

  • OTP approach :- sending an OTP to all the email addresses we have and the ones coming new to the system and asking the user to enter the OTP once they log in to the app.

  • A programmatic approach :- Verifying existing set of emails we have in our DB as well to mark them valid/invalid and follow the same approach for the newly coming emails on the fly in a pythonic way.

  • Using an existing solution :- Using an existing library such as email-validator that will help us with the validation and verification of emails.

The first approach was supposed to ask the user to re-login to the application which was discouraged by the initial set of beta users on whom this approach was tested. The third approach, however the most reliable one, did not offer much learning for me. Hence, I decided to go with the second approach.

Approach

The email verifier contains two parts:-

  • Email Syntax Validation :- the part where we check if the email getting checked, obeys the basic syntax for correct email.

  • Email Delivery Part :- the part where we check if the email actually exists in the records and the delivery is actually possible.

Email Syntax Validation:-

The first thing was to check the emails present in the system and what is coming through user interaction were legit emails and not some junk value.

For the ones coming through user interaction, a simple regex check on the form field helped users to enter clean and syntax-correct email addresses. For the emails in the database, we built the script starting with a regex check.


regex = '^[_a-z0-9-]+(\.[_a-z0-9-]+)*@[a-z0-9-]+(\.[a-z0-9-]+)*(\.[a-z]{2,})$'

Email Delivery Part:-

Post the email syntax verification, the next part was to check the actual validity of the email. For this, we need to start with MX Record(Mail Exchange)for the domain in the email address. To resolve the MX record, I used dnspython library.


import dns.resolver

a_records = dns.resolver.query(email_address_to_check)
mx_record = str(records[0].exchange)

Once we get the MX record for a domain, the next thing for us was to check if the email exists by issuing SMTPcommands HELO, MAIL FROM and RCPT TO. I used python’s smtplibLibrary for this. The idea was to check the response of RCPT TO command. If the status is 250 we can confirm that the email is a valid one and exists in the system. If we get any other status codes we are safe to say that the email does not exist on that particular system.

import smtplib
server = smtplib.SMTP(timeout=6)
server.set_debuglevel(0)
from_address = 'email@domain.com'

domain = str(addressToVerify.split('@')[1])
        try:
            records = dns.resolver.query(domain, 'MX')
            mx_record = str(records[0].exchange)
            server.connect(mx_record)
            server.helo(server.local_hostname) 
            server.mail(from_address)
            code, message = server.rcpt(str(addressToVerify))
            server.quit()
        except:
            return False
    if code == 250:
        return True
    else:
        return False

The above two steps combined helped me in verifying close to 100K email addresses with a ~2% false negative.

Challenges and Takeaways:-

While this method worked for me, given the time and a limited number of emails to verify. I observed many cases where the script was getting blocked due to spam rules or returning False for a valid email. Now in those cases, I manually updated the email address to be valid.

This method will not be helpful in a production-grade environment or in cases where we are processing millions of email addresses to check their validity.

Cheers, Pratik

Did you find this article valuable?

Support Pratik Anurag by becoming a sponsor. Any amount is appreciated!