Information about E Mail Authentication
Ensuring a valid identity on an e-mail has become a vital first step in stopping spam, forgery, fraud, and even more serious crimes. An essential second step would be ensuring the entity has a good reputation. Unfortunately, the Simple Mail Transfer Protocol (SMTP) that handles most e-mail today was designed in the early 1980's when most Internet users were honest "techies" who expected others to be equally honest. This article will explain how e-mail identities are forged and the steps that are being taken now to prevent it.
[1].
Other than the sender's IP address, there is no verification of any information in an e-mail. It is quite easy for a spammer to make an exact copy of an e-mail from smithbarney.com, including a long complicated sequence of headers and a genuine logo in the body of an e-mail, then change the content to send readers to a website that appears to be genuine, but is actually a phishing scam designed to capture names, passwords, and credit card numbers.
So why can't the sender's IP address be used to identify the spammer? There are two problems. One is that spammers often work through forwarders to hide their IP addresses (see below). Another is that the sender is often a zombie that has been infected by a computer virus, and is programmed to send spam without the owner even knowing about it. There are millions of insecure home computers, and they have become a major source of spam.
Attempts to stop spam by blacklisting sender's IP addresses still allows a small percentage through[3]. Most IP addresses are dynamic, i.e. they are frequently changing. An ISP, or any organization directly connected to the Internet, gets a block of real Internet addresses when they register in the DNS. Within that block, they assign individual addresses to customers as needed. A dial-up customer may get a new IP address each time they connect. By the time that address appears on blacklists all over the world, the spammer will have new addresses for the next run. There are 4 billion possible IPv4 addresses on the Internet. The game of keeping up with these rapidly changing IP addresses has been facetiously called "whack-a-mole".
There are a number of things that ISPs have done to stop zombies and deliberate spamming by their customers. Infected computers can be cleared of viruses and patched to resist further infection. Outgoing e-mail can be monitored for any sudden increase in flow or in content that is typical of spam. Some ISPs have been quite successful[4], but others don't care to make the effort. With spam now over 80% of all e-mail traffic[5], we can expect that there will always be ISPs who are willing to provide services for spammers.
There are a number of ways to authenticate a sender's domain name ( SPF, CSV, SenderID, DomainKeys, DKIM ). All are very effective in stopping the kind of forgery now prevalent. None exclude the use of other methods, although SPF / DKIM, SPF / CSV, SPF / SenderID, and SenderID / DomainKeys pairwise appear to be competing for the same niches. The most widely used will likely be the ones that require the least effort on the part of ISPs and others currently operating public mail servers.
SPF, CSV, and SenderID authenticate just a domain name. DomainKeys and DKIM use a digital signature to authenticate a domain name and the entire content of a message. SPF and CSV can reject a forgery before any data transfer. SenderID and DomainKeys must see at least the headers, so the entire message must be transmitted. SPF has a problem with forwarders (that Sender Rewriting Scheme defines a fix for), SenderID also with mailing lists (see below). CSV is only about the HELO identity.
SPF, CSV, and SenderID work by tying a temporary IP address to a claimed domain name. Every incoming e-mail has an IP address that cannot be forged[6], a bunch of domain names in the e-mail headers, and a few more in the commands from the sender's SMTP server. The methods differ in which of these names to use as the sender's domain name. All of them can be faked, but what cannot be faked is a domain name held by a DNS server for that section of the Internet[7].
The simplest and by far most widely deployed authentication scheme begins with a reverse DNS lookup of the connecting IP address. If there is no answer, it's a safe bet that the address is not a legitimate sender. If there is an answer, a forward DNS lookup of that answer authenticates the sender if it returns the connecting IP address. In other words, we look up the name of the connecting IP address, and look up the IP address of that name, and they must match.
The procedure to authenticate is basically simple. When a request to deliver an e-mail arrives, the claimed sender's domain name is sent in a query to a high-level DNS server. That DNS server in turn, refers to lower level servers until an answer is found that is authoritative for the domain in question. The answer returned to the receiver includes the information to authenticate the e-mail. For SPF and SenderID, the query returns the IP addresses which are authorized to send mail on behalf of that domain. Typically there will be very few authorized SMTP sending addresses, even from a domain with millions of dynamically assignable IP addresses. For DomainKeys, the query returns the public key for the domain, which then validates the signature in an e-mail header. A successful validation proves the email originated from the same people responsible for the DNS servers for that domain (as only the domain owners would have proper access to the private key that matches the public key published in DNS), and neither the headers nor the body of the e-mail were altered on its way from the sender.
A spammer has no access to any of the connections between these DNS servers. Even if he were to falsify records in the DNS server for his own domain, he would not be able to forge records for someone else's domain name. When a spammer tries to send an e-mail claiming to be from amazon.com, for example, the receiver queries the .com DNS server, then a server in a secure building at Amazon. The IP address on the message from the spammer won't match any of Amazon's authorized IP addresses, and the e-mail can be rejected. Alternatively, the DomainKey will show the signature in the e-mail is invalid because it is highly unlikely the spammer has access to the amazon.com private key.
Use of the DNS database to register authentication information for a domain is relatively new. The new information is added to existing DNS records, and queries for this information are handled the same way as any other DNS query. Publishing authentication records in DNS is voluntary, and many domains probably won't bother. However, any legitimate domain, even those that don't intend to operate public mail servers, will most likely want to block others from using their name to forge e-mails. A simple code in their DNS record will tell the world, "Block all mail claiming to be from our domain. We have no public mail servers."
CSV limits its focus to one-hop authentications. SPF and SenderID have in essence the same limitation, they don't work directly behind the "border" ( MX ) of the receiver. For SPF forwarders to third parties could rewrite the Return-Path (MAIL FROM) in a similar way like mailing lists. This approach emulates the SMTP behaviour before RFC 1123 deprecated source routes; for a technical explanation see SRS.
For SenderID, forwarders to third parties and mailing lists are asked to add a Sender: or Resent-Sender: header. For many mailing lists, the former is already the case, but other forwarders avoid any modifications of the mail in addition to the mandatory Received-timestamp line.
Use of a forwarder prevents the receiver from directly seeing the sender's IP address. The incoming IP packets have only the forwarder's IP address. Two solutions are possible if one can trust all forwarders. Either one trusts the forwarder to authenticate the sender, or one trusts the forwarder to at least accurately record the incoming IP address and pass it on, so one can do their own authentication.
The situation gets complicated when there is more than one forwarder. A sender can explicitly authorize a forwarder to send on its behalf, in effect extending its boundary to the public Internet. A receiver can trust a forwarder that it pays to handle e-mail, in effect designating a new receiver. There may be additional "MTA relays" in the middle, however. These are sometimes used for administrative control, traffic aggregation, and routing control. All it takes is one broken link in the chain-of-trust from sender to receiver, and it is no longer possible to authenticate the sender.
Forwarders have one other responsibility, and that is to route Delivery Status Notices (DSNs) and spam bounces properly. Normal DSNs should be sent straight to an address chosen by the sender. Spam bounces should not be sent to any address that may be forged[8]. These bounces may go back by the same path they came, if that path has been authenticated.
..... Click the link for more information.
..... Click the link for more information.
Mail transfer
In a simple mail transfer, there are four key players: the author or originator of the e-mail, the sender or agent who first puts the e-mail on the public Internet, the receiver or agent who receives the e-mail from the Internet, and the recipient who is the person intended to read the e-mail.[2])
Other than the sender's IP address, there is no verification of any information in an e-mail. It is quite easy for a spammer to make an exact copy of an e-mail from smithbarney.com, including a long complicated sequence of headers and a genuine logo in the body of an e-mail, then change the content to send readers to a website that appears to be genuine, but is actually a phishing scam designed to capture names, passwords, and credit card numbers.
So why can't the sender's IP address be used to identify the spammer? There are two problems. One is that spammers often work through forwarders to hide their IP addresses (see below). Another is that the sender is often a zombie that has been infected by a computer virus, and is programmed to send spam without the owner even knowing about it. There are millions of insecure home computers, and they have become a major source of spam.
Attempts to stop spam by blacklisting sender's IP addresses still allows a small percentage through[3]. Most IP addresses are dynamic, i.e. they are frequently changing. An ISP, or any organization directly connected to the Internet, gets a block of real Internet addresses when they register in the DNS. Within that block, they assign individual addresses to customers as needed. A dial-up customer may get a new IP address each time they connect. By the time that address appears on blacklists all over the world, the spammer will have new addresses for the next run. There are 4 billion possible IPv4 addresses on the Internet. The game of keeping up with these rapidly changing IP addresses has been facetiously called "whack-a-mole".
There are a number of things that ISPs have done to stop zombies and deliberate spamming by their customers. Infected computers can be cleared of viruses and patched to resist further infection. Outgoing e-mail can be monitored for any sudden increase in flow or in content that is typical of spam. Some ISPs have been quite successful[4], but others don't care to make the effort. With spam now over 80% of all e-mail traffic[5], we can expect that there will always be ISPs who are willing to provide services for spammers.
Authenticating senders
E-mail authentication greatly simplifies and automates the process of identifying senders. After identifying and verifying a claimed domain name, it is possible to treat suspected forgeries with suspicion, reject known forgeries, and block e-mail from known spamming domains. It is also possible to "whitelist" e-mail from known reputable domains, and bypass content-based filtering, which always loses some valid e-mails in the flood of spam. The fourth category, e-mail from unknown domains, can be treated like we now treat all e-mail – increasingly rigorous filtering, return challenges to the sender, etc. Success of a domain-rating system should encourage reputable ISPs to stop their outgoing spam and get a good rating.There are a number of ways to authenticate a sender's domain name ( SPF, CSV, SenderID, DomainKeys, DKIM ). All are very effective in stopping the kind of forgery now prevalent. None exclude the use of other methods, although SPF / DKIM, SPF / CSV, SPF / SenderID, and SenderID / DomainKeys pairwise appear to be competing for the same niches. The most widely used will likely be the ones that require the least effort on the part of ISPs and others currently operating public mail servers.
SPF, CSV, and SenderID authenticate just a domain name. DomainKeys and DKIM use a digital signature to authenticate a domain name and the entire content of a message. SPF and CSV can reject a forgery before any data transfer. SenderID and DomainKeys must see at least the headers, so the entire message must be transmitted. SPF has a problem with forwarders (that Sender Rewriting Scheme defines a fix for), SenderID also with mailing lists (see below). CSV is only about the HELO identity.
SPF, CSV, and SenderID work by tying a temporary IP address to a claimed domain name. Every incoming e-mail has an IP address that cannot be forged[6], a bunch of domain names in the e-mail headers, and a few more in the commands from the sender's SMTP server. The methods differ in which of these names to use as the sender's domain name. All of them can be faked, but what cannot be faked is a domain name held by a DNS server for that section of the Internet[7].
The simplest and by far most widely deployed authentication scheme begins with a reverse DNS lookup of the connecting IP address. If there is no answer, it's a safe bet that the address is not a legitimate sender. If there is an answer, a forward DNS lookup of that answer authenticates the sender if it returns the connecting IP address. In other words, we look up the name of the connecting IP address, and look up the IP address of that name, and they must match.
)
The procedure to authenticate is basically simple. When a request to deliver an e-mail arrives, the claimed sender's domain name is sent in a query to a high-level DNS server. That DNS server in turn, refers to lower level servers until an answer is found that is authoritative for the domain in question. The answer returned to the receiver includes the information to authenticate the e-mail. For SPF and SenderID, the query returns the IP addresses which are authorized to send mail on behalf of that domain. Typically there will be very few authorized SMTP sending addresses, even from a domain with millions of dynamically assignable IP addresses. For DomainKeys, the query returns the public key for the domain, which then validates the signature in an e-mail header. A successful validation proves the email originated from the same people responsible for the DNS servers for that domain (as only the domain owners would have proper access to the private key that matches the public key published in DNS), and neither the headers nor the body of the e-mail were altered on its way from the sender.
A spammer has no access to any of the connections between these DNS servers. Even if he were to falsify records in the DNS server for his own domain, he would not be able to forge records for someone else's domain name. When a spammer tries to send an e-mail claiming to be from amazon.com, for example, the receiver queries the .com DNS server, then a server in a secure building at Amazon. The IP address on the message from the spammer won't match any of Amazon's authorized IP addresses, and the e-mail can be rejected. Alternatively, the DomainKey will show the signature in the e-mail is invalid because it is highly unlikely the spammer has access to the amazon.com private key.
Use of the DNS database to register authentication information for a domain is relatively new. The new information is added to existing DNS records, and queries for this information are handled the same way as any other DNS query. Publishing authentication records in DNS is voluntary, and many domains probably won't bother. However, any legitimate domain, even those that don't intend to operate public mail servers, will most likely want to block others from using their name to forge e-mails. A simple code in their DNS record will tell the world, "Block all mail claiming to be from our domain. We have no public mail servers."
Difficulties with e-mail forwarding
There are some additional details when an e-mail forwarder is involved. Forwarders perform a useful service in allowing you to have one simple permanent address, even if you change jobs or ISPs. List servers perform a similar function, forwarding e-mail to many receivers on behalf of one sender. Forwarders pose no problem for an end-to-end authentication method like DKIM and DomainKeys, as long as the signed message is not modified (some lists do this).CSV limits its focus to one-hop authentications. SPF and SenderID have in essence the same limitation, they don't work directly behind the "border" ( MX ) of the receiver. For SPF forwarders to third parties could rewrite the Return-Path (MAIL FROM) in a similar way like mailing lists. This approach emulates the SMTP behaviour before RFC 1123 deprecated source routes; for a technical explanation see SRS.
For SenderID, forwarders to third parties and mailing lists are asked to add a Sender: or Resent-Sender: header. For many mailing lists, the former is already the case, but other forwarders avoid any modifications of the mail in addition to the mandatory Received-timestamp line.
)
Use of a forwarder prevents the receiver from directly seeing the sender's IP address. The incoming IP packets have only the forwarder's IP address. Two solutions are possible if one can trust all forwarders. Either one trusts the forwarder to authenticate the sender, or one trusts the forwarder to at least accurately record the incoming IP address and pass it on, so one can do their own authentication.
The situation gets complicated when there is more than one forwarder. A sender can explicitly authorize a forwarder to send on its behalf, in effect extending its boundary to the public Internet. A receiver can trust a forwarder that it pays to handle e-mail, in effect designating a new receiver. There may be additional "MTA relays" in the middle, however. These are sometimes used for administrative control, traffic aggregation, and routing control. All it takes is one broken link in the chain-of-trust from sender to receiver, and it is no longer possible to authenticate the sender.
Forwarders have one other responsibility, and that is to route Delivery Status Notices (DSNs) and spam bounces properly. Normal DSNs should be sent straight to an address chosen by the sender. Spam bounces should not be sent to any address that may be forged[8]. These bounces may go back by the same path they came, if that path has been authenticated.
Criticism
- “Authentication cannot stop spam, unless the cop/Reputation Service/Certificate Authority in charge revokes certificates for spamming. If that could happen, then ISPs would also be willing and even enthusiastic about terminating accounts or otherwise controlling (e.g. port block) their spammers. If ISPs would do that, then there would be no spam to need authentication to stop spam and so need for a CA playing cop. As long as ISPs remain unwilling to police their own spamming customers, they would never deal with a CA willing to play cop.
- Authentication involving TLS, SMTP-AUTH, or S/MIME cannot stop backscatter for the same reasons SPF, DKIM, and the rest were, are, and always will be powerless against it. Some of those reasons are why Yahoo still does not sign DKIM on all outgoing mail, Hotmail still publishes whishywashy SPF RRs and neither requires their snakeoil forgery solution on incoming mail.?
- ::--Vernon Schryver (Distributed Checksum Clearinghouse operator)
References
1. ^ This is a complex and controversial topic. See also the original simpler article by David MacQuigg - [1]
2. ^ How mail flows through the Internet [2] (PNG)
3. ^ Spamhaus - effective spam filtering [3]
4. ^ America Online claims to have eliminated outgoing spam. A small sample of reports from SpamCop seems to validate this.
5. ^ Junk mail statistics [4] and [5]
6. ^ IP Address forgery is possible, but generally involves a lower level of criminal behavior ( breaking and entering, wiretapping, etc.), and these crimes are too risky for a typical hacker or spammer.
7. ^ There have been attacks on DNS servers, but doing this on a large scale over a long period of time may be orders of magnitude more difficult than spreading zombie infections among millions of insecure home computers. The much smaller number of DNS servers could be upgraded to use DNSSEC if such attacks were to become commonplace.
8. ^ SpamCop FAQ about spam bounces [6]
2. ^ How mail flows through the Internet [2] (PNG)
3. ^ Spamhaus - effective spam filtering [3]
4. ^ America Online claims to have eliminated outgoing spam. A small sample of reports from SpamCop seems to validate this.
5. ^ Junk mail statistics [4] and [5]
6. ^ IP Address forgery is possible, but generally involves a lower level of criminal behavior ( breaking and entering, wiretapping, etc.), and these crimes are too risky for a typical hacker or spammer.
7. ^ There have been attacks on DNS servers, but doing this on a large scale over a long period of time may be orders of magnitude more difficult than spreading zombie infections among millions of insecure home computers. The much smaller number of DNS servers could be upgraded to use DNSSEC if such attacks were to become commonplace.
8. ^ SpamCop FAQ about spam bounces [6]
See also
- SPF Sender Policy Framework
- CSV Certified Server Validation
- Sender ID
- DomainKeys Identified Mail DKIM
- DomainKeys
- Ident
- E-mail encryption
Spamming is the abuse of electronic messaging systems to indiscriminately send unsolicited bulk messages. While the most widely recognized form of spam is e-mail spam, the term is applied to similar abuses in other media: instant messaging spam, Usenet newsgroup spam, Web search
..... Click the link for more information.
..... Click the link for more information.
Simple Mail Transfer Protocol (SMTP) is the de facto standard for e-mail transmissions across the Internet. Formally SMTP is defined in RFC 821 (STD 10) as amended by RFC 1123 (STD 3) chapter 5. The protocol used today is also known as ESMTP and defined in RFC 2821.
..... Click the link for more information.
..... Click the link for more information.
phishing is an attempt to criminally and fraudulently acquire sensitive information, such as usernames, passwords and credit card details, by masquerading as a trustworthy entity in an electronic communication. eBay, PayPal and online banks are common targets.
..... Click the link for more information.
..... Click the link for more information.
zombie computer (often abbreviated zombie) is a computer attached to the Internet that has been compromised by a Hacker, a computer virus, or a trojan horse. Generally, a compromised machine is only one of many in a "botnet", and will be used to perform malicious tasks of
..... Click the link for more information.
..... Click the link for more information.
A computer virus is a computer program that can copy itself and infect a computer without permission or knowledge of the user. The original virus may modify the copies, or the copies may modify themselves, as occurs in a metamorphic virus.
..... Click the link for more information.
..... Click the link for more information.
A DNS Blacklist, or DNSBL (definition below), is a means by which an Internet site may publish a list of IP addresses that some people may want to avoid and in a format which can be easily queried by computer programs on the Internet.
..... Click the link for more information.
..... Click the link for more information.
On the Internet, the Domain Name System (DNS) associates various sorts of information with so-called domain names; most importantly, it serves as the "phone book" for the Internet by translating human-readable computer hostnames, e.g. en.wikipedia.
..... Click the link for more information.
..... Click the link for more information.
Internet Protocol version 4 is the fourth iteration of the Internet Protocol (IP) and it is the first version of the protocol to be widely deployed. IPv4 is the dominant network layer protocol on the Internet and apart from IPv6 it is the only standard internetwork-layer protocol
..... Click the link for more information.
..... Click the link for more information.
For the House episode, see .
Whac-A-Mole is a popular arcade redemption game invented in 1971 by Aaron Fechter of Creative Engineering, Inc. Fechter designed the first Whac-a-Mole and sold it to a carnival operator who sold it to Bob's Space Racers...... Click the link for more information.
In computing, Sender Policy Framework (SPF) is an extension to the Simple Mail Transfer Protocol (SMTP). SPF allows software to identify and reject forged addresses in the SMTP MAIL FROM (Return-Path), a typical nuisance in e-mail spam.
..... Click the link for more information.
..... Click the link for more information.
Certified Server Validation (CSV) is a technical method of Email authentication intended to fight spam. Its focus is the SMTP HELO-identity of Mail transfer agents.
CSV was designed to address the problems of MARID and the ASRG, as defined in detail as the intent of
..... Click the link for more information.
CSV was designed to address the problems of MARID and the ASRG, as defined in detail as the intent of
..... Click the link for more information.
DomainKeys is an e-mail authentication system designed to verify the DNS domain of an e-mail sender and the message integrity. The DomainKeys specification has adopted aspects of Identified Internet Mail to create an enhanced protocol called DomainKeys Identified Mail (DKIM).
..... Click the link for more information.
..... Click the link for more information.
digital signature or digital signature scheme is a type of asymmetric cryptography used to simulate the security properties of a signature in digital, rather than written, form.
..... Click the link for more information.
..... Click the link for more information.
An MX record or Mail exchanger record is a type of resource record in the Domain Name System (DNS) specifying how Internet e-mail should be routed. MX records point to the servers that should receive an e-mail, and their priority relative to each other.
..... Click the link for more information.
..... Click the link for more information.
..... Click the link for more information.
Distributed Checksum Clearinghouse (also referred to as DCC), is a hash sharing method of spam email detection. The basic logic in DCC is that most spam mails have several copies floating around.
..... Click the link for more information.
..... Click the link for more information.
SpamCop (www.SpamCop.net) is a free spam reporting service, allowing recipients of unsolicited bulk email (UBE) and unsolicited commercial email (UCE) to report the offense to the sender's Internet Service Provider (ISP), and sometimes their web host.
..... Click the link for more information.
..... Click the link for more information.
The Domain Name System Security Extensions (DNSSEC) are a suite of IETF specifications for securing certain kinds of information provided by the Domain Name System (DNS) as used on Internet Protocol (IP) networks.
..... Click the link for more information.
..... Click the link for more information.
In computing, Sender Policy Framework (SPF) is an extension to the Simple Mail Transfer Protocol (SMTP). SPF allows software to identify and reject forged addresses in the SMTP MAIL FROM (Return-Path), a typical nuisance in e-mail spam.
..... Click the link for more information.
..... Click the link for more information.
Certified Server Validation (CSV) is a technical method of Email authentication intended to fight spam. Its focus is the SMTP HELO-identity of Mail transfer agents.
CSV was designed to address the problems of MARID and the ASRG, as defined in detail as the intent of
..... Click the link for more information.
CSV was designed to address the problems of MARID and the ASRG, as defined in detail as the intent of
..... Click the link for more information.
Sender ID is an anti-spam proposal from the former MARID IETF working group that joined Sender Policy Framework (SPF) and Caller ID. It was accepted as an Experimental RFC by the Internet Engineering Task Force, as is, on December 8th, 2005.
..... Click the link for more information.
..... Click the link for more information.
DomainKeys is an e-mail authentication system designed to verify the DNS domain of an e-mail sender and the message integrity. The DomainKeys specification has adopted aspects of Identified Internet Mail to create an enhanced protocol called DomainKeys Identified Mail (DKIM).
..... Click the link for more information.
..... Click the link for more information.
Ident Protocol, specified in RFC 1413, is an Internet protocol that helps identify the user of a particular TCP connection. One popular daemon program for providing the ident service is identd.
..... Click the link for more information.
..... Click the link for more information.
E-mail encryption refers to encryption, and often authentication, of e-mail messages. E-mail encryption usually relies on public-key cryptography.
..... Click the link for more information.
E-mail encryption protocols
Popular protocols for e-mail encryption include:- S/MIME
- OpenPGP
..... Click the link for more information.
This article is copied from an article on Wikipedia.org - the free encyclopedia created and edited by online user community. The text was not checked or edited by anyone on our staff. Although the vast majority of the wikipedia encyclopedia articles provide accurate and timely information please do not assume the accuracy of any particular article. This article is distributed under the terms of GNU Free Documentation License.
Herod_Archelaus
