25-06-2012, 05:00 PM
Phinding Phish: Evaluating Anti-Phishing Tools
Phinding Phish Evaluating Anti-Phishing Tools.pdf (Size: 1.39 MB / Downloads: 28)
Abstract
There are currently dozens of freely available tools
to combat phishing and other web-based scams, many
of which are web browser extensions that warn users
when they are browsing a suspected phishing site. We
developed an automated test bed for testing antiphishing
tools. We used 200 verified phishing URLs
from two sources and 516 legitimate URLs to test the
effectiveness of 10 popular anti-phishing tools. Only
one tool was able to consistently identify more than
90% of phishing URLs correctly; however, it also
incorrectly identified 42% of legitimate URLs as phish.
The performance of the other tools varied considerably
depending on the source of the phishing URLs. Of
these remaining tools, only one correctly identified
over 60% of phishing URLs from both sources.
Performance also changed significantly depending on
the freshness of the phishing URLs tested. Thus we
demonstrate that the source of phishing URLs and the
freshness of the URLs tested can significantly impact
the results of anti-phishing tool testing. We also
demonstrate that many of the tools we tested were
vulnerable to simple exploits. In this paper we describe
our anti-phishing tool test bed, summarize our
findings, and offer observations about the effectiveness
of these tools as well as ways they might be improved.
1. Introduction
Over the past few years we have seen an increase in
“semantic attacks” — computer security attacks that
exploit human vulnerabilities rather than software
vulnerabilities. Phishing is a type of semantic attack in
which victims are sent emails that deceive them into
providing account numbers, passwords, or other
personal information to an attacker. Typical phishing
emails falsely claim to be from a reputable business
where victims might have an account. Victims are
directed to a spoofed web site where they enter
information such as credit card numbers or Social
Security Numbers. There were 9,255 unique phishing
sites reported in June of 2006 alone [1]. Billions of
dollars are lost each year due to unsuspecting users
entering personal information into fraudulent web
sites. To respond to this threat, software vendors and
companies with a vested interest in preventing
phishing attacks have released a variety of “antiphishing
tools.” For example, eBay offers a free tool
that can positively identify the eBay site, and Google
offers a free tool aimed at identifying any fraudulent
site [9], [12]. As of September 2006, the free software
download site Download.com, listed 84 anti-phishing
tools. Unfortunately, few empirical studies have been
performed to examine the effectiveness of these tools.
Thus, while many anti-phishing tools exist, it is not
clear how well they actually work.
Previous studies have examined the extent to which
users fall for phishing scams and whether users benefit
from the information provided by anti-phishing tools.
These studies have shown that most users are likely to
fall for phishing scams, and that many users ignore
warnings provided by anti-phishing tools [7], [8], [13],
[25]. However, little empirical data is available on the
accuracy of these tools or on the effectiveness of the
various approaches to detecting phishing sites.
Towards that end, this paper makes three research
contributions. First, we describe the design and
implementation of a test bed for automatically
evaluating anti-phishing tools. Second, we describe the
results of experiments that assess the accuracy of 10
popular anti-phishing tools that use differing
techniques to identify phishing sites. Third, we
describe techniques we developed for circumventing
many of the tools tested. Our paper provides the antiphishing
community with insights into the
effectiveness of several approaches to combating
phishing as well as a methodology for testing antiphishing
tools.
2. Overview of Anti-Phishing Tools
There are a variety of methods that can be used to
identify a web page as a phishing site, including
whitelists (lists of known safe sites), blacklists (lists of
known fraudulent sites), heuristics, and community
ratings. The tools examined in this study employ
differing combinations of these methods. We used
publicly available information provided on the tool
download web sites as well as our observations to get a
basic understanding of how each tool functions.
2.1. CallingID Toolbar
The CallingID Toolbar, shown in Figure 1, boasts
its use of 54 different verification tests in order to
determine the legitimacy of a given site. Like many of
the other toolbars, CalingID relies on passive visual
indicators. These indicators change from green—to
represent a known-good site; to yellow—to represent a
site that is “low risk;” to red—to represent a site that is
“high risk,” and therefore probably a phishing site.
Some of the heuristics used include examining the
site’s country of origin, length of registration,
popularity, user reports, and blacklist data. The
CallingID Toolbar runs on Microsoft Windows
98/NT/2000/XP with Internet Explorer [2].
2.2. Cloudmark Anti-Fraud Toolbar
The Cloudmark Anti-Fraud Toolbar, shown in Figure
2, relies on user ratings [4]. When visiting a site, users
have the option of reporting the site as good or bad.
Accordingly, the toolbar will display a colored icon for
each site visited. Green icons indicate that the site has
been rated as legitimate, red icons indicate that the site
has been determined to be fraudulent, and yellow icons
indicate that not enough information is known to make
a determination. Additionally, the users themselves are
rated according to their record of correctly identifying
phishing sites. Each site’s rating is computed by
aggregating all ratings given for that site, with each
user’s rating of a site weighted according to that user’s
reputation. No other heuristics are used in determining
a site’s rating. Sites determined to be fraudulent are
blocked and users are redirected to an information page
and given the option of overriding the block. The
Cloudmark Anti-Fraud Toolbar runs on Microsoft
Windows 98/NT/2000/XP with Internet Explorer.
After our study began we learned that Cloudmark is no
longer supporting this toolbar. Cloudmark has since
removed this toolbar from their web site. They now
offer a phishing URL feed for other toolbars and
similar applications and a tool called Cloudmark
Desktop that works in conjunction with the Microsoft
Outlook and Microsoft Outlook Express email clients
and labels phishing emails based on millions of reports
from users each day. We have not tested Cloudmark
Desktop.
2.3. EarthLink Toolbar
The EarthLink Toolbar, shown in Figure 3, appears
to rely on a combination of heuristics, user ratings, and
manual verification. Little information is presented on
the EarthLink website; however, we used the toolbar
and observed how it functions. The toolbar allows
users to report suspected phishing sites to EarthLink.
These sites are then verified and added to a blacklist.
The toolbar also appears to examine domain
Figure 1: The CallingID Toolbar indicating a low-risk site.
Figure 2: The Cloudmark Anti-Fraud Toolbar indicating a legitimate site.
Figure 3: The EarthLink Toolbar indicating a legitimate site.
registration information such as the owner, age, and
country. The toolbar displays a thumb that changes
color and position. A green thumbs up represents a
verified legitimate site, whereas a gray thumbs up
means that the site is not suspicious, but it has not been
verified. The red thumbs down means that a site has
been verified to be fraudulent, whereas the yellow
thumbs down means that the site is “questionable.”
Sites determined to be fraudulent are sometimes
blocked, in which case users are redirected to an
information page and given the option of overriding
the block (and a green thumb is displayed on the
information page). The EarthLink Toolbar runs under
Internet Explorer as well as Firefox [10].
2.4. eBay Toolbar
The eBay Tool, shown in Figure 4, uses a
combination of heuristics and blacklists [9]. The
Account Guard indicator has three modes: green, red,
and gray. The icon is displayed with a green
background when the user visits a site known to be
operated by eBay (or PayPal). The icon is displayed
with a red background when the site is a known
phishing site. The icon is displayed with a gray
background when the site is not operated by eBay and
not known to be a phishing site. Known phishing sites
are blocked and a pop-up appears, giving users the
option to override the block. The toolbar also gives
users the ability to report phishing sites, which will
then be verified before being blacklisted. The eBay
Toolbar runs under Microsoft Windows
98/ME/NT/2000/XP with Internet Explorer.