05-05-2010, 09:56 AM
WEB SPAM.ppt (Size: 825.5 KB / Downloads: 387)
WEB SPAM
Economic considerations
Search has become the default gateway to the web
Very high premium to appear on the first page of search results
e.g., e-commerce sites
advertising-driven sites
What is web spam
Spamming = any deliberate action solely in order to boost a web pageâ„¢s position in search engine results, regardless of the pageâ„¢s real value
Spam = web pages that are the result of spamming
Spammer
Approximately 10-15% of web pages are spam
Web spam taxonomy
Boosting techniques
Techniques to achieve high relevance or importance for some pages
Hiding techniques
Techniques to hide the adopted boosting techniques from human web users and crawlers
Boosting techniques
Term spamming
Manipulating the text/fields of web pages in order to appear relevant to queries
Link spamming
Creating link structures that boost page rank
Term spamming
Target algorithms
[b]
Techniques[/b]
Repetition
of one or a few specific terms e.g., free, cheap
increase relevance for a document with respect to a small number of query terms
Dumping
of a large number of unrelated terms
e.g., copy entire dictionaries
Weaving
Copy legitimate pages and insert spam terms at random positions
Phrase Stitching
Glue together sentences and phrases from different sources
Term spam targets
Body of web page
Title
HTML meta tags
Anchor text
URL
Link spam
There are three kinds of web pages from a spammerâ„¢s point of view
Inaccessible pages
Accessible pages
e.g., web log comments pages
spammer can post links to his pages
Own pages
Completely controlled by spammer
May span multiple domain names
Target algorithms
Hypertext Induced Topic Search (HITS) algorithm
¢ Hub scores
¢ A spammer should add many outgoing links to the target page t to increase its hub score
¢ Authority scores
¢ Having many incoming links from presumably important hubs
PageRank
¢ Uses incoming link information to assign numerical weights to all pages on the web
¢ Numerical weight that it assigns to any given element E is also called the PageRank of E and denoted by PR(E)
¢ Spammers manipulate the algorithm using links
Targets and Techniques
Outgoing links
¢ Directory cloning
Incoming links
¢ Create a honey pot
¢ Infiltrate a web directory
¢ Post links to unmoderated message boards or guest books
¢ Participate in link exchange
¢ Create own spam farm
Link Farms
A link farm is a densely connected set of pages, created explicitly with the purpose of deceiving a link-based ranking algorithm (such as Googleâ„¢s PageRank)
Spammerâ„¢s goal
Maximize the page rank of target page t
Technique
Get as many links from accessible pages as possible to target page t
Construct link farm to get page rank multiplier effect
Link Farms
One of the most common and effective organizations for a link farm
Hiding techniques
Content hiding
Use same color for text and page background
Spam terms or links on a page can be made invisible when the browser renders the page
Cloaking
Return different page to crawlers and browsers
Redirection
Alternative to cloaking
Redirects are followed by browsers but not crawlers
Detecting Web Spam
Term spamming
Analyze text using statistical methods
Similar to email spam filtering
Also useful: detecting approximate duplicate pages
Link spamming
Open research area
One approach: TrustRank
TrustRank idea
Basic principle: approximate isolation
It is rare for a good page to point to a bad (spam) page
Sample a set of seed pages from the web
Have an oracle (human) identify the good pages and the spam pages in the seed set
Expensive task, so must make seed set as small as possible
Trust Propagation
Call the subset of seed pages that are identified as good the trusted pages
Set trust of each trusted page to 1
Propagate trust through links
Each page gets a trust value between 0 and 1
Use a threshold value and mark all pages below the trust threshold as spam
Picking the seed set
Human has to inspect each seed page, so seed set must be as small as possible
Must ensure every good page gets adequate trust rank, so need make all good pages reachable from seed set by short paths
Approaches to picking seed set
Suppose we want to pick a seed set of k pages
PageRank
Pick the top k pages by page rank
Assume high page rank pages are close to other highly ranked pages
We care more about high page rank good pages
Rules for trust propagation
Trust attenuation
The degree of trust conferred by a trusted page decreases with distance
Trust splitting
The larger the number of outlinks from a page, the less scrutiny the page author gives each outlink
Trust is split across outlinks
Fighting against web spam
Identify instances of spam
Prevent spamming
Counterbalance
THANK YOU