01-03-2013, 02:27 PM
The Usability of Electronic Voting Machines and How Votes Can Be Changed Without Detection
[attachment=52313]
ABSTRACT
The problems in the 2000 election in Florida focused national attention on the
need for usable voting systems. As a result, the Help America Vote Act (HAVA) of 2002
provided funding for updating voting equipment and many states purchased direct
recording electronic (DRE) systems. Although these electronic systems have been widely
adopted, they have not been empirically proven to be more usable than their predecessors
in terms of ballot completion times, error rates, or satisfaction levels for the average
voter. The series of studies reported here provides usability data on DREs to compare
with that of previous voting technologies (paper ballots, punch cards, and lever
machines). Results indicate that there are not differences between DREs and older
methods in efficiency or effectiveness. However, in terms of user satisfaction, the DREs
are significantly better than the older methods. Paper ballots also perform well, but
participants are much more satisfied with their experiences voting on the DREs.
These studies also go beyond usability comparisons and test whether voters notice
if their final ballots on the DRE reflect choices other than what the voters selected.
Results indicate that over 60% of voters do not notice if their votes as shown on the
review screen are different than how they were selected.
INTRODUCTION
The problems in the 2000 election in Florida focused national attention on the
need for usable voting systems. As the country became familiar with terms such as
“butterfly ballot” and “hanging chads,” many states realized the importance of replacing
these systems to avoid such problems in future elections. The Help America Vote Act
(HAVA) of 2002 provided funding for updating voting equipment and intended for states
to replace their outdated voting methods with newer, more reliable systems. As a result of
this legislation, many states have spent millions to purchase direct recording electronic
(DRE) systems to replace older technologies such as punch cards and lever machines.
In the 2006 election, it was estimated that over 66 million people would be voting
on DREs in 34% of the nation’s counties. While this is less than the percent of counties
using paper ballots and optical scan technology (54%), DREs were much more widely
used than punch cards at 4% of counties and also lever machines at 4%. The replacement
of older voting systems with newer, electronic ones has been steadily increasing since
2000 when they were used in a mere 10% of counties, to 18% in 2002, to 21% in 2004,
and finally to the previously mentioned 34% in 2006 (Election Data Services, 2006).
VOTING METHODS
Traditional Methods and History
Before discussing current and future directions in voting, it is worth surveying
what voting methods have previously been used. One of the oldest and most traditional
forms of voting, dating back to 1629 in the United States alone, is to cast a vote using a
paper ballot. While paper ballots may seem simple, there are numerous forms of these
ballots and they have undergone tremendous changes. Former versions of voting by paper
ballot have required the voter to write in the candidate’s name and have allowed political
parties to distribute pre-printed ballots. For this latter form of paper ballot, the only
actions required on the part of the voter were to bring this pre-printed ballot to the polling
place and deposit it in a ballot box (Jones, 2003).
In 1888, Australian ballots were introduced for the first time in the United States.
These were early versions of the current paper ballots. The Australian ballot lists all races
and all the candidates running in these contests. Various forms of the paper ballot have
required voters to scratch out the names of candidates they do not want, or more
commonly, to put an X or other mark besides the name of their selected candidate (Jones,
2003). Three forms of paper ballots currently used in the United States include the openresponse
ballot, arrow ballot, and bubble ballot. Examples of these ballots can be seen in
Figures 1, 2, and 3. A special form of paper ballot, the optical scan ballot, can be read by
a machine and does not require hand counting.
USABILITY AND HUMAN FACTORS
Why Is Usability Important?
When discussing voting systems, it is immediately obvious that issues such as
accuracy and security are important. However, there is another facet of voting systems
that can be overlooked: usability. Why is usability so critical in voting systems? To
ensure the integrity of elections, voters must actually be able to cast their votes as
intended. Unintentional undervotes (i.e., not casting a vote in a race), overvotes (i.e.,
voting for more candidates in a race than is allowed), or votes for the wrong candidates
can substantially impact elections, as evidenced in the 2000 election upset in Florida.
Wand et al. (2001) showed that the butterfly ballot used in Palm Beach County, Florida
caused over 2,000 voters to vote for Pat Buchanan instead of Al Gore, tipping the
election to George W. Bush. In another analysis of this election, Mebane (2004) used
ballot-level data to show that 50,000 votes intended for Bush or Gore were lost to
overvotes. He claims that had technology been available to warn voters of overvotes,
Gore would have won by more than 30,000 votes. Similarly, Herron and Sekhon (2003)
focused on presidential overvotes in the 2000 election and found that many more of the
overvotes came from voters with a tendency to vote Democratically further down the
ballot. In a user test performed soon after the 2000 election, Sinclair, Mark, Moore,
Lavis, and Soldat (2000) found experimental evidence that the butterfly ballot was indeed
more confusing than a single column ballot and that its use led to systematic errors in a
mock election.
Assessing Usability
To assess the usability of voting systems, the National Institute of Standards and
Technology (NIST) recommends using the International Standards and Organization’s
(ISO 9241-11, 1998) metrics of efficiency, effectiveness, and satisfaction (Laskowski,
Autry, Cugini, Killam, & Yen, 2004). As NIST is responsible for setting voting system
testing standards, it is important for research on the usability of such systems to use these
metrics.
The efficiency of a system can be determined by studying the relationship
between the level of effectiveness achieved and the amount of resources used (Industry
Usability Reporting Project, 2001). The level of efficiency of a system can be measured
objectively, usually by recording time on task. In the voting domain, efficient systems
will take an acceptable amount of time and effort by the voters to cast votes for intended
candidates.
SECURITY
Security Background
To ensure the integrity of our elections, the voting process and its outcomes must
be accurate and secure. Although DREs have been touted as the best solution for accurate
elections, many computer security experts have expressed major concerns about their
susceptibility to fraud and errors (e.g., Dill 2005). A study of the source code for one type
of DRE, the Diebold AccuVote-TS revealed many security flaws that could be exploited
to affect an election outcome (Kohno, Stubblefield, Rubin, & Wallach, 2004). These
included bugs that allowed voters to cast multiple votes without being later detected and
allowing voters administrator access to the systems, as discussed below. Another in-depth
study of the AccuVote-TS revealed that malicious code could be easily installed on
voting machines on Election Day, and could spread like a computer virus, infecting other
machines through normal voting activities (Feldman, Halderman, & Felten, 2006).
The Hack-a-Vote project from a computer security class at Rice University
(Bannet, Price, Rudys, Singer, & Wallach, 2004) demonstrated how easily even students
could introduce hacks into a voting system. Students incorporated several types of hacks
into voting machine code that could affect the final result. Hackers then switched to the
role of auditors and had difficulty finding the bugs in others’ software. This project
demonstrated what many security experts have been concerned about: how easily
attackers could insert malicious code into systems and steal elections.
[attachment=52313]
ABSTRACT
The problems in the 2000 election in Florida focused national attention on the
need for usable voting systems. As a result, the Help America Vote Act (HAVA) of 2002
provided funding for updating voting equipment and many states purchased direct
recording electronic (DRE) systems. Although these electronic systems have been widely
adopted, they have not been empirically proven to be more usable than their predecessors
in terms of ballot completion times, error rates, or satisfaction levels for the average
voter. The series of studies reported here provides usability data on DREs to compare
with that of previous voting technologies (paper ballots, punch cards, and lever
machines). Results indicate that there are not differences between DREs and older
methods in efficiency or effectiveness. However, in terms of user satisfaction, the DREs
are significantly better than the older methods. Paper ballots also perform well, but
participants are much more satisfied with their experiences voting on the DREs.
These studies also go beyond usability comparisons and test whether voters notice
if their final ballots on the DRE reflect choices other than what the voters selected.
Results indicate that over 60% of voters do not notice if their votes as shown on the
review screen are different than how they were selected.
INTRODUCTION
The problems in the 2000 election in Florida focused national attention on the
need for usable voting systems. As the country became familiar with terms such as
“butterfly ballot” and “hanging chads,” many states realized the importance of replacing
these systems to avoid such problems in future elections. The Help America Vote Act
(HAVA) of 2002 provided funding for updating voting equipment and intended for states
to replace their outdated voting methods with newer, more reliable systems. As a result of
this legislation, many states have spent millions to purchase direct recording electronic
(DRE) systems to replace older technologies such as punch cards and lever machines.
In the 2006 election, it was estimated that over 66 million people would be voting
on DREs in 34% of the nation’s counties. While this is less than the percent of counties
using paper ballots and optical scan technology (54%), DREs were much more widely
used than punch cards at 4% of counties and also lever machines at 4%. The replacement
of older voting systems with newer, electronic ones has been steadily increasing since
2000 when they were used in a mere 10% of counties, to 18% in 2002, to 21% in 2004,
and finally to the previously mentioned 34% in 2006 (Election Data Services, 2006).
VOTING METHODS
Traditional Methods and History
Before discussing current and future directions in voting, it is worth surveying
what voting methods have previously been used. One of the oldest and most traditional
forms of voting, dating back to 1629 in the United States alone, is to cast a vote using a
paper ballot. While paper ballots may seem simple, there are numerous forms of these
ballots and they have undergone tremendous changes. Former versions of voting by paper
ballot have required the voter to write in the candidate’s name and have allowed political
parties to distribute pre-printed ballots. For this latter form of paper ballot, the only
actions required on the part of the voter were to bring this pre-printed ballot to the polling
place and deposit it in a ballot box (Jones, 2003).
In 1888, Australian ballots were introduced for the first time in the United States.
These were early versions of the current paper ballots. The Australian ballot lists all races
and all the candidates running in these contests. Various forms of the paper ballot have
required voters to scratch out the names of candidates they do not want, or more
commonly, to put an X or other mark besides the name of their selected candidate (Jones,
2003). Three forms of paper ballots currently used in the United States include the openresponse
ballot, arrow ballot, and bubble ballot. Examples of these ballots can be seen in
Figures 1, 2, and 3. A special form of paper ballot, the optical scan ballot, can be read by
a machine and does not require hand counting.
USABILITY AND HUMAN FACTORS
Why Is Usability Important?
When discussing voting systems, it is immediately obvious that issues such as
accuracy and security are important. However, there is another facet of voting systems
that can be overlooked: usability. Why is usability so critical in voting systems? To
ensure the integrity of elections, voters must actually be able to cast their votes as
intended. Unintentional undervotes (i.e., not casting a vote in a race), overvotes (i.e.,
voting for more candidates in a race than is allowed), or votes for the wrong candidates
can substantially impact elections, as evidenced in the 2000 election upset in Florida.
Wand et al. (2001) showed that the butterfly ballot used in Palm Beach County, Florida
caused over 2,000 voters to vote for Pat Buchanan instead of Al Gore, tipping the
election to George W. Bush. In another analysis of this election, Mebane (2004) used
ballot-level data to show that 50,000 votes intended for Bush or Gore were lost to
overvotes. He claims that had technology been available to warn voters of overvotes,
Gore would have won by more than 30,000 votes. Similarly, Herron and Sekhon (2003)
focused on presidential overvotes in the 2000 election and found that many more of the
overvotes came from voters with a tendency to vote Democratically further down the
ballot. In a user test performed soon after the 2000 election, Sinclair, Mark, Moore,
Lavis, and Soldat (2000) found experimental evidence that the butterfly ballot was indeed
more confusing than a single column ballot and that its use led to systematic errors in a
mock election.
Assessing Usability
To assess the usability of voting systems, the National Institute of Standards and
Technology (NIST) recommends using the International Standards and Organization’s
(ISO 9241-11, 1998) metrics of efficiency, effectiveness, and satisfaction (Laskowski,
Autry, Cugini, Killam, & Yen, 2004). As NIST is responsible for setting voting system
testing standards, it is important for research on the usability of such systems to use these
metrics.
The efficiency of a system can be determined by studying the relationship
between the level of effectiveness achieved and the amount of resources used (Industry
Usability Reporting Project, 2001). The level of efficiency of a system can be measured
objectively, usually by recording time on task. In the voting domain, efficient systems
will take an acceptable amount of time and effort by the voters to cast votes for intended
candidates.
SECURITY
Security Background
To ensure the integrity of our elections, the voting process and its outcomes must
be accurate and secure. Although DREs have been touted as the best solution for accurate
elections, many computer security experts have expressed major concerns about their
susceptibility to fraud and errors (e.g., Dill 2005). A study of the source code for one type
of DRE, the Diebold AccuVote-TS revealed many security flaws that could be exploited
to affect an election outcome (Kohno, Stubblefield, Rubin, & Wallach, 2004). These
included bugs that allowed voters to cast multiple votes without being later detected and
allowing voters administrator access to the systems, as discussed below. Another in-depth
study of the AccuVote-TS revealed that malicious code could be easily installed on
voting machines on Election Day, and could spread like a computer virus, infecting other
machines through normal voting activities (Feldman, Halderman, & Felten, 2006).
The Hack-a-Vote project from a computer security class at Rice University
(Bannet, Price, Rudys, Singer, & Wallach, 2004) demonstrated how easily even students
could introduce hacks into a voting system. Students incorporated several types of hacks
into voting machine code that could affect the final result. Hackers then switched to the
role of auditors and had difficulty finding the bugs in others’ software. This project
demonstrated what many security experts have been concerned about: how easily
attackers could insert malicious code into systems and steal elections.