05-05-2011, 03:13 PM
Abstract
As smart phones grow in popularity, manufacturersare in a race to pack an increasingly rich set of featuresinto these tiny devices. This brings additional complexity inthe system software that has to fit within the constraintsof the devices (chiefly memory, stable storage, and powerconsumption) and hence, new bugs are revealed. How thisevolution of smartphones impacts their reliability is a questionthat has been largely unexplored till now. With the releaseof open source OSes for hand-held devices, such as, Android(open sourced in October 2008) and Symbian (open sourced inFebruary 2010), we are now in a position to explore the abovequestion. In this paper, we analyze the reported cases of failuresof Android and Symbian based on bug reports posted by thirdpartydevelopers and end users and documentation of bugfixes from Android developers. First, based on 628 developerreports, our study looks into the manifestation of failures indifferent modules of Android and their characteristics, suchas, their persistence and dependence on environment. Next,we analyze similar properties of Symbian bugs based on 153failure reports. Our study indicates that Development Tools,Web Browsers, and Multimedia applications are most errorpronein both these systems. We further analyze 233 bugfixes for Android and categorized the different types of codemodifications required for the fixes. The analysis shows that77% of errors required minor code changes, with the largestshare of these coming from modifications to attribute valuesand conditions. Our final analysis focuses on the relationbetween customizability, code complexity, and reliability inAndroid and Symbian. We find that despite high cyclomaticcomplexity, the bug densities in Android and Symbian aresurprisingly low. However, the support for customizability doesimpact the reliability of mobile OSes and there are cautionarytales for their further development.Keywords-smartphone, reliability, root cause, bug fix, customizability
I. INTRODUCTION
In an interview with BBC in February 2008, AndyRubin, Google’s director of mobile platforms, commented“There should be nothing that users can access on theirdesktop that they can’t access on their cell phone.” [1].This vision of bridging the gap between desktop computersand hand-held devices, brings with it new challenges inthe expanding smartphone market. As dreams turn intoreality, the new age mobile devices have transformed intominiaturized entertainment consoles connected to the globalinformation backbone, the Internet. However, every newfeature comes with a cost—cost of new software, cost ofmemory, cost of battery consumption, user frustration dueto failures, etc. In this paper, we analyze the “cost” ofsmartphones from the point of view of failures. We seekto answer the question how is the reliability of a mobileoperating system impacted when the developers offer afeature-rich and highly configurable system, such as Android(from Google) and Symbian (from Nokia and SymbianFoundation). The relation between complexity and reliabilityin traditional operating systems has been well studied [2],[3], [4], [5], [6]. But how the evolution of smartphones,with the increase in complexity, impacts their reliability is aquestion that has been largely unexplored till now. This kindof system introduces an important additional dimension tothe question, namely, the significant resource constraints—computation power, memory, battery, and display real estate.Their interface with an wide range of sensors, e.g. camera,accelerometer, and GPS, some of which may bring in copiousstreams of data, puts additional demands on the mobileOS. With the release of Android, an open-source operatingsystem for mobile devices (open sourced in October 2008)and Symbian (open sourced in February 2010), we arenow in a position to analyze their dependability and seewhat lessons we can draw for modern, highly customizable,mobile operating systems.Our case study focuses on Android and Symbian inpart due to the enormous interest shown by cellphonemanufacturers and application developers in them. Symbianat present holds the largest market share for mobile OSeson smartphones (46.9%) [7], whereas Android is predictedto be the second largest mobile OS by 2012 [8]. Theprogrammability features of both these platforms empowerus with a high degree of flexibility unforeseen in hand-helddevices. We believe that reliability analysis of such platformswill be of value to the developers and researchers in directingtheir efforts at building a user-friendly and robust frameworkunder tight resource constraints.Several researchers have analyzed failure characteristicsof popular operating systems like Windows and Linux [3],[4]. But evaluations of mobile operating systems are rarelyseen. A notable exception is the study of failures in SymbianOS-based smart phones [9]. However, since Symbian wasnot open source at the time of the analysis, the authorsof this earlier study were limited in what they could do.
Download full report
https://engineering.purdue.edu/dcsl/publ...submit.pdf
As smart phones grow in popularity, manufacturersare in a race to pack an increasingly rich set of featuresinto these tiny devices. This brings additional complexity inthe system software that has to fit within the constraintsof the devices (chiefly memory, stable storage, and powerconsumption) and hence, new bugs are revealed. How thisevolution of smartphones impacts their reliability is a questionthat has been largely unexplored till now. With the releaseof open source OSes for hand-held devices, such as, Android(open sourced in October 2008) and Symbian (open sourced inFebruary 2010), we are now in a position to explore the abovequestion. In this paper, we analyze the reported cases of failuresof Android and Symbian based on bug reports posted by thirdpartydevelopers and end users and documentation of bugfixes from Android developers. First, based on 628 developerreports, our study looks into the manifestation of failures indifferent modules of Android and their characteristics, suchas, their persistence and dependence on environment. Next,we analyze similar properties of Symbian bugs based on 153failure reports. Our study indicates that Development Tools,Web Browsers, and Multimedia applications are most errorpronein both these systems. We further analyze 233 bugfixes for Android and categorized the different types of codemodifications required for the fixes. The analysis shows that77% of errors required minor code changes, with the largestshare of these coming from modifications to attribute valuesand conditions. Our final analysis focuses on the relationbetween customizability, code complexity, and reliability inAndroid and Symbian. We find that despite high cyclomaticcomplexity, the bug densities in Android and Symbian aresurprisingly low. However, the support for customizability doesimpact the reliability of mobile OSes and there are cautionarytales for their further development.Keywords-smartphone, reliability, root cause, bug fix, customizability
I. INTRODUCTION
In an interview with BBC in February 2008, AndyRubin, Google’s director of mobile platforms, commented“There should be nothing that users can access on theirdesktop that they can’t access on their cell phone.” [1].This vision of bridging the gap between desktop computersand hand-held devices, brings with it new challenges inthe expanding smartphone market. As dreams turn intoreality, the new age mobile devices have transformed intominiaturized entertainment consoles connected to the globalinformation backbone, the Internet. However, every newfeature comes with a cost—cost of new software, cost ofmemory, cost of battery consumption, user frustration dueto failures, etc. In this paper, we analyze the “cost” ofsmartphones from the point of view of failures. We seekto answer the question how is the reliability of a mobileoperating system impacted when the developers offer afeature-rich and highly configurable system, such as Android(from Google) and Symbian (from Nokia and SymbianFoundation). The relation between complexity and reliabilityin traditional operating systems has been well studied [2],[3], [4], [5], [6]. But how the evolution of smartphones,with the increase in complexity, impacts their reliability is aquestion that has been largely unexplored till now. This kindof system introduces an important additional dimension tothe question, namely, the significant resource constraints—computation power, memory, battery, and display real estate.Their interface with an wide range of sensors, e.g. camera,accelerometer, and GPS, some of which may bring in copiousstreams of data, puts additional demands on the mobileOS. With the release of Android, an open-source operatingsystem for mobile devices (open sourced in October 2008)and Symbian (open sourced in February 2010), we arenow in a position to analyze their dependability and seewhat lessons we can draw for modern, highly customizable,mobile operating systems.Our case study focuses on Android and Symbian inpart due to the enormous interest shown by cellphonemanufacturers and application developers in them. Symbianat present holds the largest market share for mobile OSeson smartphones (46.9%) [7], whereas Android is predictedto be the second largest mobile OS by 2012 [8]. Theprogrammability features of both these platforms empowerus with a high degree of flexibility unforeseen in hand-helddevices. We believe that reliability analysis of such platformswill be of value to the developers and researchers in directingtheir efforts at building a user-friendly and robust frameworkunder tight resource constraints.Several researchers have analyzed failure characteristicsof popular operating systems like Windows and Linux [3],[4]. But evaluations of mobile operating systems are rarelyseen. A notable exception is the study of failures in SymbianOS-based smart phones [9]. However, since Symbian wasnot open source at the time of the analysis, the authorsof this earlier study were limited in what they could do.
Download full report
https://engineering.purdue.edu/dcsl/publ...submit.pdf