System, Method and Apparatus for Efficient Testing of Mobile Applications against a Plurality of Environments
Publication Date: 2016-Nov-08
The IP.com Prior Art Database
We propose an effective solution to the problem of testing an application against a plurality of environments and faulty inputs, which consists of two principal steps: (1) Fingerprinting: the act of checking how the behavior of the program may be sensitive to, or influenced by, distinctions pertaining to the environment. (2) Forking: The act of splitting the testing scenario into two or more ?child? scenarios, where each of the resulting scenarios is specialized toward a given set of assumptions regarding the environment. Our solution also has significant advantages in the context of fuzz-testing.
Page 01 of 3
Method and Apparatus for Efficient Testing of Mobile Applications against a
Method and Apparatus for Efficient Testing of Mobile Applications against a Plurality of Environments
Background. Software systems are often deployed on more than one platform and/or hardware infrastructure. Perhaps the most notable example is mobile applications, and in particular hybrid mobile applications, which are expected to support over a thousand different hardware
devices. Added to this, there are approximately 20 different versions of the Android
platform currently in (popular) use, which yields an overwhelming number of testing configurations to ensure that a native Android application behaves adequately in realworld deployment scenarios.
This situation is well beyond what can be managed via manual testing. A team of QA engineers cannot be expected to scale its effort to thousands of different environments. At the same time, there is no automated solution at present that features the ability to perform this style of analysis [CROSS-PLAT].
Clearly any naive approach to this problem, whereby the application is tested against some or all of the candidate environments is unsatisfactory. If only few environments are tried, then coverage is poor. If, on the other hand, all environments are tried, then performance would become the bottleneck, which is particularly problematic given the mobile dev ecosystem is characterized by short release cycles.
Summary. We propose an effective solution to the problem stated above of testing an application against a plurality of environments, which consists of two principal steps:
1. Fingerprinting: The act of checking how the behavior of the program may be sensitive
to, or influenced by, distinctions pertaining to the environment (e.g., if the program checks whether it's being executed on an emulator or a physical device)
2. Forking: The act of splitting the testing scenario into two or more "child" scenarios, where each of the resulting scenarios is specialized toward a given set of assumptions regarding the environment
Combined, these two steps yield a procedure whereby the testing system responds to fingerprinting observations via forking, such that distinctions drawn by the application w.r.t. its environment are handled on demand without the need to either limit coverage arbitrarily or eagerly test against all possible environments.
Specifically, in the context of fuzz testing, there are significant advantages to this approach:
1. Different faulty inputs can be tried thanks to the forking mechanism, where fingerprinting guides the selection of those inputs.
2. Thanks to forking, there is no need to recreate the execution trace up to the injection point. The test payloads can be injected directly.
We note, importantly, that we have implemented our invention for Android. Our implementation is based on a Java implementation of the Dalvik VM [DALVIK]. This
enables efficient VM-level forking without needing to rely...