DARKFLEECE: Probing the Dark Side of Android Subscription Apps

Chang Yue

1,2

, Chen Zhong

3

, Kai Chen

1,2

*, Zhiyu Zhang

1,2

School of Cyber Security, University of Chinese Academy of Sciences, China

3

University of Tampa, USA

leading to substantial financial consequences. These apps’

ambiguous nature, closely resembling legitimate subscrip-

tion apps, complicates their detection in app markets. To

address this, our study aims to devise an automated method,

named DARKFLEECE, to identify fleeceware through their

prevalent use of dark patterns. By recruiting domain ex-

perts, we curated the first-ever fleeceware feature library,

based on dark patterns extracted from user interfaces (UI).

A unique extraction method, which integrates UI elements,

layout, and multifaceted extraction rules, has been devel-

different levels of fleeceware, totaling around 5 billion down-

loads. Our results are consistent with user reviews on Google

Play. Our detailed exploration into the implications of our

results for ethical app developers, app users, and app market

regulators provides crucial insights for different stakeholders.

This underscores the need for proactive measures against the

rise of fleeceware.

1 Introduction

The integration of subscription models into mobile apps has

witnessed significant growth in recent years. Subscription

*

The corresponding authors.

may subscribe to apps without their consent, fail to realize

they will be charged after a free trial, or struggle to cancel

a subscription. Recent reports have highlighted subscription

scams [14, 30, 47, 50, 58]. According to Avast, 69 fleece-

ware apps generated $38.5 million in revenue in 2019 [4].

fleeceware is imperative to minimize its harmful effects.

However, these apps typically do not contain malicious code

ual, which is neither timely nor scalable. This work is ded-

icated to investigating fleeceware at scale through the devel-

opment of an automatic fleeceware detection system.

dark patterns [10, 50], which are deceptive interface designs

intended to manipulate users into paying subscription fees

beyond their initial intention. Therefore, we mainly focus on

identifying these dark patterns in UIs for fleeceware detec-

tion. An app with dark pattern features doesn’t automatically

qualify as fleeceware, so we utilize a model that combines

DARKFLEECE, we mainly addressed two challenges.

C1: Constructing a Fleeceware Feature Library. Imple-

menting an automatic fleeceware detection system requires

recognizing the distinct features of fleeceware. There is a

lack of research on identifying detectable features that can

accurately describe fleeceware, and the availability of a well-

labeled dataset for fleeceware is limited. Despite extensive

research on dark patterns, the automated detection of fleece-

ware remains elusive due to the complexities and variabilities

intrinsic to fleeceware patterns. Drawing a clear line between

fleeceware and legitimate apps is far from straightforward.

Android and front-end development. They analyzed 1,486

user comments on fleeceware, identified seven common com-

plaints, and executed 79 fleeceware samples, uncovering var-

ious phenomena in subscription UI. Finally, we extracted en-

tities and attributes from the phenomena and identified 19 UI

features for fleeceware detection.

guage. The most intricate aspect of the detection is extract-

ing the values of the identified features from an app’s UI.

Most of our features are based on the information in the form

standing of the relationship between the UI elements and the

overall layout, as well as the capability to handle subscrip-

tion information displayed in diverse formats.

To address C2, we developed a method that factors in both

the UI elements and their layout information, incorporating

multiple extraction rules. This method allows us to derive

necessary information from the UI. Specifically, we devel-

oped a novel layout-based approach to link related subscrip-

tion information by identifying neighboring widgets, which

are descendant widgets of a parent widget like “ViewGroup”

to describe subscription information, with numerical values

consistently positioned within the information. We then con-

structed a collection of regular expressions to extract sub-

scription details in various forms. This combined approach

accurately extracts the target information for each UI feature.

With C1 and C2 addressed, we built a decision tree model

for fleeceware detection. We chose this model because of its

ability to incorporate our domain knowledge during feature

engineering and its inherent model interpretability. To make

it more intuitive for ordinary users to identify which inter-

face features may present potential fleeceware risks, we em-

ployed SHAP, an Explainable Artificial Intelligence (XAI)

technique, to provide and visualize the explanations.

Fleeceware Measurement. Owing to our automated detec-

tion capabilities, we are able to quantify the presence of

fleeceware in the wild. We downloaded 13,597 apps from

Google Play, spanning across all app categories, and iden-

tified 589 subscription-based apps. Within an approximate

time frame of 10.12 minutes per app, we classified 443 of

these apps, representing 75.21%, as suspected fleeceware,

which may potentially cause unanticipated subscription costs

Google’s app policy violation reporting platform to bring the

problems to their attention.

Contributions. To sum up, the contributions are three-fold.

• We construct the first UI feature library for fleeceware

detection, which is the result of collaboration with domain

experts who possess in-depth knowledge of user interac-

tions with fleeceware behaviors. They meticulously analyzed

fleeceware samples and integrated insights from studies on

dark patterns and platform-specific UI design requirements.

• We develop a novel technique that merges layout-based in-

from around $1.5 billion to $17.1 billion [12]. As subscrip-

tions gain popularity, they also become vulnerable to abuse

developer [49, 55], static scanning [60, 64], and dynamic ex-

ecution [11, 25], to detect common malware. In addition,

researchers have developed machine learning-based methods

to detect new types of malware [17, 35, 41, 61]. However, the

majority of these methods are ineffective at detecting fleece-

ware as fleeceware mainly utilizes hidden or contradictory

content on the UI to confuse users, and exhibit coding pat-

terns that closely resemble legitimate apps, without resorting

sively studied [10, 19, 21, 29, 40] in the human-computer

interaction (HCI) and user experience (UX) fields. They are

carefully crafted, deceptive interface design patterns that ma-

nipulate users into taking actions or making decisions they

did not intend to make [10]. Gray et al. [29] have proposed

a comprehensive categorization of dark patterns, including

nagging, obstruction, sneaking, interface interferences, and

forced action, as shown in Table 2.

mation difficult to notice using small font sizes and incon-

spicuous colors (Interface Interference), which gives the im-

pression that the app will not automatically charge fees. In

sum, fleeceware leverages dark patterns to orchestrate sub-

scription app scams, with most of these dark patterns observ-

able in the subscription UI, leading to users’ financial detri-

ment. This observation motivates us to detect fleeceware by

identifying these dark patterns within the subscription UI.

3 DARKFLEECE

3.1 Threat Model

Our system is specifically designed to address a growing con-

cern in the landscape of subscription apps: the use of decep-

tive UI designs (i.e., dark patterns) by app developers. These

practices exploit gaps in user awareness and cognitive biases,

leading to unwanted subscriptions and financial losses for

users. To address these fleeceware problems, our work fo-

ware Detector), which methodically examines the UI of sub-

scription apps to detect any existing dark patterns. The sys-

tem framework is depicted in Figure 2. Feature Library

is a key component, as the effectiveness of the system can

greatly depend on its quality. This library is constructed by

domain experts, drawing from observations of user interac-

tions with fleeceware behaviors and a review of fleeceware

samples, and integrates insights from dark pattern studies

vest target subscription information and obtain the feature

values. Then a Detector module applies a trained classifier

to detect potential fleeceware and utilizes an Explainable Ar-

tificial Intelligence (XAI) technique to provide explanations

for the outcomes to make it easier for users to notice and un-

derstand the issues on the UI. As users provide feedback, an

Evaluator module continually optimizes and updates the fea-

ture library to meet user needs. We will provide a detailed

Table 3: Typical fleeceware behaviors users complain about

No. Behaviors Reviews from Google Play

component below.

3.3 Knowledge-Based Feature Construction

Fleeceware identification necessitates a meticulous examina-

tion of fleeceware behaviors along with a solid understand-

ing of dark patterns. Our first step involves gathering data on

fleeceware behavior by accumulating user observations. Us-

ing these observations as a basis, we enlist domain experts to

analyze user-observed behaviors. Additionally, experts an-

alyze and execute a set of fleeceware to unearth any poten-

tial fleeceware patterns that may not have been observed by

experts. This group was composed of two contributing au-

carefully selected through our professional networks to en-

sure a diverse yet highly specialized skill set was represented.

The group consisted of four researchers and six engineers,

Behaviors

Violation of

Requirements

Category of

Dark Patterns

1 The entire interface lacks price information. B1, B2 R1, R2 sneaking

2 The entire interface lacks billing frequency information. B1, B2, B3 R1, R2 sneaking

3 If there is a free trial, the entire interface does not inform about the trial period. B1, B2 R2, R4 sneaking

4 The entire interface does not indicate whether the subscription will auto-renew. B4 R1, R2, R4 sneaking

5 If there is a free trial, it does not specify the charges after the trial. B1, B2 R4 sneaking

6 The total cost for a complete billing cycle is not provided. B3 R1 sneaking

7 The total charge is not highlighted when multiple price formats are presented. B3 R1 interface interference

between subscription information and the buttons.

sneaking,

10

The subscription information is not prominent (e.g., small font size, similar color to the

background, lack of emphasis in font style, buried within lengthy paragraphs).

B1, B2, B3 R1

sneaking,

11 The close button is not clearly visible. B6 R1

obstruction,

forced action

12

B7 - interface interference

ples alongside the documented fleeceware behaviors and met

twice to discuss them. During the initial meeting, the experts

summarized the documented fleeceware behaviors using the

dark patterns in Table 3 to gain an understanding of the dis-

tribution of dark patterns in user observations. After the

meeting, to discover potentially unobserved dark patterns,

the experts further examined and ran 79 fleeceware samples,

comprising 34 iOS apps and 45 Android apps. The results

were discussed in the second meeting, and in cases of dif-

fering results, the experts engaged in discussions to reach an

and whether the subscription will automatically renew. It is

imperative that these critical pieces of information be explic-

itly stated as required by the platforms (refer to the require-

ments in Table 1), as they play a crucial role in enabling

tion choices. The lack of these subscription details is iden-

tified in the subscription interface of the fleeceware samples

(No.1-No.4 in Table 4).

redirect users’ attention away from critical subscription in-

formation (No.5-No.10). This can result in users inadver-

tently overlooking or misunderstanding the subscription de-

tails provided. Figure 3(d) showcases an interface that only

presents a monthly price for a 12-month subscription (No.6),

potentially misleading users regarding the long-term costs

they will incur upon subscribing. Similarly, if a UI high-

lights the weekly price rather than the total price of the cur-

rent subscription (No.7), it potentially causes users to under-

estimate the actual charges involved. Figure 3(b) claims a

tons are essential elements within a UI, serving as guides for

user actions and triggering specific functionalities or naviga-

fied by experts, extract the relevant entities and attributes,

and formalize them into 19 features, as outlined in Table 6

in the Appendix C. Each feature serves as an indicator for

specific aspects of the fleeceware subscription UIs. Specif-

ically, Features F

the presence of key subscription information for phenom-

ena No.1-No.5 and No.9, respectively. Feature F

rived from Phenomenon No.6 and signifies the observation

that these interfaces often display more text describing the

billing frequency than the actual price. Phenomenon No.7 is

assessed using Feature F

, which calculates the ratio of the

font size of the total price to other prices, as well as examines

whether the font is bold, indicating if the total price informa-

tion is adequately highlighted. To determine the presence of

Phenomenon No.8, we utilize F

, which compares whether

these two time are consistent. To evaluate the proximity be-

the visibility and noticeability of the information. Further-

more, Features F

utilize optical character recognition

ingly. (1) Handling introductory screens: In some apps,

users are initially presented with a series of screens highlight-

ing the app’s features. To progress to the next page leading

to the subscription UI, users need to click on buttons like

teract with these buttons first. (2) Scrolling down the inter-

face: During the introductory screens or other app UIs, users

may need to scroll down to reveal the button or content neces-

sary to proceed. To ensure that all contents are loaded before

interacting, we prioritize the “ScrollEvent” in DroidBot’s in-

teractive events, allowing it to scroll down the page if neces-

sary. (3) Handling user input: Some app UIs require users

to fill in or select specific information before proceeding to

the subscription UI. To handle this, we give higher priority to

the “SetTextEvent” that fills text boxes in DroidBot, which

ensures that DroidBot enters the required information before

keywords include terms like “subscribe”, “upgrade”, “VIP”,

Each feature is composed of specific UI information, such as

the presence of certain subscription details, text length, font

size, etc. For example, to calculate F

all subscription price information on the UI and extract the

length or font size of the information and the whole sentence.

concatenate these discrete pieces of information from the en-

tire UI. (2) Variable Display Formats: For example, the

charge information can be presented as “6 Months: $59.99”

or “59.99 USD/6-Months”. The diversity of information ex-

pression forms and the lack of dataset make it challenging for

machine learning-based natural language processing meth-

ods to accurately differentiate between different subscription

information and extract the values from them.

ing widgets. These widgets are descendants of a parent wid-

get whose widget class is “ViewGroup”, “RelativeLayout”,

gets containing charge information are neighboring widgets,

box). Therefore, by searching for all the neighboring widgets

of each widget, we can link all the related information.

rule-based approach to extract target information, avoiding

subscription information, and the numerical values appear

in specific positions within the information. For example,

the price and billing cycle information always appear to-

gether, within which, there will always be numerical values,

currency symbols (e.g., “$”, “USD”), and time units (e.g.,

der. Therefore, we construct a collection of regular expres-

sions (available on our website[44]) to extract subscription

details expressed in various forms. We detail how to gener-

ate the regular expressions and how to use the layout-based

formation, we need to combine all the related subscription

information in a UI. For each widget, we find its closest par-

ent widget with the target class, i.e., if a widget’s parent w

does not fall into the specified class, we continue searching

for the parent of w

descendant widgets are considered as neighboring widgets.

ply the regular expressions to get the target information.

However, directly applying the regular expressions to the

combined texts may lead to many mismatches. This is be-

cause irrelevant information can add noise to the original

texts and impact the accuracy of extraction. Moreover, when

multiple useful pieces of information appear together, it may

cause either information loss or incorrect information com-

bination. To prevent mismatches, we consider performing

real-time target information extraction during the process of

widget with a small view ID comes first in a UI. We link the

text in widgets in sequence, one by one. Assuming the pre-

Visual information extraction. The proposed features in-

clude not only textual information but also visual informa-

tion about the UI elements, such as font size, font style, and

visibility. With the help of DroidBot, we managed to ob-

Levenshtein distance measures the minimum number of edit

operations required to convert one string into the other, and

the Levenshtein ratio is calculated by

sum−ldist

, where sum is

the total length of the two strings and ldist is the Levenshtein

distance between them. If the Levenshtein distance or Lev-

enshtein ratio is below (or above) a predetermined threshold,

we consider the strings to be a match and the target informa-

tion to be clearly visible, and we can get the font size and

style from the outputs of the OCR. If no strings match, we

consider the information invisible.

Edge Detection technique [5, 22] to detect the icon in the

designated area. If no edges are detected, the icon is con-

sidered invisible. Specifically, we identify the widget that

represents the target icon by searching for keywords (e.g.,

of different scales on the screenshots processed by the Canny

operator and calculate the matching scores. Finally, we col-

lect the icon detection results based on empirical thresholds

for each template and vote for the final decision. If no tem-

plates match, the icon is regarded as invisible.

3.6 Detector

To account for which kind of UI may deceive users and make

them overlook or misunderstand certain subscription infor-

mation, a single feature may not be sufficient. We utilize

machine learning techniques to determine the combinations

the UI, we need to provide explanations for the detection re-

sults. Although the Decision Tree inherently offers a certain

level of interpretability, such interpretability might not be in-

tuitive enough for ordinary users to understand the detection

results, We integrated an additional step to address this is-

sue. We employed SHAP (Shapley Additive exPlanations)

[37, 38], an XAI technique, to visualize the most significant

features contributing to the prediction results for each data

sample. The integration of the XAI technique enables users

to intuitively grasp the rationale behind individual classifica-

each week to meticulous annotation. Additionally, these ex-

perts participated in an hour-long discussion weekly to en-

sure a consistent and accurate labeling process. The experts

from the 45 Android fleeceware apps and another 45 sub-

scription apps from Google Play), and their task was to in-

dependently assess whether each UI was a suspected fleece-