2021
|
Botacin, Marcus; Aghakhani, Hojjat; Ortolani, Stefano; Kruegel, Christopher; Vigna, Giovanni; Oliveira, Daniela; Geus, Paulo Lício De; Grégio, André One Size Does Not Fit All: A Longitudinal Analysis of Brazilian Financial Malware Journal Article ACM Trans. Priv. Secur., 24 (2), 2021, ISSN: 2471-2566. Abstract | Links | BibTeX @article{10.1145/3429741,
title = {One Size Does Not Fit All: A Longitudinal Analysis of Brazilian Financial Malware},
author = {Marcus Botacin and Hojjat Aghakhani and Stefano Ortolani and Christopher Kruegel and Giovanni Vigna and Daniela Oliveira and Paulo Lício De Geus and André Grégio},
url = {https://doi.org/10.1145/3429741
https://secret.inf.ufpr.br/papers/marcus_tops_br.pdf},
doi = {10.1145/3429741},
issn = {2471-2566},
year = {2021},
date = {2021-01-01},
journal = {ACM Trans. Priv. Secur.},
volume = {24},
number = {2},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
abstract = {Malware analysis is an essential task to understand infection campaigns, the behavior of malicious codes, and possible ways to mitigate threats. Malware analysis also allows better assessment of attackers’ capabilities, techniques, and processes. Although a substantial amount of previous work provided a comprehensive analysis of the international malware ecosystem, research on regionalized, country-, and population-specific malware campaigns have been scarce. Moving towards addressing this gap, we conducted a longitudinal (2012-2020) and comprehensive (encompassing an entire population of online banking users) study of MS Windows desktop malware that actually infected Brazilian banks’ users. We found that the Brazilian financial desktop malware has been evolving quickly: it started to make use of a variety of file formats instead of typical PE binaries, relied on native system resources, and abused obfuscation techniques to bypass detection mechanisms. Our study on the threats targeting a significant population on the ecosystem of the largest and most populous country in Latin America can provide invaluable insights that may be applied to other countries’ user populations, especially those in the developing world that might face cultural peculiarities similar to Brazil’s. With this evaluation, we expect to motivate the security community/industry to seriously consider a deeper level of customization during the development of next-generation anti-malware solutions, as well as to raise awareness towards regionalized and targeted Internet threats.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Malware analysis is an essential task to understand infection campaigns, the behavior of malicious codes, and possible ways to mitigate threats. Malware analysis also allows better assessment of attackers’ capabilities, techniques, and processes. Although a substantial amount of previous work provided a comprehensive analysis of the international malware ecosystem, research on regionalized, country-, and population-specific malware campaigns have been scarce. Moving towards addressing this gap, we conducted a longitudinal (2012-2020) and comprehensive (encompassing an entire population of online banking users) study of MS Windows desktop malware that actually infected Brazilian banks’ users. We found that the Brazilian financial desktop malware has been evolving quickly: it started to make use of a variety of file formats instead of typical PE binaries, relied on native system resources, and abused obfuscation techniques to bypass detection mechanisms. Our study on the threats targeting a significant population on the ecosystem of the largest and most populous country in Latin America can provide invaluable insights that may be applied to other countries’ user populations, especially those in the developing world that might face cultural peculiarities similar to Brazil’s. With this evaluation, we expect to motivate the security community/industry to seriously consider a deeper level of customization during the development of next-generation anti-malware solutions, as well as to raise awareness towards regionalized and targeted Internet threats. |
Botacin, Marcus; Ceschin, Fabricio; Sun, Ruimin; Oliveira, Daniela; Grégio, André Challenges and Pitfalls in Malware Research Journal Article Computers & Security, pp. 102287, 2021, ISSN: 0167-4048. Abstract | Links | BibTeX @article{BOTACIN2021102287,
title = {Challenges and Pitfalls in Malware Research},
author = {Marcus Botacin and Fabricio Ceschin and Ruimin Sun and Daniela Oliveira and André Grégio},
url = {https://www.sciencedirect.com/science/article/pii/S0167404821001115
https://secret.inf.ufpr.br/papers/marcus_challenges.pdf},
doi = {https://doi.org/10.1016/j.cose.2021.102287},
issn = {0167-4048},
year = {2021},
date = {2021-01-01},
journal = {Computers & Security},
pages = {102287},
abstract = {As the malware research field became more established over the last two decades, new research questions arose, such as how to make malware research reproducible, how to bring scientific rigor to attack papers, or what is an appropriate malware dataset for relevant experimental results. The challenges these questions pose also brings pitfalls that affect the multiple malware research stakeholders. To help answering those questions and to highlight potential research pitfalls to be avoided, in this paper, we present a systematic literature review of 491 papers on malware research published in major security conferences between 2000 and 2018. We identified the most common pitfalls present in past literature and propose a method for assessing current (and future) malware research. Our goal is towards integrating science and engineering best practices to develop further, improved research by learning from issues present in the published body of work. As far as we know, this is the largest literature review of its kind and the first to summarize research pitfalls in a research methodology that avoids them. In total, we discovered 20 pitfalls that limit current research impact and reproducibility. The identified pitfalls range from (i) the lack of a proper threat model, that complicates paper’s evaluation, to (ii) the use of closed-source solutions and private datasets, that limit reproducibility. We also report yet-to-be-overcome challenges that are inherent to the malware nature, such as non-deterministic analysis results. Based on our findings, we propose a set of actions to be taken by the malware research and development community for future work: (i) Consolidation of malware research as a field constituted of diverse research approaches (e.g., engineering solutions, offensive research, landscapes/observational studies, and network traffic/system traces analysis); (ii) design of engineering solutions with clearer, direct assumptions (e.g., positioning solutions as proofs-of-concept vs. deployable); (iii) design of experiments that reflects (and emphasizes) the target scenario for the proposed solution (e.g., corporation, user, country-wide); (iv) clearer exposition and discussion of limitations of used technologies and exercised norms/standards for research (e.g., the use of closed-source antiviruses as ground-truth).},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
As the malware research field became more established over the last two decades, new research questions arose, such as how to make malware research reproducible, how to bring scientific rigor to attack papers, or what is an appropriate malware dataset for relevant experimental results. The challenges these questions pose also brings pitfalls that affect the multiple malware research stakeholders. To help answering those questions and to highlight potential research pitfalls to be avoided, in this paper, we present a systematic literature review of 491 papers on malware research published in major security conferences between 2000 and 2018. We identified the most common pitfalls present in past literature and propose a method for assessing current (and future) malware research. Our goal is towards integrating science and engineering best practices to develop further, improved research by learning from issues present in the published body of work. As far as we know, this is the largest literature review of its kind and the first to summarize research pitfalls in a research methodology that avoids them. In total, we discovered 20 pitfalls that limit current research impact and reproducibility. The identified pitfalls range from (i) the lack of a proper threat model, that complicates paper’s evaluation, to (ii) the use of closed-source solutions and private datasets, that limit reproducibility. We also report yet-to-be-overcome challenges that are inherent to the malware nature, such as non-deterministic analysis results. Based on our findings, we propose a set of actions to be taken by the malware research and development community for future work: (i) Consolidation of malware research as a field constituted of diverse research approaches (e.g., engineering solutions, offensive research, landscapes/observational studies, and network traffic/system traces analysis); (ii) design of engineering solutions with clearer, direct assumptions (e.g., positioning solutions as proofs-of-concept vs. deployable); (iii) design of experiments that reflects (and emphasizes) the target scenario for the proposed solution (e.g., corporation, user, country-wide); (iv) clearer exposition and discussion of limitations of used technologies and exercised norms/standards for research (e.g., the use of closed-source antiviruses as ground-truth). |
Botacin, Marcus; Moia, Vitor Hugo Galhardo; Ceschin, Fabricio; Henriques, Marco Amaral A; Grégio, André Understanding uses and misuses of similarity hashing functions for malware detection and family clustering in actual scenarios Journal Article Forensic Science International: Digital Investigation, 38 , pp. 301220, 2021, ISSN: 2666-2817. Abstract | Links | BibTeX @article{BOTACIN2021301220,
title = {Understanding uses and misuses of similarity hashing functions for malware detection and family clustering in actual scenarios},
author = {Marcus Botacin and Vitor Hugo Galhardo Moia and Fabricio Ceschin and Marco A Amaral Henriques and André Grégio},
url = {https://www.sciencedirect.com/science/article/pii/S2666281721001281
https://secret.inf.ufpr.br/papers/marcus_similarity_hashing.pdf},
doi = {https://doi.org/10.1016/j.fsidi.2021.301220},
issn = {2666-2817},
year = {2021},
date = {2021-01-01},
journal = {Forensic Science International: Digital Investigation},
volume = {38},
pages = {301220},
abstract = {An everyday growing number of malware variants target end-users and organizations. To reduce the amount of individual malware handling, security analysts apply techniques for finding similarities to cluster samples. A popular clustering method relies on similarity hashing functions, which create short representations of files and compare them to produce a score related to the similarity level between them. Despite the popularity of those functions, the limits of their application to malware samples have not been extensively studied so-far. To help in bridging this gap, we performed a set of experiments to characterize the application of these functions on long-term, realistic malware analysis scenarios. To do so, we introduce SHAVE, an ideal model of similarity hashing-based antivirus engine. The evaluation of SHAVE consisted of applying two distinct hash functions (ssdeep and sdhash) to a dataset of 21 thousand actual malware samples collected over four years. We characterized this dataset based on the performed clustering, and discovered that: (i) smaller groups are prevalent than large ones; (ii) the threshold value chosen may significantly change the conclusions about the prevalence of similar samples in a given dataset; (iii) establishing a ground-truth for similarity hashing functions comparison has its issues, since the clusters originated from traditional AV labeling routines may result from a completely distinct approach; (iv) the application of similarity hashing functions improves traditional AVs’ detection rates by up to 40%; and finally (v) taking specific binary regions into account (e.g., instructions), leads to better classification results than hashing the entire binary file.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
An everyday growing number of malware variants target end-users and organizations. To reduce the amount of individual malware handling, security analysts apply techniques for finding similarities to cluster samples. A popular clustering method relies on similarity hashing functions, which create short representations of files and compare them to produce a score related to the similarity level between them. Despite the popularity of those functions, the limits of their application to malware samples have not been extensively studied so-far. To help in bridging this gap, we performed a set of experiments to characterize the application of these functions on long-term, realistic malware analysis scenarios. To do so, we introduce SHAVE, an ideal model of similarity hashing-based antivirus engine. The evaluation of SHAVE consisted of applying two distinct hash functions (ssdeep and sdhash) to a dataset of 21 thousand actual malware samples collected over four years. We characterized this dataset based on the performed clustering, and discovered that: (i) smaller groups are prevalent than large ones; (ii) the threshold value chosen may significantly change the conclusions about the prevalence of similar samples in a given dataset; (iii) establishing a ground-truth for similarity hashing functions comparison has its issues, since the clusters originated from traditional AV labeling routines may result from a completely distinct approach; (iv) the application of similarity hashing functions improves traditional AVs’ detection rates by up to 40%; and finally (v) taking specific binary regions into account (e.g., instructions), leads to better classification results than hashing the entire binary file. |
Botacin, Marcus; Domingues, Felipe Duarte; Ceschin, Fabrício; Machnicki, Raphael; Alves, Marco Antonio Zanata; de Geus, Paulo Lício; Grégio, André AntiViruses under the Microscope: A Hands-On Perspective Journal Article Computers & Security, pp. 102500, 2021, ISSN: 0167-4048. Abstract | Links | BibTeX @article{BOTACIN2021102500,
title = {AntiViruses under the Microscope: A Hands-On Perspective},
author = {Marcus Botacin and Felipe Duarte Domingues and Fabrício Ceschin and Raphael Machnicki and Marco Antonio Zanata Alves and Paulo Lício de Geus and André Grégio},
url = {https://www.sciencedirect.com/science/article/pii/S0167404821003242
https://secret.inf.ufpr.br/papers/marcus_av_handson.pdf},
doi = {https://doi.org/10.1016/j.cose.2021.102500},
issn = {0167-4048},
year = {2021},
date = {2021-01-01},
journal = {Computers & Security},
pages = {102500},
abstract = {AntiViruses (AVs) are the main defense line against attacks for most users and much research has been done about them, especially proposing new detection procedures that work in academic prototypes. However, as most current and commercial AVs are closed-source solutions, in practice, little is known about their real internals: information such as what is a typical AV database size, the detection methods effectively used in each operation mode, and how often on average the AVs are updated are still unknown. This prevents research work from meeting the industrial practices more thoroughly. To fill this gap, in this work, we systematize the knowledge about AVs. To do so, we first surveyed the literature and identified existing knowledge gaps in AV internals’ working. Further, we bridged these gaps by analyzing popular (Windows, Linux, and Android) AV solutions to check their operations in practice. Our methodology encompassed multiple techniques, from tracing to fuzzing. We detail current AV’s architecture, including their multiple components, such as browser extensions and injected libraries, regarding their implementation, monitoring features, and self-protection capabilities. We discovered, for instance, a great disparity in the set of API functions hooked by the distinct AV’s libraries, which might have a significant impact in the viability of academically-proposed detection models (e.g., machine learning-based ones).},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
AntiViruses (AVs) are the main defense line against attacks for most users and much research has been done about them, especially proposing new detection procedures that work in academic prototypes. However, as most current and commercial AVs are closed-source solutions, in practice, little is known about their real internals: information such as what is a typical AV database size, the detection methods effectively used in each operation mode, and how often on average the AVs are updated are still unknown. This prevents research work from meeting the industrial practices more thoroughly. To fill this gap, in this work, we systematize the knowledge about AVs. To do so, we first surveyed the literature and identified existing knowledge gaps in AV internals’ working. Further, we bridged these gaps by analyzing popular (Windows, Linux, and Android) AV solutions to check their operations in practice. Our methodology encompassed multiple techniques, from tracing to fuzzing. We detail current AV’s architecture, including their multiple components, such as browser extensions and injected libraries, regarding their implementation, monitoring features, and self-protection capabilities. We discovered, for instance, a great disparity in the set of API functions hooked by the distinct AV’s libraries, which might have a significant impact in the viability of academically-proposed detection models (e.g., machine learning-based ones). |
2020
|
Ceschin, Fabricio; Botacin, Marcus; Lüders, Gabriel; Gomes, Heitor Murilo; Oliveira, Luiz; Gregio, Andre No Need to Teach New Tricks to Old Malware: Winning an Evasion Challenge with XOR-Based Adversarial Samples Inproceedings Reversing and Offensive-Oriented Trends Symposium, pp. 13–22, Association for Computing Machinery, Vienna, Austria, 2020, ISBN: 9781450389747. Abstract | Links | BibTeX @inproceedings{10.1145/3433667.3433669,
title = {No Need to Teach New Tricks to Old Malware: Winning an Evasion Challenge with XOR-Based Adversarial Samples},
author = {Fabricio Ceschin and Marcus Botacin and Gabriel Lüders and Heitor Murilo Gomes and Luiz Oliveira and Andre Gregio},
url = {https://doi.org/10.1145/3433667.3433669
https://secret.inf.ufpr.br/papers/roots_mlsec20.pdf},
doi = {10.1145/3433667.3433669},
isbn = {9781450389747},
year = {2020},
date = {2020-11-01},
booktitle = {Reversing and Offensive-Oriented Trends Symposium},
pages = {13–22},
publisher = {Association for Computing Machinery},
address = {Vienna, Austria},
series = {ROOTS'20},
abstract = {Adversarial attacks to Machine Learning (ML) models became such a concern that tech companies (Microsoft and CUJO AI’s Vulnerability Research Lab) decided to launch contests to better understand their impact on practice. During the contest’s first edition (2019), participating teams were challenged to bypass three ML models in a white box manner. Our team bypassed all the three of them and reported interesting insights about models’ weaknesses. In the second edition (2020), the challenge evolved to an attack-and-defense model: the teams should either propose defensive models and attack other teams’ models in a black box manner. Despite the difficulty increase, our team was able to bypass all models again. In this paper, we describe our insights for this year’s contest regarding on attacking models, as well defending them from adversarial attacks. In particular, we show how frequency-based models (e.g., TF-IDF) are vulnerable to the addition of dead function imports, and how models based on raw bytes are vulnerable to payload-embedding obfuscation (e.g., XOR and base64 encoding).},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Adversarial attacks to Machine Learning (ML) models became such a concern that tech companies (Microsoft and CUJO AI’s Vulnerability Research Lab) decided to launch contests to better understand their impact on practice. During the contest’s first edition (2019), participating teams were challenged to bypass three ML models in a white box manner. Our team bypassed all the three of them and reported interesting insights about models’ weaknesses. In the second edition (2020), the challenge evolved to an attack-and-defense model: the teams should either propose defensive models and attack other teams’ models in a black box manner. Despite the difficulty increase, our team was able to bypass all models again. In this paper, we describe our insights for this year’s contest regarding on attacking models, as well defending them from adversarial attacks. In particular, we show how frequency-based models (e.g., TF-IDF) are vulnerable to the addition of dead function imports, and how models based on raw bytes are vulnerable to payload-embedding obfuscation (e.g., XOR and base64 encoding). |
Botacin, Marcus; Ceschin, Fabricio; de Geus, Paulo; Grégio, André We Need to Talk About AntiViruses: Challenges & Pitfalls of AV Evaluations Journal Article Computers & Security, pp. 101859, 2020, ISSN: 0167-4048. Abstract | Links | BibTeX @article{BOTACIN2020101859,
title = {We Need to Talk About AntiViruses: Challenges & Pitfalls of AV Evaluations},
author = {Marcus Botacin and Fabricio Ceschin and Paulo de Geus and André Grégio},
url = {http://www.sciencedirect.com/science/article/pii/S0167404820301310
https://secret.inf.ufpr.br/papers/marcus_av.pdf},
doi = {https://doi.org/10.1016/j.cose.2020.101859},
issn = {0167-4048},
year = {2020},
date = {2020-04-29},
journal = {Computers & Security},
pages = {101859},
abstract = {Security evaluation is an essential task to identify the level of protection accomplished in running systems or to aid in choosing better solutions for each specific scenario. Although antiviruses (AVs) are one of the main defensive solutions for most end-users and corporations, AV’s evaluations are conducted by few organizations and often limited to compare detection rates. Moreover, other important factors of AVs’ operating mode (e.g., response time and detection regression) are usually underestimated. Ignoring such factors create an “understanding gap” on the effectiveness of AVs in actual scenarios, which we aim to bridge by presenting a broader characterization of current AVs’ modes of operation. In our characterization, we consider distinct file types, operating systems, datasets, and time frames. To do so, we daily collected samples from two distinct, representative malware sources and submitted them to the VirusTotal (VT) service for 30 consecutive days. In total, we considered 28,875 unique malware samples. For each day, we retrieved the submitted samples’ detection rates and assigned labels, resulting in more than 1M distinct VT submissions overall. Our experimental results show that: (i) phishing contexts are a challenge for all AVs, turning malicious Web pages detectors less effective than malicious files detectors; (ii) generic procedures are insufficient to ensure broad detection coverage, incurring in lower detection rates for particular datasets (e.g., country-specific) than for those with world-wide collected samples; (iii) detection rates are unstable since all AVs presented detection regression effects after scans in different time frames using the same dataset and (iv) AVs’ long response times in delivering new signatures/heuristics create a significant attack opportunity window within the first 30 days after we first identified a malicious binary. To address the effects of our findings, we propose six new metrics to evaluate the multiple aspects that impact the effectiveness of AVs. With them, we hope to assess corporate (and domestic) users to better evaluate the solutions that fit their needs more adequately.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Security evaluation is an essential task to identify the level of protection accomplished in running systems or to aid in choosing better solutions for each specific scenario. Although antiviruses (AVs) are one of the main defensive solutions for most end-users and corporations, AV’s evaluations are conducted by few organizations and often limited to compare detection rates. Moreover, other important factors of AVs’ operating mode (e.g., response time and detection regression) are usually underestimated. Ignoring such factors create an “understanding gap” on the effectiveness of AVs in actual scenarios, which we aim to bridge by presenting a broader characterization of current AVs’ modes of operation. In our characterization, we consider distinct file types, operating systems, datasets, and time frames. To do so, we daily collected samples from two distinct, representative malware sources and submitted them to the VirusTotal (VT) service for 30 consecutive days. In total, we considered 28,875 unique malware samples. For each day, we retrieved the submitted samples’ detection rates and assigned labels, resulting in more than 1M distinct VT submissions overall. Our experimental results show that: (i) phishing contexts are a challenge for all AVs, turning malicious Web pages detectors less effective than malicious files detectors; (ii) generic procedures are insufficient to ensure broad detection coverage, incurring in lower detection rates for particular datasets (e.g., country-specific) than for those with world-wide collected samples; (iii) detection rates are unstable since all AVs presented detection regression effects after scans in different time frames using the same dataset and (iv) AVs’ long response times in delivering new signatures/heuristics create a significant attack opportunity window within the first 30 days after we first identified a malicious binary. To address the effects of our findings, we propose six new metrics to evaluate the multiple aspects that impact the effectiveness of AVs. With them, we hope to assess corporate (and domestic) users to better evaluate the solutions that fit their needs more adequately. |
