His thesis is entitled “Data flow analysis to build control flow graph of obfuscated codes”, his presentation will be held in French.

Abstract:

The increase in cyber attacks around the world makes malicious code analysis a priority research area. This software uses various protection methods, also known as obfuscations, to bypass antivirus software and slow down the analysis process. In this context, this thesis provides a solution to build the Control Float Graph (CFG) of obfuscated binary code. We developed the BOA platform (Basic blOck Analysis) which performs a static analysis of a protected binary code. For this, we have defined a semantics based on the BINSEC tool to which we have added continuations. These allow on one hand to control the self-modifications, and on the other hand to simulate the operating system to handle system calls and interruptions. The static analysis is done by symbolically executing the binary code and calculating the values of the system states using SMT solvers. Thus, we perform a data flow analysis to build the CFG by calculating the transfer addresses. Finally, loop handling is performed by transforming a CFG into a pushdown automaton. BOA is able to compute dynamic jump addresses, to detect opaque predicates, to compute return addresses on a stack even if they have been falsified, to manage interrupt handler falsifications, to rebuild import tables on the fly, and finally, to manage self-modifications. We validated the BOA correction using the Tigress code obfuscator. Then, we tested BOA on 35 known packers and showed that in 30 cases, BOA was able to completely or partially rebuild the initially protected binary. Finally, we detected the opaque predicates protecting XTunnel, a malware used during the 2016 U.S. elections, and we partially unpacked a sample of the Emotet Trojan, which on 14/10/2020 was detected by only 7 antivirus programs out of the 63 offered by VirusTotal This work contributes to the development of tools for static analysis of malicious code. In contrast to dynamic methods, this solution allows an analysis without executing the binary, which offers a double advantage : on the one hand, a static approach is easier to deploy, and on the other hand, since the malicious code is not executed, it cannot warn its author.
Keywords: Malware, Obfuscation, Data flow, Symbolic execution, Control flow graph.

Jury members:

Referes:
Valérie Viet Triem Tong – CentraleSupélec Rennes 
José Fernandez – Polytechnique Montréal

Examiners:
Nadia Tawbi – Université Laval
Sarah Zennou – Airbus
Stephan Merz – Université de Lorraine

Guest:

Supervisors: