Observe and Detection Methods of Backdoors in Windows executables (.exe Files)

Eshan Harshana Agalawatta
7 min readMay 25, 2021

In cybersecurity backdoor refers to any method by which authorized and unauthorized users are able to get around normal security measures and gain high-level user access (a.k.a root access) on a computer system, network or software application.

we can divide backdoors into three main types in windows PE files. (PE means Potable Executables, Ex: .exe, .dll, .ocx, etc.). This article mainly focuses to discuss identifying methods for these three types of backdoors in software applications. Let’s discuss it one by one.

Hard-Coded Credentials

There are two major scenarios in which developers use plain text hard-coded credentials while working on an application.

  1. Application has to connect to external services that require authorization, such as DBMS (DataBase Management System) or LDAP (Lightweight Directory Access Protocol).
  2. 2. Application functions that authentication and client authorization for access to nonpublic functions. For example login function.

Method

Strings are located in either resource or in read-only data sections of the .exe file. So these hard-coded strings usually present in .text/.data/.rdata/.rsrc sections of the .exe file (It depends on binary format). Investigators can identify these hard-coded credentials /plain text URLs/IP address using static analysis techniques. These credentials can find by extracting ASCII/ Unicode character sequences from an executable file using the binary-totext encoding method. A binary-to-text encoding is the encoding of data in plain text. More accurately, it is an encoding of binary data in a sequence of printable characters.

In this example, shows searching character sets using of hex dump of the executable file. (There are several tools for string search like Sysinternals Strings, HxD Hex Editor v2.3).

Most bytes in a binary code are non-printable. Both ASCII and Unicode store characters in a sequence that ended with a NULL byte.

Example: Hello is represented as 48 00 65 00 6C 00 6C 00 6F 00 in a byte array. To find a string, first converted the .exe file as a byte array. Then use Encoding.ASCII method to extracts ASCII/Unicode characters. Non-printable characters will be ignored in this method.

If we found an IP address in these strings, we can use the Whois web service to get details about IP addresses and their details.

Limitations

It may be hard to extract strings in packed or encrypted executable programs.

Also, it could be hard to find hardcoded strings by inserting unwanted characters in a specific pattern (Obfuscated Strings).

It depends on the coding pattern of the application. For example, Developers can use arrays to store a single character of hardcoded username/password or IP address and in execution time they will add together. Developers use complex methods to do such things.

Mitigation

Doing proper source code audit (Manual Analysis) in SDLC.

Store the login credentials in a DBMS using a suitable hashing method.

Store the IP address/3rd party URLs in a separate configuration file that is not pushed to the code repository.

If it is necessary to add usernames or passwords to source code, must use their hash values.

Undocumented Malicious Network Activity

This type of backdoor commonly,

  1. Listen on an undocumented port
  2. 2.Makes outbound connection To leak sensitive data over the network or unauthorizedly gain access.

Method

In the beginning can extract ASCII/Unicode character sequences in the .exe file for finding hardcoded IP addresses, URLs, etc. that want to network activity like reverse or bind shells.

If the victim program access WSock32.dll, wininet.dll or Ws2_32.dll (Windows socket APIs) it may be performed network-related tasks. In these DLLs contains Win32 functions such as connect(), bind(), accept(), sendto(), listen() and recvfrom() that responsible for establishing connections or sending/receiving data. These functions commonly used to listen to the ports or make outbound connections.

Example: If photo-editing application that uses these DLLs to bind/listen to a port or establishing a connection with a 3rd party IP address may be backdoored.

After compiling the above code file, it was analyzed by APIMonitor v1.4. The output will be figured as below. It shows that .exe file use WS2_32.dll (send(),recv(),setsockopt() API calls). These Win32 APIs can send and receive data to the connected socket.

Also, Investigators can use “Sysinternals TCPView” for the detection of undocumented network connections. It is a dynamic analysis method because it wants to execute the .exe file and after give output as below including remote address/port and many other details.

Limitations

It may be hard to identify malicious network activity from legitimate network activity in the application by using static analysis techniques. Because these connections will be established in the execution time of the .exe file.

Mitigation

Properly document all the outbound connections and ports at the developing stage of the application.

Undocumented Hidden Functions

Undocumented hidden function backdoors allow the attacker to issue commands/execute pre-defined tasks without performing the designed authentication procedure. Identification of this type of backdoors in the compiled program is harder than other types because this type of backdoors was created as normal functions of the application and can’t correctly identify through disassembly code or other methods. It is easy to identify these backdoors by manual source code analysis. In the source code level can get a clear idea of application flow and how application functions work.

In compiled application, by identifying referenced DLLs and Win32 API calls in .idata section and import table, can get the idea that the application was backdoored or not.

For example : user32.dll contains Win32 APIs to detect keystrokes. This DLL can be used to build keyloggers. Developers can implement command injections by using Win32API functions wherein Kernel32.dll.

In User32.dll file contain hooking functions like SetWindowsHookExA(), UnhookWindowsHookEx(), etc. SetWindowsHookExA() can be used to inject a DLL into another process. Also, Kernel32.dll have Win32APIs such as VirtualProtectEx(), VirtualAlloc(), WriteProcessMemory() that are commonly used in shellcodes. In the Backdoor detection process, must care about these Win32API combinations.

The below table indicates suspicious APIs that are commonly used in backdoors.

To find import DLLs and their APIs, first, find the address of the Import Directory. Then go to address and search an array of IMAGE_IMPORT_DESCRIPTOR. After getting the member of this array that relates to the mapped image by inspecting the strings pointed to by the name fields.

Function addresses in the binary file of DLLs are not static, as new versions come out they are destined to change, so applications cannot be built using a hardcoded function address. They use pointers to a specific function. A pointer is a memory address of a function.

Extras

Suspicious Sections

Suspicious section names are also an indicator of backdoored files. A standard compiler gives well-defined name to each section like .data, .text, .rdata, etc. In the .exe file, can find all the section names and their locations in the section table. By looking section table can identify suspicious sections. In below .exe file injected a new section called “.test” and it has executable (code), readable, writable characteristic. In this .exe file has two executable (code) sections. It is atypical. Normally the compiler will make only a section named “.text” with the correct permissions for executable code.

AddressOfEntryPoint being the starting address of the program, the code at that address will be executed. After backdooring, mostly AddressOfEntryPoint value changed to injected section address.

In this injected section can be presented embedded shellcodes. Shellcode is a payload of raw executable code. The shellcode payload is usually found by looking for the typical process injection APIs like VirtualAllocEx, WriteProcessMemory, CreateProcess and CreateRemoteThread by analyzing relevant assembly code (CALL address).

Conclusion

When backdoor analysis in .exe file, there are two major analysis techniques. They are Static Analysis and Dynamic Analysis. In this article, mainly focused on Static Analysis techniques for the detection process. For detection of backdoors in the compiled application, the program must be reverse-engineered first.

References

  1. Ahmed, S., Wen, R., Backes, M., Ma, S. & Zhang, Y. (2020) Dynamic Backdoor Attacks Against Machine Learning Models [Online] Available from: [05 May 2020].
  2. Veracode Inc. (2020) Static Detection of Application Backdoors [Online] Available from: [20 February 2020].
  3. Villeneuve, N. & Bennett, J. (2014) Detecting APT Activity With Network Traffic Analysis [Online] Available from: [21 February 2020].
  4. Zhang, Y. & Paxson, V. (2001) Detecting Backdoors [Online] Available from: [21 February 2020].

--

--