process-forest: processes Microsoft Windows EVTX event logs

by do son · Published November 3, 2018 · Updated November 3, 2018

process-forest is a tool that processes Microsoft Windows EVTX event logs that contain process accounting events and reconstructs the historical process hierarchies. That is, it displays the parent-child relationships among programs. When using this tool during an incident response engagement, identifying a reverse shell process quickly leads to the processes launched by the operator, and insight into how it may be maintaining persistence.

process-forest relies on the process accounting events that may be enabled and captured in the Security event log or Sysmon Operational log. When a new process is spawned, the event logs capture event 4688 (Security) or 1 (Sysmon) that includes the process path, username information, PID and parent PID, etc. Events with EID 4689 (Security) or 5 (Sysmon) signal that a process has exited. By walking these events in order, keeping track of PID “liveness”, and watching PID/PPID links, this tool reconstructs the relationships among all processes captured in the event log.

Download

git clone https://github.com/williballenthin/process-forest.git
cd process-forest
python setup.py install

Usage

Example

Summarize process lifetime events:

> python process_forest.py SECURITY.evtx summary



first event: 2014-11-26T18:16:27.004395

last event: 2014-12-23T19:18:54.895493

-------------------------

path counts

  - C:\Windows\System32\conhost.exe: 4256

  - C:\Windows\System32\wbem\WmiPrvSE.exe: 2930

  - C:\Windows\SysWOW64\CCM\SMSCliUI.exe: 1725

  - C:\Windows\System32\taskeng.exe: 1298

  ...

Display process trees for all process lifetime events:

> python process_forest.py SECURITY.evtx ts all



Process(C:\Windows\explorer.exe, cmd=UNKNOWN, hashes=UNKNOWN, pid=ce8, ppid=0, begin=0001-01-01T00:00:00, end=0001-01-01T00:00:00: Fake Parent: This is a faked process created since a ppid didn't exist

  Process(C:\Windows\System32\cmd.exe, cmd="C:\Windows\system32\cmd.exe" , hashes=UNKNOWN, pid=968, ppid=ce8, begin=2016-03-19T20:47:43.846725, end=0001-01-01T00:00:00

    Process(C:\Windows\System32\conhost.exe, cmd=\??\C:\Windows\system32\conhost.exe 0xffffffff -ForceV1, hashes=UNKNOWN, pid=10b0, ppid=968, begin=2016-03-19T20:47:43.930553, end=0001-01-01T00:00:00

    Process(C:\Windows\System32\PING.EXE, cmd=ping  192.168.2.2, hashes=UNKNOWN, pid=13f8, ppid=968, begin=2016-03-19T20:47:48.418451, end=2016-03-19T20:47:51.509247

    Process(C:\Windows\System32\net.exe, cmd=net  time, hashes=UNKNOWN, pid=b78, ppid=968, begin=2016-03-19T20:47:53.295649, end=2016-03-19T20:48:00.207243

      Process(C:\Windows\System32\net1.exe, cmd=C:\Windows\system32\net1  time, hashes=UNKNOWN, pid=f0c, ppid=b78, begin=2016-03-19T20:47:53.306993, end=2016-03-19T20:48:01.142002

    Process(C:\Windows\System32\calc.exe, cmd=calc, hashes=UNKNOWN, pid=920, ppid=968, begin=2016-03-19T20:48:29.693279, end=2016-03-19T20:48:32.034391

    Process(C:\Windows\System32\notepad.exe, cmd=notepad, hashes=UNKNOWN, pid=d14, ppid=968, begin=2016-03-19T20:48:35.440413, end=0001-01-01T00:00:00

Process(C:\Windows\System32\svchost.exe, cmd=UNKNOWN, hashes=UNKNOWN, pid=338, ppid=0, begin=0001-01-01T00:00:00, end=0001-01-01T00:00:00: Fake Parent: This is a faked process created since a ppid didn't exist

  Process(C:\Windows\System32\dllhost.exe, cmd=C:\Windows\system32\DllHost.exe /Processid:{AB8902B4-09CA-4BB6-B78D-A8F59079A8D5}, hashes=UNKNOWN, pid=1020, ppid=338, begin=2016-03-19T20:47:43.038507, end=2016-03-19T20:47:48.193815

  Process(C:\Windows\System32\ApplicationFrameHost.exe, cmd=C:\Windows\system32\ApplicationFrameHost.exe -Embedding, hashes=UNKNOWN, pid=678, ppid=338, begin=2016-03-19T20:48:30.616486, end=0001-01-01T00:00:00

  Process(C:\Program Files\WindowsApps\Microsoft.WindowsCalculator_10.1601.49020.0_x64__8wekyb3d8bbwe\Calculator.exe, cmd="C:\Program Files\WindowsApps\Microsoft.WindowsCalculator_10.1601.49020.0_x64__8wekyb3d8bbwe\Calculator.exe" -ServerName:App.AppXsm3pg4n7er43kdh1qp4e79f1j7am68r8.mca, hashes=UNKNOWN, pid=3c0, ppid=338, begin=2016-03-19T20:48:31.711050, end=0001-01-01T00:00:00

    ...



> python process_forest.py Microsoft-Windows-Sysmon%4Operational.evtx ts all



Process(C:\Windows\System32\services.exe, cmd=UNKNOWN, hashes=UNKNOWN, pid=500, ppid=0, begin=0001-01-01T00:00:00, end=0001-01-01T00:00:00: Fake Parent: This is a faked process created since a ppid didn't exist

  Process(C:\Windows\Sysmon.exe, cmd=C:\Windows\Sysmon.exe, hashes=MD5=2E5F6BB9692F7FF20CFCFC9AF097D9FC,IMPHASH=CDFE7352C4CC5D5EFCFFAAAC26E91D60, pid=2560, ppid=500, begin=2016-03-23T17:07:58.651699, end=0001-01-01T00:00:00

  Process(C:\Windows\System32\taskhost.exe, cmd=taskhost.exe $(Arg0), hashes=MD5=639774C9ACD063F028F6084ABF5593AD,IMPHASH=D9C431646227DBA4B6B2A1313802ED63, pid=1424, ppid=500, begin=2016-03-23T18:00:30.337669, end=2016-03-23T18:02:30.478363

  Process(C:\Windows\System32\taskhost.exe, cmd=taskhost.exe SYSTEM, hashes=MD5=639774C9ACD063F028F6084ABF5593AD,IMPHASH=D9C431646227DBA4B6B2A1313802ED63, pid=2252, ppid=500, begin=2016-03-23T18:07:09.324966, end=2016-03-23T18:07:09.340567

  Process(C:\Windows\System32\taskhost.exe, cmd=taskhost.exe $(Arg0), hashes=MD5=639774C9ACD063F028F6084ABF5593AD,IMPHASH=D9C431646227DBA4B6B2A1313802ED63, pid=2620, ppid=500, begin=2016-03-23T19:03:00.504421, end=2016-03-23T19:05:00.646055

  Process(C:\Windows\System32\taskhost.exe, cmd=taskhost.exe $(Arg0), hashes=MD5=639774C9ACD063F028F6084ABF5593AD,IMPHASH=D9C431646227DBA4B6B2A1313802ED63, pid=1252, ppid=500, begin=2016-03-23T20:05:47.669977, end=2016-03-23T20:07:47.810583

  Process(C:\Windows\System32\taskhost.exe, cmd=taskhost.exe $(Arg0), hashes=MD5=639774C9ACD063F028F6084ABF5593AD,IMPHASH=D9C431646227DBA4B6B2A1313802ED63, pid=1192, ppid=500, begin=2016-03-24T00:06:11.779625, end=2016-03-24T00:08:16.182283

  Process(C:\Windows\System32\taskhost.exe, cmd=taskhost.exe $(Arg0), hashes=MD5=639774C9ACD063F028F6084ABF5593AD,IMPHASH=D9C431646227DBA4B6B2A1313802ED63, pid=1200, ppid=500, begin=2016-03-24T01:09:20.788097, end=2016-03-24T01:11:20.913534

Process(C:\Windows\System32\winlogon.exe, cmd=UNKNOWN, hashes=UNKNOWN, pid=444, ppid=0, begin=0001-01-01T00:00:00, end=0001-01-01T00:00:00: Fake Parent: This is a faked process created since a ppid didn't exist

  Process(C:\Windows\System32\taskmgr.exe, cmd=taskmgr.exe /2 , hashes=MD5=09F7401D56F2393C6CA534FF0241A590,IMPHASH=68E56344CAB250384904953E978B70A9, pid=1424, ppid=444, begin=2016-03-23T17:17:34.770151, end=2016-03-23T17:17:46.470600

Process(C:\Windows\System32\csrss.exe, cmd=UNKNOWN, hashes=UNKNOWN, pid=408, ppid=0, begin=0001-01-01T00:00:00, end=0001-01-01T00:00:00: Fake Parent: This is a faked process created since a ppid didn't exist

  Process(C:\Windows\System32\conhost.exe, cmd=\??\C:\Windows\system32\conhost.exe "-2596793011012622666521040507-794026175-12198051332145736009-124749138470921422", hashes=MD5=94C5B49D3E89CE9E02A6D6133A4F4321,IMPHASH=BA6498D069813141251615FFFC2A69D3, pid=1824, ppid=408, begin=2016-03-23T17:09:55.095173, end=2016-03-23T17:14:32.319595

Process(C:\Windows\System32\svchost.exe, cmd=UNKNOWN, hashes=UNKNOWN, pid=612, ppid=0, begin=0001-01-01T00:00:00, end=0001-01-01T00:00:00: Fake Parent: This is a faked process created since a ppid didn't exist

  Process(C:\Windows\System32\mobsync.exe, cmd=C:\Windows\System32\mobsync.exe -Embedding, hashes=MD5=509E88FF7B257885775791FAF0965D6A,IMPHASH=F714D092385CEE7898887F01B2072F4B, pid=2044, ppid=612, begin=2016-03-23T21:03:26.431755, end=2016-03-24T00:06:37.388523

  Process(C:\Windows\System32\mobsync.exe, cmd=C:\Windows\System32\mobsync.exe -Embedding, hashes=MD5=509E88FF7B257885775791FAF0965D6A,IMPHASH=F714D092385CEE7898887F01B2072F4B, pid=2280, ppid=612, begin=2016-03-24T00:16:23.524853, end=2016-03-24T00:16:41.803833

  Process(C:\Windows\System32\dllhost.exe, cmd=C:\Windows\system32\DllHost.exe /Processid:{AB8902B4-09CA-4BB6-B78D-A8F59079A8D5}, hashes=MD5=A8EDB86FC2A4D6D1285E4C70384AC35A,IMPHASH=3D806EF1101283F4E5D20F0C4F83B8FD, pid=1920, ppid=612, begin=2016-03-24T01:56:11.389832, end=0001-01-01T00:00:00

Process(C:\Windows\System32\SearchIndexer.exe, cmd=UNKNOWN, hashes=UNKNOWN, pid=2528, ppid=0, begin=0001-01-01T00:00:00, end=0001-01-01T00:00:00: Fake Parent: This is a faked process created since a ppid didn't exist

  Process(C:\Windows\System32\SearchProtocolHost.exe, cmd="C:\Windows\system32\SearchProtocolHost.exe" Global\UsGthrFltPipeMssGthrPipe12_ Global\UsGthrCtrlFltPipeMssGthrPipe12 1 -2147483646 "Software\Microsoft\Windows Search" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT; MS Search 4.0 Robot)" "C:\ProgramData\Microsoft\Search\Data\Temp\usgthrsvc" "DownLevelDaemon" , hashes=MD5=D9E21CBF9E6A87847AFFD39EA3FA28EE,IMPHASH=3E21C2C0BFB7FA9AAD5C782DBF401846, pid=2948, ppid=2528, begin=2016-03-23T17:11:58.822676, end=2016-03-23T17:13:08.917692

  Process(C:\Windows\System32\SearchFilterHost.exe, cmd="C:\Windows\system32\SearchFilterHost.exe" 0 516 520 528 65536 524 , hashes=MD5=49A3AD5CE578CD77F445F3D244AEAB2D,IMPHASH=48476C179FBF9FC0CC7AE2A8A5FB1073, pid=2400, ppid=2528, begin=2016-03-23T17:11:58.838276, end=2016-03-23T17:13:08.917692

Process(C:\Windows\System32\svchost.exe, cmd=UNKNOWN, hashes=UNKNOWN, pid=748, ppid=0, begin=0001-01-01T00:00:00, end=0001-01-01T00:00:00: Fake Parent: This is a faked process created since a ppid didn't exist

  Process(C:\Windows\System32\audiodg.exe, cmd=C:\Windows\system32\AUDIODG.EXE 0x1a4, hashes=MD5=6E974F1C384615DEB0710E44F4847351,IMPHASH=2A6BF191EDFE97CC30EBB8D1AEB3A6B6, pid=1640, ppid=748, begin=2016-03-24T01:56:09.687605, end=0001-01-01T00:00:00

Process(C:\Windows\explorer.exe, cmd=UNKNOWN, hashes=UNKNOWN, pid=1448, ppid=0, begin=0001-01-01T00:00:00, end=0001-01-01T00:00:00: Fake Parent: This is a faked process created since a ppid didn't exist

  Process(C:\Windows\System32\cmd.exe, cmd="C:\Windows\system32\cmd.exe" , hashes=MD5=5746BD7E255DD6A8AFA06F7C42C1BA41,IMPHASH=D0058544E4588B1B2290B7F4D830EB0A, pid=1100, ppid=1448, begin=2016-03-23T17:09:55.095173, end=2016-03-23T17:14:32.319595

    Process(C:\Windows\System32\notepad.exe, cmd=notepad, hashes=MD5=B32189BDFF6E577A92BAA61AD49264E6,IMPHASH=FCCD5E915D9C361A1F0ECCBF0B8B66ED, pid=940, ppid=1100, begin=2016-03-23T17:09:58.152889, end=2016-03-23T17:10:03.936321

    Process(C:\Windows\System32\calc.exe, cmd=calc, hashes=MD5=10E4A1D2132CCB5C6759F038CDB6F3C9,IMPHASH=CA7337BD1DFA93FD45FF30B369488A37, pid=2004, ppid=1100, begin=2016-03-23T17:10:02.099840, end=2016-03-23T17:10:05.168768

    Process(C:\Windows\System32\net.exe, cmd=net  time, hashes=MD5=63DD6FBAABF881385899FD39DF13DCE3,IMPHASH=96B4B43C2313DC3C3237F7C32A9F8812, pid=1676, ppid=1100, begin=2016-03-23T17:10:07.290449, end=2016-03-23T17:10:24.107897

      Process(C:\Windows\System32\net1.exe, cmd=C:\Windows\system32\net1  time, hashes=MD5=3B6928BC39E5530CEAD1E99269E7B1EE,IMPHASH=72AA515B1963995C201E36DE48594F61, pid=2272, ppid=1676, begin=2016-03-23T17:10:07.290449, end=2016-03-23T17:10:24.107897

    Process(C:\Windows\System32\ipconfig.exe, cmd=ipconfig, hashes=MD5=CF45949CDBB39C953331CDCB9CEC20F8,IMPHASH=BBBA00511B8BEF70143B0EEBBB337273, pid=1304, ppid=1100, begin=2016-03-23T17:11:37.542912, end=2016-03-23T17:11:37.558514

    Process(C:\Windows\System32\mstsc.exe, cmd=mstsc, hashes=MD5=8E75B1112C374EBDF18FD640DA2F0655,IMPHASH=A4508E6BF5CA8E66A9003310D569E036, pid=2712, ppid=1100, begin=2016-03-23T17:11:58.760271, end=2016-03-23T17:12:01.459244

    Process(C:\Windows\System32\cmd.exe, cmd=cmd  /c ipconfig, hashes=MD5=5746BD7E255DD6A8AFA06F7C42C1BA41,IMPHASH=D0058544E4588B1B2290B7F4D830EB0A, pid=2956, ppid=1100, begin=2016-03-23T17:12:21.085302, end=2016-03-23T17:12:21.116505

      Process(C:\Windows\System32\ipconfig.exe, cmd=ipconfig, hashes=MD5=CF45949CDBB39C953331CDCB9CEC20F8,IMPHASH=BBBA00511B8BEF70143B0EEBBB337273, pid=1536, ppid=2956, begin=2016-03-23T17:12:21.100903, end=2016-03-23T17:12:21.116505

    Process(C:\Windows\System32\PING.EXE, cmd=ping  192.168.2.2, hashes=MD5=5FB30FE90736C7FC77DE637021B1CE7C,IMPHASH=33BEE540593D1AD78C69C59B9D26DECF, pid=1668, ppid=1100, begin=2016-03-23T17:12:28.994989, end=2016-03-23T17:12:32.037167

    Process(C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe, cmd=powershell.exe  Invoke-Command -ScriptBlock {Get-Help Invoke-Command -full}, hashes=MD5=852D67A27E454BD389FA7F02A8CBE23F,IMPHASH=F2C0E8A5BD10DBC167455484050CD683, pid=2980, ppid=1100, begin=2016-03-23T17:13:53.551691, end=2016-03-23T17:13:54.940161

  Process(C:\Program Files\Windows Media Player\wmpnscfg.exe, cmd="C:\Program Files\Windows Media Player\wmpnscfg.exe", hashes=MD5=6699A112A3BDC9B52338512894EBA9D6,IMPHASH=CE1A36A2A999517CA9B1C36DF3B7E240, pid=764, ppid=1448, begin=2016-03-23T21:03:26.993387, end=2016-03-23T21:03:27.040190

...

For large EVTX files, the parsing can take a while (due to the implementation in Python). To save time as you explore the tool, you can preprocess the log file into a “.pt” (Process Tree) file, which speeds up subsequent invocations. Use the .pt file instead of the EVTX file, and the tool will handle the details.

Here’s the time it takes to process a 20MB EVTX file on one computer:

> time python process_forest.py SECURITY.evtx ts all

... snip ...

python process_forest.py ts all:  38.85s user 0.55s system 97% cpu 40.375 total

To speed multiple future invocations up, we precompute some data and save it off:

> python process_forest.py SECURITY.evtx serialize SECURITY.pt

Now when we run the listings again, it goes much faster:

> time python process_forest.py SECURITY.pt ts all

... snip ...

python process_forest.py ts all:  3.04s user 0.49s system 76% cpu 4.610 total

limitations

this tool currently only supports Security.EVTX and Microsft-Windows-Sysmon%4Operational.evtx files. EVT support is coming later.
this tool only captures information from the event log. If important entries are missing, then it can only do a best-effort job to reconstruct the process trees. For example:
- if a log does not span the time since a system was starting, it will not contain the creation event for the core Windows processes.
- if a portion of the log was cleared, or events missing, then there may be collisions in process “liveness”. That is, the tool may identify two processes with the same PID at the same time. The tool alerts the user when data is missing.

Source: https://github.com/williballenthin/

process-forest: processes Microsoft Windows EVTX event logs

Search

Brilliantly

Content & Links