LogViewer v0.1.2 releases: viewing and searching large text files
LogViewer is designed to work with any large text files so that even very large files can be opened, viewed and searched.
Its original use case is for DFIR cases that involve log analysis. Whilst I use grep (well actually I use sift to extract data from logs, it is handy to be able to view log files, search for terms, hide lines whilst you get an idea what the log file contains, what actions are being performed.
The use of the custom control would make debugging any future issues a lot harder, so after a bit of thought, I used the ObjectListView library. The ObjectListView library is a custom list view control for use with .Net projects, I have used it extensively as it is easy to use and works with large datasets.
The core operation of LogViewer works in the same way as Highlighter e.g. parse the file, find the line offsets and line lengths, then when a line is needed for display, an existing file stream is used to seek to the offset, and then read X bytes.
I tested the v0.0.1 release of LogViewer against v1.1.3 of Mandiant Highlighter. My test log file was 1.2 GB and had 4.4 million rows. The following shows the operation and duration of the operation to compare:
- Load (LogViewer): 15s
- Load (Highlighter): 42s
- Search (LogViewer): 1m 5s
- Search (Highlighter): 2m 15s
- Show Only Highlighted (LogViewer): 2s (+ the search operation above 1m 5s) Total: 1m 7s
- Show Only Highlighted (Highlighter): Killed after 35m
The main reasons for this being faster is that it has removed some functionality and I have optimised the file load code so that there is less memory allocation and unnecessary checks/logic, plus Highlighter does some Md5 calcs etc.
- Very fast
- Supports huge files
- Cumulative search
- Can disable/enable search terms that are cumulative and the results are displayed instantly
- Export current view
- Show/Hide matched lines
- Four search modes (SubString Case Insensitive, SubString Case Sensitive, Regex Case Insensitive, Regex Case Sensitive)
- To stop an action such as load, search, export, you double click on the progress bar, located in the status bar
- The context menu holds the majority of actions
- Lots of stuff to be fixed/added! 🙂
- Fixed searching bug
- Improved search speed