velociraptor v0.6.8-1 releases: Endpoint visibility and collection tool

Velociraptor – Endpoint visibility and collection tool.

Velociraptor is a unique, advanced open-source endpoint monitoring, digital forensic, and cyber response platform.

It was originally developed by DFIR professionals who needed a powerful and efficient way to hunt and monitor activities across fleets of endpoints for specific artifacts, in a wide range of digital forensic and cyber incident response investigations such as:

• Responding to data breaches
• Reconstructing attacker activities through digital forensic analysis
• Hunting for evidence of sophisticated adversaries
• Investigating malware outbreaks and other suspicious network activities
• Continual monitoring for suspicious user activities, such as files copies to USB devices
• Disclosure of confidential information outside the network
• Gathering endpoint data over time, for use in threat hunting and future investigations.

Velociraptor is actively being used by DFIR professionals across cases such as these and continues to grow and develop based on their feedback and ideas.

The design goals of Velociraptor that we’re working towards, are to be:

• Useful – each artefact and use case must return valuable information to the user
• Simple – the design and interface must be easy for a person to navigate and use
• Guided – users don’t need to be DFIR experts, since all elements should provide informative descriptions and guidance
• Powerful – the user should not have to perform too much additional work to achieve their objectives
• Quick – performance should be speedy and resource impact low, while allowing performance to be managed when needed
• Reliable – each feature and artefact should work as expected and be relatively free of bugs and issues

Changelog v0.6.8

This release introduces an upgraded communication protocol between client and server. Previously the server would keep track of various collection statistics but now these are tracked on the client, making the server a lot more efficient. The new communication protocol is a lot faster and allows the server to support a larger number of clients. When using a 0.6.8 server with older clients, the older protocol is still used to maintain backwards compatibility.

Notable features

• Send ClientInfo messages all the time. Previously client information like hostname was collected during interrogation by collecting Generic.Client.Info. In this version, Velociraptor will send periodic updates every day to keep client information in sync.
• Use pageable tables for the VFS: The previous VFS view would store the entire directory listing in a single table making it impossible to view very large directories like C:\Windows or System32. This version updates the way the VFS is handled to transfer directory listing more efficiently, and uses a pageable table to view arbitrarily large directories. Performance is much improved for example recursive directory sync on C:\Windows tables about 90 sec to sync 250k files.
• Refactored hunt and collection export code – exporting a hunt or collection creates a zip file containing the files collected and query results. The code for exporting this zip file was optimized to produce this export much faster and with less memory requirements. The GUI is also much improved.
• Batch client log messages into JSONL groups – Previously clients would send each query log in a separate message, in order to ensure they get to the server ASAP. This release batches these into larger messages reducing the server’s load and improving efficiency.
• Implemented a preview Column renderer – Sometimes it is handy to send from the client some raw data to provide context around e.g. yara hits. Previously this was only possible to view in the data was encoded within the result set JSON causing very large JSON payloads. In this version it is possible to use the upload() function to upload an arbitrary sized file to the server, and mark the column as type “upload_preview” . The GUI will then render a preview of the file (if it is an image, it will be shown as a thumbnail) otherwise a hex viewer.
• Uploads are now deduplicated on store_as_name: Previously each call to the upload() function would result in the file being uploaded which made using upload() a bit tricky in order to avoid uploading the same file multiple times. Now this is not necessary as upload will be automatically deduplicated so can be called multiple times on the same file.
• Added a tempfile based materializer to have safe queries : Some VQL queries would materialize a query into an array, e.g. with an expression like

Previously this expanded the query into memory and would cause huge memory requirements and possible out of memory crashes. In this release VQL implements a temp file implementation of the LET operator which automatically switches to a temp file when the variable receives more than 1000 rows by default. This keeps memory use controlled and allows these kinds of queries to be run safely.

• Allow client side collections to be traced: We often get questions about what happened to a collection that seems to be hung? Previously the only way to gather client side information was to collect a Generic.Client.Profile collection. This required running it at just the right time and did not guarantee that we would get helpful insight of what the query and the client binary were doing. In this release it is possible to specify a trace collection for any collection to automatically collect client side state as the collection is progressing.
• Added progress reporting to offline collector: When copying large files it is useful to get some kind of feedback as the copy operation is progressing.
• Track tool definitions by defining artifact: Previously once a tool was defined, the admin could only override the tool but not reset it to a new version. If a tool was defined by 2 artifacts it might be that each artifact declares a different version. The release allows the admin to reset the tool definition to any one of the artifacts that define it.

Including many bug fixes and stability improvements.

Install & Use

Copyright (C) 2019 Velocidex