Feature Update: Vulnerability Query Builders
- May 25, 2023
- Michael Tucker
NopSec is proud to announce the release of a new feature this week aimed at enhancing our users’ workflows! We met with many of our users and worked to understand where they were facing pain points during their workflows. The end result of this latest addition is a new and improved querying system for Unified VRM.
The following were the main drivers we aimed at solving with the new Query Builder.
This often caused users to turn to the text based editor over time.
Below you’ll find a quick overview list of all the new features that we’ll go over in this post related to our updated query builder:
Starting in mid-May we will do the initial release of UVRM 5.5 which will include the new query for users to choose when creating reports. You’ll be able to do so by clicking the “New Report V2” button.
Later this summer we’ll be releasing UVRM 6.0. In this version update, the new query builder will become the default and only option for all users to select.
It’s important to note up front that all queries adhere to each user’s data access controls. This means users will only be able to see data they have been given access to see regardless of the query they provide. This applies to type-ahead search recommendations, drop-downs, and report table data.
The first thing you’ll notice when using the new query builder is that the default will be the text based query builder over a GUI based one.
A few reasons for this. First, one of the core pain points users faced was based on the ability to quickly edit a query. The text based version allows for users to quickly modify their query without having to click any buttons or switch context. This includes the ability to quickly add a new filter as well. We understand that the learning curve may be high at first. To access the GUI query builder click the icon (pictured below) on the far right of the text input window.
This will open the GUI query builder.
Over time, we hope that the query syntax and the way we’ve designed the filters will be easy to learn and hope that you’ll use the text editor natively. That said, we’ll continue improving the GUI query builder as well as there are several use cases that will only work in that space.
We worked hard to find a query system that was used across many different applications and use cases and landed with Google’s Common Expression Language (CEL) as the main compiler for our query system.
This means that our queries are performant and support some pretty complex logical operators out of the box. We are aiming to make our query syntax straightforward and simple with filters using a common format to quickly find and remember so that you can quickly learn to use the query builder without any UI help. Below is how our query language is configured.
The format is generally the following:
The above query can be condensed down in the following ways:
Here the query can be condensed down to one rule because the “ORs” were compared against the same filter; asset.ip. So the values become a List of values similar to many programming languages consisting of bookended brackets and values in quotes separated by commas. Ex. [“windows”, “mac”, “unix”]
The above query can be further condensed down by removing the bookending parentheses and changing the query from an “IN” operator to an “=” operator.
An internal rule “OR” is supported by placing a comma in the string, however, if the string you’re looking for contains a “,” then it is best to use the bookending brackets instead.
One of the key features UVRM provides clients is the ability to aggregate VM data from across all of their scanners. In order to operationalize this data users need the ability to efficiently query for it or filter it out to create reports and remediation plans.
In order to support users ability to learn the text based query system and provide some organization we are leveraging filter categories. New Categories will be:
You can download the Query Syntax Cheat Sheet here.
To open up the GUI helper you click on the icon located in the far right to open the drawer.
You’ll now see our new GUI based query builder.
You should notice that many of the components from the text based editor format are now visible in the GUI.
Today, when using the GUI query builder many of the filters support type-ahead search to help you discover your own data and find specific items you’re looking for. A great user experience improvement we’ve implemented is to filter down your type-ahead suggestions to the data already filtered by your query rules. This means that as you add new filters you will be presented with specific options to deep dive further into your dataset. You can override this by typing your own value and selecting the “Search by …” option in the dropdown menu. We’ll support type-ahead support in the text based query system in the future.
Editing queries is much simpler than before. In the old query system you had to delete the filter, add a new filter, find your filter, and re-configure the values if you had made a mistake and wanted to edit the filter. Today, you’ll be able to directly edit the values directly in the text based query editor or in the GUI query builder within two clicks or less.
Date pickers are something that is normal in any query system and now they are available in the GUI query builder whenever the filter is a Date based filter. You can override this by typing the value as well.
Relative time support was one of the most requested features for reports as there was no way of configuring a report to use relative times. Today this is supported in the text based query editor and it will be supported in the GUI query builder in the next iteration. See the Cheat Sheet for examples but here’s a simple example; vuln.detected_date > “-6m” which translates to vuln.detected_date > 12/15/2023.
For the query system we will continue releasing new filters to continue to support complex querying to build highly specific reports to meet your needs. We will also listen to your feedback and provide improvements to syntax, categories, and usage in order to provide user experience improvements such as better type-ahead support in the text based query editor or in the GUI query builder.