Technical preview

To gain clearer insights into real threats, you can tune the anomaly results. The following procedures help to reduce the number of false positives:

• Tune results for rare applications and processes
• Define an anomaly threshold for a job

Filter out anomalies from rarely used applications and processes

When anomalies include results from a known process that only runs occasionally, you can filter out the unwanted results.

For example, to filter out results from a housekeeping process, named maintenanceservice.exe, that only executes occasionally you need to:

1. Create a filter list
2. Add the filter to the relevant job
3. Clone and rerun the job (optional)

Create a filter list

1. Go to Machine learning → Anomaly Detection → Settings.

2. Click Filter Lists and then Create.

   The Create new filter list pane is displayed.

3. Enter a filter list ID.

4. Enter a description for the filter list (optional).

5. Click Add item.

6. In the Items textbox, enter the name of the process for which you want to filter out anomaly results (maintenanceservice.exe in our example).

   

   Click to open this image in fullscreen mode

7. Click Add and then Save.

   The new filter appears in the Filter List and can be added to relevant jobs.

Add the filter to the relevant job

1. Go to Machine learning → Anomaly Detection → Anomaly Explorer.

2. Navigate to the job results for which the filter is required. If the job results are not listed, click Edit job selection and select the relevant job.

3. In the actions column, click the gear icon and then select Configure rules.

   The Create Rule window is displayed.

   

   Click to open this image in fullscreen mode

4. Select:

   1. Add a filter list to limit where the rule applies.

   2. The WHEN statement for the relevant detector (process.name in our example).

   3. The IS IN statement.

   4. The filter you created as part of the Create a filter list procedure.

      Tip

      For more information, see Customizing detectors with custom rules.

5. Click Save.

Clone and rerun the job

If you want to remove all the previously detected results for the process, you must clone and run the cloned job.

1. Go to Machine learning → Anomaly Detection → Jobs.
2. Navigate to the job for which you configured the rule.

1. Optionally, expand the job row and click JSON to verify the configured filter appears under custom rules in the JSON code.

2. In the actions column, click the options menu () and select Clone job.

   The Configure datafeed page is displayed.

3. Click Data Preview and check the data is displayed without errors.

1. Click Next until the Job details page is displayed.

2. Enter a Job ID for the cloned job that indicates it is an iteration of the original one. For example, append a number or a username to the original job name, such as windows-rare-network-process-2.

   

   Click to open this image in fullscreen mode

3. Click Next and check the job validates without errors. You can ignore warnings about multiple influencers.

4. Click Next and then Create job.

   The Start <job name> window is displayed.

   

   Click to open this image in fullscreen mode

5. Select the point of time from which the job will analyze anomalies.

1. Click Start.

   After a while, results will start to appear on the Anomaly Explorer page.

Define an anomaly threshold for a job

Certain jobs use a high-count function to look for unusual spikes in process events. For some processes, a burst of activity is a normal, such as automation and housekeeping jobs running on server fleets. However, sometimes a high-delta event count is unlikely to be the result of routine behavior. In these cases, you can define a minimum threshold for when a high-event count is considered an anomaly.

Depending on your anomaly detection results, you may want to set a minimum event count threshold for the packetbeat_dns_tunneling job:

1. Go to Machine learning → Anomaly Detection → Anomaly Explorer.

2. Navigate to the job results for the packetbeat_dns_tunneling job. If the job results are not listed, click Edit job selection and select packetbeat_dns_tunneling.

3. In the actions column, click the gear icon and then select Configure rules.

   The Create Rule window is displayed.

   

   Click to open this image in fullscreen mode

4. Select Add numeric conditions for when the rule applies and the following when statement:

   WHEN actual IS GREATER THAN <X>

   Where <X> is the threshold above which anomalies are detected.

5. Click Save.

6. To apply the new threshold, rerun the job (Job Management → Actions → Start datafeed).