Description
This function allows to filter a dataset based on the values of a column. The filtering operations available will differ for numerical and categorical columns.
Application
Custom Filter can help to reduce the size of the data and focus on relevant information. For example, focus on high performing products, extreme values or specific operating conditions.
How to use
Create the step and assign a tabular dataset to it in the field Data.
Select a Column on which to apply the filter.
- If the selected column contains categorical values, available operations are:
-
Contains: Use this option to filter results that include a specific substring. The search is case-sensitive by default, and glob patterns (e.g.,
% for any sequence of characters, _ for a single character) are supported, allowing for flexible pattern matching.
Does not contain: This option excludes any results containing a specific substring. Like “Contains,” the search is case-sensitive by default, and glob patterns are supported, ensuring that any result matching the specified pattern is omitted.
-
Is part of list: This function is a positive filter on a categorical column; all selected values will be kept in the dataset. User can define their list from all existing values in the dataset.
-
Is not part of list: This function is a negative filter on a categorical column; all selected values will be removed from the dataset. User can define their list from all existing values in the dataset.
- If the selected column contains numerical values, available operations are:
- < (Less Than): Filters values that are strictly less than the specified number.
- <= (Less Than or Equal To): Filters values that are less than or equal to the specified number.
- > (Greater Than): Filters values that are strictly greater than the specified number.
- >= (Greater Than or Equal To): Filters values that are greater than or equal to the specified number.
- = (Equal To): Filters values that are exactly equal to the specified number. This option might be difficult to apply on real numbers due to rounding errors.
- != (Not Equal To): Filters values that are not equal to the specified number. This option might be difficult to apply on real numbers due to rounding errors.
- Within value range: Filters values that fall within a specified percentile range (e.g. range [-100,250]).
- Outside value range: Filters values that fall outside a specified percentile range (e.g. range [-100,250]).
- Within percentile range: Filters values that fall within a specified percentile range (e.g. between the 25th and 75th percentiles).
- Outside percentile range: Filters values that fall outside a specified percentile range (e.g. outside the 25th and 75th percentiles).
- Is empty/missing: Filters out rows where the value is empty/missing.
- Is not empty/missing: Filters out rows where the value is present (not empty/missing).
You can save the resulting dataset under a new name. Enable Save output under different name to do so. If that option is unticked the existing dataset will be overwritten.
Click Apply to filter the data.
Once the step is completed, the filtering operation is displayed at the top of the step, and at the bottom of the step an info message will be shown which displays the number of rows before and after applying the filter (“Filtered from X to Y rows”).
Example
Filtering column number_of_cycles so that the value is exactly equal to 100:
Last modified on July 23, 2026
