DataPrime Expression Language (DPXL)

Overview

DataPrime Expression Language, or DPXL, is an expression language based on DataPrime expression syntax. Leverage it across the Coralogix platform to define rich expression-based filters, such as when setting up Scopes.

General syntax

DPXL expressions are a subset of DataPrime expressions, such as those used in the filter operator.

DPXL expressions are versioned to maintain predictability and stability of execution. Versioning allows enhancing DPXL over time without changing the semantics of existing expressions.

Each DPXL expression starts with a version identifier <vX>, currently <v1>. This is followed by the actual boolean expression, including literals, logical and comparison constructs, keypath access, and functions.

<v1> <boolean-expression>

Note that the UI provides the <v1> prefix automatically, allowing you to focus on the actual boolean expressions. However, when used in an API, a DPXL expression must start with the <v1> prefix.

Some of the examples in this document omit the <v1> prefix for brevity.

Examples

# A filter that returns true if my_text field has the value 'example'
<v1> $d.my_text == 'example'
# A filter that returns true if the event's timestamp is before the beginning of the year 2023
<v1> $m.timestamp < @'2023-01-01T00:00:00'
# A filter that returns true if the application name starts with 'dev-'
<v1> $l.applicationName.startsWith('dev-')
# A filter that returns true if the field region_id is us-east-1 or us-east-2
<v1> region_id:string.in('us-east-1', 'us-west-2')

Literals and data types

Here’s a list of data types and example literals:

Data Type	Example Literals
`string`	`‘us-east-1’ ’dev-’`
`number`	`23 -12.32`
`bool`	`true false`
`timestamp`	`@(’2023-01-01T00:00Z’) @’now’`
Regular Expression	`/H.o$/ /^prod-./`
`severity`	`VERBOSE DEBUG INFO WARNING ERROR CRITICAL`

In addition, there’s a null literal, which can be used with all other types.

Logical constructs

Operator	Meaning	Example	Example Description
`&&`	Logical AND	`cloud == ‘aws’ && region == ‘us-east-1’`	Will return true if both the cloud is `aws` and the region is `us-east-1`
			Logical OR
`!`	Logical NOT	`!region.contains(’us-’)`	Will return true if the region does not contain `us-`

Ordering

You can control the evaluation order inside an expression using parenthesis. For example:

region.startsWith('us-') && (cloud == 'aws' || cloud == 'gcp')

(country == 'us' && (age > 40 || age < 10)) || (country == 'il' && age > 25)

Comparison constructs

Operator	Meaning	Examples
`>`	Greater than	`duration > 40.5`
`$m.timestamp > @(’2023-01-01T00:00:00’)`
`>=`	Greater than or equals	`duration >= 40.5`
`<`	Smaller than	`age < 20`
`<=`	Smaller than or equals	`age <= 20`
`lastName <= ‘Smith’`
`==`	Equals	`cloud == 'aws'`
`!=`	Not equal to	`first_name != 'joe'`

Keypath access

Keypaths are divided into three different parts, each with a separate prefix:

$m : Metadata
$l : Labels, such as applicationName or subsystemName
$d : Actual user data (default prefix)

$m - Metadata keypaths

Keypath	Data Type	Description
`$m.timestamp`	`timestamp`	Contains the timestamp of the event
`$m.severity`	`severity`	Contains the severity of the event

$l - Label keypaths

For logs:

Keypath	Data Type
`$l.applicationname`	`string`
`$l.subsystemname`	`string`

For spans:

Keypath	Data Type
`$l.applicationName`	`string`
`$l.subsystemName`	`string`
`$l.serviceName`	`string`
`$l.operationName`	`string`

$d - User Data keypaths

Any user keypath can be accessed using $d.<keypath> including nested keypaths.

$d is the default prefix. Any keypath that does not contain a prefix will be considered a user data field.

Functions

Functions provide additional capabilities within DPXL expressions.

Function	Description	Example
`<s>.startsWith(<substr>):bool`	Checks if a string `s` starts with the specified substring `substr`	`region.startsWith('us-')`
`<s>.endsWith(<substr>):bool`	Checks if a string `s` ends with the specified substring `substr`	`firstName.endsWith(’Jo’)`
`<s>.contains(<substr>):bool`	Checks if a string `s` contains the specified substring `substr`	`stream.contains(’err’)`
`<s>.matches(<regex>):bool`	Checks if a string `s` matches the specified pattern provided by `regex`	`hostname.matches(/prod-.*/)`
`<value>.in(<value1>,<value2>,...)`	Checks if `value` is one of the provided values `value1-valueN`	`cloud.in(’aws’,’gcp’)`

Data types

DPXL strives to infer the expected datatype of keypaths. For example, when writing age > 50, it will infer that age is expected to be a number. In cases where DPXL cannot infer the data type for a keypath, it will require a cast to provide the necessary information about the type. For example:

'123':number

region1:string == region2

my_key:number > my_other_key

Usage

Hit the ground running with DPXL based on these examples.

  # For the entity logs, allow access only to logs where the application name is "production"
  <v1> $l.applicationname == 'production'

  # For the entity logs, allow access only to logs in which app name starts with dev, or the field "r egion_id" in the data is us-east-1
  <v1> $l.applicationname.startsWith('dev-') && region_id == 'us-east-1'

  # For the entity logs, allow access only to logs in which the field "country" is not one of the listed below. 
  <v1> !$d.country:string.in('us','il','gr')

  # For the entity logs, allow access only to logs in which cloud is aws and the region field either ends with "-2" or is speficially set to "eu-west-1"
  <v1> cloud == 'aws' && (region == 'eu-west-1' || region.endsWith('-2'))

  # For the entity logs, allow access only to logs where the pod name matches the regex provided
  <v1> kubernetes.pod_name.matches(/^kafka-[0-9]+/)

  # For the entity logs, allow access only to logs that don't have a DEBUG severity
  <v1> $m.severity != DEBUG

  # For the entity logs, allow access only to logs in which some query duration is very large
  <v1> query_duration_seconds > 100

  # For the entity logs, allow access only to logs up to the beginning of the year 2023
  <v1> $m.timestamp < @'2023-01-01T00:00:00'

  # For the entity logs, disallow access to all logs entirely
  <v1> false

  # For the entity spans, allow access only to spans in which the applicationName is not "dev"
  <v1> $l.applicationName != 'dev'

  # For the entity spans, allow access only to spans that have a service name that starts with "alerts-"
  <v1> $l.serviceName.startsWith('alerts-')

  # For the entity spans, allow access only to spans that have the tag "db.system" set to mysql, and whose operationName contains the string "query"
  <v1> $l.operationName.contains('query') && tags['db.system'] == 'mysql'

  # For the entity spans, allow access to all spans
  <v1> true

Limitations

Expressions

Maximum number: 150
Maximum depth: 20

`in` Function type inference

The in function currently cannot automatically infer the expected type of a keypath. In order to use it, cast the keypath to the proper type. For example:

<v1> !$d.country:string.in('us','il','gr')

Additional resources

Documentation

DataPrime Cheat Sheet

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by emailing [email protected].

AI Query Assistant

Next DataPrime Quick-Start Guide