Parsing Rules API

This guide will help you use our Parsing Rules API to create, read, update or delete parsing rules and rule groups via API.

In order to use the Parsing Rules API, first locate your:

Alerts, Rules, and Tags API key - You can find your key in the Coralogix UI under Data Flow –> API Keys.
Coralogix domain

To send an external request, the request headers should contain the following Headers:

Content-Type: application/JSON
Authorization: Bearer {{Alerts, Rules, and Tags API key}}

Alerts, Rules, and Tags API key can be found under Data Flow –> API Keys.

NOTE - Only users with Admin privileges have access to the Rules API and keys.

This guide will help you use our Rules API to create, read, update or delete parsing rules and rule groups via API. Parsing rules are organized inside Rule Groups. Each group has a name and a set of parsing rules with a logical AND/OR relationship between them.

Logs are processed according to the order of the Rule Groups (top to bottom) and then by the order of rules within the Rule Group and according to the logical operators between them (AND/OR).

Group Name	Sub-Group / Condition	Rule Name
My group 1		My rule 1-1-a
	OR
		My rule 1-1-b
	AND
		My rule 1-2-a
	OR
		My rule 1-2-b
My group 2		My rule 2-1-a

Create a Rule

To create a parsing rule, there is a need to define a rule group and the parsing rule itself:

Method: POST
URL: https://api./api/v1/external/rule/rule-set
Body (the following fields are required/optional):

"name": (string)(mandatory) - the name of the rules group,
"description": (string)(optional - default is an empty string) - the description for the rules group,
"enabled": (boolean)(optional) - indicates rather the group is active or not when created - if fields is missing or value is empty - default is True,
"creator": (string)(optional - default is “Coralogix external API”) - the name of the group creator,
"order": (number)(optional) - the rule group order number. If it is not provided, the rule group will be added at the end of the list of rule groups. If the group with the provided order number already exists, new group will be added one below and the order will be changed to +1 automatically,
"ruleMatchers":(array of objects)(mandatory) - the conditions which the group of rules works on - application/subsystem/severity of logs. Each object in the array should contain:
- "field": (string) can be any of the following - applicationName/ subsystemName/ severity,
- "constraint": (string)- the value for the chosen field (if the field is severity, the value should be one: debug, verbose, info, warning, error, critical)
"rulesGroups":(array of objects)(optional) - It contains two fields:
- "order": (number) - it is the number of the sub-group. If rules are under two different sub-groups then it means there is AND condition between them
- "rules": (array of object) - each object is a rule

The rules object keys:

"name": (string)(mandatory) - the name of rule,
"description": (string)(optional - default is an empty string) - the description for the rule,
"enabled": (boolean)(optional) - indicates rather the rule is active or not when created - if fields is missing or value is empty - default is True,
"type": (string)(mandatory) - type of the rule - (block, extract, parse, jsonextract, replace, timestampextract, removefields),
"sourceField": (string)(optional) - If not stated in the body then the default source field will be the log text. If you want to run the rule against any internal log JSON field use Text.field_name.field_name2…
"destinationField":(string)(optional) - If not stated in the body then the default destination field will be the log text. If you want to run the rule against any internal log JSON field use Text.field_name.field_name2…
"order": (number)(optional) - if empty then the rule is added at the end
"rule": (string)(mandatory - except in “removefields” and “timestampextract” - for more info look at restrictions) - the regex of the rule (must be a regex that can be compiled)

Rule Type	Description / Restrictions
BLOCK, EXTRACT	API request body cannot have: "ReplaceNewVal" OR "DestinationField"
PARSE	API request body cannot have: "ReplaceNewVal"
JSONEXTRACT	API request body cannot have: "ReplaceNewVal". API request body must have: "DestinationField":(string) - need to be one of: “category”, “className”, “methodName”, “severity”, “threadId”
REPLACE	API request body must have: "ReplaceNewVal"
REMOVEFIELDS	API request body must have: "rule":(string) - comma-separated list of json fields you would like to remove
TIMESTAMPEXTRACT	API request body cannot have: "rule" or "destinationField" fields. API request body must have:"timeFormat"(string) - you can see our suggestion for each type of standard, normally you will need to change the format to match your time field string pattern with the while comply to the standard "formatStandard":(string) = need to be one of: “javasdf”, “golang”, “strftime”, “secondsts”, “millits”, “microts”, “nanots”). API request body should contain a meaningful value for "sourceField": (string) - a value is not specified the default source files will be the text field and the rule will not work.You should set the value as Text.`<json\_key\_with\_time\_field>`)
STRINGIFY JSON FIELD	Convert JSON object to JSON string. Find out more here.
PARSE JSON FIELD	Convert JSON string to JSON object. Find out more here.

Examples

A group with one parsing rule

{
    "name": "My test group with one rule - parse",
    "description": "It is an example how to create a group with a parsing rule",
    "enabled": true,
    "order": 1,
    "creator": "[email protected]",
    "ruleMatchers": [],
    "rulesGroups": [
        {
            "order": 1,
            "rules": [
                {
                    "name": "Test parse rule",
                    "description": "This is the first parsing rule",
                    "enabled": true,
                    "type": "parse",
                    "sourceField": "text",
                    "destinationField": "text",
                    "order": 1,
                    "rule": "^(?P<timestamp>\\\\d{2}-\\\\d{2}-\\\\d{4}\\\\s*\\\\d{2}:\\\\d{2}:\\\\d{2}\\\\.\\\\d+)\\\\s+(?P<message>.*)$"
                }
            ]
        }
    ]
}

A group with two parsing rules and an AND condition between them

{
    "name": "My test group with one rule per each sub-group",
    "description": "It is an example how to create a group with two rules with AND condition between them ",
    "enabled": true,
    "order": 1,
    "creator": "[email protected]",
    "ruleMatchers": [],
    "rulesGroups": [
        {
            "order": 1,
            "rules": [
                {
                    "name": "Test parse rule",
                    "description": "This is the first parsing rule",
                    "enabled": true,
                    "type": "parse",
                    "sourceField": "text",
                    "destinationField": "text",
                    "order": 1,
                    "rule": "^(?P<timestamp>\\\\d{2}-\\\\d{2}-\\\\d{4}\\\\s*\\\\d{2}:\\\\d{2}:\\\\d{2}\\\\.\\\\d+)\\\\s+(?P<message>.*)$"
                }
            ]
        },
        {
            "order": 2,
            "rules": [
                {
                    "name": "Test replace rule",
                    "description": "This is the first replace rule",
                    "enabled": true,
                    "type": "replace",
                    "sourceField": "text.message",
                    "destinationField": "text.message",
                    "order": 1,
                    "rule": "host",
                    "replaceNewVal": "hostname"
                }
            ]
        }
    ]
}

A group with 3 parsing rules

{
    "name": "My test group with three rules",
    "description": "It is an example how to create a group with 3 rules",
    "enabled": true,
    "order": 5,
    "creator": "[email protected]",
    "ruleMatchers": [],
    "rulesGroups": [
        {
            "order": 1,
            "rules": [
                {
                    "name": "Test parse rule",
                    "description": "This is the first parsing rule",
                    "enabled": true,
                    "type": "parse",
                    "sourceField": "text",
                    "destinationField": "text",
                    "order": 1,
                    "rule": "^(?P<timestamp>\\\\d{2}-\\\\d{2}-\\\\d{4}\\\\s*\\\\d{2}:\\\\d{2}:\\\\d{2}\\\\.\\\\d+)\\\\s+(?P<message>.*)$"
                }
            ]
        },
        {
            "order": 2,
            "rules": [
                {
                    "name": "Test replace rule 1",
                    "description": "This is the first replace rule",
                    "enabled": true,
                    "type": "replace",
                    "sourceField": "text.message",
                    "destinationField": "text.message",
                    "order": 1,
                    "rule": "host",
                    "replaceNewVal": "hostname"
                },
                {
                    "name": "Test replace rule 2",
                    "description": "This is the second replace rule",
                    "enabled": true,
                    "type": "replace",
                    "sourceField": "text.message",
                    "destinationField": "text.message",
                    "order": 2,
                    "rule": "server",
                    "replaceNewVal": "hostname"
                }
            ]
        }
    ]
}

Get All Rules

To get all parsing rules:

Method: GET
URL: https://api.

Parsing Rules API

Create a Rule

Examples

Get All Rules

Example

Get a Rule Group

Example

Modify Rules

Delete Rules

Delete a Group

Delete a Parsing Rule

Export Multiple Rules and Groups

Legacy REST API Methods

Groups API

Create a Group

Read a Group

Update a Group

Delete a Group

Rules API

Create a Rule

Read a Rule

Update a Rule

Delete a Rule

Support