Skip to content

OpenSearch API

Coralogix provides an OpenSearch API that allows you to query your hosted OpenSearch instances securely and with ease. 

Our OpenSearch API, which supports REST API, requires backward compatibility with Elasticsearch from v6.0 to v7.10. Find more information here.

Query your Coralogix OpenSearch API

To use the OpenSearch API, it is necessary to:

STEP 1. Add a Coralogix authorization key with each HTTP request. Navigate to Data Flow > API Keys > Logs Query Key from your Coralogix toolbar.

STEP 2. Select the OpenSearch API endpoint associated with your Coralogix domain.

STEP 3. Note the index as a variable in the endpoint. For example, you can put the name of the log2metrics index (:_log_metrics*) if you are querying the Log2metrics index, or (*) if you are querying the regular index.

The API request should contain the following:

HeaderValue
AuthorizationBearer <Logs Query Key>
Content-typeapplication/json

Examples:

curl "<CX_ENDPOINT_URL>/_search" -H "Authorization: Bearer <Logs Query Key>" -H "Content-type: application/json" -d '{
    "query": {
        "bool": {
            "must":
            [
                {
                    "term": {
                        "coralogix.metadata.applicationName": "PROD"
                    }
                },
                {
                    "range": {
                        "coralogix.timestamp": {
                            "gte": "now-15m",
                            "lt": "now"
                        }
                    }
                }
            ]
        }
    },
    "aggs": {
        "severities": {
            "terms": {
                "field": "coralogix.metadata.severity"
            }
        }
    }
}'

curl "<CX_ENDPOINT_URL>/_search" -H "Authorization: Bearer <Logs Query Key>" -H "Content-type: application/json" -d '{
    "query": {
        "bool": {
            "must":
            [
        {
            "match": {
                "text": "created"
            }
        },
                {
            "range": {
                "coralogix.timestamp": {
                    "gte": "now-15m",
                    "lt": "now"
                    }
                }
             }
            ]
        }
    }
}'

curl "<CX_ENDPOINT_URL>/_search" -H "Authorization: Bearer <Logs Query Key>" -H "Content-type: application/json" -d '{
    "query": {
        "bool": {
            "filter":[{
                "query_string": {
                    "query": "YOUR QUERY"
                }
             },
                {
            "range": {
                "coralogix.timestamp": {
                    "gte": "2019-10-23T14:00:00",
                    "time_zone": "+03:00"
                }
            }
                }
            ]
        }
    }
}'

Using the OpenSearch Scroll API

curl -H "Authorization: Bearer <Logs Query Key>" -H "Content-type: application/json" -d '{
    "size": 1000,
    "query": {
        "bool": {
            "filter": [{
                    "query_string": {
                        "query": "YOUR QUERY"
                    }
                },
                {
                    "range": {
                        "coralogix.timestamp": {
                            "gte": "now-24h",
                            "lt": "now"
                        }
                    }
                }
            ]
        }
    }
}'

You will receive the first batch of logs along with a new field in the root of the response named _scroll_id.

That same scroll_id should be used in subsequent requests to create the pagination, and obtain the next log batch. You should repeat the second request until all logs are retrieved.

Second and subsequent scroll API requests, should contain the Headers and URL endpoint. Note that an index is not specified in the endpoint for the second, and subsequent requests.

Example #1:

curl -H "Authorization: Bearer <Logs Query Key>" -H "Content-type: application/json" -d '{

    "scroll": "5m",
    "scroll_id": "YOUR_SCROLL_ID"

}'

Example #2:

curl --location --request POST "<CX_ENDPOINT_URL>" \
--header "Authorization: Bearer <Logs Query Key>" \
--header 'Content-Type: application/json' \
--data-raw '{
    "scroll": "5m",
    "scroll_id": "YOUR_SCROLL_ID"

}'

The last log page retrieved will show and empty array at the bottom of the page:

"hits": []

This indicates that there are no more log pages to retrieve.

curl --location --request POST '<CX_ENDPOINT_URL>' \
--header 'Authorization: Bearer <Logs Query Key>' \
--header 'Content-Type: application/json' \
--header 'Content-Type: application/x-ndjson' \
--data-raw '{"index":"*"}
{"query":{"bool":{"filter":[{"match_phrase":{"coralogix.metadata.applicationName":{"query":"coralogix"}}},{"match_phrase":{"coralogix.metadata.subsystemName":{"query":"coralogix"}}},{"range":{"coralogix.timestampsdsd":{"gte":"now-2h","lte":"now"}}}]}}}

{"index":"*"}
{"query":{"bool":{"filter":[{"match_phrase":{"coralogix.metadata.applicationName":{"query":"coralogix"}}},{"match_phrase":{"coralogix.metadata.subsystemName":{"query":"coralogix"}}},{"range":{"coralogix.timestampsdsd":{"gte":"now-4h","lte":"now-2h"}}}]}}}'
Every query should be in online format and start with {"index":"*"}

Limitations

The Coralogix OpenSearch API has the following limitations:

  • Supports only POST requests.

  • Supported top-level elements of the Search API: query, from, size, 
    sort, _source, post_filter, aggs, aggregations.

  • The sum of the top-level elements ‘from' and ‘size' cannot be
    greater than 12000.

  • allow_leading_wildcard element in query_string query is not allowed.

  • Wildcard queries can’t start with '*' or '?'

  • RegEx queries can’t start with '.*' or '.?'

  • max_determinized_states element inside regex queries is not allowed.

  • Size element for bucket aggregations cannot be greater than 1200.

  • The bucket aggregation of the type significant_terms is not allowed.

  • Nesting of the following bucket aggregations 3 or more times is not allowed: date_histogram, geohash_grid, histogram, 
    ip_ranges, and terms.

  • fuzzy_max_expansions element in query_string query is not allowed.

  • Max_expansions element in a fuzzy query is not allowed.

  • When specifying the URL query param ’scroll’ it can not be greater than 6m.

  • To retrieve the accurate number of hits of your query
    add to your request: "track_total_hits":true.

  • If you are running OpenSearch API requests with scripts
    note that there is a 160 requests limit per 30 seconds.

When using the scroll API _search/scroll

  • Supported top-level elements of the scroll API: size, scroll, scroll_ids.

  • Scroll element cannot be greater than 6m. This value is the maximum amount of time the _scroll_id will be valid after the last scroll API request. If for example a script is used for the second request, over and over again, then this time limit will have no effect. The countdown starts when the scroll API requests stop. When the time expires, let's say it was set originally to 6 minutes, then if an additional scroll API request is sent, it will fail, as the _scroll_id is no longer valid.

  • Size element cannot be greater than 12000.

Additional resources

OpenSearchOpenSearch API documentation

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].

Previous Previous Coralogix Terraform Provider
Home
User Guides User Guides
Integrations Integrations
OpenTelemetry OpenTelemetry
Developer Portal Developer Portal