Archive Query

Archive Query enables you to directly query your logs and spans from your S3 archive using any text or a wide range of syntax queries. Query data irrespective of priority, daily quota, or its time frame.

Overview

Archive Query enables you to directly query your logs and spans from your S3 archive.

Use this feature to:

• Take advantage of various querying options. This includes text-based queries or specific query languages such as Lucene or DataPrime.

• Enjoy multiple data sources. Using our innovative DataPrime syntax language, query both logs and spans.

• Query data irrespective of priority and daily quota. Only blocked logs are not sent to the archive.

• Query data with unlimited time frames. There are no restrictions on how far back in time your data can go.

• Save on costs while maintaining interactive query times. Store more of your data in our Monitoring and Compliance priority levels. By prioritizing logs at the Monitoring priority level, for example, you can view and query them in LiveTail, receive real-time alerts, utilize parsing rules, Loggregation and Events2Metrics, and query them without indexing your data. All of this is available at 40% of the cost.

Prerequisites

• Read/ Write permissions enabled in your AWS S3 bucket

Archive Query Setup

STEP 1. In your Coralogix toolbar, navigate to Data Flow > Archive Queries.

STEP 2. Click ARCHIVE QUERY.

STEP 3. Define a New Archive Query.

• Enter a Query Name and Description.

• Search Query. Enter a text search query, Lucene, or DataPrime syntax query to match a subset of logs from your S3 bucket.

Notes:

• Only the data matching the query will be presented.

• Using our innovative DataPrime syntax language, query both logs and spans.

Query Examples

1. A query to find logs with the field ClientIP_geoip.continent_name:”Europe” and the field ClientIP_geoip.country_name with values other than: Czechia, United Kingdom or Germany:ClientIP_geoip.continent_name:”Europe” NOT (ClientIP_geoip.country_name:”Czechia” OR ClientIP_geoip.country_name:”United Kingdom” OR ClientIP_geoip.country_name:”Germany”)

2. A query to find logs with words status and get:status get

3. A query to find only logs with HTTP method post:“http_method”:”post”

• Source. Select CX-Data.

• Select the Timeframe, Applications, Subsystems, and Severity for the query.

STEP 4. Click RUN ARCHIVE QUERY. Once you have set up and run your query, a test will be run to validate your setup.

View Archive Query Results

View your query results in one of three formats: Logs Preview, Download TSV, or Clone.

Logs Preview

This option allows you to view your logs without ever indexing your data.

Download TSV

Download a TSV file to view query results.

Clone

Duplicate your current query by clicking on the Clone button.  In the new duplicated query, click RUN ARCHIVE QUERY.

If you wish to share an archive query with another teammate, click on the chain-link icon in the query of choice. This will copy to your clipboard the link to that same archive query.

After some time, the archive query you created will expire so you can no longer view or download the data. Click Clone and duplicate the same query with the same criteria instead of recreating the query from scratch.

Limitations

The limitations placed on queries are described below.

Warnings

Once a limit is reached, a warning message is displayed. Refine your query results to avoid reaching a limit.

Refine Your Query Results

Refine your query results using any of the following methods:

• Apply more selective filters to your queries (for example, on application or subsystem).

• If using the Dataprime extract operator and subsequently filtering its results, create a parsing rule and filter on the parsed field instead.

• Avoid regular expressions or wildcards in filters.

• In Dataprime, switch from using the contains operator on strings to the free text search operator (~).

Additional Resources

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected]

Previous Access CX-Data Directly

Next Archive Query from the Explore Screen

Theme

Light

light

dark