Archive Query from the Explore Screen

Directly query logs and spans in your S3 archive from your Explore Screen. Using text-based queries or specific query languages such as DataPrime or Lucene, you may query your data irrespective of its priority or timeframe and regardless of your daily quota - all with the ease of familiar functionalities.

Overview

Archive Query allows you to query the data in your S3 archive directly from your Explore screen.

Use this feature to:

Query your archive in the Coralogix UI using any text or Lucene or DataPrime syntax query, irrespective of log priority or daily quota.
Enjoy various data sources. Using DataPrime, you may query logs or spans.
Query data with unlimited time frames. There are no restrictions on how far back in time your data can go.
View query results alongside live data streams and with the use of familiar functionalities.
Store more of your data in our Monitoring and Compliance priority levels, while maintaining interactive query times and saving on costs.

Prerequisites

Read/ Write permissions enabled in your AWS S3 bucket

Getting Started

STEP 1. From your Coralogix toolbar, navigate to Explore > Logs tab.

STEP 2. Select Archive the upper left-hand corner of your Logs screen.

STEP 3. Select CX-Data in the dropdown menu.

Archive Query Explore Screen

Explore Screen Functionalities

Query

Query using any text or Lucene or DataPrime syntax query.

Filter

Filter your logs using application, subsystem, and severity filters.

Manage Settings

Archive Query Explore Screen

Manage your settings by clicking on SETTINGS in the upper right-hand corner of your screen.

Select ACTIONS to trigger 3rd party services based on your search results and / or values under specific keys. Find out more here.

Limitations

The following limitations on Explore Screen archive queries are described below.

Limitation	Description
Bytes processed	Up to 30% of daily ingested bytes
Parquet files scanned	Up to 500k files
Total log results	Up to 2k results
While aggregations are available across all the data, we only pull up to 2,000 raw logs to display in the main logs grid.
Exported log results	Up to 2k results
Exporting the data in the logs grid from an archive query is limited to the top 20 pages (100 logs per page), so you can export a max of 2000 logs at a time. In order to export all archive query logs, create an archive query by navigating Data Flow > Archive Queries.

Archive Query Explore Screen

Notes:

You may incur slight delays when querying archived data, when compared with other Explore screen queries.
It is possible to use the same query syntax as queries run on the Archive Query page.

Warnings

Once a limit is reached, a warning message is displayed. Refine your query results to avoid reaching a limit.

Refine Your Query Results

Refine your query results using any of the following methods:

Apply more selective filters to your queries (for example, on application or subsystem).
If using the Dataprime extract operator and subsequently filtering its results, create a parsing rule and filter on the parsed field instead.
Avoid regular expressions or wildcards in filters.
In Dataprime, switch from using the contains operator on strings to the free text search operator (~).

Additional Resources

Documentation

Archive Query

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].

Archive Query

Next Archive Query with AWS Athena