Imperva: A successful migration from Graylog to Coralogix

The challenge

When Ram Saiegh, who leads Imperva’s infrastructure engineering team, first joined the company in 2021, they were struggling with their Graylog open-source observability tooling. 

Ram and his team are responsible for Imperva’s SaaS production environment. This environment is based on 60 private data centers where Imperva’s DDOS scrubbing service checks all customer traffic before letting it pass to any actual customer website or application. Ram’s team also is responsible for their AWS environments where they host less critical functions as well as their internal development platform that Imperva engineers use for their day-to-day work. These responsibilities are their core focus with limited resources for anything else. 

However, when Ram first joined Imperva in 2021, they employed a full-time engineer just to manage the 150 elasticache nodes needed for the Graylog deployment as well as handle ongoing maintenance for Graylog itself. This arrangement was prohibitively costly and inefficient.

The solution

In search of a turnkey solution, Ram and his team evaluated Splunk, Graylog Enterprise, and Coralogix. Despite Splunk’s reputation as a leading observability platform and despite the ease of transitioning to Graylog Enterprise from the open source version of Graylog they already were using, Imperva opted for Coralogix, drawn by its superior capabilities.

“From the first demo with Coralogix, we knew it was a winner. The ability to rapidly query logs straight from archived storage, create integrated alerts and dashboards using both logs and metrics, plus affordable, long-term log retention, influenced our decision to move ahead with Coralogix.” 

Ram Saiegh, DevOps leader of Infrastructure Operations at Imperva

While previously Imperva limited themselves to 3 month hot storage for their most important logs, Coralogix’s in-stream analysis, value-driven data routing and rapid archive query, has eliminated the need for expensive indexing and hot storage. This translates into Imperva doubling the amount of logs ingested and establishing one year retention as the new standard for all data. 

This has positively impacted productivity at Imperva with, for example, application teams better able to analyze long-term trends and support teams easily troubleshooting customer transactions that span longer periods, with full access to any relevant logs going back a year. 

Ram shared another example where Coralogix shines. “We sometimes see DDOS attacks with over 100 million IP addresses.  Before Coralogix, this would create extra load on our Prometheus setup with those IP addresses being defined as metrics. Now this is solved by sending all the related logs to Coralogix which smoothly handles the sudden load.” He added, “Coralogix’s parsing rules enable our Threat Detection team to easily extract key fields and values from logs which are sent to our dashboards alongside relevant metrics for faster root cause analysis and troubleshooting.” Ram concluded, “Coralogix delivers an engineer’s dream. Observability for all your logs, metrics and traces. Intuitive, fast and affordable.”

“Coralogix delivers an engineer’s dream. Observability for all your logs, metrics and traces. Intuitive, fast and affordable.”

Ram Saiegh, DevOps leader of Infrastructure Operations at Imperva

The benefits

Imperva’s choice of Coralogix as their observability solution brought them many positive benefits including: 

Rapid archive querying and affordable long-term retention

Coralogix’s unique architecture and in-stream analysis eliminates the need for indexing and hot storage for most logs. Imperva takes advantage of this and now sends roughly 60% of all logs directly to their archive storage for up to one year minimum retention and with Coralogix making all that data readily available via query, alerts, dashboards and more. This setup also ensures that Imperva don’t have to reindex their archived data which in the past was an extremely painful process. 

Customer support that goes above and beyond

All of the 500+ Imperva engineers using Coralogix know that they can count on the Coralogix support team to be available to them 24/7 with expert knowledge on all observability issues, large or small. On-site training sessions are also well attended and enjoyed by the Imperva team. Furthermore, feature requests from Imperva have often been delivered within weeks of initial request. 

Faster MTTR

Ram and the broader engineering team at Imperva found that their mean time to resolution was much faster and smoother with Coralogix than it had been with Graylog. With Coralogix’s full integration of logs, metrics and traces, Imperva’s troubleshooting exercises always yield the full context and interdependencies of any event as well as correlation with potentially related issues. 

Extracting pearls with Events2Metrics

Coralogix’s Event2Metrics feature has empowered Imperva to generate metrics and extract the most relevant data from logs for inclusion in multiple dashboards. This has helped reduce system load as well as reducing the associated log storage cost without missing critical information.

Built-in cost optimization equals more observability

TCO optimization allows Ram to determine which logs are indexed and sent to hot storage, which logs will undergo Coralogix’s in-stream analysis and then be sent to archive and which are sent straight to archive. 

This capability has resulted in Imperva doubling the amount of logs being monitored from 4TB daily to 8TB daily all without increasing their observability budget. 

Summary

In summary, the move from Graylog to Coralogix with its integrated approach to logs, metrics, and traces, led to doubling log volume ingested without increased costs. This shift not only streamlined Imperva’s operations but also significantly enhanced their troubleshooting and long-term trend analysis capabilities benefiting not only the over 700 engineers using Coralogix, but ultimately also all Imperva customers and partners.