Rsyslog

Seamlessly send Coralogix your logs with Rsyslog using TCP (recommended), UDP, or manual installation.

Parameters

Private Key: Your Send-Your-Data API key is a unique ID that represents your company.

Company ID: A unique number that represents your company. Access your Company ID from the settings tab in your Coralogix dashboard.

Application Name & SubSystem Name: The name of your main application and subsystem(s). These are required in order to organize and query your data.

Coralogix Syslog Endpoint: Select one of the following TLS / TCP Syslog endpoints on the basis of your Coralogix domain.

TCP (Recommended)

STEP 1. Open /etc/rsyslog.d/ and create a file named coralogix.rsyslog.conf.

STEP 2. Paste the lines below in this file.

Input your application and subsystem name, private key, and company i.d.
Input your endpoint. The example below adopts an endpoint for the domain coralogix.com.

 $template CoralogixSyslogFormat,"{\"fields\": {\"private_key\":\"put-your-private-key\",\"company_id\":\"company-id\",\"app_name\":\"name-for-application\",\"subsystem_name\":\"%programname%\"},\"message\": {\"message\":\"%msg:::json%\",\"program_name\":\"%programname%\",\"pri_text\":\"%pri-text%\",\"hostname\":\"%HOSTNAME%\"}}"

*.* @@syslog.<span class="domain-value"></span>:6514;CoralogixSyslogFormat

STEP 3. Restart rsyslog. Ensure that port 6514 is open in your firewall to allow the connection.

Notes:

In the Coralogix rsyslog integration configuration, "hostname":"%HOSTNAME%" does not correspond to a "hostname" field that will appear in the syslog messages sent to Coralogix. This is the name of the server sending the messages.
To see the "hostname" in your Explore screen, select the computer field as a column.
- Click COLUMNS on the top right-hand sidebar of the UI. Drag the computer field to the list of fields "in use" on the right. Click APPLY.

UDP Installation

Determine Your Syslog Type

Coralogix only supports rsyslog, not syslog-ng.

STEP 1. Open your terminal window and type:

ls -d /etc/*syslog*

STEP 2. If you see rsyslog.d, you are using rsyslog. If you see syslog-ng, you are using syslog-ng, which has been deprecated.

If you do not see any of these options, install rsyslog. Most Linux distributions already have this syslog package included, so you should refer to the documentation of your Linux distribution for installation guidelines.

Rsyslog Configuration

STEP 1. Open your terminal window and open the file rsyslog.conf with your editor of choice.

#!bash
vi /etc/rsyslog.conf

STEP 2. Locate a line containing the parameter $RepeatedMsgReduction. If it is configured to ‘on’ then turn it off. If the line is commented, uncomment it. If you can’t find this parameter, add it. After the modification, it should look like this:

#!bash
$RepeatedMsgReduction off

STEP 3. Download the coralogix rsyslog configuration file rsyslog.conf (right-click link & save as) and save it in /etc/rsyslog.d/ folder.

#!bash
cd /etc/rsyslog.d &amp;&amp; wget

https://coralogix-public.s3-eu-west-1.amazonaws.com/integrations/rsyslog/coralogix.rsyslog.conf

#!bash
vi /etc/rsyslog.d/coralogix.rsyslog.conf

Template Configuration

Navigate to the template section and update the CoralogixSyslogFormat parameter with your specific values. Change only the following parameters: [YOUR COMPANY ID], [YOUR COMPANY KEY], [YOUR APPLICATION NAME], [YOUR APPLICATION SUBSYSTEM NAME].

#!bash
#*****************************************************************#
#                        TEMPLATE SECTION                         #
#*****************************************************************#
$template CoralogixSyslogFormat,"{"<wbr>fields": {"private_key":"YOUR COMPANY KEY","company_id":"YOUR COMPANY ID","app_name":"YOUR APPLICATION NAME","subsystem_name":"<wbr>YOUR APPLICATION SUBSYSTEM NAME"},"message": {&nbsp; &nbsp; &nbsp; &nbsp; "message":"%msg:::json%","<wbr>program_name":"%programname%<wbr>","pri_text":"%pri-text%"<wbr>,"hostname":"%HOSTNAME%",<wbr>"tag":"%syslogtag%"}}n"

Notes:

If you have several applications or subsystem components writing to the same syslog, create this template for each and give each a unique name. For instance, if you have an application with the name myapp with 2 subsystems - mydal and myclient - running on the same host, the template configuration should look like this:

#!bash
$template CoralogixSyslogForma1,"{"<wbr>fields": {"private_key":"530e925d-<wbr>be9e-****-****-75884f54efbe",<wbr>"company_id":"****","app_<wbr>name":"prod","subsystem_<wbr>name":"nginx"},"message": {&nbsp; &nbsp; &nbsp; &nbsp; "message":"%msg:::json%","<wbr>program_name":"%programname%<wbr>","pri_text":"%pri-text%"<wbr>,"hostname":"%HOSTNAME%",<wbr>"tag":"%syslogtag%"}}n"

Filter Configuration

STEP 1. Navigate to the filter section of the file.

#*****************************************************************#
#                        FILTER SECTION                           #
#*****************************************************************#
#Filter messages and send only the relevant one
#For more information and other filter options please refer to:
#http://www.rsyslog.com/doc/v8-stable/configuration/filters.html
#This will filter messages and send only the one with program name equal to: myApp
#:programname, isequal, "myapp" 
#This will filter messages and send only the one with facility equal to: user
#:syslogfacility-text, isequal, "user"

STEP 2. If you do not want to send the entirety of your syslog data, configure syslog to send logs only from your application, rather than entire messages coming from your Linux OS. Using rsyslog filters, forward only those messages that successfully pass your filter.

For example, to filter only application with the name myapp:

#!bash
:programname, isequal, "myapp"

To filter only applications writing to facility user:

#!bash
:syslogfacility-text, isequal, "user"

You can also filter by other parameters and you can use regular expressions as well. Find out more here.

Destination Configuration

STEP 1. Navigate to destination section.

#!bash
#*****************************************************************#
#                      DESTINATION SECTION                        #
#*****************************************************************#
#Send with UDP
*.* @<cx_syslog_endpoint>;CoralogixSyslogFormat

#Print messages locally. Great for debugging #*.* /var/log/messages;CoralogixSyslogFormat

STEP 2. Configure rsyslog to send logs via UDP protocol using port 5140.

STEP 3. [Optional] Redirect your syslog messages to your local file to see the exact data that is being sent to Coralogix. This step allows you to:

debug
check how logs are written locally
filter your syslog data based on your application name, but are unsure of the exact name of the process

Uncomment this line:

#!bash
*.* /var/log/messages;CoralogixSyslogFormat

Example

Here is an example for the log output of an application myapp sending log: Hello World!:

#!bash
my-user@test-machine:~$ tail -f /var/log/messages
Nov 10 21:10:06 127.0.0.1/127.0.0.1 crx=1 crxversion=1 crxtype=syslog crxcompid=1
crxpkey=11111111-1111-1111-1111-1111111111 crxapp=myapp crxsubsys=mydal crxhostname=hostname1
crxtag='' &lt;CRX.TIME_STAMP=1478812206820706&gt; &lt;CRX.PRI=daemon.err&gt; &lt;CRX.CATEGORY_REWRITE=''&gt;
&lt;CRX.SEVERITY_REWRITE=''&gt; &lt;CRX.MSG_REWRITE=''&gt; &lt;CRX.PROGRAM_NAME=my-app1&gt; &lt;CRX.MSG=Hello World!&gt;

crxapp=myapp: name of the application

<CRX.PROGRAM_NAME=my-app1>: process/program name that sent the log line to syslog

If you defined several templates for each program/process name, then instead of using one generic redirect rule

#!bash
*.* @@<cx_syslog_endpoint>;CoralogixSyslogFormat

use a conditional redirect, such as:

#!bash
if $programname == 'mydal' then @@<cx_syslog_endpoint>;CoralogixSyslogFormat1
if $programname == 'myclient' then @@<cx_syslog_endpoint>;CoralogixSyslogFormat2

Note:

By default, syslog listens for messages on a LOCAL Unix domain socket. In you are sending messages to your local syslog using UDP, you are required to enable this option by following these instructions.

STEP 4. Save the file and restart rsyslog. The command to restart rsyslog daemon can vary from one Linux distribution to another, but in most cases it will be:

#!bash
sudo service rsyslog restart

Test Your Configuration

STEP 1. To send a test message, use the Linux logger command:

#!bash
logger -p info Hello World!

This should send a message “Hello World!” with severity information. If you enabled the option to redirect your syslog messages to a local file, you should see this message with the command:

#!bash
tail -f /var/log/messages

If you don’t see the message, check your configuration.

STEP 2. Navigate to the ‘Log Query’ menu in your Coralogix Explore UI. Press the Go button to search for logs from the last 15 minutes. If you see your logs, you are now connected to Coralogix. If not, book an implementation session.

Docker Syslog Configuration

This section describes how to work with Docker syslog driver and redirect your messages to Coralogix server.

Docker provides several log drivers that can redirect console output logs to a log server. View a complete list of log drivers here.

STEP 1. Ensure that the host running your Docker container has a syslog daemon up and running:

ps aux | grep syslog

STEP 2. Configure your syslog to forward messages to Coralogix server. Refer to the configuration steps shared earlier in this document.

STEP 3. Run your docker with the –log-driver option, for instance:

docker run -d –log-driver=syslog ubuntu /bin/sh -c “while true; do echo hello world; sleep 1; done”

If you are using docker-compose V2, you can alternatively use:

my-container1:

image: ubuntu

entrypoint: /bin/sh -c “while true; do echo hello world; sleep 1; done”

logging:

driver: “syslog”

Your logs should now appear in Coralogix dashboard.

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].

SyslogNG

Next OpenTelemetry