Kubernetes with Filebeat

kubernetes versions

The following tutorial demonstrates how to integrate Coralogix with your Kubernetes cluster using Filebeat.

See other tutorials for:

Prerequisites

Before you will begin, make sure that you already have:

Installed Kubernetes Cluster
Enabled RBAC authorization mode support

Installation

First, you should create Kubernetes secret with Coralogix credentials:

kubectl -n kube-system create secret generic coralogix-filebeat-account-secrets \
    --from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX \
    --from-literal=COMPANY_ID=XXXX \
    --from-literal=CLUSTER_NAME=cluster.local

You should receive something like:

secret "coralogix-filebeat-account-secrets" created

Then you need to create filebeat-coralogix-logger resources on your Kubernetes cluster with this manifests:

$ kubectl create -k https://github.com/coralogix/integrations-docs/integrations/filebeat/kubernetes

Output:

serviceaccount "coralogix-filebeat-service-account" created
clusterrole "coralogix-filebeat-cluster-role" created
clusterrolebinding "coralogix-filebeat-cluster-role-binding" created
configmap "coralogix-filebeat-config" created
secret "coralogix-filebeat-certificate" created
daemonset "coralogix-filebeat-daemonset" created
service "coralogix-filebeat-service" created

Now filebeat-coralogix-logger collects logs from your Kubernetes cluster.

Here is an example log record:

{
    "cloud": {
        "availability_zone": "projects/379343634745/zones/us-central1-a",
        "instance_id": "7653580772456904060",
        "instance_name": "gke-coralogix-test-default-pool-4d86c144-sbkd",
        "machine_type": "projects/379343634745/machineTypes/n1-standard-1",
        "project_id": "coralogix-test",
        "provider": "gce"
    },
    "kubernetes": {
        "container": {
            "name": "prometheus-to-sd"
        },
        "labels": {
            "k8s-app": "kube-dns",
            "pod-template-hash": "989689126"
        },
        "namespace": "kube-system",
        "node": {
            "name": "minikube",
            "labels": {
                "kubernetes_io/arch": "amd64",
                "kubernetes_io/os": "linux"
            },
            "hostname": "minikube"
        },
        "pod": {
            "name": "kube-dns-fdfbdf56b-jbbw2",
            "uid": "56584469-534d-11e9-8bcd-42010a800179"
        },
        "replicaset": {
            "name": "kube-dns-fdfbdf56b"
        }
    },
    "@timestamp": "2019-03-31T00:45:53.973Z",
    "@version": "1",
    "host": {
        "name": "filebeat-coralogix"
    },
    "beat": {
        "hostname": "filebeat-coralogix-daemonset-98wxr",
        "name": "filebeat-coralogix",
        "version": "6.7.0"
    },
    "message": "E0331 00:45:53.970719 1 stackdriver.go:58] Error while sending request to Stackdriver Post /v3/projects/coralogix-test/timeSeries?alt=json: unsupported protocol scheme \"\"",
    "tags": [
        "kubernetes",
        "containers",
        "beats_input_codec_plain_applied"
    ]
}

To generate installation manifests for future customization, just execute:

kubectl kustomize https://github.com/coralogix/integrations-docs/integrations/filebeat/kubernetes

Uninstall

If you want to remove filebeat-coralogix-logger from your cluster, execute this:

kubectl -n kube-system delete secret filebeat-coralogix-account-secrets
kubectl delete -k https://github.com/coralogix/integrations-docs/integrations/filebeat/kubernetes

Kubernetes with Fluentd (Without Helm)

Next MongoDB Atlas