Google Workspace Data Ingestion - GCP

Note! This integration has been replaced. Please refer to our new integration here.

Overview

The following tutorial demonstrates how to successfully integrate Google Workspace with Coralogix and send us your logs using Filebeat and Google report API.

This document includes cluster dependent URLs. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches the top level domain of your Coralogix account (.com, .in etc.). Replace the variable part of the URL with this entry.

[table id=51 /]

Set up a GCP Service Account

Follow the official Google tutorial for setting up a service account.

Grant access to the Admin SDK API

Follow the official Google Workspace tutorial for granting access to the Admin API.

Note: to gather Google Alert Center, enable the Google Workspace Alert Center API on the GCP project.

Delegate domain-wide authority to your service account

Go to your Google Workspace domain’s Admin console.
Go to Main menu > Security > Access and data control > API controls.
Under the Domain-wide delegation pane, select Manage Domain Wide Delegation.
Click Add new, and fill in the details:
- Client ID - Enter the service account’s Client ID - you can find it in the service account’s details under Unique ID. It is also found in the client_id field of the credentials file that was auto-downloaded when you created a new key for your service account.
- OAuth Scopes - Enter https://www.googleapis.com/auth/admin.reports.audit.readonly
- Click Authorize to confirm your changes.

Once done with the configuration of Google Workspace, now we should start with filebeat.

Filebeat Setup

Please follow the link for Coralogix filebeat setup (Make sure to use the latest version)

Please make sure to have at least version 7.12 and up for filebeat.

Filebeat Google Workspace Configuration

ignore_older: 3h
filebeat.modules:
- module: google_workspace
  saml:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  user_accounts:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  login:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  admin:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  drive:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  groups:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"

fields_under_root: true
fields:
  PRIVATE_KEY: "Send-Your-Data API Key"
  COMPANY_ID: your_company_Id
  APP_NAME: "App_name for example google"
  SUB_SYSTEM: "Sub_system_name"

processors:
- drop_fields:
    fields:
    - event.original
    ignore_missing: true

logging:
  level: debug
  to_files: true
  files:
  path: /var/log/filebeat
  name: filebeat.log
  keepfiles: 10
  permissions: 0644

output.logstash:
  enabled: true
  hosts: ["logstashserver.Cluster URL:5015"] 
  tls.certificate_authorities: ["/etc/filebeat/ssl/coralogix.crt"]
  ssl.certificate_authorities: ["/etc/filebeat/ssl/coralogix.crt"]

After applying the changes. Start your filebeat service.

SAML	View users’ successful and failed sign-ins to SAML applications.
User Accounts	Audit actions carried out by users on their own accounts including password changes, account recovery details and 2-Step Verification enrollment.
Login	Track user sign-in activity to your domain.
Admin	View administrator activity performed within the Google Admin console.
Drive	Record user activity within Google Drive including content creation in such as Google Docs, as well as content created elsewhere that your users upload to Drive such as PDFs and Microsoft Word files.
Groups	Track changes to groups, group memberships and group messages.

If you want to learn more about the Google Workspace plugin please see the link.

Please note that Google Workspace defaults to a 2-hour polling interval because Google reports can go from some minutes up to 3 days of delay.
For more details on this, you can read more here.

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].

Bitbucket Data Ingestion

Next GELF