An approach for deploying simple File Integrity Monitoring (FIM) using Sumo Logic

(for Linux)

ยท

6 min read

Introduction

Although Sumo Logic doesn't offer a pre-packaged FIM solution, it's now feasible to establish a basic FIM by utilizing Sumo Logic's Script Source feature.

With the help of some custom bash scripts, Sumo Logic can monitor file states (including md5 or SHA1 checksums, and attributes) and identify alterations to those states over time.
The changes detected are saved to a Sumo Logic index, which keeps track of these events. By utilizing the contents of this index, users can generate alerts, visualizations, and reports.

This approach creates a Script Source within Sumo Logic Collectors, which will run a bash script to monitor a list of files and directories for changes (FIM Compliance).
If any changes are detected, it will log them into the Sumo Logic platform.

Constraints

  • Since this is not a pre-built solution and relies on customized scripts to generate File Integrity Information for tracking purposes, these scripts are simply provided as examples 'as is'.

  • Determining which files should be monitored it's up to the deployer. You can also follow FIM best practices.

  • This custom implementation keeps track of file hashes and attributes. It records if a file has been changed, and describes what specific changes have occurred within the file. If you want to go deeper, that is something that you will need to investigate separately.

Yes. This is a bare-bones/workaround FIM solution for Sumo Logic.

Deployment

Configuring a 'Script Source' on the hosts you wish to monitor

The Sumo Collector agent does not allow using Script Source by default unless adding enableScriptSource into the user.properties file located in <SUMOCOLLECTOR_PATH>/config

To begin monitoring, we need to install a Sumo Logic Collector onto the machine(s) we want to track and set up the Script Source like the following bash script example:

#!/bin/bash

#List of files and directories to monitor
FILES=(/etc/hosts /etc/passwd /etc/shadow)

#Location to store checksums
CHECKSUM_DIR=/var/log/fim

#Create a directory for checksums if not exists
if [ ! -d $CHECKSUM_DIR ]; then
 mkdir -p $CHECKSUM_DIR
fi

#Loop through files and directories and calculate checksums
for file in "${FILES[@]}"
 do
 #Get the file basename
 basename=$(basename $file)
 #Calculate the checksum of the file
 checksum=$(sha256sum $file)
 #Create a file containing the checksum for the file
 echo "$checksum $basename" > $CHECKSUM_DIR/$basename.sha256
done

#Compare current checksums to previous checksums
for file in "${FILES[@]}"
 do
 #Get the file basename
 basename=$(basename $file)
 #Calculate the current checksum of the file
 current_checksum=$(sha256sum $file)
 #Get the previous checksum of the file
 previous_checksum=$(cat $CHECKSUM_DIR/$basename.sha256)
 #Compare current and previous checksums
 if [ "$current_checksum" != "$previous_checksum" ]; then
 #Generate Log
 message="$(date "+%Y-%m-%d %H:%M:%S") - The file $basename has been modified. CHANGES: "
 message+=$(diff --unified $file <(cat $CHECKSUM_DIR/$basename.sha256) | grep -v "sha256sum")
 echo -e $message

 #Update the checksum file with the new checksum
 echo "$current_checksum $basename" > $CHECKSUM_DIR/$basename.sha256
 fi
done

This bash script monitors the integrity of files and directories by calculating and comparing their checksums.
It uses the diff --unified command to compare the current file to its previous version, and appends the differences to a Log message.

Log Example:
2023-03-16 19:45:00 - The file hosts have been modified. CHANGES: --- /etc/hosts 2023-03-09 19:31:00.245760937 +0000 +++ /dev/fd/63 2023-03-16 19:45:00.324639261 +0000 @@ -1,3 +1 @@ -127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 -::1 localhost6 localhost6.localdomain6 -192.168.1.1 test_fim123456789 +ffe6gf0d20abe99999999ddcf999216308ba7ebga1999b2d4bb6a6bd3d5a7ed /etc/hosts hosts

NOTE: The previous bash script is already embedded within another Python script in the following steps below to ease deployment.

Log Explanation:

This log message shows that the hosts file has modifications. It also shows the changes that were made, for instance:

  • It shows the difference between the original file data /etc/hosts (from March 9, 2023) and the modified data at the time of the log message.

  • The text starting with "---" shows the original data, and the text starting with "+++" shows the modified data.

  • Also, text with a "-" symbol indicates removed/modified data, and text with a "+" symbol indicates added data.

  • The text between "@@" means that changes start on line 1 and end on line 3. (1,3)

In this case, the change was made to modify the following: 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost6 localhost6.localdomain6 192.168.1.1 test_fim123456789

The new line contains the SHA256 checksum of the modified file: ffe6f5f0d20abe10715715ddcf836216308ba7ebba1468b2d4bb6a6db3d5a7ed /etc/hosts hosts

Functionality

If you're monitoring multiple machines, make sure that every Script Source is collecting data using the same _sourceCategory tag. To deploy the Script Source into every host I've developed a Python script to do so in bulk:

import requests
import os

# Creates a Script Source based on payload
def createScriptSource(headers, collectorID, access_id, access_key):
  session = requests.session()

  data = {
    "source": {
        "name": "fim-auto-1",
        "category": "/fim",
        "description": "",
        "sourceType": "Script",
        "messagePerRequest": "true",
        "commands":["/bin/bash"],
        "script": "#!/bin/bash\n\n#List of files and directories to monitor\nFILES=(/etc/passwd /etc/shadow /etc/hosts)\n\n#Location to store checksums\nCHECKSUM_DIR=/var/log/fim\n\n#Create a directory for checksums if not exists\nif [ ! -d $CHECKSUM_DIR ]; then\n mkdir -p $CHECKSUM_DIR\nfi\n\n#Loop through files and directories and calculate checksums\nfor file in \"${FILES[@]}\"\n do\n #Get the file basename\n basename=$(basename $file)\n #Calculate the checksum of the file\n checksum=$(sha256sum $file)\n #Create a file containing the checksum for the file\n echo \"$checksum $basename\" > $CHECKSUM_DIR/$basename.sha256\ndone\n\n#Compare current checksums to previous checksums\nfor file in \"${FILES[@]}\"\n do\n #Get the file basename\n basename=$(basename $file)\n #Calculate the current checksum of the file\n current_checksum=$(sha256sum $file)\n #Get the previous checksum of the file\n previous_checksum=$(cat $CHECKSUM_DIR/$basename.sha256)\n #Compare current and previous checksums\n if [ \"$current_checksum\" != \"$previous_checksum\" ]; then\n #Generate Log\n message=\"$(date \"+%Y-%m-%d %H:%M:%S\") - AUTO - The file $basename has been modified. CHANGES: \"\n message+=$(diff $file <(cat $CHECKSUM_DIR/$basename.sha256) | grep -v \"sha256sum\")\n echo -e $message\n\n #Update the checksum file with the new checksum\n echo \"$current_checksum $basename\" > $CHECKSUM_DIR/$basename.sha256\n fi\ndone",
        "cronExpression":"0 0/15 * 1/1 * ? *"
        }
    }

  response = session.post(f'<YOUR_SUMOLOGIC_ENDPOINT>/api/v1/collectors/{collectorID}/sources',
    headers=headers, json=data, auth=(access_id, access_key))

  print(response.status_code, response.text)


# Set Header
headers = {'Content-Type': 'application/json'}

# Set Sumo Logic API credentials
access_id = "<YOUR_ACCESS_ID>"
access_key = "<YOUR_ACCESS_KEY>"

# Get Sumo Collector IDs
response = requests.get('<YOUR_SUMOLOGIC_ENDPOINT>/api/v1/collectors', headers=headers, auth=(access_id, access_key))

# Iterates through each Collector ID calling "createScriptSource" to create a Source for each Sumo Collector
for col in response.json()["collectors"]:
  collID = col["id"]
  createScriptSource(headers, collID, access_id, access_key)

The Python script performs the following tasks:

  • Sends a request to get the Sumo Collector IDs

  • Iterates through each Collector ID calling the createScriptSource function with the necessary parameters to create a Source for each Sumo Collector

  • The function creates a new Script Source in every Sumo Logic Collector using the information provided in the payload

Conclusion

The Python code can be used to automate the process of creating a Script Source in the Sumo Logic platform to monitor files and directories for changes.

By setting a cron expression, the script can be executed at regular intervals to ensure that the files and directories are continuously monitored.

Now, you know! ๐Ÿ˜‰

ย