OSSEC Ubuntu Server
OSSEC is a useful tool in monitoring for malicious activity across various servers. It's lightweight, and easy to install an agent and have it reporting to the master server within minutes. Unfortunately, there is no automated solution to configuring agents remotely via Ansible or other tools that I am aware of.
OSSEC Official Downloads provides the official download sources, and after selecting one the file can be downloaded right to your working directory using wget -
# Download the tar for linux server / agent installation
wget https://github.com/ossec/ossec-hids/archive/3.3.0.tar.gz
# Extract it
tar xf 3.3.0.tar.gz
Now, we have created the below directory -
ossec-hids-3.3.0/
BUGS CONFIG INSTALL README.md active-response doc install.sh
CHANGELOG CONTRIBUTORS LICENSE SUPPORT.md contrib etc src
OSSEC Server Configuration
Its important to note that we are installing the server in these instructions, and not an agent manager. An Agent manager is a much lighter installation from the same tarball that allows connecting to this server and reporting alerts through one host.
Creating OSSEC User
Once you are logged in to the host you wish to act as the server sending email alerts and recieving reports from agents and create a new user to manage OSSEC -
admin@host:~$ git clone https://github.com/shaunrd0/klips
Cloning into 'klips'...
remote: Enumerating objects: 295, done.
remote: Counting objects: 100% (295/295), done.
remote: Compressing objects: 100% (187/187), done.
remote: Total 295 (delta 109), reused 255 (delta 72), pack-reused 0
Receiving objects: 100% (295/295), 47.48 KiB | 3.96 MiB/s, done.
Resolving deltas: 100% (109/109), done.
admin@host:~$ cp klips/scripts/adduser.sh .
admin@host:~$ sudo ./adduser.sh ossec 5555
Adding user `ossec' ...
Adding new group `ossec' (5555) ...
Adding new user `ossec' (5555) with group `ossec' ...
Creating home directory `/home/ossec' ...
Copying files from `/etc/skel' ...
Enter 1 if ossec should have sudo privileges. Any other value will continue and make no changes
1
Configuring sudo for ossec...
Enter 1 to set a password for ossec, any other value will exit with no password set
1
Changing password for ossec...
New password:
Retype new password:
passwd: password updated successfully
Dependencies / Installation Files
Now that we have our user created, lets become them and prepare to install the OSSEC server
admin@host:~$ sudo -iu ossec
To run a command as administrator (user "root"), use "sudo <command>".
see "man sudo_root" for details.
ossec@host:~$ wget https://github.com/ossec/ossec-hids/archive/3.3.0.tar.gz
--2019-08-31 19:40:50-- https://github.com/ossec/ossec-hids/archive/3.3.0.tar.gz
Resolving github.com (github.com)... 192.30.253.113
Connecting to github.com (github.com)|192.30.253.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/ossec/ossec-hids/tar.gz/3.3.0 [following]
--2019-08-31 19:40:51-- https://codeload.github.com/ossec/ossec-hids/tar.gz/3.3.0
Resolving codeload.github.com (codeload.github.com)... 140.82.114.10
Connecting to codeload.github.com (codeload.github.com)|140.82.114.10|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1680499 (1.6M) [application/x-gzip]
Saving to: ‘3.3.0.tar.gz’
3.3.0.tar.gz 100%[===================================>] 1.60M --.-KB/s in 0.06s
2019-08-31 19:40:51 (28.0 MB/s) - ‘3.3.0.tar.gz’ saved [1680499/1680499]
ossec@host:~$ tar xf 3.3.0.tar.gz ossec-hids-3.3.0/
ossec@host:~$ sudo apt install build-essential gcc make
Installing OSSEC Server
Now we have our user created, permissions granted, and dependencies / files we need to install OSSEC. Navigate within the ossec-hids-3.3.0/ directory and run sudo ./install.sh. You will be prompted to select preferred settings for this installation. Pay attention to the prompts and respond accordingly, this is where the difference is seen in installing an Agent vs installing the OSSEC Monitoring Server.
You may see the below error for a missing dependency - pcre2.
5- Installing the system
- Running the Makefile
cd external/pcre2-10.32/ && \
./configure \
--prefix=/home/kossec/ossec-hids-3.3.0/src/external/pcre2-10.32//install \
--enable-jit \
--disable-shared \
--enable-static && \
make install-libLTLIBRARIES install-nodist_includeHEADERS
/bin/sh: 1: cd: can't cd to external/pcre2-10.32/
Makefile:770: recipe for target 'external/pcre2-10.32//install/lib/libpcre2-8.a' failed
make: *** [external/pcre2-10.32//install/lib/libpcre2-8.a] Error 2
Error 0x5.
Building error. Unable to finish the installation.
Continue on by running the below commands, which will add the required files to your extracted ossec-hids-3.3.0/ directory -
# Error - build fails because of missing pcre2
# Run these commands within the installation directory
cd ossec-hids-3.3.0
wget https://ftp.pcre.org/pub/pcre/pcre2-10.32.tar.gz
tar xzf pcre2-10.32.tar.gz -C src/external
After fixing the above error, you may see another when attempting to install again. The error below is due to the missing libz-dev dependency -
os_zlib/os_zlib.c:13:10: fatal error: zlib.h: No such file or directory
#include <zlib.h>
^~~~~~~~
compilation terminated.
Makefile:727: recipe for target 'os_zlib/os_zlib.o' failed
make: *** [os_zlib/os_zlib.o] Error 1
Error 0x5.
Building error. Unable to finish the installation.
If you see this error, you'll need to install zlib using the below command
# Error Making os_auth
sudo apt install -y libz-dev
Enable EMail Notifications
To enable email alerting from the OSSEC server and the remote agents it monitors, follow the instructions on Configuring Postfix and then make the appropriate changes to the mail settings in /var/ossec/etc/ossec.conf below -
<ossec_config>
<global>
<email_notification>yes</email_notification>
<email_to>someone@somedomain.com</email_to>
<smtp_server>127.0.0.1</smtp_server>
<email_from>ossec@host</email_from>
</global>
There is no need to point smtp_server to a mail server directly, handling mail this way simply bounces the messages off the localhost's configuration we already have setup and validated working from Configuring Postfix.
If you can send mail with the below command, chances are the issue is within OSSEC and not your servers postifx configuration or gmail authentication.
echo "This email confirms that Postfix is working" | mail -s "Testing Posfix" emailuser@example.com
If this command fails, go back and check that you've configured Postfix correctly with GMail, and once you have that verified come back here to finish up with OSSEC.
Managing Agents
To use the manage_agent utility that comes with OSSEC, run /var/ossec/bin/manage_agents either as root or with sudo (sudo /var/ossec/bin/manage_agents)
If you are running the manage_agents utility on an OSSEC Server -
****************************************
* OSSEC HIDS v3.3.0 Agent manager. *
* The following options are available: *
****************************************
(A)dd an agent (A).
(E)xtract key for an agent (E).
(L)ist already added agents (L).
(R)emove an agent (R).
(Q)uit.
Choose your action: A,E,L,R or Q: q
If you are running the manage_agents utility on an OSSEC Agent -
****************************************
* OSSEC HIDS v3.3.0 Agent manager. *
* The following options are available: *
****************************************
(I)mport key from the server (I).
(Q)uit.
Choose your action: I or Q:
This tool is used to define an agent on the server, export its key, then import the server's key within the ossec-agent on the remote host, allowing the connection. Its important to follow these steps carefully, as any discrepency in IP or client.keys will result in a connection failing.
Defining Agents
To start, on the OSSEC Server, run the manage_agents utility and add an agent -
****************************************
* OSSEC HIDS v3.3.0 Agent manager. *
* The following options are available: *
****************************************
(A)dd an agent (A).
(E)xtract key for an agent (E).
(L)ist already added agents (L).
(R)emove an agent (R).
(Q)uit.
Choose your action: A,E,L,R or Q: a
- Adding a new agent (use '\q' to return to the main menu).
Please provide the following:
* A name for the new agent: test
* The IP Address of the new agent: 0.0.0.0
* An ID for the new agent[007]: 007
Agent information:
ID:007
Name:test
IP Address:0.0.0.0
Confirm adding it?(y/n): y
Agent added.
Extracting Server Keys (Server)
We've define our agent with the local OSSEC Server, and prepared it for the connection. Now we need to continue through the prompts and extract the key for the agent to copy over onto our remote host -
****************************************
* OSSEC HIDS v3.3.0 Agent manager. *
* The following options are available: *
****************************************
(A)dd an agent (A).
(E)xtract key for an agent (E).
(L)ist already added agents (L).
(R)emove an agent (R).
(Q)uit.
Choose your action: A,E,L,R or Q: e
Provide the ID of the agent to extract the key (or '\q' to quit): 007
Agent key information for '007' is:
MDA2Ig4LjQzIDkxZDYmRlZGN5mOG5NzY5Nd325dmFmMTU0NzZkddaDM1ND456431MWY1ODhhMDyjukuMDYzg4MzA5MmM=
** Press ENTER to return to the main menu.
Exit the prompts, and copy this key or temporarily store it in the text file. We will need it to register the agent with its remote server.
Importing Server Keys (Agent)
Open a terminal on the host you have installed the OSSEC Agent on, and run the manage_agents utility just as we did on the last host (sudo /var/ossec/bin/manage_agents) -
****************************************
* OSSEC HIDS v3.3.0 Agent manager. *
* The following options are available: *
****************************************
(I)mport key from the server (I).
(Q)uit.
Choose your action: I or Q: i
* Provide the Key generated by the server.
* The best approach is to cut and paste it.
*** OBS: Do not include spaces or new lines.
Paste it here (or '\q' to quit): MDA2Ig4LjQzIDkxZDYmRlZGN5mOG5NzY5Nd325dmFmMTU0NzZkddaDM1ND456431MWY1ODhhMDyjukuMDYzg4MzA5MmM
Agent information:
ID:007
Name:test
IP Address:0.0.0.0
Confirm adding it?(y/n): y
Added.
** Press ENTER to return to the main menu.
Starting OSSEC Agent
That's it! Press enter and exit through the prompts, then just sudo /var/ossec/bin/ossec-control restart to apply our changes on both the server and the agent.
Sometimes, when initially starting an agent on a new host you will get like the below -
ossec-syscheckd(1226): ERROR: Error reading XML file '/var/ossec/etc/shared/agent.conf': XMLERR: File '/var/ossec/etc/shared/agent.conf' not found. (line 99).
ossec-syscheckd(1226): ERROR: Error reading XML file '/var/ossec/etc/shared/agent.conf': XMLERR: File '/var/ossec/etc/shared/agent.conf' not found. (line 99).
Just touch /var/ossec/etc/shared/agent.conf and sudo /var/ossec/bin/ossec-control restart / sudo /var/ossec/bin/ossec-control start again. The services should start normally. This is an old bug I came across in this GitHub issue.
Troubleshooting Agent Connections
If an agent isnt connecting, try the following commands to check for common problems -
Verify Agent Server Configuration
When attempting to start the OSSEC agent, you may see errors like the following -
Deleting PID file '/var/ossec/var/run/ossec-logcollector-23324.pid' not used...
Deleting PID file '/var/ossec/var/run/ossec-agentd-23320.pid' not used...
ossec-logcollector not running ..
ossec-syscheckd not running ..
ossec-agentd not running ..
Killing ossec-execd ..
OSSEC HIDS v2.9.0 Stopped
Starting OSSEC HIDS v2.9.0 (by Trend Micro Inc.)...
Started ossec-execd...
2019/11/04 17:17:16 ossec-agentd: INFO: Using notify time: 600 and max time to reconnect: 1800
Started ossec-agentd...
Started ossec-logcollector...
2019/11/04 17:17:19 ossec-syscheckd(1210): ERROR: Queue '/var/ossec/queue/ossec/queue' not accessible: 'Connection r
efused'.
2019/11/04 17:17:19 rootcheck(1210): ERROR: Queue '/var/ossec/queue/ossec/queue' not accessible: 'Connection refused
'.
Checking the logs within /var/ossec/logs/ossec.log reveals that no there is no valid OSSEC server configuration. To resolve this, ensure that you used the correct IP during installation for your OSSEC Server, then use the /var/ossec/bin/manage_agents tool on both the server and the agent to Add / Export Agent Keys and then run sudo /var/ossec/bin/ossec-control restart to restart the agent.
Check Agent Status on Server
There are many ways to check the agent status on the OSSEC Server host itself. One of which is within the manage_agents utility used to register new agents. Simply run sudo /var/ossec/bin/manage_agents and select L to list added agents and their current status with the server.
****************************************
* OSSEC HIDS v3.3.0 Agent manager. *
* The following options are available: *
****************************************
(A)dd an agent (A).
(E)xtract key for an agent (E).
(L)ist already added agents (L).
(R)emove an agent (R).
(Q)uit.
Choose your action: A,E,L,R or Q: l
Available agents:
ID: 001, Name: host, IP: 100.200.300.400
ID: 002, Name: host2, IP: 200.200.300.400
ID: 003, Name: host3, IP: 300.200.300.400
ID: 004, Name: host4, IP: 400.200.300.400
** Press ENTER to return to the main menu.
We can also list all agents with the built in OSSEC utility list_agents, run the following command to list all agents, even if they are disconnected or have never been connected in the first place -
admin@host:~# /var/ossec/bin/list_agents -a
host-100.200.300.400 is active.
host2-200.200.300.400 is active.
host3-300.200.300.400 is active.
host4-400.200.300.400 is active.
Check OSSEC Logs
OSSEC is an HIDS, which means it takes a ton of logs. Use them to your advantage, in this care we can easily check for general OSSEC errors within the /var/ossec/logs/ossec.log
Logs such as the below could indicate an incorrect client.key, which is configured when adding the agent to the OSSEC server, and again on the remote host when importing the generated key. If you see errors like this, try going back and removing the agent from the OSSEC server, create a new one and be sure you are using the correct IP for your agent.
2019/08/31 18:14:05 ossec-remoted(1213): WARN: Message from '100.200.300.400' not allowed.
2019/08/31 18:14:43 ossec-remoted(1213): WARN: Message from '100.200.300.400' not allowed.
2019/08/31 18:14:49 ossec-remoted(1213): WARN: Message from '100.200.300.400' not allowed.
2019/08/31 18:14:53 ossec-remoted(1213): WARN: Message from '100.200.300.400' not allowed.
2019/08/31 18:14:58 ossec-remoted(1213): WARN: Message from '100.200.300.400' not allowed
It's important to note that when installing the agent on the remote host you wish to monitor, you are prompted to input the Server IP - this is an important step and if done incorrectly just re-run the installation as if you were starting on a fresh host, and when prompted to update OSSEC input NO. Follow the prompts and install in the same location you did previously, and OSSEC will prompt you to remove the previous installation before reinstalling. Do so, and input the correct IP of the host running your OSSEC Server.
A useful trick when debugging mail issues with ossec-maild -
tail -f /var/ossec/logs/ossec.log | grep ossec-maild
For a live feed, filtered log on ossec-maild within the base OSSEC server logs.
Check IPTables
You may need to allow traffic on ports used by OSSEC with iptables, to do so, run the following commands. If you make any changes to iptables, be sure to sudo apt install iptables-persistent, which will enable iptables to carry over your settings if your host is restarted.
View current iptables -
iptables -nL
Allow the Agent to connect to our OSSEC Server host on the specified port, run -
iptables -A INPUT -p UDP --dport 1514 -s your_agent_ip -j ACCEPT
Allow the OSSEC Server to connect to our agent on the specified port, run -
iptables -A INPUT -p UDP --dport 1514 -s your_server_ip -j ACCEPT
Allow all outbound traffic, assuming no malicious activity will come from within -
iptables -A OUTPUT -j ACCEPT
Check Network Traffic
Still not sure why your agent isn't connecting? Try to monitor network traffic on the ports you're running OSSEC across - this could give you some idea of if there is traffic moving on one host and not the other, and lead you to where it is being stopped.
The output below is healthy traffic from my OSSEC server monitoring a few hosts -
tcpdump -i eth0 port 1514
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:48:48.984246 IP 100.200.300.400.53359 > monitor.1514: UDP, length 265
21:48:48.984314 IP 100.200.300.400.53359 > monitor.1514: UDP, length 265
21:48:49.991446 IP 100.200.300.400.53359 > monitor.1514: UDP, length 233
21:48:49.992233 IP monitor.1514 > 159.65.224.81.53359: UDP, length 73
21:48:53.914955 IP 100.200.300.400.48049 > monitor.1514: UDP, length 265
21:48:54.990058 IP 200.200.300.400.53359 > monitor.1514: UDP, length 249
Duplicate Counter Errors
Sometimes, when attempting an update or reinstall / reconfiguration of the OSSEC server or agents you may see the errors below within your /var/ossec/logs/ossec.log -
2019/11/05 01:58:16 ossec-remoted: WARN: Duplicate error: global: 0, local: 7316, saved global: 0, saved local:7317
2019/11/05 01:58:16 ossec-remoted(1407): ERROR: Duplicated counter for 'box1'.
2019/11/05 01:59:20 ossec-remoted: WARN: Duplicate error: global: 0, local: 5910, saved global: 0, saved local:5911
2019/11/05 01:59:20 ossec-remoted(1407): ERROR: Duplicated counter for 'box2'.
OSSEC uses a counter system to avoid repeat alerts on the same attacks. If you recently refactored your monitoring, it is easy to get your wires crossed in the process and knock the OSSEC server out of sync with its agents. This can happen any number of ways, but a resolution can be found in the OSSEC FAQ Docs. The process is detailed below.
On every agent:
stop ossec
go to: .../ossec/queue/rids (or ossec-agent/rids on Windows) and remove every file in there.
Go to the server:
Stop ossec
Remove the rids file with the same name as the agent id that is reporting errors.
Restart the server
Restart the agents.
More links: