Suricata Integration for Elastic

Serverless Observability Serverless Security Stack 9.0.0

The Suricata integration for Elastic enables the collection of logs from Suricata, a high-performance Network Intrusion Detection and Prevention System (IDS/IPS) and Network Security Monitoring (NSM) engine. This integration ingests Suricata's Extensible Event Format (EVE) JSON logs, allowing you to monitor, analyze, and alert on network security events within your Elastic deployment.

This integration facilitates several key security use cases, including:

• Intrusion Detection and Prevention (IDS/IPS)
• Network traffic analysis and threat hunting
• Security event correlation and alerting

This integration is compatible with Suricata versions that support EVE JSON output, including versions 5.x, 6.x, and 7.x.

This integration is compatible with Elastic Stack version 8.11.0 or higher.

This integration works by collecting data from the EVE JSON log file (eve.json) that Suricata produces. An Elastic Agent must be installed on the Suricata host, or have access to the log file. The agent is configured to monitor this log file, and it ships the collected events to your Elastic deployment for processing and analysis.

The Suricata integration collects events from Suricata's EVE JSON log. The specific event types can be configured in Suricata but typically include:

• alert: Security alerts generated by rules.
• anomaly: Protocol anomalies detected in traffic.
• flow: Network flow records.
• dns, http, smb, tls, ssh, dhcp, nfs, smtp, kerberos: Protocol-specific transaction logs.
• fileinfo: Information about files extracted from network traffic.
• stats: Periodic statistics about Suricata's performance.

• Real-time Threat Detection: Use Elastic SIEM to detect and respond to threats identified in Suricata alerts and events.
• Network Traffic Analysis: Leverage Kibana dashboards to visualize and analyze network traffic patterns, helping to identify anomalies and investigate security incidents.
• Threat Hunting: Search and correlate across various network protocol logs to proactively hunt for threats and indicators of compromise.
• Security Auditing: Maintain a searchable, long-term archive of network events for compliance and forensic analysis.

• A running instance of Suricata.
• Administrative access to the Suricata host to modify configuration files and restart the service.
• An Elastic Agent installed on the Suricata host, enrolled in a policy.

Elastic Agent must be installed on the host where Suricata is running. For more details, check the Elastic Agent installation instructions. You can install only one Elastic Agent per host.

Elastic Agent is required to read the log file and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines.

1. Locate and open the Suricata configuration file, suricata.yaml. The default location is typically /etc/suricata/suricata.yaml on Linux systems.

2. In the outputs section of the file, find the eve-log logger.

3. Ensure the eve-log output is enabled and configured to write to a file. For a comprehensive event log, you can enable various event types as shown below.

   outputs:
     # Extensible Event Format (nicknamed EVE) event log in JSON format
     - eve-log:
         enabled: yes
         filetype: regular #regular|syslog|unix_dgram|unix_stream
         filename: eve.json
         # In addition to alerts, you can enable other event types for full visibility.
         types:
           - alert
           - anomaly
           - dhcp
           - dns
           - fileinfo
           - flow
           - http
           - kerberos
           - nfs
           - smb
           - smtp
           - ssh
           - stats
           - tls
   		

   1. The log file to be collected

4. After saving your changes to suricata.yaml, restart the Suricata service for the configuration to take effect.

   sudo systemctl restart suricata
   		

5. The log file to be collected will be located in the default Suricata log directory, typically /var/log/suricata/eve.json.

1. In Kibana, navigate to Management > Integrations.
2. Search for "Suricata" and click on the result.
3. Click Add Suricata.
4. Configure the integration settings. Under Log file path, provide the absolute path to the Suricata log file (e.g., /var/log/suricata/eve.json).
5. Choose an existing or create a new agent policy.
6. Click Save and continue, then Save and deploy changes to apply the policy to any agents enrolled in it.

1. On the Suricata host, generate traffic that will trigger a test alert. You can use curl for this purpose.

   curl http://testmyids.com
   		

2. Check the Suricata eve.json log file to confirm that new event data is being written.

   tail -f /var/log/suricata/eve.json
   		

3. In Kibana, navigate to the Discover tab or open the pre-built [Logs Suricata] Overview dashboard.
4. Filter for Suricata data by using the KQL query event.dataset : "suricata.eve".
5. Verify that new log events from the Suricata host are appearing. You should see an alert related to the testmyids.com traffic.

• Issue: No data is being ingested into Elastic.
  • Solution:
    1. Verify the Suricata service is running on the host: sudo systemctl status suricata.
    2. Ensure the eve-log output is set to enabled: yes in suricata.yaml.
    3. Check that the file path configured in the Elastic integration policy matches the filename specified in suricata.yaml. The default is /var/log/suricata/eve.json.
    4. Verify file permissions. The user running the Elastic Agent must have read permissions for the eve.json file and execute permissions for its parent directories (/var/log/ and /var/log/suricata/).
    5. Check the Elastic Agent logs for any "permission denied" or "file not found" errors related to the Suricata log file.

• Issue: Fields are not parsed correctly or error.message is present in some documents.
  • Solution: This may occur if the EVE JSON output format has been customized in a way that the integration does not expect. Ensure your eve-log configuration in suricata.yaml produces standard EVE JSON. Compare your output with the format described in the official Suricata documentation.

• Suricata User Guide
• Troubleshooting Common Suricata Issues
• Suricata EVE JSON Output Documentation

Suricata's performance is highly dependent on the hardware it runs on, the volume of network traffic being monitored, and the complexity of the enabled rule sets. Suricata is multi-threaded and can be scaled by optimizing its configuration (e.g., run modes, CPU affinity) to match the underlying hardware. For high-traffic environments, ensure the disk I/O for the eve.json log file can keep up with the event generation rate.

For more information on architectures that can be used for scaling this integration, check the Ingest Architectures documentation.

The eve data stream collects all event types from Suricata's EVE JSON log output.

{
    "@timestamp": "2018-07-05T19:01:09.820Z",
    "agent": {
        "ephemeral_id": "58adcb6e-5d0e-4822-98a4-8d93557f8f2e",
        "id": "0a5c1566-c6fd-4e91-b96d-4083445a000e",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.9.0"
    },
    "data_stream": {
        "dataset": "suricata.eve",
        "namespace": "ep",
        "type": "logs"
    },
    "destination": {
        "address": "192.168.253.112",
        "ip": "192.168.253.112",
        "port": 22
    },
    "ecs": {
        "version": "8.17.0"
    },
    "elastic_agent": {
        "id": "0a5c1566-c6fd-4e91-b96d-4083445a000e",
        "snapshot": false,
        "version": "8.9.0"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "network"
        ],
        "created": "2023-08-08T15:09:13.171Z",
        "dataset": "suricata.eve",
        "ingested": "2023-08-08T15:09:14Z",
        "kind": "event",
        "type": [
            "protocol"
        ]
    },
    "input": {
        "type": "log"
    },
    "log": {
        "file": {
            "path": "/tmp/service_logs/eve-small.ndjson"
        },
        "offset": 0
    },
    "network": {
        "community_id": "1:NLm1MbaBR6humQxEQI2Ai7h/XiI=",
        "protocol": "ssh",
        "transport": "tcp"
    },
    "related": {
        "ip": [
            "192.168.86.85",
            "192.168.253.112"
        ]
    },
    "source": {
        "address": "192.168.86.85",
        "ip": "192.168.86.85",
        "port": 55406
    },
    "suricata": {
        "eve": {
            "event_type": "ssh",
            "flow_id": "298824096901438",
            "in_iface": "en0",
            "ssh": {
                "client": {
                    "proto_version": "2.0",
                    "software_version": "OpenSSH_7.6"
                },
                "server": {
                    "proto_version": "2.0",
                    "software_version": "libssh_0.7.0"
                }
            }
        }
    },
    "tags": [
        "forwarded",
        "suricata-eve"
    ]
}
		

These inputs can be used with this integration:

<details> <summary>logfile</summary>

For more details about the logfile input settings, check the Filebeat documentation.

To collect logs via logfile, select Collect logs via the logfile input and configure the following parameter:

• Paths: List of glob-based paths to crawl and fetch log files from. Supports glob patterns like /var/log/*.log or /var/log/*/*.log for subfolder matching. Each file found starts a separate harvester.

</details>

This integration includes one or more Kibana dashboards that visualizes the data collected by the integration. The screenshots below illustrate how the ingested data is displayed.

← →

×

×