Suricata drop

[pawsitivtyBE]

Version

2.4.141

Installation Method

Security Onion ISO image

Description

configuration

Installation Type

Distributed

Location

airgap

Hardware Specs

Exceeds minimum requirements

CPU

112

RAM

269

Storage for /

75Gb

Storage for /nsm

13Tb

Network Traffic Collection

tap

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

Hey,

I keep getting suricata loss (or drops?) with the low amount of 1.3Gbps traffic as you can see in screenshot.
I've already played around with the ring-size, threads, cpu-affinity and making sure the CPU workers are set to the NUMA node my NIC is connected to.
Any help on this would be appreciated. I've been searching for a few days now.

I use tcpreplay to send a pcap file from a multitude of hosts, which are fiber tapped.

cpu-affinity: yes
worker-cpu: 2-56
max-pending packets: 65534
threads: 70
ring-size: 4096

Doing the TOP command, yes I can see CPU 100%+, but even when I got it at around 60% by adjusting the thread count I still got loss.

Kind regards,

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[cm-ops]

How are your CPUs pinned?

[pawsitivtyBE]

All to NUMA node 0, which my NIC is connected to.
Changing any of the these values I found to have no impact with htop.

    NUMA node(s):        2
    NUMA node0 CPU(s):   0-27,56-83
    NUMA node1 CPU(s):   28-55,84-111

I also have 1-9 set for managent-cpus for suricata.

I did some further testing today and am noticing that when adding "--unique-ip" for tcpreplay, which should give every loop unique IP adresses that I am getting to 1.4Gbps at 0% now. What do you think about this? Though with insane Capture Loss, which might just be a pcap issue I guess.

At this moment I cannot progress further than 1.4Gbps it seems, I'll work on that on monday.

Kind regards

[cm-ops]

Have you tried not pinning the CPUs? What is the result?

[pawsitivtyBE]

Yeah, originally it is configured like that out of the box, didn't work.
I found the solution in the meantime:
Not enough cores for the management.
I was only looking at the workers and changing that, leaving the management (which handles the overhead of all threads created on the worker cores) at 1 core.
I increased that to 9 cores and it works fine so far. Gonna test further now.