flow timeout issue

[Kesslarr]

Hi there, here is code of function addPacket in class FlowGenerator:

public void addPacket(BasicPacketInfo packet) {
        if (packet == null) {
            return;
        }

        BasicFlow flow;
        long currentTimestamp = packet.getTimeStamp();
        String id;

        if (this.currentFlows.containsKey(packet.fwdFlowId()) || this.currentFlows.containsKey(packet.bwdFlowId())) {

            if (this.currentFlows.containsKey(packet.fwdFlowId())) {
                id = packet.fwdFlowId();
            } else {
                id = packet.bwdFlowId();
            }

            flow = currentFlows.get(id);
            // Flow finished due flowtimeout:
            // 1.- we move the flow to finished flow list
            // 2.- we eliminate the flow from the current flow list
            // 3.- we create a new flow with the packet-in-process
            if ((currentTimestamp - flow.getFlowStartTime()) > flowTimeOut) {
                if (flow.packetCount() > 1) {
                    if (mListener != null) {
                        mListener.onFlowGenerated(flow);
                    } else {
                        finishedFlows.put(getFlowCount(), flow);
                    }
                    //flow.endActiveIdleTime(currentTimestamp,this.flowActivityTimeOut, this.flowTimeOut, false);
                }
                currentFlows.remove(id);
                currentFlows.put(id, new BasicFlow(bidirectional, packet, flow.getSrc(), flow.getDst(), flow.getSrcPort(), flow.getDstPort(), this.flowActivityTimeOut));

                int cfsize = currentFlows.size();
                if (cfsize % 50 == 0) {
                    logger.debug("Timeout current has {} flow", cfsize);
                }

                // Flow finished due FIN flag (tcp only):
                // 1.- we add the packet-in-process to the flow (it is the last packet)
                // 2.- we move the flow to finished flow list
                // 3.- we eliminate the flow from the current flow list
            } else if (packet.hasFlagFIN()) {
                logger.debug("FlagFIN current has {} flow", currentFlows.size());
                flow.addPacket(packet);
                if (mListener != null) {
                    mListener.onFlowGenerated(flow);
                } else {
                    finishedFlows.put(getFlowCount(), flow);
                }
                currentFlows.remove(id);
            } else {
                flow.updateActiveIdleTime(currentTimestamp, this.flowActivityTimeOut);
                flow.addPacket(packet);
                currentFlows.put(id, flow);
            }
        } else {
            currentFlows.put(packet.fwdFlowId(), new BasicFlow(bidirectional, packet, this.flowActivityTimeOut));
        }
    }

As I know, there is no kind of FIN flag in UDP flow, to check UDP flow is ended or not, we will set a time for timeout, if next packet's fw/bw flowId is matched and currentTimestamp - flow.getFlowStartTime()) > flowTimeOut, means it flow is end. But I have some questions to ask:

1. Why didn't add packet into flow when it is timeout?
2. After add timeout flow into finishFlow and remove from currentFlow, why add a new one into currentFlow again?
3. It seems every kinds of packets(TCP/UDP/ICMP.....), all of them have run into this if-else, is that possible have this situation that a packet which protocol is TCP and is in the line with timeout but did't have FIN flag?

Also, I want to ask the code in class BasicFlow:

    public BasicFlow(boolean isBidirectional, BasicPacketInfo packet, byte[] flowSrc, byte[] flowDst, int flowSrcPort, int flowDstPort, long activityTimeout) {
        super();
        this.activityTimeout = activityTimeout;
        this.initParameters();
        this.isBidirectional = isBidirectional;
        this.firstPacket(packet);
        this.src = flowSrc;
        this.dst = flowDst;
        this.srcPort = flowSrcPort;
        this.dstPort = flowDstPort;
    }

    public void initParameters() {
        this.forward = new ArrayList<BasicPacketInfo>();
        this.backward = new ArrayList<BasicPacketInfo>();
        this.flowIAT = new SummaryStatistics();
        this.forwardIAT = new SummaryStatistics();
        this.backwardIAT = new SummaryStatistics();
        this.flowActive = new SummaryStatistics();
        this.flowIdle = new SummaryStatistics();
        this.flowLengthStats = new SummaryStatistics();
        this.fwdPktStats = new SummaryStatistics();
        this.bwdPktStats = new SummaryStatistics();
        this.flagCounts = new HashMap<String, MutableInt>();
        initFlags();
        this.forwardBytes = 0L;
        this.backwardBytes = 0L;
        this.startActiveTime = 0L;
        this.endActiveTime = 0L;
        this.src = null;
        this.dst = null;
        this.fPSH_cnt = 0;
        this.bPSH_cnt = 0;
        this.fURG_cnt = 0;
        this.bURG_cnt = 0;
        this.fHeaderBytes = 0L;
        this.bHeaderBytes = 0L;

    }

 public void firstPacket(BasicPacketInfo packet) {
        updateFlowBulk(packet);
        detectUpdateSubflows(packet);
        checkFlags(packet);
        ....
}
public void updateFlowBulk(BasicPacketInfo packet) {

        if (this.src == packet.getSrc()) {
            updateForwardBulk(packet, blastBulkTS);
        } else {
            updateBackwardBulk(packet, flastBulkTS);
        }

    }

In function BasicFlow, it initialize parameters, it seems doesn't have any chance match to this.src == packet.getSrc() this condition? Any answers are welcome and grateful, thanks.

[ltkhang]

Hi,

For the first block of questions:

1. Why did we have to add packets to flow when it "time out"?
2. In the time-out case, the tail of the previous flow is the head of the flowing flow. This makes sense because the packets just keep coming to the monitor (in realtime, this happens really fast), so the code try to do not miss any long flow by separate it into series of flows by timeout limitation.
3. Yup, some DDoS attacks based on the duration of TCP like slowris TCP, they just keep the connection and never complete it.

For the second block of questions:

Actually, it does, I was working with IDS 2018 dataset and everything went fine.

My code for applying ml methods you can find here https://github.com/ltkhang/ddos-detection-deeplearning and the code to apply cicflowmeter (python) in realtime here: https://github.com/ltkhang/sdn-ids-ddos-defense