use tcnl.Class().Add(), but received "netlink receive: invalid argument"

[SMALL-head]

Describe the issue

I am learning TC recently. I want to do something like $TC class add dev {ifName} parent {parent} classid {classid} htb rate {limit}, but received netlink receive: invalid argument as a result.

Minimal code example to reproduce the issue

Because I don't want to change the network config in host, I call AddHTBClass in Linux network namespace(ip nets add ns1, BTW, I'm using Ubuntu20.04). I noticed that tc.Config has NetNS attribute, but I'm not sure how to deal with it, so I use another way to make the func works in NS correctly. To make the ns environment, use the bash below:

ip netns add ns1
ovs-vsctl add-br br0
ifconfig br0 up
ip link add v-ns1 type veth peer name b-ns1
ovs-vsctl add-port br0 b-ns1
ifconfig b-ns1 up
ip link set v-ns1 netns ns1
ip netns exec ns1 ifconfig v-ns1 up

Here's my main func. in this function, v-ns1 is one side of a veth pair where the other side b-ns1 is set on an ovs in the host .

package main

import (
        "net"

	"github.com/containernetworking/plugins/pkg/ns"
	"github.com/florianl/go-tc"
	"github.com/florianl/go-tc/core"
	"github.com/sirupsen/logrus"
        "github.com/mdlayher/netlink"
	"golang.org/x/sys/unix"
)

const (
	path_ns = "/var/run/netns"
)

// AddHTBToInterface  acts like
// $TC qdisc add dev {ifName} root handle 1:0 htb
func AddHTBToInterface(ifName string) error {
	ifByName, err := net.InterfaceByName(ifName)
	if err != nil {
		return err
	}

	qdiscHTB := createHTBObject(uint32(ifByName.Index), 
							core.BuildHandle(0x1, 0x0), 
							tc.HandleRoot, 
							nil, &tc.HtbGlob{Version: 0x3, Rate2Quantum: 0xa,})

	tcnl, err := tc.Open(&tc.Config{})
	if err != nil {
		logrus.Errorf("err creating tcnl, err = %s", err)
		return err
	}
	defer tcnl.Close()
	if err := tcnl.SetOption(netlink.ExtendedAcknowledge, true); err != nil {
		logrus.Warnf("EXT_ACK set failed, err = %v", err)
	}

	if err := tcnl.Qdisc().Add(qdiscHTB); err != nil {
		return err
	}
	return nil
}

// AddHTBClass acts as
// $TC class add dev {ifName} parent {parent} classid {classid} htb rate {limit}
func AddHTBClass(ifName string, parent uint32, classid uint32, limit uint64) error {
	ifByName, err := net.InterfaceByName(ifName)
	if err != nil {
		logrus.Errorf("cannot find %s interface", ifName)
		return err
	}

	rate := limit	
	htbObject := 
		createHTBObject(uint32(ifByName.Index), 
					classid,
					parent,
					&rate, &tc.HtbGlob{Version: 0x3, Rate2Quantum: 0xa,})

	tcnl, err := tc.Open(&tc.Config{})
	if err != nil {
		logrus.Errorf("err creating tcnl, err = %s", err)
		return err
	}
	defer tcnl.Close()
	if err := tcnl.SetOption(netlink.ExtendedAcknowledge, true); err != nil {
		logrus.Warnf("EXT_ACK set failed, err = %v", err)
	}
	if err := tcnl.Class().Add(htbObject); err != nil {
		return err
	}
	
	return nil
}

func createHTBObject(ifIndex uint32, handle uint32, parent uint32, rate *uint64, htbInit *tc.HtbGlob) *tc.Object {
	return &tc.Object{
		Msg: tc.Msg{
			Family:  unix.AF_UNSPEC,
			Ifindex: uint32(ifIndex),
			Handle:  handle,
			Parent:  parent,
			Info:    0,
		},
		Attribute: tc.Attribute{
			Kind: "htb",
			Htb: &tc.Htb{
				Init: htbInit,
				Rate64: rate,
			},
		},
	}
}

func createHTB() {

	ns1, err := ns.GetNS(path_ns + "/ns1")
	if err != nil {
		logrus.Fatalf("can not open ns1, err=%v", err)
	}

	err = ns1.Do(func(hostNS ns.NetNS) error {
		return AddHTBToInterface("v-ns1")
	})

	if err != nil {
		logrus.Errorf("ns1 tc get err, err = %s", err)
	}

}

func createClass() {
	ns1, err := ns.GetNS(path_ns + "/ns1")
	if err != nil {
		logrus.Fatalf("can not open ns1, err=%v", err)
	}


	err = ns1.Do(func(hostNS ns.NetNS) error {
		return AddHTBClass("v-ns1", core.BuildHandle(1,0), core.BuildHandle(1, 1), 100000)
	})

	if err != nil {
		logrus.Errorf("ns1 tc get err, err = %s", err)
	}
}

func main() {
	//createHTB()   // run successfully 
	createClass()
	
}

after running createHTB(), I use ip netns exec ns1 tc -s qdisc ls dev v-ns1 to check v-ns1, the result seems good:

qdisc htb 1: root refcnt 9 r2q 10 default 0 direct_packets_stat 5 direct_qlen 1000
 Sent 350 bytes 5 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0

but when running createClass(), I get the err err = netlink receive: invalid argument.

[florianl]

Thanks for your report. I will try to find some time in the coming days to take a look at it and come back.

[florianl]

Sorry for the delay :pray:

I'm not sure, how github.com/containernetworking/plugins/pkg/ns is supposed to work, so I removed it from further looking into the topic.

The handling of Msg.Handle and Msg.Parent looks incorrect to me. While the qdisc requires Htb.Init{} to be populated, the class requirers Htb.Parms{}.

In the following gist you can find an example of a HTB qdisc with two classes attached: https://gist.github.com/florianl/f8917929ed9ebe7fa819663a853742be

Hope this helps. So far, it doesn't look like a bug but rather a incorrectly configured handling of elements.

[SMALL-head]

Thank you for your answer. The code you provided offers practical guidance for me. And using Htb.Parms{} does help.