Find distance between each bounding boxes

[samahwaleed]

❓ Questions & Help

I have a text file has labeled, bbox coordinator and score for each detected object in the image like this format: label x1 y1 x2 y2 score.

I represented each bounding boxes as a node. I want to ask how we can find and represent the distance between the nodes as an edge?

Thanks

[rusty1s]

You can compute the centroid of each bounding box, leading to a tensor of shape pos = [num_bounding_boxes, 2]. You can then create a graph from it as follows:

from torch_geometric.data import Data
import torch_geometric.transforms as T

data = Data(pos=pos, label=label, score=score)

transform = T.Compose([T.KNNGraph(k=6), T.Distance(norm=False)])
data = transform(data)

[samahwaleed]

ok, Thank you.

[samahwaleed]

what does this code do. does it compute edges? how does the current code compute the edges?

`from torch_geometric.data import Data import torch_geometric.transforms as T

data = Data(pos=pos, label=label, score=score)

transform = T.Compose([T.KNNGraph(k=6), T.Distance(norm=False)]) data = transform(data)`

[rusty1s]

Sure, it will

1. Create a data object containing the bounding boxes as nodes (Data(pos=pos, ...))
2. It will transform this data object such that
   1. for each node, it will connect to its k-nearest nodes (T.KNNGraph)
   2. for each created edge, it will compute the pair-wise distance between the two nodes and writes them as edge features edge_attr to the data object (T.Distance)

[samahwaleed]

I have calculated the centroid of each bounding boxes, but it result to a tensor of shape pos = [2, num_bounding_boxes] like this:

Data(pos=[2, 17], pixel_values=[249, 358, 3], score=[17], edge_index=[2, 2], edge_attr=[2, 1])

[rusty1s]

You can input its transpose (pos.t()) into the data object.

[samahwaleed]

Sir, I want to represent the angle between the bounding boxes as an edge. I calculated the centroid and pixel values of the bounding boxes.

I want to ask if it's possible or not?

if possible, how to compute the angle between the bounding boxes, and what will be the result of pos tensor??

Regards

[rusty1s]

If you want to store angles between nodes as edge features, you can use the Polar transform. Please see its code for more details.

[samahwaleed]

I have calculated the centroid of each bounding boxes and pixel values for each detected object. I represented each pixel values as a node and centroid as edges. I got this result.

Data(pos=[12, 2], pixel_values=[71, 69, 3], score=[12], edge_index=[2, 72], edge_attr=[72, 1])

I still don't understand the edge_index and edge_attr.

[rusty1s]

Super, this looks great. The KNNGraph transform will connect nodes to the k-nearest nodes based on their spatial positions (pos). With k=6, this result in a total of 72 edges (12*6=72), which is also indicates in the shapes of edge_index and edge_attr. edge_index will hold the node pair of an edge for each of the 72 edges, and edge_attr holds the distance between node pairs for each edge.

[samahwaleed]

Ok. Why k=6 not other numbers?. I’m wondering if there is another way to build edge_index without using KNNGraph?

[rusty1s]

k=6 is just arbitrary. You can use different values of k or use a different graph generation procedure as well, e.g., ball queries via Radius(). However, the only way to generate a graph from a "point cloud" is to utilize such synthetic graph generation tools, as we do not have any other information about relationships of nodes in advance.

[samahwaleed]

Ok. How is the distance between node pairs for each edge (edge_attr) calculated? When I change the norm to True I got different results of edge_attr??

[rusty1s]

The norm flag will normalize all distances to be in interval [0, 1], which can be useful to force some scale-invariance. Other-wise, it is not necessarily needed.

[samahwaleed]

Ok. I try to visualize the graph using networkx but I got the graph with randomly distances between the nodes. How can I pass the centroid which I calculated to nx.draw?

import networkx as nx
import torch_geometric


g = torch_geometric.utils.to_networkx(data, to_undirected=True)
nx.draw(g, with_labels=True)

[rusty1s]

networkx.draw supports the pos argument (see here, which denotes the position of nodes, i.e., the centroids of your bounding boxes.

[samahwaleed]

thank you for your kind response. I have 800 images and I want to create a graph for each image, but I got KeyError: 1.

this is my code:

data = CustomDataset(transform=transforms.ToTensor())

# for each image
for i in range(data.__len__()):
  centroid, cropped_image_pixels, detection_score = data.__getitem__(i)
  print('At image ', i, ', I have ', len(centroid), ' centroids.. ', len(cropped_image_pixels), ' subimages')
  data = Data(pos=centroid, pixel_values=cropped_image_pixels, detection_score= detection_score) 
  transform = T.Compose([T.KNNGraph(k=3), T.Distance(norm=False)])
  data = transform(data)

the error:

/usr/local/lib/python3.7/dist-packages/torch_geometric/data/storage.py in __getitem__(self, key)
     66 
     67     def __getitem__(self, key: str) -> Any:
---> 68         return self._mapping[key]
     69 
     70     def __setitem__(self, key: str, value: Any):

KeyError: 1

[rusty1s]

It seems like you are overriding data within your loop, so that you are losing access to the original image dataset.

[samahwaleed]

How can I solve this? Any suggestion please,

[rusty1s]

dataset = CustomDataset(transform=transforms.ToTensor())

# for each image
for data in dataset:
  centroid, cropped_image_pixels, detection_score = data
  data = Data(pos=centroid, pixel_values=cropped_image_pixels, detection_score= detection_score) 
  transform = T.Compose([T.KNNGraph(k=3), T.Distance(norm=False)])
  data = transform(data)

[samahwaleed]

Sir, I'm trying to create a graph for each data but it creates all the graphs in a single graph.

graphData.append(data)
for i in range(len(graphData)):
    print("An element of data: ", i ,graphData[i])
    g = torch_geometric.utils.to_networkx(graphData[i], to_undirected=True)
    nx.draw(g, with_labels=True)

[rusty1s]

Can you clarify what you mean? I don't see any problem in your code.

[samahwaleed]

I have 800 data looks like this:

Data(pos=[15, 2], pixel_values=[15], detection_score=[15], edge_index=[2, 45], edge_attr=[45, 1]) Data(pos=[12, 2], pixel_values=[12], detection_score=[12], edge_index=[2, 36], edge_attr=[36, 1]) .......

I need to create a graph for each one so I should get 800 graphs at the end, but I got only one graph including all the graphs

[rusty1s]

Can you show me a minimal example of what you are doing? Given that graphData[i] only contains a single Data object, the networkx.Graph will only hold a single graph as well.

[samahwaleed]

Thank you. I got it Sir.

[samahwaleed]

Sir, I am trying to get some information about the dataset. But I got an error when I try to get the num_features and num_classes.

my code:

print(f'Dataset: {dataset}:')
print('====================')
print(f'Number of graphs: {len(dataset)}')
print(f'Number of features: {dataset.num_features}')
print(f'Number of features: {dataset.num_classes}')

data = graphData[0]  # Get the first graph object.
 
print('====================')
print(f'Number of edges: {data.num_edges}')
print(f'Average node degree: {data.num_edges / data.num_nodes:.2f}')
print(f'Contains isolated nodes: {data.contains_isolated_nodes()}')
print(f'Contains self-loops: {data.contains_self_loops()}')
print(f'Is undirected: {data.is_undirected()}')

the error:

AttributeError                            Traceback (most recent call last)
<ipython-input-36-edaea90b0e31> in <module>()
      2 print('====================')
      3 print(f'Number of graphs: {len(dataset)}')
----> 4 print(f'Number of features: {dataset.num_features}')
      5 print(f'Number of classes: {dataset.num_classes}')
      6 

/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataset.py in __getattr__(self, attribute_name)
     81             return function
     82         else:
---> 83             raise AttributeError
     84 
     85     @classmethod

AttributeError: 

[rusty1s]

It looks like your dataset is just a list of data objects, so you don't have access to additional functionality. The following should work as well:

print(dataset[0].num_features)

[samahwaleed]

ok, the graphData is a list of data objects. It returns num_node_features = 0 for each data. For num_classes it shows me error: AttributeError: 'GlobalStorage' object has no attribute 'num_classes'

my code:

for i in range(100):
      data = graphData[i]
      print(f'Dataset: {dataset}:')
      print('====================')
      print(f'Number of graphs: {len(dataset)}')
      print(f'Number of features: {data.num_node_features}')
      print(f'Number of classes: {data.num_classes}')  

the error:

Dataset: <__main__.CustomDataset object at 0x7f9f1005e650>:
====================
Number of graphs: 701
Number of features: 0
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/torch_geometric/data/storage.py in __getattr__(self, key)
     47         try:
---> 48             return self[key]
     49         except KeyError:

3 frames
KeyError: 'num_classes'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/torch_geometric/data/storage.py in __getattr__(self, key)
     49         except KeyError:
     50             raise AttributeError(
---> 51                 f"'{self.__class__.__name__}' object has no attribute '{key}'")
     52 
     53     def __setattr__(self, key: str, value: Any):

AttributeError: 'GlobalStorage' object has no attribute 'num_classes'

[rusty1s]

Yes, this is to be expected as there exists no property num_classes for a single data object. I suggest to simply drop this call.

[samahwaleed]

Sir, I want to implement GCN on my dataset for the node classification task. but I got error: Sizes of tensors must match except in dimension 0.

theload_data_graphreturns a list of data objects.

this is my code:

data_size = 701
train_size = 560
batch_size = 100
epoch_num = 150

class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = GCNConv(2, 16)
        self.conv2 = GCNConv(16, 32)
        self.conv3 = GCNConv(32, 48)
        self.conv4 = GCNConv(48, 64)
        self.conv5 = GCNConv(64, 96)
        self.conv6 = GCNConv(96, 128)
        self.linear1 = torch.nn.Linear(128,64)
        self.linear2 = torch.nn.Linear(64,10)

    def forward(self, data):
        x, edge_index = data.x, data.edge_index
        x = self.conv1(x, edge_index)
        x = F.relu(x)
        x = self.conv2(x, edge_index)
        x = F.relu(x)
        x = self.conv3(x, edge_index)
        x = F.relu(x)
        x = self.conv4(x, edge_index)
        x = F.relu(x)
        x = self.conv5(x, edge_index)
        x = F.relu(x)
        x = self.conv6(x, edge_index)
        x = F.relu(x)
        x, _ = scatter_max(x, data.batch, dim=0)
        x = self.linear1(x)
        x = F.relu(x)
        x = self.linear2(x)
        return x


def main():
    data_list = load_data_graph(data_size=data_size)
    device = torch.device('cuda')
    model = Net().to(device)
    trainset = data_list[:train_size]
    optimizer = torch.optim.Adam(model.parameters())
    trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True)
    testset = data_list[train_size:]
    testloader = DataLoader(testset, batch_size=batch_size)
    criterion = nn.CrossEntropyLoss()
    history = {
        "train_loss": [],
        "test_loss": [],
        "test_acc": []
    }

    print("Start Train")
    
    model.train()
    for epoch in range(epoch_num):
        train_loss = 0.0
        for i, batch in enumerate(trainloader):
            batch = batch.to("cuda")
            optimizer.zero_grad()
            outputs = model(batch)
            loss = criterion(outputs,batch.t)
            loss.backward()
            optimizer.step()
            
            train_loss += loss.cpu().item()
            if i % 10 == 9:
                progress_bar = '['+('='*((i+1)//10))+(' '*((train_size//100-(i+1))//10))+']'
                print('\repoch: {:d} loss: {:.3f}  {}'
                        .format(epoch + 1, loss.cpu().item(), progress_bar), end="  ")

        print('\repoch: {:d} loss: {:.3f}'
            .format(epoch + 1, train_loss / (train_size / batch_size)), end="  ")
        history["train_loss"].append(train_loss / (train_size / batch_size))

        correct = 0
        total = 0
        batch_num = 0
        loss = 0
        with torch.no_grad():
            for data in testloader:
                data = data.to(device)
                outputs = model(data)
                loss += criterion(outputs,data.t)
                _, predicted = torch.max(outputs, 1)
                total += data.t.size(0)
                batch_num += 1
                correct += (predicted == data.t).sum().cpu().item()

        history["test_acc"].append(correct/total)
        history["test_loss"].append(loss.cpu().item()/batch_num)
        endstr = ' '*max(1,(train_size//1000-39))+"\n"
        print('Test Accuracy: {:.2f} %%'.format(100 * float(correct/total)), end='  ')
        print(f'Test Loss: {loss.cpu().item()/batch_num:.3f}',end=endstr)


    print('Finished Training')

    correct = 0
    total = 0
    with torch.no_grad():
        for data in testloader:
            data = data.to(device)
            outputs = model(data)
            _, predicted = torch.max(outputs, 1)
            total += data.t.size(0)
            correct += (predicted == data.t).sum().cpu().item()
    print('Accuracy: {:.2f} %%'.format(100 * float(correct/total)))

if __name__=="__main__":
    main()