Generating Scene Graphs for random Images

[ritwickchaudhry]

@rowanz , I wanted to generate scene graphs for images not belonging to the Visual Genome dataset, using your model. Can I do this using the existing codebase? If yes, can you please guide me through it?

[rowanz]

You should be able to! Just change the dataloader to load from your own dataset. You’ll need to provide bounding boxes if using SGCls mode, boxes and labels if Predcls, or just use SGDet (nothing needed)

On Mon, Oct 1, 2018 at 2:35 AM Ritwick Chaudhry [email protected] wrote:

@rowanz https://github.com/rowanz , I wanted to generate scene graphs for images not belonging to the Visual Genome dataset, using your model. Can I do this using the existing codebase? If yes, can you please guide me through it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rowanz/neural-motifs/issues/26, or mute the thread https://github.com/notifications/unsubscribe-auth/ABWJx5HqWfvddvDAKRBDv_YoDPJKFgjtks5ugeHIgaJpZM4XBsuk .

[ritwickchaudhry]

Thanks a lot for the help @rowanz . I'll try to get it to work, and get back. Thanks!

[ritwickchaudhry]

@rowanz , I'm still facing some issues. For example, in SGDet mode, there should be no Ground Truth right? (I just want to produce scene graphs from images, and I don't have any ground truth available).

The data loaders by default load the gt_boxes and gt_relations. What exactly am I supposed to do about those in the data loader. The blobs also process these for the collate. I'm currently stuck there. In SGDet mode, functions like get_counts also access the gt elements.

Can you please help me through this?

Also, the rel_model_stanford seems to not have gt elements in sgdet mode, but the checkpoint isn't available. Can you please provide that?

Thanks!

[rowanz]

I'm still facing some issues. For example, in SGDet mode, there should be no Ground Truth right? (I just want to produce scene graphs from images, and I don't have any ground truth available).

The data loaders by default load the gt_boxes and gt_relations. What exactly am I supposed to do about those in the data loader. The blobs also process these for the collate. I'm currently stuck there.

Yep, you can replace the gt boxes with zeros and that should be okay. You don't need get_counts for running MotifNet.

Also, the rel_model_stanford seems to not have gt elements in sgdet mode, but the checkpoint isn't available. Can you please provide that?

Not sure what you mean by this (none of the models should). I don't support the checkpoints for this model, but you can reproduce my results by following the training script for that :)

[hengqiali]

@ritwickchaudhry I also want to generate scene graphs for COCO datasets without any groundtruth.Did you succeeded??

[Thoudancer]

@ritwickchaudhry hello, I met the same question as you. I want to use the trained sgdet model to generate sg, but i own only image, without files like image_data.json, VG-SGG.h5, VG-SGG-dicts.json and proposals.h5. If I set None to these, suffered error about get_counts(). Would you like to share the modification about the code?

[ankitsamal]

@rowanz , I'm still facing some issues. For example, in SGDet mode, there should be no Ground Truth right? (I just want to produce scene graphs from images, and I don't have any ground truth available).

The data loaders by default load the gt_boxes and gt_relations. What exactly am I supposed to do about those in the data loader. The blobs also process these for the collate. I'm currently stuck there. In SGDet mode, functions like get_counts also access the gt elements.

Can you please help me through this?

Also, the rel_model_stanford seems to not have gt elements in sgdet mode, but the checkpoint isn't available. Can you please provide that?

Thanks!

Hey Ritwik,

Were you able to figure this out. I am stuck in the same place trying to figure out how to input custom images

[forbiddenname]

@rowanz , I'm still facing some issues. For example, in SGDet mode, there should be no Ground Truth right? (I just want to produce scene graphs from images, and I don't have any ground truth available). The data loaders by default load the gt_boxes and gt_relations. What exactly am I supposed to do about those in the data loader. The blobs also process these for the collate. I'm currently stuck there. In SGDet mode, functions like get_counts also access the gt elements. Can you please help me through this? Also, the rel_model_stanford seems to not have gt elements in sgdet mode, but the checkpoint isn't available. Can you please provide that? Thanks!

Hey Ritwik,

Were you able to figure this out. I am stuck in the same place trying to figure out how to input custom images

hi ！Have you solved this problem? tks

[forbiddenname]

Thanks a lot for the help @rowanz . I'll try to get it to work, and get back. Thanks!

hi ！Have you solved this problem? tks

[forbiddenname]

@ritwickchaudhry I also want to generate scene graphs for COCO datasets without any groundtruth.Did you succeeded??

hi ！Have you solved this problem? tks

[alice-cool]

https://github.com/rowanz/neural-motifs/issues/26#issuecomment-427087900 but at now the project only use 150 object classes. My detector detect more object class, but the project will fail to more object classes.

[Yudezhi]

Same issue here. comment for subscribe.