grid_label minus values ?

[4kssoft]

I prepared data for 21 classes And when i check uniques values in grid_label: np.unique(grid_label)

array([-124, -116, -114, -103,  -93,  -82,  -61,    0,    1,    3,    4,
          5,    6,    8,   10,   11,   12,   13,   14,   15,   16,   17,
         19,   20,   22,   24,   25,   27,   33,   35,   36,   37,   38,
         43,   45,   46,   48,   54,   56,   57,   58,   64,   66,   69,
         75,   77,   78,   79,   90,   98,  100,  111,  119,  121],
      dtype=int8)

I have minus values. I dont now why? Training crashes on the loss function.

[4kssoft]

When i changed format like in example 2 https://github.com/vsymbol/CUTIE/issues/7#issuecomment-599129949 I have no more errors and no minus values.

But validation accuracy does not improve:

Iter: 400/1000, total loss: 3.3000, model loss: 3.1069, regularization loss: 0.1931
LOSS CURVE: 0:11.364 >100:3.514 >200:3.411 >300:3.328 >400:3.300
TRAINING ACC CURVE: 0:0.016 >100:0.933 >200:0.958 >300:0.989 >400:0.994
TRAINING ACC (Recall/Acc): 0.994 / 0.994 (0.994) | highest 0.994 / 0.994 (0.994)
VALIDATION ACC (STRICT) CURVE: 0:0.032 >100:0.161 >200:0.157 >300:0.154 >400:0.146
VALIDATION ACC (SOFT) CURVE: 0:0.200 >100:0.204 >200:0.218 >300:0.221 >400:0.232
TRAINING RECALL CURVE: 0:0.29 >100:0.97 >200:0.98 >300:0.99 >400:0.99
VALIDATION RECALL CURVE: 0:0.22 >100:0.34 >200:0.39 >300:0.40 >400:0.43
VALIDATION Statistic 400(100) (Recall/Acc): 0.427 / 0.146 (0.232) | highest 0.339 / 0.161 (0.204) 

[vsymbol]

When i changed format like in example 2 #7 (comment) I have no more errors and no minus values.

But validation accuracy does not improve:

Iter: 400/1000, total loss: 3.3000, model loss: 3.1069, regularization loss: 0.1931
LOSS CURVE: 0:11.364 >100:3.514 >200:3.411 >300:3.328 >400:3.300
TRAINING ACC CURVE: 0:0.016 >100:0.933 >200:0.958 >300:0.989 >400:0.994
TRAINING ACC (Recall/Acc): 0.994 / 0.994 (0.994) | highest 0.994 / 0.994 (0.994)
VALIDATION ACC (STRICT) CURVE: 0:0.032 >100:0.161 >200:0.157 >300:0.154 >400:0.146
VALIDATION ACC (SOFT) CURVE: 0:0.200 >100:0.204 >200:0.218 >300:0.221 >400:0.232
TRAINING RECALL CURVE: 0:0.29 >100:0.97 >200:0.98 >300:0.99 >400:0.99
VALIDATION RECALL CURVE: 0:0.22 >100:0.34 >200:0.39 >300:0.40 >400:0.43
VALIDATION Statistic 400(100) (Recall/Acc): 0.427 / 0.146 (0.232) | highest 0.339 / 0.161 (0.204) 

0. check your dictionary if it has words for your target receipts, otherwise generate one with the provided code
1. use the recommended backbone as analyzed in the paper
2. play with parameters in the main_train_json.py file
3. check the TrainingStatistic.xlsx file, where you may infer good parameters for your target job please note that the model performance will have better performance for receipts with less words (about less than 50 words in a line) than receipts with too much words.

[nitinmlvya]

When i changed format like in example 2 #7 (comment) I have no more errors and no minus values.

But validation accuracy does not improve:

Iter: 400/1000, total loss: 3.3000, model loss: 3.1069, regularization loss: 0.1931
LOSS CURVE: 0:11.364 >100:3.514 >200:3.411 >300:3.328 >400:3.300
TRAINING ACC CURVE: 0:0.016 >100:0.933 >200:0.958 >300:0.989 >400:0.994
TRAINING ACC (Recall/Acc): 0.994 / 0.994 (0.994) | highest 0.994 / 0.994 (0.994)
VALIDATION ACC (STRICT) CURVE: 0:0.032 >100:0.161 >200:0.157 >300:0.154 >400:0.146
VALIDATION ACC (SOFT) CURVE: 0:0.200 >100:0.204 >200:0.218 >300:0.221 >400:0.232
TRAINING RECALL CURVE: 0:0.29 >100:0.97 >200:0.98 >300:0.99 >400:0.99
VALIDATION RECALL CURVE: 0:0.22 >100:0.34 >200:0.39 >300:0.40 >400:0.43
VALIDATION Statistic 400(100) (Recall/Acc): 0.427 / 0.146 (0.232) | highest 0.339 / 0.161 (0.204) 

I've used both the formats which are available in the example link. I'm training my model on 13 classes.

Tried both the format but it gets failed with the given below error:

Caused by op 'SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits', defined at:
  File "main_train_json.py", line 148, in <module>
    model_loss, regularization_loss, total_loss, model_logits, model_output = network.build_loss()
  File "C:\Users\703254816\OneDrive - Genpact\code_mania\CUTIE\model_cutie.py", line 113, in build_loss
    cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels, logits=cls_logits)
  File "C:\Users\703254816\Documents\virtual_envs\CUTIE\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 2679, in sparse_softmax_cross_entropy_with_logits
    precise_logits, labels, name=name)
  File "C:\Users\703254816\Documents\virtual_envs\CUTIE\lib\site-packages\tensorflow\python\ops\gen_nn_ops.py", line 8759, in sparse_softmax_cross_entropy_with_logits
    labels=labels, name=name)
  File "C:\Users\703254816\Documents\virtual_envs\CUTIE\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\703254816\Documents\virtual_envs\CUTIE\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "C:\Users\703254816\Documents\virtual_envs\CUTIE\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op
    op_def=op_def)
  File "C:\Users\703254816\Documents\virtual_envs\CUTIE\lib\site-packages\tensorflow\python\framework\ops.py", line 1801, in __init__
    self._traceback = tf_stack.extract_stack()

**InvalidArgumentError (see above for traceback): Received a label value of 25 which is outside the valid range of [0, 13).**

Could you please help me on this?

[4kssoft]

You have one class per field_name in json file? I think that in your case there should be a maximum of 12 items in the "fields" table ( 13 - 1 "DontCare" class)

[nitinmlvya]

You have one class per field_name in json file? I think that in your case there should be a maximum of 12 items in the "fields" table ( 13 - 1 "DontCare" class)

Yes. One class per field_name but I didn't include the "DontCare" class in the "fields" list in the JSON file.

Should I need to add the "DontCare" class? If yes, then what would be their coordinates into the "bbox" of "text_boxes" in the JSON file?

I've also added "DontCare" in the self.classes variable into the file which is required, I guess.

[4kssoft]

You have one class per field_name in json file? I think that in your case there should be a maximum of 12 items in the "fields" table ( 13 - 1 "DontCare" class)

Yes. One class per field_name but I didn't include the "DontCare" class in the "fields" list in the JSON file.

Should I need to add the "DontCare" class? If yes, then what would be their coordinates into the "bbox" of "text_boxes" in the JSON file?

I've also added "DontCare" in the self.classes variable into the file which is required, I guess.

Do not add DontCare class in fields. In text_boxes add all words and in self.classes first class should be DontCare

[nitinmlvya]

You have one class per field_name in json file?

I think that in your case there should be a maximum of 12 items in the "fields" table ( 13 - 1 "DontCare" class)

Yes. One class per field_name but I didn't include the "DontCare" class in the "fields" list in the JSON file.

Should I need to add the "DontCare" class? If yes, then what would be their coordinates into the "bbox" of "text_boxes" in the JSON file?

I've also added "DontCare" in the self.classes variable into the file which is required, I guess.

Do not add DontCare class in fields.

In text_boxes add all words and in self.classes first class should be DontCare

@4kssoft: According to you, fields key contains all the class names only and text_boxes key contains all the words(class name words & non-class name words as well) and their coordinates.

Am I correct?

I would be grateful to you if you provide a sample training file along with your image file(on which you have created that .json training file). It’ll be easier for me to understand. Thank you.

[githubofjosh]

@4kssoft As @nitinmlvya said, a sample file would be great for our better understanding!

[4kssoft]

@nitinmlvya I prepared a sample data file https://github.com/4kssoft/CUTIE/blob/master/invoice_data/Faktura1.pdf_0.json

[Ibmaria]

@4kssoft Hi , I need help , my training acc curve does not improve . Thanks in advance

Iter: 600/40000, total loss: 0.9532, model loss: 0.7458, regularization loss: 0.2075 LOSS CURVE: 0:7.280 >100:1.317 >200:1.019 >300:0.971 >400:0.950 >500:0.936 >600:0.953 TRAINING ACC CURVE: 0:0.000 >100:0.000 >200:0.000 >300:0.000 >400:0.000 >500:0.083 >600:0.000 TRAINING ACC (Recall/Acc): 1.000 / 0.000 (1.000) | highest 1.000 / 0.083 (1.000) VALIDATION ACC (STRICT) CURVE: 0:0.000 >100:0.000 >200:0.000 >300:0.000 >400:0.000 >500:0.000 >600:0.000 VALIDATION ACC (SOFT) CURVE: 0:0.833 >100:0.792 >200:0.583 >300:0.792 >400:0.875 >500:0.792 >600:0.708 TRAINING RECALL CURVE: 0:0.53 >100:1.00 >200:1.00 >300:1.00 >400:1.00 >500:0.92 >600:1.00 VALIDATION RECALL CURVE: 0:0.96 >100:0.94 >200:0.80 >300:0.87 >400:0.95 >500:0.88 >600:0.79 VALIDATION Statistic 600(0) (Recall/Acc): 0.788 / 0.000 (0.708) | highest 0.958 / 0.000 (0.833)

[4kssoft]

@4kssoft Hi , I need help , my training acc curve does not improve . Thanks in advance

Iter: 600/40000, total loss: 0.9532, model loss: 0.7458, regularization loss: 0.2075 LOSS CURVE: 0:7.280 >100:1.317 >200:1.019 >300:0.971 >400:0.950 >500:0.936 >600:0.953 TRAINING ACC CURVE: 0:0.000 >100:0.000 >200:0.000 >300:0.000 >400:0.000 >500:0.083 >600:0.000 TRAINING ACC (Recall/Acc): 1.000 / 0.000 (1.000) | highest 1.000 / 0.083 (1.000) VALIDATION ACC (STRICT) CURVE: 0:0.000 >100:0.000 >200:0.000 >300:0.000 >400:0.000 >500:0.000 >600:0.000 VALIDATION ACC (SOFT) CURVE: 0:0.833 >100:0.792 >200:0.583 >300:0.792 >400:0.875 >500:0.792 >600:0.708 TRAINING RECALL CURVE: 0:0.53 >100:1.00 >200:1.00 >300:1.00 >400:1.00 >500:0.92 >600:1.00 VALIDATION RECALL CURVE: 0:0.96 >100:0.94 >200:0.80 >300:0.87 >400:0.95 >500:0.88 >600:0.79 VALIDATION Statistic 600(0) (Recall/Acc): 0.788 / 0.000 (0.708) | highest 0.958 / 0.000 (0.833)

Hi @Ibmaria

What model do you use and what parameters you set ? I trained on a model CUTIERes from model_cutie_aspp Params:

    use_cutie2=False
    text_case=False
    tokenize=True
    # data manipulation
    segment_grid=False
    rows_segment=72 
    cols_segment=72 
    augment_strategy=1
    positional_mapping_strategy=1
    rows_target=80 
    cols_target=80 
    rows_ulimit=80
    cols_ulimit=80
    fill_bbox=False
    data_augmentation_extra=True 
    data_augmentation_dropout=1
    data_augmentation_extra_rows=16 
    data_augmentation_extra_cols=16 
    # training
    batch_size=5
    iterations=40000
    lr_decay_step=13000
    learning_rate=0.0001
    # learning_rate=0.001
    lr_decay_factor=0.1 
    # loss optimization
    hard_negative_ratio=3 
    use_ghm=0
    ghm_bins=30
    ghm_momentum=0

    embedding_size=256
    weight_decay=0.0005 
    eps=1e-6

My last training was like this

Iter: 7400/40000, total loss: 2.9161, model loss: 2.8293, regularization loss: 0.0868
TRAINING ACC (Recall/Acc): 0.995 / 0.918 (0.976) | highest 1.000 / 1.000 (1.000)
VALIDATION Statistic 7400(5900) (Recall/Acc): 0.809 / 0.543 (0.727) | highest 0.810 / 0.623 (0.723) 

[Neelesh1121]

@4kssoft, Thanks for your support, I am facing an issue, In the result i am getting mostly # and 0, is it possible to get the correct output, or you if you can just explain why # and 0 are coming?