Crash upon error allocating Tier 1 handle...

[sebras]

While testing what happens when opj_calloc() returns NULL using a custom allocator in OpenJPEG 2.3.0 I found a crash that doesn't seem to have been addressed in OpenJPEG on master.

The relevant part of backtrace for the crash is:

    #0 0x5616e78a855d in opj_t1_clbl_decode_processor openjpeg/src/lib/openjp2/t1.c:1663
    #1 0x5616e78378f3 in opj_thread_pool_submit_job openjpeg/src/lib/openjp2/thread.c:835
    #2 0x5616e78aa93c in opj_t1_decode_cblks openjpeg/src/lib/openjp2/t1.c:1901
    #3 0x5616e782dfa0 in opj_tcd_t1_decode openjpeg/src/lib/openjp2/tcd.c:1985
    #4 0x5616e782bdf4 in opj_tcd_decode_tile openjpeg/src/lib/openjp2/tcd.c:1639
    #5 0x5616e77e52d6 in opj_j2k_decode_tile openjpeg/src/lib/openjp2/j2k.c:8947
    #6 0x5616e77f21f7 in opj_j2k_decode_tiles openjpeg/src/lib/openjp2/j2k.c:10735
    #7 0x5616e77df8fd in opj_j2k_exec openjpeg/src/lib/openjp2/j2k.c:8105
    #8 0x5616e77f4dae in opj_j2k_decode openjpeg/src/lib/openjp2/j2k.c:11081

This is the backtrace of where I had opj_calloc() return NULL later causing the crash above:

#0  0x0000555555833224 in opj_calloc (n=1, size=280)
#1  0x00005555561a7ad9 in opj_t1_create (isEncoder=0) at openjpeg/src/lib/openjp2/t1.c:1541
#2  0x00005555561a84b5 in opj_t1_clbl_decode_processor (user_data=0x608000009228, tls=0x603000004518) at openjpeg/src/lib/openjp2/t1.c:1660
#3  0x00005555561378f4 in opj_thread_pool_submit_job (tp=0x6080000091a8, job_fn=0x5555561a7d07 <opj_t1_clbl_decode_processor>, user_data=0x608000009228) at openjpeg/src/lib/openjp2/thread.c:835
#4  0x00005555561aa93d in opj_t1_decode_cblks (tcd=0x60b000000258, pret=0x7fffffff93e0, tilec=0x613000001e08, tccp=0x61f000007088, p_manager=0x6100000005a8, p_manager_mutex=0x0, check_pterm=0) at openjpeg/src/lib/openjp2/t1.c:1901
#5  0x000055555612dfa1 in opj_tcd_t1_decode (p_tcd=0x60b000000258, p_manager=0x6100000005a8) at openjpeg/src/lib/openjp2/tcd.c:1985
#6 0x000055555612bdf5 in opj_tcd_decode_tile (p_tcd=0x60b000000258, win_x0=0, win_y0=0, win_x1=640, win_y1=480, numcomps_to_decode=0, comps_indices=0x0, p_src=0x7ffff3e9f808 "H\241\343\345\355\213\035\211\031\210ZU\357T0R\355\363Y\376\242\355(X\256\071\177V\003e\276ڬ%\322g\357\032\351\016\244\065\357\254\027ߠcL\223P\366\253L\026\372\r\033\372e\237U\242\210\216\233w {\b\275\377\n\257\306\376\202[\247\v%\243\327\034\200\216n*\210\f\004\026\004:\nE\352il}\236\254\275\356\211\327\346\273\373\210\t\343\031ki\261\220\223Lnoè\234\065\373\003~yd,E\002C\034\223\366E)\313\346@iD\343\027k\244J\350n\331\356<]b\356\253\023\066\262 \357\067\313\336\035i\301(\250\022\006^!w\366J\033\320\016vIN\216\317\316\032\023L`\362\306ȧZ\243\312\305Y"..., p_max_length=259641, p_tile_no=0, p_cstr_index=0x606000007a08, p_manager=0x6100000005a8) at openjpeg/src/lib/openjp2/tcd.c:1639
#7 0x00005555560e52d7 in opj_j2k_decode_tile (p_j2k=0x613000001548, p_tile_index=0, p_data=0x0, p_data_size=0, p_stream=0x60c000008808, p_manager=0x6100000005a8) at openjpeg/src/lib/openjp2/j2k.c:8947
#8 0x00005555560f21f8 in opj_j2k_decode_tiles (p_j2k=0x613000001548, p_stream=0x60c000008808, p_manager=0x6100000005a8) at openjpeg/src/lib/openjp2/j2k.c:10735
#9 0x00005555560df8fe in opj_j2k_exec (p_j2k=0x613000001548, p_procedure_list=0x6030000044e8, p_stream=0x60c000008808, p_manager=0x6100000005a8) at openjpeg/src/lib/openjp2/j2k.c:8105
#10 0x00005555560f4daf in opj_j2k_decode (p_j2k=0x613000001548, p_stream=0x60c000008808, p_image=0x606000007ac8, p_manager=0x6100000005a8) at openjpeg/src/lib/openjp2/j2k.c:11081

Looking at the source code it is evident that the return value NULL from opj_t1_create() is not handled.