ML
ML copied to clipboard
wala
Missing `putfield` in IR when traversing non-scalar iterable datasets
Consider the following Python code:
# test_tf2_dataset8.py
import tensorflow as tf
def add(a, b):
return a + b
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train)).shuffle(10000).batch(32)
for images, labels in dataset:
c = add(images, images)
And corresponding IR:
callees of node Lscript tf2_test_dataset8.py : [import, add, from_tensor_slices, shuffle, batch]
IR of node 2, context CallStringContext: [ com.ibm.wala.FakeRootClass.fakeRootMethod()V@2 ]
<Code body of function Lscript tf2_test_dataset8.py>
CFG:
BB0[-1..-2]
-> BB1
BB1[0..98]
-> BB2
-> BB9
BB2[99..115]
-> BB3
-> BB9
BB3[116..117]
-> BB4
-> BB9
BB4[118..119]
-> BB5
-> BB9
BB5[120..121]
-> BB6
BB6[122..133]
-> BB7
-> BB9
BB7[134..142]
-> BB8
-> BB9
BB8[143..144]
-> BB6
BB9[-1..-2]
Instructions:
BB0
BB1
0 global:global script tf2_test_dataset8.py = v1<no information>
1 v3 = new <PythonLoader,Lwala/builtin/enumerate>@1<no information> [3=[enumerate]]
2 v4 = new <PythonLoader,Lwala/builtin/int>@2<no information> [4=[int]]
3 v5 = new <PythonLoader,Lwala/builtin/round>@3<no information> [5=[round]]
4 v6 = new <PythonLoader,Lwala/builtin/len>@4<no information> [6=[len]]
5 v7 = new <PythonLoader,Lwala/builtin/list>@5<no information> [7=[list]]
6 v8 = new <PythonLoader,Lwala/builtin/range>@6<no information> [8=[range]]
7 v9 = new <PythonLoader,Lwala/builtin/sorted>@7<no information> [9=[sorted]]
8 v10 = new <PythonLoader,Lwala/builtin/str>@8<no information> [10=[str]]
9 v11 = new <PythonLoader,Lwala/builtin/sum>@9<no information> [11=[sum]]
10 v12 = new <PythonLoader,Lwala/builtin/type>@10<no information> [12=[type]]
11 v13 = new <PythonLoader,Lwala/builtin/zip>@11<no information> [13=[zip]]
12 v14 = new <PythonLoader,Lwala/builtin/slice>@12<no information> [14=[slice]]
13 v15 = new <PythonLoader,Lwala/builtin/__delete__>@13<no information> [15=[__delete__]]
14 v16 = new <PythonLoader,Lwala/builtin/print>@14<no information> [16=[print]]
15 v19 = invokestatic < PythonLoader, LBaseException, import()LBaseException; > @15 exception:v20<no information> [19=[BaseException]]
16 v22 = invokestatic < PythonLoader, LDeprecationWarning, import()LDeprecationWarning; > @16 exception:v23<no information> [22=[DeprecationWarning]]
17 v25 = invokestatic < PythonLoader, LException, import()LException; > @17 exception:v26<no information> [25=[Exception]]
18 v28 = invokestatic < PythonLoader, LFutureWarning, import()LFutureWarning; > @18 exception:v29<no information> [28=[FutureWarning]]
19 v31 = invokestatic < PythonLoader, LNameError, import()LNameError; > @19 exception:v32<no information> [31=[NameError]]
20 v34 = invokestatic < PythonLoader, LNone, import()LNone; > @20 exception:v35<no information> [34=[None]]
21 v37 = invokestatic < PythonLoader, LRuntimeError, import()LRuntimeError; > @21 exception:v38<no information> [37=[RuntimeError]]
22 v40 = invokestatic < PythonLoader, LStopIteration, import()LStopIteration; > @22 exception:v41<no information> [40=[StopIteration]]
23 v43 = invokestatic < PythonLoader, LTypeError, import()LTypeError; > @23 exception:v44<no information> [43=[TypeError]]
24 v46 = invokestatic < PythonLoader, LUserWarning, import()LUserWarning; > @24 exception:v47<no information> [46=[UserWarning]]
25 v49 = invokestatic < PythonLoader, LValueError, import()LValueError; > @25 exception:v50<no information> [49=[ValueError]]
26 v52 = invokestatic < PythonLoader, L__doc__, import()L__doc__; > @26 exception:v53<no information> [52=[__doc__]]
27 v55 = invokestatic < PythonLoader, L__file__, import()L__file__; > @27 exception:v56<no information> [55=[__file__]]
28 v58 = invokestatic < PythonLoader, L__name__, import()L__name__; > @28 exception:v59<no information> [58=[__name__]]
29 v61 = invokestatic < PythonLoader, Labs, import()Labs; > @29 exception:v62<no information> [61=[abs]]
30 v64 = invokestatic < PythonLoader, Lall, import()Lall; > @30 exception:v65<no information> [64=[all]]
31 v67 = invokestatic < PythonLoader, Lany, import()Lany; > @31 exception:v68<no information> [67=[any]]
32 v70 = invokestatic < PythonLoader, Lbin, import()Lbin; > @32 exception:v71<no information> [70=[bin]]
33 v73 = invokestatic < PythonLoader, Lbool, import()Lbool; > @33 exception:v74<no information> [73=[bool]]
34 v76 = invokestatic < PythonLoader, Lbytes, import()Lbytes; > @34 exception:v77<no information> [76=[bytes]]
35 v79 = invokestatic < PythonLoader, Lcallable, import()Lcallable; > @35 exception:v80<no information> [79=[callable]]
36 v82 = invokestatic < PythonLoader, Lchr, import()Lchr; > @36 exception:v83<no information> [82=[chr]]
37 v85 = invokestatic < PythonLoader, Lcomplex, import()Lcomplex; > @37 exception:v86<no information> [85=[complex]]
38 v88 = invokestatic < PythonLoader, Ldel, import()Ldel; > @38 exception:v89<no information> [88=[del]]
39 v91 = invokestatic < PythonLoader, Ldict, import()Ldict; > @39 exception:v92<no information> [91=[dict]]
40 v94 = invokestatic < PythonLoader, Ldir, import()Ldir; > @40 exception:v95<no information> [94=[dir]]
41 v97 = invokestatic < PythonLoader, Ldivmod, import()Ldivmod; > @41 exception:v98<no information> [97=[divmod]]
42 v100 = invokestatic < PythonLoader, Leval, import()Leval; > @42 exception:v101<no information> [100=[eval]]
43 v103 = invokestatic < PythonLoader, Lexec, import()Lexec; > @43 exception:v104<no information> [103=[exec]]
44 v106 = invokestatic < PythonLoader, Lexit, import()Lexit; > @44 exception:v107<no information> [106=[exit]]
45 v109 = invokestatic < PythonLoader, Lfilter, import()Lfilter; > @45 exception:v110<no information> [109=[filter]]
46 v112 = invokestatic < PythonLoader, Lfloat, import()Lfloat; > @46 exception:v113<no information> [112=[float]]
47 v115 = invokestatic < PythonLoader, Lformat, import()Lformat; > @47 exception:v116<no information> [115=[format]]
48 v118 = invokestatic < PythonLoader, Lfrozenset, import()Lfrozenset; > @48 exception:v119<no information> [118=[frozenset]]
49 v121 = invokestatic < PythonLoader, Lget_ipython, import()Lget_ipython; > @49 exception:v122<no information> [121=[get_ipython]]
50 v124 = invokestatic < PythonLoader, Lgetattr, import()Lgetattr; > @50 exception:v125<no information> [124=[getattr]]
51 v127 = invokestatic < PythonLoader, Lglobals, import()Lglobals; > @51 exception:v128<no information> [127=[globals]]
52 v130 = invokestatic < PythonLoader, Lhasattr, import()Lhasattr; > @52 exception:v131<no information> [130=[hasattr]]
53 v133 = invokestatic < PythonLoader, Lhelp, import()Lhelp; > @53 exception:v134<no information> [133=[help]]
54 v136 = invokestatic < PythonLoader, Lhex, import()Lhex; > @54 exception:v137<no information> [136=[hex]]
55 v139 = invokestatic < PythonLoader, Lid, import()Lid; > @55 exception:v140<no information> [139=[id]]
56 v142 = invokestatic < PythonLoader, Linput, import()Linput; > @56 exception:v143<no information> [142=[input]]
57 v145 = invokestatic < PythonLoader, Lisinstance, import()Lisinstance; > @57 exception:v146<no information> [145=[isinstance]]
58 v148 = invokestatic < PythonLoader, Liter, import()Liter; > @58 exception:v149<no information> [148=[iter]]
59 v151 = invokestatic < PythonLoader, Llocals, import()Llocals; > @59 exception:v152<no information> [151=[locals]]
60 v154 = invokestatic < PythonLoader, Lmap, import()Lmap; > @60 exception:v155<no information> [154=[map]]
61 v157 = invokestatic < PythonLoader, Lmax, import()Lmax; > @61 exception:v158<no information> [157=[max]]
62 v160 = invokestatic < PythonLoader, Lmin, import()Lmin; > @62 exception:v161<no information> [160=[min]]
63 v163 = invokestatic < PythonLoader, Lnext, import()Lnext; > @63 exception:v164<no information> [163=[next]]
64 v166 = invokestatic < PythonLoader, Lobject, import()Lobject; > @64 exception:v167<no information> [166=[object]]
65 v169 = invokestatic < PythonLoader, Lopen, import()Lopen; > @65 exception:v170<no information> [169=[open]]
66 v172 = invokestatic < PythonLoader, Lord, import()Lord; > @66 exception:v173<no information> [172=[ord]]
67 v175 = invokestatic < PythonLoader, Lpow, import()Lpow; > @67 exception:v176<no information> [175=[pow]]
68 v178 = invokestatic < PythonLoader, Lprint, import()Lprint; > @68 exception:v179<no information> [178=[print]]
70 v181 = invokestatic < PythonLoader, Lproperty, import()Lproperty; > @70 exception:v182<no information> [181=[property]]
71 v184 = invokestatic < PythonLoader, Lrepr, import()Lrepr; > @71 exception:v185<no information> [184=[repr]]
72 v187 = invokestatic < PythonLoader, Lreversed, import()Lreversed; > @72 exception:v188<no information> [187=[reversed]]
73 v190 = invokestatic < PythonLoader, Lset, import()Lset; > @73 exception:v191<no information> [190=[set]]
74 v193 = invokestatic < PythonLoader, Lsuper, import()Lsuper; > @74 exception:v194<no information> [193=[super]]
75 v196 = invokestatic < PythonLoader, Ltuple, import()Ltuple; > @75 exception:v197<no information> [196=[tuple]]
76 v199 = invokestatic < PythonLoader, Lvars, import()Lvars; > @76 exception:v200<no information> [199=[vars]]
77 v202 = invokestatic < PythonLoader, LNotImplementedError, import()LNotImplementedError; > @77 exception:v203<no information> [202=[NotImplementedError]]
78 v205 = invokestatic < PythonLoader, LWarning, import()LWarning; > @78 exception:v206<no information> [205=[Warning]]
79 v208 = invokestatic < PythonLoader, Lcd, import()Lcd; > @79 exception:v209<no information> [208=[cd]]
80 v211 = invokestatic < PythonLoader, Lclear, import()Lclear; > @80 exception:v212<no information> [211=[clear]]
81 v214 = invokestatic < PythonLoader, Lpylab, import()Lpylab; > @81 exception:v215<no information> [214=[pylab]]
82 v217 = invokestatic < PythonLoader, LRuntimeWarning, import()LRuntimeWarning; > @82 exception:v218<no information> [217=[RuntimeWarning]]
83 v220 = invokestatic < PythonLoader, Lhist, import()Lhist; > @83 exception:v221<no information> [220=[hist]]
84 v223 = invokestatic < PythonLoader, Lmatplotlib, import()Lmatplotlib; > @84 exception:v224<no information> [223=[matplotlib]]
85 v226 = invokestatic < PythonLoader, Lrecall, import()Lrecall; > @85 exception:v227<no information> [226=[recall]]
86 v229 = invokestatic < PythonLoader, Lhistory, import()Lhistory; > @86 exception:v230<no information> [229=[history]]
87 v232 = invokestatic < PythonLoader, Ltime, import()Ltime; > @87 exception:v233<no information> [232=[time]]
88 v235 = invokestatic < PythonLoader, LKeyError, import()LKeyError; > @88 exception:v236<no information> [235=[KeyError]]
89 v238 = invokestatic < PythonLoader, Ldisplay, import()Ldisplay; > @89 exception:v239<no information> [238=[display]]
90 v241 = invokestatic < PythonLoader, Ltensorflow, import()Ltensorflow; > @90 exception:v242tf2_test_dataset8.py [1:0] -> [1:23] [241=[tf]]
91 v246 = new <PythonLoader,Lscript tf2_test_dataset8.py/add>@91<no information> [246=[add]]
92 global:global script tf2_test_dataset8.py/add = v246<no information> [246=[add]]
93 putfield v1.< PythonLoader, LRoot, add, <PythonLoader,LRoot> > = v246<no information> [246=[add]]
94 v252 = fieldref v241.v253:#keras tf2_test_dataset8.py [8:39] -> [8:41] [241=[tf]]
95 v251 = fieldref v252.v254:#datasets tf2_test_dataset8.py [8:39] -> [8:41]
96 v250 = fieldref v251.v255:#mnist tf2_test_dataset8.py [8:39] -> [8:41]
97 v249 = fieldref v250.v256:#load_data tf2_test_dataset8.py [8:39] -> [8:41]
98 v248 = invokeFunction < PythonLoader, LCodeBody, do()LRoot; > v249 @98 exception:v257tf2_test_dataset8.py [8:39] -> [8:74]
BB2
99 v260 = fieldref v248.v258:#0 tf2_test_dataset8.py [8:0] -> [8:1]
100 v262 = fieldref v260.v258:#0 tf2_test_dataset8.py [8:1] -> [8:8] [262=[x_train]]
102 v264 = fieldref v260.v259:#1 tf2_test_dataset8.py [8:10] -> [8:17] [264=[y_train]]
104 v265 = fieldref v248.v259:#1 tf2_test_dataset8.py [8:0] -> [8:1]
105 v267 = fieldref v265.v258:#0 tf2_test_dataset8.py [8:21] -> [8:27] [267=[x_test]]
107 v269 = fieldref v265.v259:#1 tf2_test_dataset8.py [8:29] -> [8:35] [269=[y_test]]
109 v277 = fieldref v241.v278:#data tf2_test_dataset8.py [9:10] -> [9:12] [241=[tf]]
110 v276 = fieldref v277.v279:#Dataset tf2_test_dataset8.py [9:10] -> [9:12]
111 v275 = fieldref v276.v280:#from_tensor_slicestf2_test_dataset8.py [9:10] -> [9:12]
112 v281 = new <PythonLoader,Ltuple>@112 tf2_test_dataset8.py [9:46] -> [9:64]
113 fieldref v281.v258:#0 = v262 = v262 tf2_test_dataset8.py [9:46] -> [9:64] [262=[x_train]]
114 fieldref v281.v259:#1 = v264 = v264 tf2_test_dataset8.py [9:46] -> [9:64] [264=[y_train]]
115 v274 = invokeFunction < PythonLoader, LCodeBody, do()LRoot; > v275,v281 @115 exception:v282tf2_test_dataset8.py [9:10] -> [9:12]
BB3
116 v273 = fieldref v274.v283:#shuffle tf2_test_dataset8.py [9:10] -> [9:12]
117 v272 = invokeFunction < PythonLoader, LCodeBody, do()LRoot; > v273,v284:#10000 @117 exception:v285tf2_test_dataset8.py [9:10] -> [9:12]
BB4
118 v271 = fieldref v272.v286:#batch tf2_test_dataset8.py [9:10] -> [9:12]
119 v270 = invokeFunction < PythonLoader, LCodeBody, do()LRoot; > v271,v287:#32 @119 exception:v288tf2_test_dataset8.py [9:10] -> [9:89] [270=[dataset, temp 3]]
BB5
BB6
122 v293 = new <PythonLoader,Ltuple>@122 tf2_test_dataset8.py [1:0] -> [12:27]
123 v295 = global:global images tf2_test_dataset8.py [11:4] -> [11:10]
124 fieldref v293.v258:#0 = v295 = v295 tf2_test_dataset8.py [1:0] -> [12:27]
125 v297 = global:global labels tf2_test_dataset8.py [11:12] -> [11:18]
126 fieldref v293.v259:#1 = v297 = v297 tf2_test_dataset8.py [1:0] -> [12:27]
127 v298 = a property name of v270 <no information> [270=[dataset, temp 3]]
128 v300 = fieldref v298.v258:#0 tf2_test_dataset8.py [11:4] -> [11:10] [300=[images]]
130 v302 = fieldref v298.v259:#1 tf2_test_dataset8.py [11:12] -> [11:18] [302=[labels]]
132 v291 = binaryop(ne) v292:#null , v298 tf2_test_dataset8.py [1:0] -> [12:27]
133 conditional branch(eq, to iindex=-1) v291,v258:#0tf2_test_dataset8.py [1:0] -> [12:27]
BB7
134 v304 = new <PythonLoader,Ltuple>@134 tf2_test_dataset8.py [1:0] -> [12:27]
135 fieldref v304.v258:#0 = v300 = v300 tf2_test_dataset8.py [1:0] -> [12:27] [300=[images]]
136 fieldref v304.v259:#1 = v302 = v302 tf2_test_dataset8.py [1:0] -> [12:27] [302=[labels]]
137 v303 = fieldref v270.v304 tf2_test_dataset8.py [1:0] -> [12:27] [270=[dataset, temp 3]]
138 v305 = fieldref v303.v258:#0 tf2_test_dataset8.py [11:4] -> [11:10] [305=[images]]
140 v306 = fieldref v303.v259:#1 tf2_test_dataset8.py [11:12] -> [11:18] [306=[labels]]
142 v307 = invokeFunction < PythonLoader, LCodeBody, do()LRoot; > v246,v305,v305 @142 exception:v308tf2_test_dataset8.py [12:8] -> [12:27] [307=[c]246=[add]305=[images]]
BB8
144 goto (from iindex= 144 to iindex = 122)tf2_test_dataset8.py [1:0] -> [12:27]
BB9
From the above, consider the following snippet:
134 v304 = new <PythonLoader,Ltuple>@134 tf2_test_dataset8.py [1:0] -> [12:27]
135 fieldref v304.v258:#0 = v300 = v300 tf2_test_dataset8.py [1:0] -> [12:27] [300=[images]]
136 fieldref v304.v259:#1 = v302 = v302 tf2_test_dataset8.py [1:0] -> [12:27] [302=[labels]]
137 v303 = fieldref v270.v304 tf2_test_dataset8.py [1:0] -> [12:27] [270=[dataset, temp 3]]
Instruction 134 creates a new tuple and assigns images and labels to elements 0 and 1, respectively. But, I am losing track of these values later down the IR. I believe that the reason is that instruction 137 is incorrect. Specifically, there is no field of v270 whose value is stored in v304. In other words, I believe that there is a missing putfield instruction to store v304 in v270. That being said, the CAst printing is still able to track the variable names later down the IR:
138 v305 = fieldref v303.v258:#0 tf2_test_dataset8.py [11:4] -> [11:10] [305=[images]]
140 v306 = fieldref v303.v259:#1 tf2_test_dataset8.py [11:12] -> [11:18] [306=[labels]]
But, I don't understand how, because the object stored v270 doesn't have a field whose value is v304; it's created in instruction 134 but never assigned. Either way, the dataflow analysis can't track it, so something is wrong here---either the IR or the dataflow analysis.
