Problem of including pdb Handle of Vector or Map in pdb Computation

[BinhangYuan]

Hi,

I got a problem when implementing the sampling method for batch.

My implementation is to first generate a map which stores all the rowIndex (stored in the key set) that should appear in the batch. Then I pass the handle of such maps to the constructor of my multiselection, and stores them as private class members.

(BTW, I need to store the sampled index to make sure the batch index is the same for all the variables stored in different pdb set.)

Then some problem occurs after the generation of TCAP.

My code is here (https://github.com/BinhangYuan/plinycompute/blob/CaffeBackEnd/applications/PlinyCaffe/backend/sharedLibraries/operatorSharedLibraries/headers/Reshape1MultiSelection.h).

The TCAP looks likes this:

inputDataForScanUserSet_0(in0) <= SCAN ('inputDB_0', 'default_workspace_db', 'ScanUserSet_0')

/* Apply MultiSelection filtering */ nativ_0OutForMultiSelectionComp1(in0,nativ_0_1OutFor) <= APPLY (inputDataForScanUserSet_0(in0), inputDataForScanUserSet_0(in0), 'MultiSelectionComp_1', 'native_lambda_0', [('lambdaType', 'native_lambda')]) filteredInputForMultiSelectionComp1(in0) <= FILTER (nativ_0OutForMultiSelectionComp1(nativ_0_1OutFor), nativ_0OutForMultiSelectionComp1(in0), 'MultiSelectionComp_1')

/* Apply MultiSelection projection */ nativ_1OutForMultiSelectionComp1 (nativ_1_1OutFor) <= APPLY (filteredInputForMultiSelectionComp1(in0), filteredInputForMultiSelectionComp1(), 'MultiSelectionComp_1', 'native_lambda_1', [('lambdaType', 'native_lambda')]) flattenedOutForMultiSelectionComp1(flattened_nativ_1_1OutFor) <= FLATTEN (nativ_1OutForMultiSelectionComp1(nativ_1_1OutFor), nativ_1OutForMultiSelectionComp1(), 'MultiSelectionComp_1')

/* Extract key for aggregation */ methodCall_0OutFor_ClusterAggregationComp2(flattened_nativ_1_1OutFor,methodCall_0OutFor__getMultiply2AggregateKey) <= APPLY (flattenedOutForMultiSelectionComp1(flattened_nativ_1_1OutFor), flattenedOutForMultiSelectionComp1(flattened_nativ_1_1OutFor), 'ClusterAggregationComp_2', 'methodCall_0', [('inputTypeName', 'TensorBlock'), ('lambdaType', 'methodCall'), ('methodName', 'getMultiply2AggregateKey'), ('returnTypeName', 'TensorBlock')]) deref_1OutForClusterAggregationComp2(flattened_nativ_1_1OutFor, methodCall_0OutFor__getMultiply2AggregateKey) <= APPLY (methodCall_0OutFor_ClusterAggregationComp2(methodCall_0OutFor__getMultiply2AggregateKey), methodCall_0OutFor_ClusterAggregationComp2(flattened_nativ_1_1OutFor), 'ClusterAggregationComp_2', 'deref_1')

/* Extract value for aggregation */ methodCall_2OutFor_ClusterAggregationComp2(methodCall_0OutFor__getMultiply2AggregateKey,methodCall_2OutFor__getMultiply2AggregateValue) <= APPLY (deref_1OutForClusterAggregationComp2(flattened_nativ_1_1OutFor), deref_1OutForClusterAggregationComp2(methodCall_0OutFor__getMultiply2AggregateKey), 'ClusterAggregationComp_2', 'methodCall_2', [('inputTypeName', 'TensorBlock'), ('lambdaType', 'methodCall'), ('methodName', 'getMultiply2AggregateValue'), ('returnTypeName', 'TensorBlock')]) deref_3OutForClusterAggregationComp2(methodCall_0OutFor__getMultiply2AggregateKey, methodCall_2OutFor__getMultiply2AggregateValue) <= APPLY (methodCall_2OutFor_ClusterAggregationComp2(methodCall_2OutFor__getMultiply2AggregateValue), methodCall_2OutFor_ClusterAggregationComp2(methodCall_0OutFor__getMultiply2AggregateKey), 'ClusterAggregationComp_2', 'deref_3')

/* Apply aggregation */ aggOutForClusterAggregationComp2 (aggOutFor2)<= AGGREGATE (deref_3OutForClusterAggregationComp2(methodCall_0OutFor__getMultiply2AggregateKey, methodCall_2OutFor__getMultiply2AggregateValue),'ClusterAggregationComp_2') aggOutForClusterAggregationComp2_out( ) <= OUTPUT ( aggOutForClusterAggregationComp2 ( aggOutFor2 ), 'X_0', 'default_workspace_db', 'WriteUserSet_3')

We debugged into the code, and found the code stuck in the the macro ENABLE_DEEP_COPY, then the stack is full of STD copy issues.

Any suggestion is helpful!

Best wishes, Binhang