HLS4ML Profiling missing dense layers

[rtspk]

Hi Guys,

I have a Qkeras Model, which I am trying to convert to hls4ml model. The accuracy has reduced drastically. I have tried to profile the hls4ml model. The problem is that, there are some layers which are missing from the plot ( activations plot after optimization) . for some models, it did plot all the layers, but for this model, it is not plotting all the layers. Here are the four plots of the HLS4ML profiler.

Could you guys help me out with this ?

---

Here are some additional details on the model. Qkeras is quantized to quantized_bits(6,0,alpha=1)

Model Precision: ap_fixed<16,6> ReuseFactor: 1 Strategy: Resource LayerName Qmodel_input Precision result: ap_fixed<16,6> Strategy: Resource Trace: True Qconv2d_L1 Precision weight: ap_fixed<6,1> bias: ap_fixed<6,1> ReuseFactor: 1 Strategy: Resource Trace: True act_L1 Precision result: ap_ufixed<6,0> ReuseFactor: 1 Strategy: Resource Trace: True max_pooling2d_6 Precision: ap_fixed<16,6> Strategy: Resource Trace: True Qconv2d_L2 Precision weight: ap_fixed<6,1> bias: ap_fixed<6,1> ReuseFactor: 1 Strategy: Resource Trace: True act_L2 Precision result: ap_ufixed<6,0> ReuseFactor: 1 Strategy: Resource Trace: True max_pooling2d_7 Precision: ap_fixed<16,6> Strategy: Resource Trace: True Qconv2d_L3 Precision weight: ap_fixed<6,1> bias: ap_fixed<6,1> ReuseFactor: 1 Strategy: Resource Trace: True act_L3 Precision result: ap_ufixed<6,0> ReuseFactor: 1 Strategy: Resource Trace: True radar_output_L4 Precision weight: ap_fixed<16,6> bias: ap_fixed<16,6> result: ap_fixed<16,6> ReuseFactor: 1 Strategy: Resource Trace: True radar_output_L4_linear Precision: ap_fixed<16,6> ReuseFactor: 1 table_size: 1024 table_t: ap_fixed<18,8> Strategy: Resource Trace: True softmax_L4 Precision: ap_fixed<16,6> ReuseFactor: 1 table_size: 1024 exp_table_t: ap_fixed<18,8,AP_RND,AP_SAT> inv_table_t: ap_fixed<18,8,AP_RND,AP_SAT> Strategy: Stable Trace: True

[rtspk]

I need to observe the profiling results, to drill down why the Qkeras Quantized version is not giving the desired accuracy on the HLS4ML model. the problem, i am having is that, the profiler is not showing the output of all the layers.

Distribution of non-zero activations before optimization (3rd Figure): All the plots are shown

Distribution of non-zero activations after optimization(4th fig): only the max_pool results are showing. the plot of other layers, mainly softmax_L4 is missing

Does anyone else have encountered this problem ? any suggestions/solutions ?

[HenningCode]

I actually think you dont need to see the layers after optimization. You should concentrate on this part of the documentation.

As a starting point, a good configuration would at least cover the box and whisker for each variable with the grey box. Make sure the box and whisker is contained to the right by using sufficient integer bits to avoid overflow. It might be that more precision is needed (grey boxes extend further to the left) to achieve satisfactory performance. In some cases, it is safe to barely cover the values and still achieve good accuracy.

So radar_output_L4 needs more integer bits.

[rtspk]

@HenningCode Thanks you very much. It did solve the problem. I have one follow up questions. I am not able to find the answer in the hls4ml publications.

1. What is the different between 'Before and After Optimization' in these plots ?

[HenningCode]

My guess is, that the values before are the ones of the QKeras Model and the the after values are the ones of the hls_model, because hls4ml has to apply PTQ to some parameters to fit the fixed_point data type.

I have a question aswell: Do you know what the radar_output_L4 layer represents in the activations plot? Because thats a dense layer and not really an activation layer right ? Also these layers always have to standard precision of the model why is that the case?

[rtspk]

@HenningCode My understanding is that, radar_output_L4 in the activation plot represents the output of the layer before the activation function.

I didnt understand your second question (Also these layers always have to standard precision of the model why is that the case?). could you elaborate it further.

[HenningCode]

@rtspk thats what I though aswell, but my understanding was that QKeras quantized everything: the input, the weights and the Output, so the Output should have the same precision as the rest of the layer, but as you can see all your Outputs of the Layers have the same precision in the activations plot, but the you specified the Conv layers to have less precision than the Dense layer?

I think now I understand your problem with before/after optimization. I doesnt make sense if I trained my whole model qunantized why is there and optimization necessary.. Basicly only the softmax layer needs to be quantized by hls4ml...