Improve handling of model parameters

[berleon]

Hi Avanti,

I modified how DeepLIFT handles model parameters. The current behaviour is to store them as numpy arrays. I converted them to tensorflow variables. This change resulted in a 10x speed up for a VGG-16.

def benchmark_deeplift(analyzer):
    start = time.time()
    analyzer.create_analyzer_model()
    print("creating analyzer: {:.4f}s".format(time.time() - start))

    start = time.time()
    analyzer.analyze(meta.ex_image)
    print("analyzing first image: {:.4f}s".format(time.time() - start))

    start = time.time()
    n = 5
    for i in range(n):
        analyzer.analyze(meta.ex_image)
    print("analyzing {} image: {:.4f}s".format(n, time.time() - start))

This code snippet gives the following numbers on (AMD Ryzen 1950X 16 Cores, 64 GB RAM, RTX 2080 TI) for a VGG-16:

With tensorflow variables:

creating analyzer: 14.2925s
analyzing first image: 6.4111s
analyzing 5 image: 13.3561s

With numpy variables:

creating analyzer: 62.4572s
analyzing first image: 164.0128s
analyzing 5 image: 95.1095s

Also the speedup is decent, the current code would break backwards compabibilty. If any code relies on Conv2d.kernel to be a numpy array. If you would consider this a problem, one could create extra variables like Conv2d._tf_kernel with the tensorflow variable.

Cheers, Leon

[berleon]

okay, the tests are failing exactly because of this backward compatibility problem. Let me know if you would prefer to put the tensorflow variables into an ._tf_kernel and ._tf_bias members. The .kernel member could then become a property that evaluates _tf_kernel. Otherwise, I could adapt the tests to work with tensorflow variables.