AttributeError: 'UserRegisteredTypeKerasTensor' object has no attribute 'mean' error raised.

[Diogenes4Sinope]

Please can anyone let me know why the following code:

from tensorflow.keras.layers import Input
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Concatenate
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam

def neg_log_likelihood(y, distr): 
    return -distr.log_prob(y) 

def my_dist(params): #723 
    return tfd.Normal(loc=params, scale=1)

inputs = Input(shape=(1,))
params = Dense(1)(inputs)
dist = tfp.layers.DistributionLambda(my_dist)(params) 
model_sd1 = Model(inputs=inputs, outputs=dist) 
model_sd1.compile(Adam(), loss=neg_log_likelihood) 

And then running the following code;

model_mean_sd1 = Model(inputs=inputs, outputs=dist.mean())

leads to the following error:

AttributeError: 'UserRegisteredTypeKerasTensor' object has no attribute 'mean'

Any help will be greatly appreciated. Thanks.

[anilkunwar]

dist = tfp.layers.DistributionLambda(my_dist)(params) ... model_mean_sd1 = Model(inputs=inputs, outputs=dist.mean())

In case of tfp version 0.13.0, I also get the same error in the above line. The code however runs perfect in tfp version 0.8.0 . Temporary solution :- installing tfp version =0.8.0 will work pip install tensorflow-probability==0.8.0

It seems there have been a change in the newer versions of tensorflow_probability library.

Which version of tensorflow_probability (tfp) is installed in your computer ?

[macot]

after version 0.12 use the "convert_to_tensor_fn" for outputs' manipulation. ex: dist1 = tfp.layers.DistributionLambda( make_distribution_fn=my_dist, convert_to_tensor_fn=tfp.distributions.Distribution.mean )(params)
model_mean_sd_1 = Model(inputs=inputs, outputs=dist1)

[asaranti]

What should one do, if one wants to get both the mean and the stddev ? Thanks in advance.

TF Version 2.13.0 TFP Version 0.20.1

[ricopicone]

@asaranti did you figure that out?

[asaranti]

@asaranti did you figure that out?

Sadly not, but if I do I will share it.

[ricopicone]

Thanks!

[auspect]

What should one do, if one wants to get both the mean and the stddev ? Thanks in advance.

TF Version 2.13.0 TFP Version 0.20.1

I'm not sure if that answers your question, but I think we can calculate the mean and variance like this :

means = model(x).mean() # not model.predict
vars = model(x).variance() # not model.predict

At least for my use case it worked fine

[egolfbr]

What should one do, if one wants to get both the mean and the stddev ? Thanks in advance. TF Version 2.13.0 TFP Version 0.20.1

I'm not sure if that answers your question, but I think we can calculate the mean and variance like this :

means = model(x).mean() # not model.predict
vars = model(x).variance() # not model.predict

At least for my use case it worked fine

I think the goal is to create a keras.Model instance where the outputs are a tfp.layers.distributionLambda function that is created by a user defined tfp.Distribution. So while your method operates on the existing model, the goal is to create a new model.....i think

[egolfbr]

This solved it for me

def NLL(y, distr):
  return -distr.log_prob(y) 

def my_dist(params): 
  return tfd.Normal(loc=params[:,0:1], scale=1e-3 + tf.math.softplus(0.05 * params[:,1:2]))# both parameters are learnable

inputs = Input(shape=(1,))
out1 = Dense(1)(inputs) #A
hidden1 = Dense(30,activation="relu")(inputs)
hidden1 = Dense(20,activation="relu")(hidden1)
hidden2 = Dense(20,activation="relu")(hidden1)
out2 = Dense(1)(hidden2) #B
params = Concatenate()([out1,out2]) #C
dist = tfp.layers.DistributionLambda(my_dist)(params) 
distmean = tfp.layers.DistributionLambda(my_dist, convert_to_tensor_fn=tfp.distributions.Distribution.mean)(params) 
diststddev = tfp.layers.DistributionLambda(my_dist, convert_to_tensor_fn=tfp.distributions.Distribution.stddev)(params) 

model_flex_sd = Model(inputs=inputs, outputs=dist)
model_flex_sd.compile(Adam(learning_rate=0.01), loss=NLL)

model_flex_sd_mean = Model(inputs=inputs, outputs=distmean)
model_flex_sd_sd = Model(inputs=inputs, outputs=diststddev)

[auspect]

This solved it for me

def NLL(y, distr):
  return -distr.log_prob(y) 

def my_dist(params): 
  return tfd.Normal(loc=params[:,0:1], scale=1e-3 + tf.math.softplus(0.05 * params[:,1:2]))# both parameters are learnable

inputs = Input(shape=(1,))
out1 = Dense(1)(inputs) #A
hidden1 = Dense(30,activation="relu")(inputs)
hidden1 = Dense(20,activation="relu")(hidden1)
hidden2 = Dense(20,activation="relu")(hidden1)
out2 = Dense(1)(hidden2) #B
params = Concatenate()([out1,out2]) #C
dist = tfp.layers.DistributionLambda(my_dist)(params) 
distmean = tfp.layers.DistributionLambda(my_dist, convert_to_tensor_fn=tfp.distributions.Distribution.mean)(params) 
diststddev = tfp.layers.DistributionLambda(my_dist, convert_to_tensor_fn=tfp.distributions.Distribution.stddev)(params) 

model_flex_sd = Model(inputs=inputs, outputs=dist)
model_flex_sd.compile(Adam(learning_rate=0.01), loss=NLL)

model_flex_sd_mean = Model(inputs=inputs, outputs=distmean)
model_flex_sd_sd = Model(inputs=inputs, outputs=diststddev)

Yes, this is indeed more helpful! Thank you for providing this piece of code.