tensorboard
tensorboard copied to clipboard
Published
20 hours ago •
tensorflow
tensorflow
Callback failing: AttributeError: 'TrackableWeightHandler' object has no attribute 'name'
Diagnostics
Diagnostics output
--- check: autoidentify
INFO: diagnose_tensorboard.py source unavailable
INFO: diagnose_tensorboard.py source unavailable
--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=7, micro=8, releaselevel='final', serial=0)
INFO: sys.version_info: sys.version_info(major=3, minor=7, micro=8, releaselevel='final', serial=0)
INFO: os.name: posix
INFO: os.name: posix
INFO: os.uname(): posix.uname_result(sysname='Darwin', nodename='MacBook-Pro-3.lan', release='19.6.0', version='Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; root:xnu-6153.141.2.2~1/RELEASE_X86_64', machine='x86_64')
INFO: os.uname(): posix.uname_result(sysname='Darwin', nodename='MacBook-Pro-3.lan', release='19.6.0', version='Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; root:xnu-6153.141.2.2~1/RELEASE_X86_64', machine='x86_64')
INFO: sys.getwindowsversion(): N/A
INFO: sys.getwindowsversion(): N/A
--- check: package_management
INFO: has conda-meta: False
INFO: has conda-meta: False
INFO: $VIRTUAL_ENV: None
INFO: $VIRTUAL_ENV: None
--- check: installed_packages
INFO: installed: tensorboard==2.4.0
INFO: installed: tensorboard==2.4.0
INFO: installed: tensorflow==2.4.0
INFO: installed: tensorflow==2.4.0
INFO: installed: tensorflow-estimator==2.4.0
INFO: installed: tensorflow-estimator==2.4.0
--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '2.4.0'
INFO: tensorboard.version.VERSION: '2.4.0'
--- check: tensorflow_python_version
INFO: tensorflow.__version__: '2.4.0'
INFO: tensorflow.__version__: '2.4.0'
INFO: tensorflow.__git_version__: 'v2.4.0-rc4-71-g582c8d236cb'
INFO: tensorflow.__git_version__: 'v2.4.0-rc4-71-g582c8d236cb'
--- check: tensorboard_binary_path
INFO: which tensorboard: b'/Users/dennisyurkevich/.pyenv/versions/3.7.8/bin/tensorboard\n'
INFO: which tensorboard: b'/Users/dennisyurkevich/.pyenv/versions/3.7.8/bin/tensorboard\n'
--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC = <AddressFamily.AF_UNSPEC: 0>
socket.SOCK_STREAM = <SocketKind.SOCK_STREAM: 1>
socket.AI_ADDRCONFIG = <AddressInfo.AI_ADDRCONFIG: 1024>
socket.AI_PASSIVE = <AddressInfo.AI_PASSIVE: 1>
Loopback flags: <AddressInfo.AI_ADDRCONFIG: 1024>
Loopback infos: [(<AddressFamily.AF_INET6: 30>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0))]
Wildcard flags: <AddressInfo.AI_PASSIVE: 1>
Wildcard infos: [(<AddressFamily.AF_INET6: 30>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 0))]
--- check: readable_fqdn
INFO: socket.getfqdn(): 'macbook-pro-3.lan'
INFO: socket.getfqdn(): 'macbook-pro-3.lan'
--- check: stat_tensorboardinfo
INFO: directory: /var/folders/gn/91n6_2ts4r5drxqt__hnn36w0000gn/T/.tensorboard-info
INFO: directory: /var/folders/gn/91n6_2ts4r5drxqt__hnn36w0000gn/T/.tensorboard-info
INFO: .tensorboard-info directory does not exist
INFO: .tensorboard-info directory does not exist
--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['/Users/dennisyurkevich/.pyenv/versions/3.7.8/lib/python3.7/site-packages']; bad_roots (0): []
INFO: tensorboard_roots (1): ['/Users/dennisyurkevich/.pyenv/versions/3.7.8/lib/python3.7/site-packages']; bad_roots (0): []
--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==0.11.0
aiohttp==3.6.2
alabaster==0.7.12
alembic==1.4.2
anyconfig==0.9.11
AppNexus-client==0.8.1
appnope==0.1.0
argcomplete==1.12.0
arrow==0.17.0
astroid==2.4.2
astunparse==1.6.3
async-timeout==3.0.1
attrs==19.3.0
Babel==2.9.0
backcall==0.2.0
backoff==1.10.0
beautifulsoup4==4.9.1
binaryornot==0.4.4
bleach==3.1.5
blis==0.4.1
bokeh==2.1.1
boto==2.49.0
boto3==1.14.30
botocore==1.17.30
CacheControl==0.12.6
cachetools==4.1.1
catalogue==1.0.0
cdx-toolkit==0.9.25
Cerberus==1.3.2
certifi==2020.6.20
cffi==1.14.1
chardet==3.0.4
click==7.1.2
cloudevents==0.3.0
cloudpickle==1.6.0
clusteval==1.0.2
colorcet==2.0.2
comcrawl==1.0.2
configparser==5.0.0
cookiecutter==1.6.0
courlan==0.2.3
coverage==5.3
crcmod==1.7
cryptography==3.0
cssselect==1.1.0
cycler==0.10.0
cymem==2.0.3
Cython==0.29.21
dask==2.25.0
datashape==0.5.2
decorator==4.4.2
defusedxml==0.6.0
dill==0.3.3
distributed==2.21.0
docker==4.3.1
docutils==0.15.2
en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz
enstop==0.2.5
entrypoints==0.3
fastcore==1.3.10
filelock==3.0.12
firebase-admin==4.4.0
Flask==1.1.2
flatbuffers==1.12
fsspec==0.6.3
functions-framework==2.0.0
funcy==1.14
furl==2.1.0
future==0.18.2
gast==0.3.3
gensim==3.8.3
google-api-core==1.23.0
google-api-python-client==1.12.3
google-auth==1.23.0
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.1
google-cloud-bigquery==2.6.0
google-cloud-bigquery-storage==2.1.0
google-cloud-core==1.4.1
google-cloud-firestore==1.9.0
google-cloud-logging==1.15.1
google-cloud-secret-manager==2.0.0
google-cloud-storage==1.30.0
google-crc32c==0.1.0
google-pasta==0.2.0
google-resumable-media==0.7.0
googleapis-common-protos==1.52.0
graphviz==0.8.4
grpc-google-iam-v1==0.12.3
grpcio==1.32.0
gunicorn==20.0.4
h5py==2.10.0
hdbscan==0.8.26
HeapDict==1.0.1
holoviews==1.13.3
html2text==2020.1.16
htmldate==0.7.2
httplib2==0.18.1
humanfriendly==8.2
ibm-cloud-sdk-core==1.5.1
ibm-watson==4.5.0
idna==2.10
imagesize==1.2.0
importlib-metadata==1.7.0
importlib-resources==3.3.0
ipykernel==5.3.4
ipython==7.16.1
ipython-genutils==0.2.0
ipywidgets==7.5.1
isort==4.3.21
itsdangerous==1.1.0
jedi==0.17.2
jellyfish==0.8.2
Jinja2==2.11.2
jinja2-time==0.2.0
jmespath==0.10.0
joblib==0.16.0
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==6.1.6
jupyter-console==6.1.0
jupyter-core==4.6.3
jusText==2.2.0
kedro==0.16.6
Keras-Preprocessing==1.1.2
kiwisolver==1.2.0
lazy-object-proxy==1.4.3
libcst==0.3.13
lightfm==1.15
llvmlite==0.34.0
locket==0.2.0
lxml==4.5.2
Mako==1.1.3
Markdown==3.2.2
MarkupSafe==1.1.1
matplotlib==3.3.0
mccabe==0.6.1
mistune==0.8.4
more-itertools==8.4.0
msgpack==1.0.0
multidict==4.7.6
multipledispatch==0.6.0
murmurhash==1.0.2
mxnet==1.7.0.post1
mypy-extensions==0.4.3
nbconvert==5.6.1
nbdev==1.1.5
nbformat==5.0.7
networkx==2.4
nltk==3.5
notebook==6.0.3
numba==0.51.2
numexpr==2.7.1
numpy==1.19.4
oauthlib==3.1.0
opt-einsum==3.3.0
orderedmultidict==1.0.1
packaging==20.4
pandas==1.0.5
pandocfilters==1.4.2
panel==0.9.7
param==1.9.3
parsel==1.6.0
parso==0.7.1
partd==1.1.0
pathtools==0.1.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==7.2.0
pip==20.3.3
pip-autoremove==0.9.1
pip-tools==5.3.0
pipx==0.15.4.0
plac==1.1.3
plotly==4.12.0
pluggy==0.13.1
ply==3.11
poyo==0.5.0
preshed==3.0.2
prometheus-client==0.8.0
promise==2.3
prompt-toolkit==3.0.5
proto-plus==1.11.0
protobuf==3.12.2
psutil==5.7.2
ptyprocess==0.6.0
py==1.9.0
-e git+https://github.com/AirGrid/py-utils.git@b89ba5a0ff82fe15510cf82613a1d1a1808baa76#egg=py_utils
py-zipkin==0.20.0
pyarrow==2.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybind11==2.5.0
pycld3==0.20
pycparser==2.20
pyct==0.4.6
pydash==4.9.0
Pygments==2.6.1
PyJWT==1.7.1
pyLDAvis==2.1.2
pylint==2.5.3
pynndescent==0.4.8
pyOpenSSL==19.1.0
pyparsing==2.4.7
pyrsistent==0.16.0
pytest==5.4.3
pytextrank==2.0.3
python-dateutil==2.8.1
python-editor==1.0.4
python-json-logger==0.1.11
pytz==2020.1
pyviz-comms==0.7.6
PyYAML==5.3.1
pyzmq==19.0.1
qtconsole==4.7.5
QtPy==1.9.0
readability-lxml==0.8.1
regex==2020.7.14
requests==2.24.0
requests-file==1.5.1
requests-oauthlib==1.3.0
retrying==1.3.3
rsa==4.6
ruamel.yaml==0.16.12
ruamel.yaml.clib==0.2.2
s3transfer==0.3.3
sacremoses==0.0.43
scikit-learn==0.23.2
scipy==1.5.2
seaborn==0.10.1
segtok==1.5.10
Send2Trash==1.5.0
sentencepiece==0.1.94
setuptools==50.0.0
six==1.15.0
sklearn==0.0
smart-open==2.1.0
snowballstemmer==2.0.0
sortedcontainers==2.2.2
soupsieve==2.0.1
spacy==2.3.2
Sphinx==3.3.1
sphinx-autodoc-typehints==1.11.1
sphinx-markdown-builder==0.5.4
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
SQLAlchemy==1.3.19
SQLAlchemy-Utils==0.36.8
srsly==1.0.2
stanza==1.1.1
tabulate==0.8.7
tbb==2019.0
tblib==1.7.0
tenacity==6.2.0
tensorboard==2.4.0
tensorboard-plugin-wit==1.7.0
tensorflow==2.4.0
tensorflow-addons==0.12.0
tensorflow-datasets==4.1.0
tensorflow-estimator==2.4.0
tensorflow-metadata==0.26.0
tensorflow-recommenders==0.3.0
termcolor==1.1.0
terminado==0.8.3
testpath==0.4.4
textmap @ git+https://github.com/TutteInstitute/TextMAP.git@150d939b4fb5b6dbcda057ef782ef442c042e7a5
thinc==7.4.1
Thingy==0.8.5
threadpoolctl==2.1.0
thriftpy2==0.4.11
tldextract==2.2.2
tokenizers==0.9.4
toml==0.10.1
toolz==0.10.0
toposort==1.5
torch==1.6.0
torchvision==0.7.0
tornado==6.0.4
tqdm==4.55.1
trafilatura==0.5.2
traitlets==4.3.3
transformers==4.1.1
typed-ast==1.4.1
typeguard==2.10.0
typing-extensions==3.7.4.2
typing-inspect==0.6.0
umap-learn==0.4.6
unify==0.5
untokenize==0.1.1
uritemplate==3.0.1
urllib3==1.25.11
userpath==1.4.1
vectorizers @ git+https://github.com/TutteInstitute/vectorizers.git@eff54167d0e68bbf35ec946a95ffd59444454c77
w3lib==1.22.0
warcio==1.7.4
wasabi==0.8.0
watchdog==0.10.3
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==0.48.0
Werkzeug==1.0.1
wheel==0.36.2
whichcraft==0.6.1
widgetsnbextension==3.5.1
wikipedia==1.4.0
Wikipedia-API==0.5.4
wordcloud==1.8.0
wrapt==1.12.1
xarray==0.16.0
yake @ git+https://github.com/LIAAD/yake@2c9fedbe0e3195a273a3c3daef3d41d4d7c5b3b3
yapf==0.30.0
yarl==1.5.1
zenserp==0.2
zict==2.0.0
zipp==3.1.0
INFO: pip freeze --all:
absl-py==0.11.0
aiohttp==3.6.2
airgrid @ git+https://github.com/AirGrid/mind.git@0a778be6e462881799b8c7b74e5d14ba05067332#subdirectory=serverless_crawler/client
alabaster==0.7.12
alembic==1.4.2
anyconfig==0.9.11
AppNexus-client==0.8.1
appnope==0.1.0
argcomplete==1.12.0
arrow==0.17.0
astroid==2.4.2
astunparse==1.6.3
async-timeout==3.0.1
attrs==19.3.0
Babel==2.9.0
backcall==0.2.0
backoff==1.10.0
beautifulsoup4==4.9.1
binaryornot==0.4.4
bleach==3.1.5
blis==0.4.1
bokeh==2.1.1
boto==2.49.0
boto3==1.14.30
botocore==1.17.30
CacheControl==0.12.6
cachetools==4.1.1
catalogue==1.0.0
cdx-toolkit==0.9.25
Cerberus==1.3.2
certifi==2020.6.20
cffi==1.14.1
chardet==3.0.4
click==7.1.2
client @ git+https://github.com/AirGrid/mind.git@482982a9dbac1275c85cd7e441feec525b041caa#subdirectory=serverless_crawler/client
cloudevents==0.3.0
cloudpickle==1.6.0
clusteval==1.0.2
colorcet==2.0.2
comcrawl==1.0.2
configparser==5.0.0
cookiecutter==1.6.0
courlan==0.2.3
coverage==5.3
crcmod==1.7
cryptography==3.0
cssselect==1.1.0
cycler==0.10.0
cymem==2.0.3
Cython==0.29.21
dask==2.25.0
datashape==0.5.2
decorator==4.4.2
defusedxml==0.6.0
dill==0.3.3
distributed==2.21.0
docker==4.3.1
docutils==0.15.2
en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz
enstop==0.2.5
entrypoints==0.3
fastcore==1.3.10
filelock==3.0.12
firebase-admin==4.4.0
Flask==1.1.2
flatbuffers==1.12
fsspec==0.6.3
functions-framework==2.0.0
funcy==1.14
furl==2.1.0
future==0.18.2
gast==0.3.3
gensim==3.8.3
google-api-core==1.23.0
google-api-python-client==1.12.3
google-auth==1.23.0
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.1
google-cloud-bigquery==2.6.0
google-cloud-bigquery-storage==2.1.0
google-cloud-core==1.4.1
google-cloud-firestore==1.9.0
google-cloud-logging==1.15.1
google-cloud-secret-manager==2.0.0
google-cloud-storage==1.30.0
google-crc32c==0.1.0
google-pasta==0.2.0
google-resumable-media==0.7.0
googleapis-common-protos==1.52.0
graphviz==0.8.4
grpc-google-iam-v1==0.12.3
grpcio==1.32.0
gunicorn==20.0.4
h5py==2.10.0
hdbscan==0.8.26
HeapDict==1.0.1
holoviews==1.13.3
html2text==2020.1.16
htmldate==0.7.2
httplib2==0.18.1
humanfriendly==8.2
ibm-cloud-sdk-core==1.5.1
ibm-watson==4.5.0
idna==2.10
imagesize==1.2.0
importlib-metadata==1.7.0
importlib-resources==3.3.0
ipykernel==5.3.4
ipython==7.16.1
ipython-genutils==0.2.0
ipywidgets==7.5.1
isort==4.3.21
itsdangerous==1.1.0
jedi==0.17.2
jellyfish==0.8.2
Jinja2==2.11.2
jinja2-time==0.2.0
jmespath==0.10.0
joblib==0.16.0
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==6.1.6
jupyter-console==6.1.0
jupyter-core==4.6.3
jusText==2.2.0
kedro==0.16.6
Keras-Preprocessing==1.1.2
kiwisolver==1.2.0
lazy-object-proxy==1.4.3
libcst==0.3.13
lightfm==1.15
llvmlite==0.34.0
locket==0.2.0
lxml==4.5.2
Mako==1.1.3
Markdown==3.2.2
MarkupSafe==1.1.1
matplotlib==3.3.0
mccabe==0.6.1
mistune==0.8.4
more-itertools==8.4.0
msgpack==1.0.0
multidict==4.7.6
multipledispatch==0.6.0
murmurhash==1.0.2
mxnet==1.7.0.post1
mypy-extensions==0.4.3
nbconvert==5.6.1
nbdev==1.1.5
nbformat==5.0.7
networkx==2.4
nltk==3.5
notebook==6.0.3
numba==0.51.2
numexpr==2.7.1
numpy==1.19.4
oauthlib==3.1.0
opt-einsum==3.3.0
orderedmultidict==1.0.1
packaging==20.4
pandas==1.0.5
pandocfilters==1.4.2
panel==0.9.7
param==1.9.3
parsel==1.6.0
parso==0.7.1
partd==1.1.0
pathtools==0.1.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==7.2.0
pip==20.3.3
pip-autoremove==0.9.1
pip-tools==5.3.0
pipx==0.15.4.0
plac==1.1.3
plotly==4.12.0
pluggy==0.13.1
ply==3.11
poyo==0.5.0
preshed==3.0.2
prometheus-client==0.8.0
promise==2.3
prompt-toolkit==3.0.5
proto-plus==1.11.0
protobuf==3.12.2
psutil==5.7.2
ptyprocess==0.6.0
py==1.9.0
-e git+https://github.com/AirGrid/py-utils.git@b89ba5a0ff82fe15510cf82613a1d1a1808baa76#egg=py_utils
py-zipkin==0.20.0
pyarrow==2.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybind11==2.5.0
pycld3==0.20
pycparser==2.20
pyct==0.4.6
pydash==4.9.0
Pygments==2.6.1
PyJWT==1.7.1
pyLDAvis==2.1.2
pylint==2.5.3
pynndescent==0.4.8
pyOpenSSL==19.1.0
pyparsing==2.4.7
pyrsistent==0.16.0
pytest==5.4.3
pytextrank==2.0.3
python-dateutil==2.8.1
python-editor==1.0.4
python-json-logger==0.1.11
pytz==2020.1
pyviz-comms==0.7.6
PyYAML==5.3.1
pyzmq==19.0.1
qtconsole==4.7.5
QtPy==1.9.0
readability-lxml==0.8.1
regex==2020.7.14
requests==2.24.0
requests-file==1.5.1
requests-oauthlib==1.3.0
retrying==1.3.3
rsa==4.6
ruamel.yaml==0.16.12
ruamel.yaml.clib==0.2.2
s3transfer==0.3.3
sacremoses==0.0.43
scikit-learn==0.23.2
scipy==1.5.2
seaborn==0.10.1
segtok==1.5.10
Send2Trash==1.5.0
sentencepiece==0.1.94
setuptools==50.0.0
six==1.15.0
sklearn==0.0
smart-open==2.1.0
snowballstemmer==2.0.0
sortedcontainers==2.2.2
soupsieve==2.0.1
spacy==2.3.2
Sphinx==3.3.1
sphinx-autodoc-typehints==1.11.1
sphinx-markdown-builder==0.5.4
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
SQLAlchemy==1.3.19
SQLAlchemy-Utils==0.36.8
srsly==1.0.2
stanza==1.1.1
tabulate==0.8.7
tbb==2019.0
tblib==1.7.0
tenacity==6.2.0
tensorboard==2.4.0
tensorboard-plugin-wit==1.7.0
tensorflow==2.4.0
tensorflow-addons==0.12.0
tensorflow-datasets==4.1.0
tensorflow-estimator==2.4.0
tensorflow-metadata==0.26.0
tensorflow-recommenders==0.3.0
termcolor==1.1.0
terminado==0.8.3
testpath==0.4.4
textmap @ git+https://github.com/TutteInstitute/TextMAP.git@150d939b4fb5b6dbcda057ef782ef442c042e7a5
thinc==7.4.1
Thingy==0.8.5
threadpoolctl==2.1.0
thriftpy2==0.4.11
tldextract==2.2.2
tokenizers==0.9.4
toml==0.10.1
toolz==0.10.0
toposort==1.5
torch==1.6.0
torchvision==0.7.0
tornado==6.0.4
tqdm==4.55.1
trafilatura==0.5.2
traitlets==4.3.3
transformers==4.1.1
typed-ast==1.4.1
typeguard==2.10.0
typing-extensions==3.7.4.2
typing-inspect==0.6.0
umap-learn==0.4.6
unify==0.5
untokenize==0.1.1
uritemplate==3.0.1
urllib3==1.25.11
userpath==1.4.1
vectorizers @ git+https://github.com/TutteInstitute/vectorizers.git@eff54167d0e68bbf35ec946a95ffd59444454c77
w3lib==1.22.0
warcio==1.7.4
wasabi==0.8.0
watchdog==0.10.3
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==0.48.0
Werkzeug==1.0.1
wheel==0.36.2
whichcraft==0.6.1
widgetsnbextension==3.5.1
wikipedia==1.4.0
Wikipedia-API==0.5.4
wordcloud==1.8.0
wrapt==1.12.1
xarray==0.16.0
yake @ git+https://github.com/LIAAD/yake@2c9fedbe0e3195a273a3c3daef3d41d4d7c5b3b3
yapf==0.30.0
yarl==1.5.1
zenserp==0.2
zict==2.0.0
zipp==3.1.0
Issue description
The keras callback is failing to write files to the folder:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-81-3f8d7320ec70> in <module>
18 verbose=0,
19 validation_data=cached_test,
---> 20 callbacks=[tensorboard_callback]
21 )
22
~/.pyenv/versions/3.7.8/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
1143 epoch_logs.update(val_logs)
1144
-> 1145 callbacks.on_epoch_end(epoch, epoch_logs)
1146 training_logs = epoch_logs
1147 if self.stop_training:
~/.pyenv/versions/3.7.8/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py in on_epoch_end(self, epoch, logs)
426 for callback in self.callbacks:
427 if getattr(callback, '_supports_tf_logs', False):
--> 428 callback.on_epoch_end(epoch, logs)
429 else:
430 if numpy_logs is None: # Only convert once.
~/.pyenv/versions/3.7.8/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py in on_epoch_end(self, epoch, logs)
2337
2338 if self.histogram_freq and epoch % self.histogram_freq == 0:
-> 2339 self._log_weights(epoch)
2340
2341 if self.embeddings_freq and epoch % self.embeddings_freq == 0:
~/.pyenv/versions/3.7.8/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py in _log_weights(self, epoch)
2395 for layer in self.model.layers:
2396 for weight in layer.weights:
-> 2397 weight_name = weight.name.replace(':', '_')
2398 summary_ops_v2.histogram(weight_name, weight, step=epoch)
2399 if self.write_images:
AttributeError: 'TrackableWeightHandler' object has no attribute 'name'
Code:
model = RecModel()
log_dir = 'logs/fit/' + datetime.datetime.now().strftime('%Y%m%d-%H%M%S')
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.compile(optimizer=tf.keras.optimizers.SGD(learning_rate=LEARNING_RATE))
history = model.fit(cached_train,
epochs=EPOCHS,
verbose=0,
validation_data=cached_test,
callbacks=[tensorboard_callback]
)
@ydennisy, In order to expedite the trouble-shooting process, please provide a complete code snippet to reproduce the issue reported here. Thanks!
i meet the same error,code like this :
# Compile the model
model.compile(optimizer = keras.optimizers.Adagrad(learning_rate=0.01),
loss = keras.losses.MeanSquaredError(),
metrics = [keras.metrics.MeanAbsoluteError()],
)
train_dataset = get_dataset_from_csv("train_data.csv", shuffle=True, batch_size=265)
log_dir="logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
test_dataset = get_dataset_from_csv("test_data.csv", batch_size=265)
# Fit the model with the training dtaa
model.fit(train_dataset, epochs=5, validation_data=test_dataset,callbacks=[tensorboard_callback])
the error is ;
Epoch 1/5
1599/1599 [==============================] - 42s 25ms/step - loss: 0.7993 - mean_absolute_error: 0.7109 - val_loss: 0.9138 - val_mean_absolute_error: 0.7630
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-78-61a5081c4849> in <module>
10 test_dataset = get_dataset_from_csv("test_data.csv", batch_size=265)
11 # Fit the model with the training dtaa
---> 12 model.fit(train_dataset, epochs=5, validation_data=test_dataset,callbacks=[tensorboard_callback])
13
14 # _,rmse = model.evaluate(test_dataset, verbose=0)
~/anaconda3/envs/tf2.4/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
1143 epoch_logs.update(val_logs)
1144
-> 1145 callbacks.on_epoch_end(epoch, epoch_logs)
1146 training_logs = epoch_logs
1147 if self.stop_training:
~/anaconda3/envs/tf2.4/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py in on_epoch_end(self, epoch, logs)
426 for callback in self.callbacks:
427 if getattr(callback, '_supports_tf_logs', False):
--> 428 callback.on_epoch_end(epoch, logs)
429 else:
430 if numpy_logs is None: # Only convert once.
~/anaconda3/envs/tf2.4/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py in on_epoch_end(self, epoch, logs)
2337
2338 if self.histogram_freq and epoch % self.histogram_freq == 0:
-> 2339 self._log_weights(epoch)
2340
2341 if self.embeddings_freq and epoch % self.embeddings_freq == 0:
~/anaconda3/envs/tf2.4/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py in _log_weights(self, epoch)
2395 for layer in self.model.layers:
2396 for weight in layer.weights:
-> 2397 weight_name = weight.name.replace(':', '_')
2398 summary_ops_v2.histogram(weight_name, weight, step=epoch)
2399 if self.write_images:
AttributeError: 'TrackableWeightHandler' object has no attribute 'name'
Ran the demo in colab getting started, which uses a very similar flow and was unable to reproduce. Suspect that somehow the issue is entangled with a choice of model?
https://colab.research.google.com/github/tensorflow/tensorboard/blob/master/docs/get_started.ipynb
...
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit(x=x_train,
y=y_train,
epochs=5,
validation_data=(x_test, y_test),
callbacks=[tensorboard_callback])
...
@HelWireless or @ydennisy , is it possible to reproduce your issue in colab and share (possibly with fake data or a minimal fake version of your model).
The issue persists when there are experimental preprocessing layers in the model.
This looks like the same issue as: https://github.com/tensorflow/tensorflow/issues/41244
EDIT:
Setting histogram_freq=0 stops this issue.
Is there a way to manually assign names to all layers?
This looks like the same issue as: tensorflow/tensorflow#41244
EDIT: Setting
histogram_freq=0stops this issue.Is there a way to manually assign names to all layers?
Setting histogram_freq=0 is a bad idea because you can't get tensorflow profiler in tensorboard if you do that .
It is a temporary solution .
@ydennisy, In order to expedite the trouble-shooting process, please provide a complete code snippet to reproduce the issue reported here. Thanks!
Here is a colab
This issue arises in particular when using categorical string layers (e.g. tensorflow.keras.layers.experimental.keras.preprocessing.StringLookup). We're looking into how this can be fixed, but it's more of an issue with the way the layer is structured than with the TensorBoard callback per se so we don't have much control over how quickly it will be resolved. (Googlers, see b/169618190.)
For now my best suggestion is to avoid using that layer; it is marked as experimental still.
@nfelt thanks for the update!
Do you have a suggestion for a workaround for that layer which works fine?
Unfortunately I'm not aware of any specific alternatives for this layer (I am not very familiar with the code in question myself, just ran into this when debugging). The most likely person to know is out of office for a little while so it may take some time to get this addressed.
@ydennisy a bit of a crude workaround to get this working is to subclass the TensorBoard Callback and explicitly check if there is a name attribute. This assumes that you do not want a histogram of those experimental layers (which is probably a fair assumption as not much learning happening there).
I fully expect there will be a more principled way to handle these experimental layers in the future but hopefully this should get you around this current issue.
class TBCallback(keras.callbacks.TensorBoard):
def _log_weights(self, epoch):
with self._train_writer.as_default():
with summary_ops_v2.always_record_summaries():
for layer in self.model.layers:
for weight in layer.weights:
if hasattr(weight, "name"):
weight_name = weight.name.replace(':', '_')
summary_ops_v2.histogram(weight_name, weight, step=epoch)
if self.write_images:
self._log_weight_as_image(weight, weight_name, epoch)
self._train_writer.flush()
FYI - I believe this is similar to https://github.com/tensorflow/tensorflow/issues/43834 as well.
Unfortunately https://github.com/tensorflow/tensorboard/issues/4530#issuecomment-774331534 is still the best suggestion I have (avoid problematic layers if using the keras callback). I've bumped the internal issue as well.
