profiler icon indicating copy to clipboard operation
profiler copied to clipboard

Profiler : No Profile Data was Found in Collab code example

Open JaShams opened this issue 3 years ago • 9 comments

I am new to Tensorflow, and was trying to figure out how to use the profiler. By following the docs ( https://www.tensorflow.org/tensorboard/tensorboard_profiling_keras ), I got tensorboard working on my remote development machine and could view everything except the profiler output.
I then tried executing the code example mentioned in the doc in Collab ( https://colab.research.google.com/github/tensorflow/tensorboard/blob/master/docs/tensorboard_profiling_keras.ipynb ) only to see the same output.

Is the code example not up to date or is there an issue somewhere else?

Thanks Screenshot 2023-04-15 000305

JaShams avatar Apr 14 '23 18:04 JaShams

I am running into the same problem after running the command tensorboard --logdir logs/ System information:

  • Azure ML compute instance
  • Linux 22.08.19

Environment information

  • python=3.8.16
  • pip= 22.3.1
  • tensorboard=2.10.1
  • tensorboard-plugin-profile=2.10.1

Files in logs directory files

Tensorboard: tb

rick020 avatar Apr 18 '23 12:04 rick020

I run into the same problem, too

gxxxh avatar Apr 26 '23 07:04 gxxxh

I can confirm that I'm experiencing the same issue 🫤

Pondiniii avatar Jul 19 '23 13:07 Pondiniii

same here. Does anybody know how to solve this? Several months have passed, without even a single reply from the developers.

caleb1010 avatar Sep 05 '23 03:09 caleb1010

Hey, just took a look, seems the log directory is missing an events file, we will look into fixing this. As a quick workaround for now, please try:

  1. download the events.out.tfevents file at https://github.com/tensorflow/profiler/tree/master/demo
  2. upload this file to the demo log directory like: Screenshot 2023-09-05 at 9 49 44 PM

then you should be able to see the profiler: Screenshot 2023-09-05 at 9 46 13 PM

zzzaries avatar Sep 06 '23 04:09 zzzaries

When using the profiler, the logs address should include the run ID. In the original example, this would be: %tensorboard --logdir='logs/20230414-182044'

lakeyk avatar Oct 11 '23 22:10 lakeyk

Hey, just took a look, seems the log directory is missing an events file, we will look into fixing this. As a quick workaround for now, please try:

  1. download the events.out.tfevents file at https://github.com/tensorflow/profiler/tree/master/demo
  2. upload this file to the demo log directory like:
Screenshot 2023-09-05 at 9 49 44 PM then you should be able to see the profiler: Screenshot 2023-09-05 at 9 46 13 PM

Hi, I have the same problem showing that "No profile data was found". However, there exists a file events.out.tfevents.1713334799.XX.profile-empty in folder logs/20240417-141935.

My TF version is also 2.10.1, but I try to profile everything in my own computer.

I even tried the advice from @lakeyk , but that still didn't work.

Seems that everything required by tensorboard is present:

│  events.out.tfevents.1713334799.XX.profile-empty
│
├─plugins
│  └─profile
│      └─2024_04_17_06_19_59
│              XX.input_pipeline.pb
│              XX.kernel_stats.pb
│              XX.memory_profile.json.gz
│              XX.overview_page.pb
│              XX.tensorflow_stats.pb
│              XX.trace.json.gz
│              XX.xplane.pb
│
├─train
│      events.out.tfevents.1713334775.XX.40004.0.v2
│
└─validation
        events.out.tfevents.1713334787.XX.40004.1.v2

I'm not sure whether it is an issue of this version of TF or not though.

ZM-J avatar Apr 17 '24 06:04 ZM-J

I recommend updating Tensorflow to the latest version. I am able to successfully load the files with the given file structure with version 2.16.1, but am not able to with version 2.10.1.

2.16.1

2.10.1

lakeyk avatar Apr 17 '24 18:04 lakeyk

@lakeyk could you share a gist/colab notebook to reproduce it?

Note that the problem in most cases is:

  1. Not using the right --logdir path
  2. Having correct logdir path but not showing Overview page
  • To solve number 1 check the correct --logdir as adviced by @lakeyk, something like %tensorboard --logdir logs/ID in Colab, or w/o % sign from CLI.

  • To solve number 2 for CPUs take a look at this code GIST, then Step-Time Graph should show up.

  • Note that for GPUs it won't show the Step-Time Graph 😭 but the Tools (dropdown menu) does show other stats. Also the framework ops are shown, but to be honest it's not clear what they show (probably Host to Device communication.)

Screenshot 2024-08-03 234856

santi-mir avatar Aug 02 '24 03:08 santi-mir