lingvo

lingvo copied to clipboard

Exits training

1

I am attempting to run the Librispeech training example (only the 100hrs data, not the 360 or 500). I have an issue where Lingvo successfully begins training and gets to...

argideritzalpea

Abnormal memory allocation of transformer with GPipe on V100-16GB

1

**Codebase Version**: - Lingvo: https://github.com/tensorflow/lingvo/tree/tf1.15 - Docker: build from https://github.com/tensorflow/lingvo/blob/tf1.15/docker/dev.dockerfile **Hardward config** - V100-16GB x 8 with NVLink **Problem Description** For default hyper-parameter as described [here](https://github.com/tensorflow/lingvo/blob/tf1.15/lingvo/tasks/lm/params/one_billion_wds.py#L174-L198), direct use of these...

fanshiqing

Using multiple GPUs for training on a single machine?

7

I am trying to train a simple 34-layer Resnet model with the Imagenet dataset on a machine with multiple GPU cards (all v100s). I am using Lingvo-0.6.2 synced from github...

xxww

Code for paper on choice of modeling unit for sequence-to-sequence speech recognition

3

Hi @rprabhavalkar @tonybruguier-google I was wondering if the code for this paper ("On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition") was open source: [https://www-i6.informatik.rwth-aachen.de/publications/download/1106/IrieKazukiPrabhavalkarRohitKannanAnjuliBruguierAntoineRybachDavidNguyenPatrick--OntheChoiceofModelingUnitforSequence-to-SequenceSpeechRecognition--2019.pdf](https://www-i6.informatik.rwth-aachen.de/publications/download/1106/IrieKazukiPrabhavalkarRohitKannanAnjuliBruguierAntoineRybachDavidNguyenPatrick--OntheChoiceofModelingUnitforSequence-to-SequenceSpeechRecognition--2019.pdf) I would be very...

argideritzalpea

Hi, I use all the data with camera labels in the waymo open dataset v1.0.0 to train the StarNet model, but after 16924 steps, the logger always print out the...

DapengFeng

When I run the script `bazel-bin/lingvo/trainer --job=evaler_test --run_locally=gpu --model=car.waymo.StarNetVehicle --logdir=/data3/fengdapeng/waymo/log --logtostderr`, I get the results `eval_test: step: 3120, error/center_distance: 0, error/height: 0, error/length: 0, error/rotation_deg: 0, error/width: 0, loss: 8.7859153e-05,...

DapengFeng

horovod support to MPI

2

jacob1017

Loading a saved Lingvo model from its .meta file

4

Hi. I am trying to load a tensorflow meta graph from a saved Lingvo checkpoint using Tensorflow version 1.15 to convert it to a SavedModel for tensorflow serving.. I am...

smdshakeelhassan

ASR model with unidirectional lstm and Monotonic attention not converging

4

I made some changes to asr model. encoder -> unidirectional lstm and monotonic attention. After around 10000 steps, I can see that the loss is around 0.1 ``` I0106 06:35:36.474763...

manish-kumar-garg

A problem of distributed training

5

sync mode ================================================== server info: controller: server0:2222, 8gpus trainer_client: server1:2222, worker0: server2:2222, 8gpus worker1: server3:2222, 8gpus worker2: server4:2222, 8gpus follow the setting of "run_distributed.py", it doesn't work. server0# bazel-bin/lingvo/trainer --cluster_spec=controller=server0:2222@trainer_client=server1:2222@worker=server2:2222,server3:2222,server4:2222...

maoxuepeng