Hope to have a colab version of run-classifier

[SchenbergZY]

Actually i can't figure out what is the {TPU_NAME} when using colab tpu. Looking forward to a colab version of the code.

[AxeldeRomblay]

Also could it be uploaded on tensorflow hub please ?

[ymcui]

If you are using TPU in Colab, then there will no TPU_NAME. However, you can do the following things to make it work. replace: https://github.com/zihangdai/xlnet/blob/master/run_classifier.py#L695 run_config = model_utils.configure_tpu(FLAGS) to

  tpu_cluster_resolver = tf.contrib.cluster_resolver.TPUClusterResolver(TPU_ADDRESS)
  is_per_host = tf.contrib.tpu.InputPipelineConfig.PER_HOST_V2
  run_config = tf.contrib.tpu.RunConfig(
      cluster=tpu_cluster_resolver,
      model_dir=OUTPUT_DIR,
      save_checkpoints_steps=2000,
      keep_checkpoint_max=2,
      tpu_config=tf.contrib.tpu.TPUConfig(
          iterations_per_loop=1000,
          num_shards=8,
          per_host_input_for_training=is_per_host))

If you are familiar with Colab, then you will know what is TPU_ADDRESS in the code above.

[SchenbergZY]

@ymcui Thank you for your help. But Actually the same problem occurs:

tensorflow.python.framework.errors_impl.InvalidArgumentError: From /job:worker/replica:0/task:0: Unsuccessful TensorSliceReader constructor: Failed to get matching files on xlnet_cased_L-24_H-1024_A-16/xlnet_model.ckpt: Unimplemented: File system scheme '[local]' not implemented (file: 'xlnet_cased_L-24_H-1024_A-16/xlnet_model.ckpt') [[node checkpoint_initializer_224 (defined at /content/model_utils.py:76) ]]

[ymcui]

It seems that you should load checkpoint file from GCS (Google Cloud Storage) instead of loading from local file system (i.e. uploading directly in Colab). But spiece.model file should load from local file system. Also see: https://github.com/zihangdai/xlnet/issues/13

[aditya-malte]

Do you have a colab ready notebook for XLNet now @ymcui . Please share it, I'm working on integrating colab with XLNet too.

[hana9090]

can I run it over Colab using GPU?

[aditya-malte]

I now have a notebook that's perfectly running with BERT TPUs, will share it shortly

[hana9090]

Thank you :) that's would be great. I appreciate that! @aditya-malte do I need a business account on google cloud to use TPU?

[CharlieBickerton]

@aditya-malte I'm currently working on integrating run-classifier with colab (GPU first then TPU). Anything you could share would be appricated!

[CharlieBickerton]

#44 Here's a link to a working classifier (imdb) colab (code)

[aditya-malte]

With Colab TPU Tested

Hello @hana9090 , @CharlieBickerton , @SchenbergZY , I have made available a colab Notebook with modified repo at the following link: https://github.com/aditya-malte/Colab-XLNet-FineTuning

Please star and share.

[SchenbergZY]

sure will star after my successful trial

[aditya-malte]

Please do :)

[h21k]

Hi all,
I also have BERT versions running on Colab. Currently also trying to use xlnet on there as well...will post once it's running.