unilm icon indicating copy to clipboard operation
unilm copied to clipboard

How to fix this error in sagemaker layoutlmv2 fine tuning

Open ha-seungwon opened this issue 2 years ago • 0 comments

I just use layoutlmv2 model for fine tuning but I got some error for detectron2

I think this problem is came from git repo how can i fix this error.... plz help me

Error

File "run_ner.py", line 600, in <module> main() File "run_ner.py", line 343, in main model = AutoModelForTokenClassification.from_pretrained( File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 447, in from_pretrained return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs) File "/opt/conda/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1493, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "/opt/conda/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py", line 1125, in __init__ self.layoutlmv2 = LayoutLMv2Model(config) File "/opt/conda/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py", line 709, in __init__ requires_backends(self, "detectron2") File "/opt/conda/lib/python3.8/site-packages/transformers/file_utils.py", line 848, in requires_backends raise ImportError("".join(failed)) ImportError: LayoutLMv2Model requires the detectron2 library but it was not found in your environment. Checkout the instructions on the installation page: https://github.com/facebookresearch/detectron2/blob/master/INSTALL.md and follow the ones that match your environment. 2023-02-02 01:46:17,069 sagemaker-training-toolkit INFO Waiting for the process to finish and give a return code. 2023-02-02 01:46:17,069 sagemaker-training-toolkit INFO Done waiting for a return code. Received 1 from exiting process. 2023-02-02 01:46:17,070 sagemaker-training-toolkit ERROR Reporting training FAILURE 2023-02-02 01:46:17,070 sagemaker-training-toolkit ERROR ExecuteUserScriptError: ExitCode

This is my sagemaker code

import sagemaker
from sagemaker.huggingface import HuggingFace

# gets role for executing training job
role = sagemaker.get_execution_role()


# hyperparameters, which are passed into the training job
hyperparameters={
                'model_name_or_path':'microsoft/layoutlmv2-base-uncased',
                'output_dir':'/opt/ml/model',
                'dataset_name':'seungwon12/layoutlmv2-data',
                 }

# git configuration to download our fine-tuning script
git_config = {'repo': 'https://github.com/huggingface/transformers.git','branch': 'v4.17.0'}

# creates Hugging Face estimator
huggingface_estimator = HuggingFace(
	entry_point='run_ner.py',
	source_dir='./examples/pytorch/token-classification',
	instance_type='ml.p3.2xlarge',
	instance_count=1,
	role=role,
	git_config=git_config,
	transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	hyperparameters = hyperparameters
)


# starting the train job
huggingface_estimator.fit({"train": "s3://sagemaker-us-east-1-862152216222/samples/datasets/layoutlm/train", "validation": "s3://sagemaker-us-east-1-862152216222/samples/datasets/layoutlm/validation","test": "s3://sagemaker-us-east-1-862152216222/samples/datasets/layoutlm/test"})

ha-seungwon avatar Feb 02 '23 02:02 ha-seungwon