diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard

Published 20 hours ago • verified publisher icon ShivamShrirao

ERROR: xformers-0.0.14.dev0-cp37-cp37m-linux_x86_64.whl is not a supported wheel on this platform.

Open jeannen opened this issue 3 years ago • 9 comments

Describe the bug

Hi!

I'm trying to run the notebook locally on Vast.ai, but when I type %pip install git+https://github.com/facebookresearch/xformers@1d31a3a#egg=xformers I get an error saying ERROR: xformers-0.0.14.dev0-cp37-cp37m-linux_x86_64.whl is not a supported wheel on this platform.

I tried to use the alternative %pip install git+https://github.com/facebookresearch/xformers@1d31a3a#egg=xformers but then I got a massive error (log section)

Here is my folder: image

Here is the accelerate config (I answered randomly, I'm just starting out and I don't know what to put there 😅)

[email protected]:/workspace$ accelerate config In which compute environment are you running? [0] Which type of machine are you using? [0] Do you want to run your training on CPU only (even if a GPU is available)? NO Do you want to use DeepSpeed?: NO Do you wish to use FP16 or BF16 (mixed precision)?: FP16

Reproduction

Default notebook with

!pip install git+https://github.com/ShivamShrirao/diffusers.git !pip install -U -r requirements.txt

at the begining

Logs

Collecting xformers
  Cloning https://github.com/facebookresearch/xformers (to revision 1d31a3a) to /tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e
  Running command git clone -q https://github.com/facebookresearch/xformers /tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e
  WARNING: Did not find branch or tag '1d31a3a', assuming revision or ref.
  Running command git checkout -q 1d31a3a
  Resolved https://github.com/facebookresearch/xformers to commit 1d31a3a
  Running command git submodule update --init --recursive -q
Requirement already satisfied: torch>=1.12 in /opt/conda/lib/python3.9/site-packages (from xformers) (1.13.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.9/site-packages (from xformers) (1.22.3)
Requirement already satisfied: pyre-extensions==0.0.23 in /opt/conda/lib/python3.9/site-packages (from xformers) (0.0.23)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.9/site-packages (from pyre-extensions==0.0.23->xformers) (4.4.0)
Requirement already satisfied: typing-inspect in /opt/conda/lib/python3.9/site-packages (from pyre-extensions==0.0.23->xformers) (0.8.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/lib/python3.9/site-packages (from typing-inspect->pyre-extensions==0.0.23->xformers) (0.4.3)
Building wheels for collected packages: xformers
  Building wheel for xformers (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-_7ydqext
       cwd: /tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/
  Complete output (197 lines):
  /opt/conda/lib/python3.9/site-packages/setuptools/dist.py:516: UserWarning: Normalizing '0.0.14.dev' to '0.0.14.dev0'
    warnings.warn(tmpl.format(**locals()))
  running bdist_wheel
  /opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.9
  creating build/lib.linux-x86_64-3.9/experimental
  copying experimental/__init__.py -> build/lib.linux-x86_64-3.9/experimental
  copying experimental/setup.py -> build/lib.linux-x86_64-3.9/experimental
  creating build/lib.linux-x86_64-3.9/xformers
  copying xformers/test.py -> build/lib.linux-x86_64-3.9/xformers
  copying xformers/utils.py -> build/lib.linux-x86_64-3.9/xformers
  copying xformers/__init__.py -> build/lib.linux-x86_64-3.9/xformers
  copying xformers/ops.py -> build/lib.linux-x86_64-3.9/xformers
  creating build/lib.linux-x86_64-3.9/experimental/ragged_inference
  copying experimental/ragged_inference/__init__.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
  copying experimental/ragged_inference/garbage_pad_ragged_acts.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
  copying experimental/ragged_inference/seq_kv_cache.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
  copying experimental/ragged_inference/test_utils.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
  copying experimental/ragged_inference/triton_v2_matmul.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
  copying experimental/ragged_inference/triton_v2_qk_dotprod.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
  copying experimental/ragged_inference/triton_v2_ragged_qk_dotprod.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
  creating build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/__init__.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_blocksparse_transformers.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_causal_blocksparse.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_core.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_encoder.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_mlp.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_multi_head_dispatch.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_nvfuser.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_nystrom_utils.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_pytorch_transformer.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_revnet.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_sddmm.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_triton_blocksparse.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_triton_dropout.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_triton_fused_linear.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_triton_layernorm.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_triton_softmax.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_triton_stride_sum.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_vit_timm.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/benchmark_mem_eff_attention.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  copying xformers/benchmarks/utils.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
  creating build/lib.linux-x86_64-3.9/xformers/components
  copying xformers/components/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components
  copying xformers/components/activations.py -> build/lib.linux-x86_64-3.9/xformers/components
  copying xformers/components/patch_embedding.py -> build/lib.linux-x86_64-3.9/xformers/components
  copying xformers/components/reversible.py -> build/lib.linux-x86_64-3.9/xformers/components
  copying xformers/components/input_projection.py -> build/lib.linux-x86_64-3.9/xformers/components
  copying xformers/components/multi_head_dispatch.py -> build/lib.linux-x86_64-3.9/xformers/components
  copying xformers/components/residual.py -> build/lib.linux-x86_64-3.9/xformers/components
  copying xformers/components/simplicial_embedding.py -> build/lib.linux-x86_64-3.9/xformers/components
  creating build/lib.linux-x86_64-3.9/xformers/factory
  copying xformers/factory/__init__.py -> build/lib.linux-x86_64-3.9/xformers/factory
  copying xformers/factory/block_configs.py -> build/lib.linux-x86_64-3.9/xformers/factory
  copying xformers/factory/block_factory.py -> build/lib.linux-x86_64-3.9/xformers/factory
  copying xformers/factory/hydra_helper.py -> build/lib.linux-x86_64-3.9/xformers/factory
  copying xformers/factory/model_factory.py -> build/lib.linux-x86_64-3.9/xformers/factory
  copying xformers/factory/weight_init.py -> build/lib.linux-x86_64-3.9/xformers/factory
  creating build/lib.linux-x86_64-3.9/xformers/helpers
  copying xformers/helpers/__init__.py -> build/lib.linux-x86_64-3.9/xformers/helpers
  copying xformers/helpers/hierarchical_configs.py -> build/lib.linux-x86_64-3.9/xformers/helpers
  copying xformers/helpers/timm_sparse_attention.py -> build/lib.linux-x86_64-3.9/xformers/helpers
  copying xformers/helpers/test_utils.py -> build/lib.linux-x86_64-3.9/xformers/helpers
  creating build/lib.linux-x86_64-3.9/xformers/sparse
  copying xformers/sparse/__init__.py -> build/lib.linux-x86_64-3.9/xformers/sparse
  copying xformers/sparse/_csr_ops.py -> build/lib.linux-x86_64-3.9/xformers/sparse
  copying xformers/sparse/csr_tensor.py -> build/lib.linux-x86_64-3.9/xformers/sparse
  copying xformers/sparse/utils.py -> build/lib.linux-x86_64-3.9/xformers/sparse
  copying xformers/sparse/blocksparse_tensor.py -> build/lib.linux-x86_64-3.9/xformers/sparse
  creating build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/__init__.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/dropout.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/fused_linear_layer.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/k_activations.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/k_dropout.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/k_fused_matmul_bw.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/k_fused_matmul_fw.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/k_layer_norm.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/k_softmax.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/k_sum.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/sum_strided.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/layer_norm.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/softmax.py -> build/lib.linux-x86_64-3.9/xformers/triton
  copying xformers/triton/utils.py -> build/lib.linux-x86_64-3.9/xformers/triton
  creating build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
  copying xformers/benchmarks/LRA/__init__.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
  copying xformers/benchmarks/LRA/batch_fetch_results.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
  copying xformers/benchmarks/LRA/batch_submit.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
  copying xformers/benchmarks/LRA/run_grid_search.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
  copying xformers/benchmarks/LRA/run_tasks.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
  copying xformers/benchmarks/LRA/run_with_submitit.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
  creating build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA/code
  copying xformers/benchmarks/LRA/code/__init__.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA/code
  copying xformers/benchmarks/LRA/code/dataset.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA/code
  copying xformers/benchmarks/LRA/code/model_wrapper.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA/code
  creating build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/_sputnik_sparse.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/attention_mask.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/attention_patterns.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/base.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/compositional.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/fourier_mix.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/global_tokens.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/lambda_layer.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/linformer.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/local.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/pooling.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/random.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/sparsity_config.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/utils.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/visual.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/blocksparse.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/core.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/favor.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/nystrom.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/ortho.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  copying xformers/components/attention/scaled_dot_product.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
  creating build/lib.linux-x86_64-3.9/xformers/components/feedforward
  copying xformers/components/feedforward/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
  copying xformers/components/feedforward/base.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
  copying xformers/components/feedforward/conv_mlp.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
  copying xformers/components/feedforward/mlp.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
  copying xformers/components/feedforward/fused_mlp.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
  copying xformers/components/feedforward/mixture_of_experts.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
  creating build/lib.linux-x86_64-3.9/xformers/components/nvfuser
  copying xformers/components/nvfuser/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
  copying xformers/components/nvfuser/bias_act_dropout.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
  copying xformers/components/nvfuser/bias_dropout_res.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
  copying xformers/components/nvfuser/bias_dropout_res_layernorm.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
  copying xformers/components/nvfuser/utils.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
  creating build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
  copying xformers/components/positional_embedding/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
  copying xformers/components/positional_embedding/base.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
  copying xformers/components/positional_embedding/param.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
  copying xformers/components/positional_embedding/rotary.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
  copying xformers/components/positional_embedding/sine.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
  copying xformers/components/positional_embedding/vocab.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
  creating build/lib.linux-x86_64-3.9/xformers/components/attention/feature_maps
  copying xformers/components/attention/feature_maps/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/attention/feature_maps
  copying xformers/components/attention/feature_maps/base.py -> build/lib.linux-x86_64-3.9/xformers/components/attention/feature_maps
  copying xformers/components/attention/feature_maps/softmax.py -> build/lib.linux-x86_64-3.9/xformers/components/attention/feature_maps
  running build_ext
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/setup.py", line 232, in <module>
      setuptools.setup(
    File "/opt/conda/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup
      return run_commands(dist)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
      dist.run_commands()
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
      self.run_command(cmd)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
      super().run_command(command)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
      cmd_obj.run()
    File "/opt/conda/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
      super().run_command(command)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
      cmd_obj.run()
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
      super().run_command(command)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
      cmd_obj.run()
    File "/opt/conda/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
      self.build_extensions()
    File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 485, in build_extensions
      compiler_name, compiler_version = self._check_abi()
    File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 869, in _check_abi
      _, version = get_compiler_abi_compatibility_and_version(compiler)
    File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 336, in get_compiler_abi_compatibility_and_version
      if not check_compiler_ok_for_platform(compiler):
    File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 290, in check_compiler_ok_for_platform
      which = subprocess.check_output(['which', compiler], stderr=subprocess.STDOUT)
    File "/opt/conda/lib/python3.9/subprocess.py", line 424, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/opt/conda/lib/python3.9/subprocess.py", line 528, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['which', 'g++']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for xformers
  Running setup.py clean for xformers
Failed to build xformers
Installing collected packages: xformers
    Running setup.py install for xformers ... error
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-nwi2pczz/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.9/xformers
         cwd: /tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/
    Complete output (201 lines):
    /opt/conda/lib/python3.9/site-packages/setuptools/dist.py:516: UserWarning: Normalizing '0.0.14.dev' to '0.0.14.dev0'
      warnings.warn(tmpl.format(**locals()))
    running install
    /opt/conda/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.9
    creating build/lib.linux-x86_64-3.9/experimental
    copying experimental/__init__.py -> build/lib.linux-x86_64-3.9/experimental
    copying experimental/setup.py -> build/lib.linux-x86_64-3.9/experimental
    creating build/lib.linux-x86_64-3.9/xformers
    copying xformers/test.py -> build/lib.linux-x86_64-3.9/xformers
    copying xformers/utils.py -> build/lib.linux-x86_64-3.9/xformers
    copying xformers/__init__.py -> build/lib.linux-x86_64-3.9/xformers
    copying xformers/ops.py -> build/lib.linux-x86_64-3.9/xformers
    creating build/lib.linux-x86_64-3.9/experimental/ragged_inference
    copying experimental/ragged_inference/__init__.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
    copying experimental/ragged_inference/garbage_pad_ragged_acts.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
    copying experimental/ragged_inference/seq_kv_cache.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
    copying experimental/ragged_inference/test_utils.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
    copying experimental/ragged_inference/triton_v2_matmul.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
    copying experimental/ragged_inference/triton_v2_qk_dotprod.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
    copying experimental/ragged_inference/triton_v2_ragged_qk_dotprod.py -> build/lib.linux-x86_64-3.9/experimental/ragged_inference
    creating build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/__init__.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_blocksparse_transformers.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_causal_blocksparse.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_core.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_encoder.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_mlp.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_multi_head_dispatch.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_nvfuser.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_nystrom_utils.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_pytorch_transformer.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_revnet.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_sddmm.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_triton_blocksparse.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_triton_dropout.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_triton_fused_linear.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_triton_layernorm.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_triton_softmax.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_triton_stride_sum.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_vit_timm.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/benchmark_mem_eff_attention.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    copying xformers/benchmarks/utils.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks
    creating build/lib.linux-x86_64-3.9/xformers/components
    copying xformers/components/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components
    copying xformers/components/activations.py -> build/lib.linux-x86_64-3.9/xformers/components
    copying xformers/components/patch_embedding.py -> build/lib.linux-x86_64-3.9/xformers/components
    copying xformers/components/reversible.py -> build/lib.linux-x86_64-3.9/xformers/components
    copying xformers/components/input_projection.py -> build/lib.linux-x86_64-3.9/xformers/components
    copying xformers/components/multi_head_dispatch.py -> build/lib.linux-x86_64-3.9/xformers/components
    copying xformers/components/residual.py -> build/lib.linux-x86_64-3.9/xformers/components
    copying xformers/components/simplicial_embedding.py -> build/lib.linux-x86_64-3.9/xformers/components
    creating build/lib.linux-x86_64-3.9/xformers/factory
    copying xformers/factory/__init__.py -> build/lib.linux-x86_64-3.9/xformers/factory
    copying xformers/factory/block_configs.py -> build/lib.linux-x86_64-3.9/xformers/factory
    copying xformers/factory/block_factory.py -> build/lib.linux-x86_64-3.9/xformers/factory
    copying xformers/factory/hydra_helper.py -> build/lib.linux-x86_64-3.9/xformers/factory
    copying xformers/factory/model_factory.py -> build/lib.linux-x86_64-3.9/xformers/factory
    copying xformers/factory/weight_init.py -> build/lib.linux-x86_64-3.9/xformers/factory
    creating build/lib.linux-x86_64-3.9/xformers/helpers
    copying xformers/helpers/__init__.py -> build/lib.linux-x86_64-3.9/xformers/helpers
    copying xformers/helpers/hierarchical_configs.py -> build/lib.linux-x86_64-3.9/xformers/helpers
    copying xformers/helpers/timm_sparse_attention.py -> build/lib.linux-x86_64-3.9/xformers/helpers
    copying xformers/helpers/test_utils.py -> build/lib.linux-x86_64-3.9/xformers/helpers
    creating build/lib.linux-x86_64-3.9/xformers/sparse
    copying xformers/sparse/__init__.py -> build/lib.linux-x86_64-3.9/xformers/sparse
    copying xformers/sparse/_csr_ops.py -> build/lib.linux-x86_64-3.9/xformers/sparse
    copying xformers/sparse/csr_tensor.py -> build/lib.linux-x86_64-3.9/xformers/sparse
    copying xformers/sparse/utils.py -> build/lib.linux-x86_64-3.9/xformers/sparse
    copying xformers/sparse/blocksparse_tensor.py -> build/lib.linux-x86_64-3.9/xformers/sparse
    creating build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/__init__.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/dropout.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/fused_linear_layer.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/k_activations.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/k_dropout.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/k_fused_matmul_bw.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/k_fused_matmul_fw.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/k_layer_norm.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/k_softmax.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/k_sum.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/sum_strided.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/layer_norm.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/softmax.py -> build/lib.linux-x86_64-3.9/xformers/triton
    copying xformers/triton/utils.py -> build/lib.linux-x86_64-3.9/xformers/triton
    creating build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
    copying xformers/benchmarks/LRA/__init__.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
    copying xformers/benchmarks/LRA/batch_fetch_results.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
    copying xformers/benchmarks/LRA/batch_submit.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
    copying xformers/benchmarks/LRA/run_grid_search.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
    copying xformers/benchmarks/LRA/run_tasks.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
    copying xformers/benchmarks/LRA/run_with_submitit.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA
    creating build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA/code
    copying xformers/benchmarks/LRA/code/__init__.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA/code
    copying xformers/benchmarks/LRA/code/dataset.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA/code
    copying xformers/benchmarks/LRA/code/model_wrapper.py -> build/lib.linux-x86_64-3.9/xformers/benchmarks/LRA/code
    creating build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/_sputnik_sparse.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/attention_mask.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/attention_patterns.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/base.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/compositional.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/fourier_mix.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/global_tokens.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/lambda_layer.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/linformer.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/local.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/pooling.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/random.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/sparsity_config.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/utils.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/visual.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/blocksparse.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/core.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/favor.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/nystrom.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/ortho.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    copying xformers/components/attention/scaled_dot_product.py -> build/lib.linux-x86_64-3.9/xformers/components/attention
    creating build/lib.linux-x86_64-3.9/xformers/components/feedforward
    copying xformers/components/feedforward/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
    copying xformers/components/feedforward/base.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
    copying xformers/components/feedforward/conv_mlp.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
    copying xformers/components/feedforward/mlp.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
    copying xformers/components/feedforward/fused_mlp.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
    copying xformers/components/feedforward/mixture_of_experts.py -> build/lib.linux-x86_64-3.9/xformers/components/feedforward
    creating build/lib.linux-x86_64-3.9/xformers/components/nvfuser
    copying xformers/components/nvfuser/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
    copying xformers/components/nvfuser/bias_act_dropout.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
    copying xformers/components/nvfuser/bias_dropout_res.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
    copying xformers/components/nvfuser/bias_dropout_res_layernorm.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
    copying xformers/components/nvfuser/utils.py -> build/lib.linux-x86_64-3.9/xformers/components/nvfuser
    creating build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
    copying xformers/components/positional_embedding/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
    copying xformers/components/positional_embedding/base.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
    copying xformers/components/positional_embedding/param.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
    copying xformers/components/positional_embedding/rotary.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
    copying xformers/components/positional_embedding/sine.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
    copying xformers/components/positional_embedding/vocab.py -> build/lib.linux-x86_64-3.9/xformers/components/positional_embedding
    creating build/lib.linux-x86_64-3.9/xformers/components/attention/feature_maps
    copying xformers/components/attention/feature_maps/__init__.py -> build/lib.linux-x86_64-3.9/xformers/components/attention/feature_maps
    copying xformers/components/attention/feature_maps/base.py -> build/lib.linux-x86_64-3.9/xformers/components/attention/feature_maps
    copying xformers/components/attention/feature_maps/softmax.py -> build/lib.linux-x86_64-3.9/xformers/components/attention/feature_maps
    running build_ext
    /opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
      warnings.warn(msg.format('we could not find ninja.'))
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/setup.py", line 232, in <module>
        setuptools.setup(
      File "/opt/conda/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
        return distutils.core.setup(**attrs)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup
        return run_commands(dist)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
        dist.run_commands()
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
        self.run_command(cmd)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
        super().run_command(command)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
        cmd_obj.run()
      File "/opt/conda/lib/python3.9/site-packages/setuptools/command/install.py", line 68, in run
        return orig.install.run(self)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 664, in run
        self.run_command('build')
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
        super().run_command(command)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
        cmd_obj.run()
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
        super().run_command(command)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
        cmd_obj.run()
      File "/opt/conda/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run
        _build_ext.run(self)
      File "/opt/conda/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
        self.build_extensions()
      File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 485, in build_extensions
        compiler_name, compiler_version = self._check_abi()
      File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 869, in _check_abi
        _, version = get_compiler_abi_compatibility_and_version(compiler)
      File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 336, in get_compiler_abi_compatibility_and_version
        if not check_compiler_ok_for_platform(compiler):
      File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 290, in check_compiler_ok_for_platform
        which = subprocess.check_output(['which', compiler], stderr=subprocess.STDOUT)
      File "/opt/conda/lib/python3.9/subprocess.py", line 424, in check_output
        return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
      File "/opt/conda/lib/python3.9/subprocess.py", line 528, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['which', 'g++']' returned non-zero exit status 1.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-0iijw5gj/xformers_5b1f993f646b4ac9b3108b3ccff7db4e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-nwi2pczz/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.9/xformers Check the logs for full command output.
Note: you may need to restart the kernel to use updated packages.

System Info

[email protected]:/workspace$ diffusers-cli env [!] Not using xformers memory efficient attention.

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

  • diffusers version: 0.7.0.dev0
  • Platform: Linux-4.15.0-189-generic-x86_64-with-glibc2.27
  • Python version: 3.9.12
  • PyTorch version (GPU?): 1.13.0 (True)
  • Huggingface_hub version: 0.10.1
  • Transformers version: 4.23.1
  • Using GPU in script?:
  • Using distributed or parallel set-up in script?:

image

jeannen avatar Oct 31 '22 11:10 jeannen

This is one of the things I did to make it run on SageMaker Studio Lab after getting the same error: %conda install -y -c pytorch -c conda-forge cudatoolkit=11.6 pytorch=1.12.1 %conda install -y xformers -c xformers/label/dev

After commenting out the present xformers lines. It may work on vast.ai too?

Miraculix200 avatar Oct 31 '22 17:10 Miraculix200

This is one of the things I did to make it run on SageMaker Studio Lab after getting the same error: %conda install -y -c pytorch -c conda-forge cudatoolkit=11.6 pytorch=1.12.1 %conda install -y xformers -c xformers/label/dev

After commenting out the present xformers lines. It may work on vast.ai too?

Was able to have this step working thanks to this! Thanks 😄 Now I got this when I run !accelerate launch train_dreambooth.py \

The following values were not passed to `accelerate launch` and had defaults used instead:
	`--num_cpu_threads_per_process` was set to `24` to improve out-of-box performance
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Traceback (most recent call last):
  File "/workspace/train_dreambooth.py", line 814, in <module>
    main(args)
  File "/workspace/train_dreambooth.py", line 529, in main
    import bitsandbytes as bnb
  File "/opt/conda/lib/python3.9/site-packages/bitsandbytes/__init__.py", line 6, in <module>
    from .autograd._functions import (
  File "/opt/conda/lib/python3.9/site-packages/bitsandbytes/autograd/_functions.py", line 5, in <module>
    import bitsandbytes.functional as F
  File "/opt/conda/lib/python3.9/site-packages/bitsandbytes/functional.py", line 13, in <module>
    from .cextension import COMPILED_WITH_CUDA, lib
  File "/opt/conda/lib/python3.9/site-packages/bitsandbytes/cextension.py", line 113, in <module>
    lib = CUDASetup.get_instance().lib
  File "/opt/conda/lib/python3.9/site-packages/bitsandbytes/cextension.py", line 109, in get_instance
    cls._instance.initialize()
  File "/opt/conda/lib/python3.9/site-packages/bitsandbytes/cextension.py", line 59, in initialize
    binary_name, cudart_path, cuda, cc, cuda_version_string = evaluate_cuda_setup()
ValueError: too many values to unpack (expected 5)
Traceback (most recent call last):
  File "/opt/conda/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.9/site-packages/accelerate/commands/accelerate_cli.py", line 43, in main
    args.func(args)
  File "/opt/conda/lib/python3.9/site-packages/accelerate/commands/launch.py", line 837, in launch_command
    simple_launcher(args)
  File "/opt/conda/lib/python3.9/site-packages/accelerate/commands/launch.py", line 354, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/opt/conda/bin/python', 'train_dreambooth.py', '--pretrained_model_name_or_path=runwayml/stable-diffusion-v1-5', '--pretrained_vae_name_or_path=stabilityai/sd-vae-ft-mse', '--output_dir=/content/stable_diffusion_weights/zwx', '--with_prior_preservation', '--prior_loss_weight=1.0', '--seed=1337', '--resolution=512', '--train_batch_size=1', '--train_text_encoder', '--mixed_precision=fp16', '--use_8bit_adam', '--gradient_accumulation_steps=1', '--learning_rate=1e-6', '--lr_scheduler=constant', '--lr_warmup_steps=0', '--num_class_images=50', '--sample_batch_size=4', '--max_train_steps=800', '--save_interval=10000', '--save_sample_prompt=photo of zwx dog', '--concepts_list=concepts_list.json']' returned non-zero exit status 1.

jeannen avatar Oct 31 '22 23:10 jeannen

I did not encounter this error on SageMaker Studio Lab. Here's the .ipynb I use, maybe it works for you too: https://github.com/Miraculix200/diffusers/blob/main/examples/dreambooth/DreamBooth_Stable_Diffusion_SageMakerSL.ipynb

Miraculix200 avatar Nov 01 '22 10:11 Miraculix200

@Miraculix200 thanks for sharing your notebook! Is using 8_bit_adam / bitsandbytes package working for you in Sagemaker?

samching avatar Nov 01 '22 15:11 samching

@samching yes, both worked. I know because at some stage during testing with a fresh environment I removed bitsandbytes from the list of packages to install, and it didn't work. This is SageMaker Studio Lab however, which is different from SageMaker

Miraculix200 avatar Nov 01 '22 15:11 Miraculix200

Thanks a lot for sharing the notebook! It seems to work better than the previous one, but I get this a few minutes after starting the training: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.

Full logs https://pastebin.com/0six3hA4

Any idea how it could be fixed? Didn't found anything on Google yet

jeannen avatar Nov 02 '22 00:11 jeannen

@jeannen I'm not sure if I got that warning. But that's not the error which stops accelerate in your case anyway. It looks like it can't find the instance images in the path provided in concepts_list (instance_data_dir). Did you skip some cells which aren't supposed to be skipped?

Miraculix200 avatar Nov 02 '22 09:11 Miraculix200

@jeannen Sorry, I didn't test the .ipynb after some changes, and I couldn't test it because SageMaker SL didn't work in the past 2 days. You have to remove the preceding / in instance_data_dir and class_data_dir. Instead of "/content/..." you have to use "content/...". That is most likely the cause of your error

Miraculix200 avatar Nov 03 '22 17:11 Miraculix200

Chiming in late but I made a {kind of} universal notebook with improvements for Shivamshrirao and thelastbens notebook... Working on vast and runpod.

https://github.com/SU1199/fastBooth

SU1199 avatar Jan 10 '23 10:01 SU1199