Run CI on Windows?

[julien-c]

[LysandreJik]

Yes!

[LysandreJik]

There's a WIP branch here: https://github.com/huggingface/huggingface_hub/tree/ci_windows-tests

[mayankjobanputra]

Can someone help me figure out how I should download a model to "tmp" folder on windows using snapshot_download so that I don't have to clean it up later?

I am getting the following error.

tmp_path = WindowsPath('C:/Users/runneradmin/AppData/Local/Temp/pytest-of-runneradmin/pytest-0/test_farm_reader_load_hf_local0')

@pytest.mark.integration
def test_farm_reader_load_hf_local(tmp_path):
    # Test Case: 2. HuggingFace downloaded (local load)

    hf_model = "hf-internal-testing/tiny-random-RobertaForQuestionAnswering"
    local_model_path = str(Path.joinpath(tmp_path, "locally_saved_hf"))

  model_path = snapshot_download(repo_id=hf_model, revision="main", cache_dir=local_model_path)

test\nodes\test_reader.py:290:

---

c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\huggingface_hub\utils_validators.py:120: in _inner_fn return fn(*args, **kwargs) c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\huggingface_hub_snapshot_download.py:235: in snapshot_download thread_map( c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\tqdm\contrib\concurrent.py:69: in thread_map return _executor_map(ThreadPoolExecutor, fn, *iterables, **tqdm_kwargs) c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\tqdm\contrib\concurrent.py:51: in _executor_map return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs)) c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\tqdm\std.py:1178: in iter for obj in iterable: c:\hostedtoolcache\windows\python\3.8.10\x64\lib\concurrent\futures_base.py:619: in result_iterator yield fs.pop().result() c:\hostedtoolcache\windows\python\3.8.10\x64\lib\concurrent\futures_base.py:437: in result return self.__get_result() c:\hostedtoolcache\windows\python\3.8.10\x64\lib\concurrent\futures_base.py:389: in __get_result raise self._exception c:\hostedtoolcache\windows\python\3.8.10\x64\lib\concurrent\futures\thread.py:57: in run result = self.fn(*self.args, **self.kwargs) c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\huggingface_hub_snapshot_download.py:212: in _inner_hf_hub_download return hf_hub_download( c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\huggingface_hub\utils_validators.py:120: in _inner_fn return fn(*args, **kwargs) c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\huggingface_hub\file_download.py:1316: in hf_hub_download _create_relative_symlink(blob_path, pointer_path, new_blob=True) c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\huggingface_hub\file_download.py:850: in _create_relative_symlink os.path.dirname(os.path.commonpath([os.path.realpath(src), os.path.realpath(dst)]))

---

paths = ('\\?\C:\Users\runneradmin\AppData\Local\Temp\pytest-of-runneradmin\pytest-0\test_farm_reader_load_hf_local...ting--tiny-random-RobertaForQuestionAnswering\snapshots\eea4a81306891746bac9e7715f805a2d9dbf4be7\pytorch_model.bin')

def commonpath(paths):
    """Given a sequence of path names, returns the longest common sub-path."""

    if not paths:
        raise ValueError('commonpath() arg is an empty sequence')

    paths = tuple(map(os.fspath, paths))
    if isinstance(paths[0], bytes):
        sep = b'\\'
        altsep = b'/'
        curdir = b'.'
    else:
        sep = '\\'
        altsep = '/'
        curdir = '.'

    try:
        drivesplits = [splitdrive(p.replace(altsep, sep).lower()) for p in paths]
        split_paths = [p.split(sep) for d, p in drivesplits]

        try:
            isabs, = set(p[:1] == sep for d, p in drivesplits)
        except ValueError:
            raise ValueError("Can't mix absolute and relative paths") from None

        # Check that all drive letters or UNC paths match. The check is made only
        # now otherwise type errors for mixing strings and bytes would not be
        # caught.
        if len(set(d for d, p in drivesplits)) != 1:

          raise ValueError("Paths don't have the same drive")

E ValueError: Paths don't have the same drive

[Wauplin]

Hi @mayankjobanputra , thanks for reporting. Could you create a new issue to describe your problem please? I'll answer to it/help you debug that once it's created. Because this is an old issue that should be closed as we already have a CI running on Windows. Thanks in advance!

[Wauplin]

Closing initial issue (done as part of https://github.com/huggingface/huggingface_hub/pull/1112)