Install issue on ARM64 / embedded GPU

[rebotnix]

Have same installation issue on a ARM64 with NVIDIA GPU, 16 GB or 64 GB, CUDA 11.2.

pip install --verbose --trusted-host fair-package-repo.s3-website-us-east-1.amazonaws.com --extra-index-url http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118 fairseq2 --verbose Using pip 23.2.1 from /opt/ssd700gb/venv/lib/python3.8/site-packages/pip (python 3.8) Non-user install because user site-packages disabled Created temporary directory: /tmp/pip-build-tracker-1winxm5u Initialized build tracking at /tmp/pip-build-tracker-1winxm5u Created build tracker: /tmp/pip-build-tracker-1winxm5u Entered build tracker: /tmp/pip-build-tracker-1winxm5u Created temporary directory: /tmp/pip-install-0uco4p7o Created temporary directory: /tmp/pip-ephem-wheel-cache-t8of0zzg Looking in indexes: https://pypi.org/simple, http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118 2 location(s) to search for versions of fairseq2:

• https://pypi.org/simple/fairseq2/
• http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2/ Fetching project page and analyzing links: https://pypi.org/simple/fairseq2/ Getting page https://pypi.org/simple/fairseq2/ Found index url https://pypi.org/simple/ Looking up "https://pypi.org/simple/fairseq2/" in the cache Request header has "max_age" as 0, cache bypassed Starting new HTTPS connection (1): pypi.org:443 https://pypi.org:443 "GET /simple/fairseq2/ HTTP/1.1" 304 0 Fetched page https://pypi.org/simple/fairseq2/ as application/vnd.pypi.simple.v1+json Found link https://files.pythonhosted.org/packages/cd/27/46c14e28e8cb0aa602660ce64d4547a37f460d382e4fcf94f2a53d47e5b0/fairseq2-0.1.0-py3-none-any.whl (from https://pypi.org/simple/fairseq2/) (requires-python:>=3.8), version: 0.1.0 Fetching project page and analyzing links: http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2/ Getting page http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2/ Found index url http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/ Looking up "http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2/" in the cache Request header has "max_age" as 0, cache bypassed Starting new HTTP connection (1): fair-package-repo.s3-website-us-east-1.amazonaws.com:80 http://fair-package-repo.s3-website-us-east-1.amazonaws.com:80 "GET /fairseq2/whl/stable/pt2.0.1/cu118/fairseq2/ HTTP/1.1" 404 388 Status code 404 not in (200, 203, 300, 301, 308) Could not fetch URL http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2/: 404 Client Error: Not Found for url: http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2/ - skipping Skipping link: not a file: https://pypi.org/simple/fairseq2/ Skipping link: not a file: http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2/ Given no hashes to check 1 links for project 'fairseq2': discarding no candidates Collecting fairseq2 Obtaining dependency information for fairseq2 from https://files.pythonhosted.org/packages/cd/27/46c14e28e8cb0aa602660ce64d4547a37f460d382e4fcf94f2a53d47e5b0/fairseq2-0.1.0-py3-none-any.whl.metadata Created temporary directory: /tmp/pip-unpack-d19v50cb Looking up "https://files.pythonhosted.org/packages/cd/27/46c14e28e8cb0aa602660ce64d4547a37f460d382e4fcf94f2a53d47e5b0/fairseq2-0.1.0-py3-none-any.whl.metadata" in the cache Current age based on date: 512 Ignoring unknown cache-control directive: immutable Freshness lifetime from max-age: 365000000 The response is "fresh", returning cached response 365000000 > 512 Using cached fairseq2-0.1.0-py3-none-any.whl.metadata (1.2 kB) 2 location(s) to search for versions of fairseq2n:
• https://pypi.org/simple/fairseq2n/
• http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2n/ Fetching project page and analyzing links: https://pypi.org/simple/fairseq2n/ Getting page https://pypi.org/simple/fairseq2n/ Found index url https://pypi.org/simple/ Looking up "https://pypi.org/simple/fairseq2n/" in the cache Request header has "max_age" as 0, cache bypassed https://pypi.org:443 "GET /simple/fairseq2n/ HTTP/1.1" 304 0 Fetched page https://pypi.org/simple/fairseq2n/ as application/vnd.pypi.simple.v1+json Skipping link: none of the wheel's tags (cp310-cp310-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/21/e8/14d332f4b412578dca7631453704ad76ff2046b29f4bcddc48ca6dbea4f7/fairseq2n-0.1.0-cp310-cp310-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/fairseq2n/) (requires-python:>=3.8) Skipping link: none of the wheel's tags (cp311-cp311-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/c2/31/22f4c8d8b09b8eca1f27a0239f7051bbe3916a4b1cae215c460fd0b13bd4/fairseq2n-0.1.0-cp311-cp311-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/fairseq2n/) (requires-python:>=3.8) Skipping link: none of the wheel's tags (cp38-cp38-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/e5/25/e0b3775ddf8fbfe678c09c8b212ddb3e43f2996ef95a1121fbc183865156/fairseq2n-0.1.0-cp38-cp38-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/fairseq2n/) (requires-python:>=3.8) Skipping link: none of the wheel's tags (cp39-cp39-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/a8/df/dd0d2a4aec37e2bb1fb0b44c30a107b34bbc773fbf2c254757bcb5a75aea/fairseq2n-0.1.0-cp39-cp39-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/fairseq2n/) (requires-python:>=3.8) Fetching project page and analyzing links: http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2n/ Getting page http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2n/ Found index url http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/ Looking up "http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2n/" in the cache Request header has "max_age" as 0, cache bypassed http://fair-package-repo.s3-website-us-east-1.amazonaws.com:80 "GET /fairseq2/whl/stable/pt2.0.1/cu118/fairseq2n/ HTTP/1.1" 404 389 Status code 404 not in (200, 203, 300, 301, 308) Could not fetch URL http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2n/: 404 Client Error: Not Found for url: http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2n/ - skipping Skipping link: not a file: https://pypi.org/simple/fairseq2n/ Skipping link: not a file: http://fair-package-repo.s3-website-us-east-1.amazonaws.com/fairseq2/whl/stable/pt2.0.1/cu118/fairseq2n/ Given no hashes to check 0 links for project 'fairseq2n': discarding no candidates INFO: pip is looking at multiple versions of fairseq2 to determine which version is compatible with other requirements. This could take a while. Will try a different candidate, due to conflict: fairseq2 0.1.0 depends on fairseq2n==0.1.0 ERROR: Could not find a version that satisfies the requirement fairseq2n==0.1.0 (from fairseq2) (from versions: none) ERROR: No matching distribution found for fairseq2n==0.1.0 Exception information: Traceback (most recent call last): File "/opt/ssd700gb/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve result = self._result = resolver.resolve( File "/opt/ssd700gb/venv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve state = resolution.resolve(requirements, max_rounds=max_rounds) File "/opt/ssd700gb/venv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 439, in resolve raise ResolutionImpossible(self.state.backtrack_causes) pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('fairseq2n==0.1.0'), parent=LinkCandidate('https://files.pythonhosted.org/packages/cd/27/46c14e28e8cb0aa602660ce64d4547a37f460d382e4fcf94f2a53d47e5b0/fairseq2-0.1.0-py3-none-any.whl (from https://pypi.org/simple/fairseq2/) (requires-python:>=3.8)'))]

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/ssd700gb/venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper status = run_func(*args) File "/opt/ssd700gb/venv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper return func(self, options, args) File "/opt/ssd700gb/venv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 377, in run requirement_set = resolver.resolve( File "/opt/ssd700gb/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 101, in resolve raise error from e pip._internal.exceptions.DistributionNotFound: No matching distribution found for fairseq2n==0.1.0 Remote version of pip: 23.2.1 Local version of pip: 23.2.1 Was pip installed by pip? True Removed build tracker: '/tmp/pip-build-tracker-1winxm5u'

[astrowar]

I was able to compile for arm aarch64, with nvidia GPU (an jetson orin nano) These errors appear from the repository as a whole. The only detail that seemed to be specific to the ARM implementation was the TBB intel that needed to be tweaked in the source code. still it was not possible to run due to lack of more yaml files that are nowhere in the repository.

[cbalioglu]

@rebotnix @astrowar we just released v0.1.1 that addresses some of the issues you have been experiencing (e.g. TBB is only installed on x86-64). I also significantly revised the installation instructions in fairseq2's README. Please let me know if you have any feedback. Thanks!