Support azureml-pipeline-steps on M1 Mac

[davystrong]

• Package Name: azureml-pipeline-steps
• Package Version: 1.39.0
• Operating System: macOS Monterey 12.3
• Python Version: 3.8.12

Describe the bug I can't install azureml-pipeline-steps on an M1 Mac because some dependencies don't support ARM. I have found four problem dependencies:

• pyarrow (azureml-pipeline-steps uses an old version which doesn't have ARM support. New versions do).
• azureml-dataprep-native, azureml-dataprep-rslex and dotnetcore2, all of which are distributed as binary wheels and don't support ARM.

To Reproduce Steps to reproduce the behavior:

1. Run pip install azureml-pipeline-steps

Expected behavior It should install correctly.

[SaurabhSharma-MSFT]

@davystrong Thanks for reporting this issue. We are redirecting it to services team to look into it.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

Issue Details

---

• Package Name: azureml-pipeline-steps
• Package Version: 1.39.0
• Operating System: macOS Monterey 12.3
• Python Version: 3.8.12

Describe the bug I can't install azureml-pipeline-steps on an M1 Mac because some dependencies don't support ARM. I have found four problem dependencies:

• pyarrow (azureml-pipeline-steps uses an old version which doesn't have ARM support. New versions do).
• azureml-dataprep-native, azureml-dataprep-rslex and dotnetcore2, all of which are distributed as binary wheels and don't support ARM.

To Reproduce Steps to reproduce the behavior:

1. Run pip install azureml-pipeline-steps

Expected behavior It should install correctly.

Author:	davystrong
Assignees:	-
Labels:	question, Machine Learning, Service Attention, Client, customer-reported, needs-team-attention
Milestone:	-

[nagydavid]

HI @SaurabhSharma-MSFT, @bandsina May I ask, is there any update regarding the support of M1 ?

[lnkirkham-datasparq]

Also have the same issue. Any update on this?

[nagydavid]

Hi @SaurabhSharma-MSFT ,@xiangyan99, @bandsina Can we have an ETA or options for possible solutions.

Thanks in advance

[xiangyan99]

@azureml-github

[davystrong]

It's almost 5 months since I opened this issue, would love an update. Is this anywhere near being fixed?

[lloydhamilton]

Install x86_64 python packages on M1 MacBook Apple Silicon

Found a temporary workaround that will allow M1 MacBook Pros to install x86 binaries.

The steps here will install x86_64 binarys of homebrew, poetry and python. I am using pyenv and poetry to manage python environments and dependencies respectively.

Configure new rosetta terminal

1. Duplicate a new terminal by going to applications>utilities>Terminal.app
2. Rename the duplicated terminal app to Rosetta Terminal.
3. Right click on Rosetta Terminal and select Get Info.
4. Tick the box Open using Rosetta

Install x86 dependencies

1. First of all to run x86 executables, we'll need to install [Rosetta]. Open a new Rosetta Terminal:

softwareupdate --install-rosetta --agree-to-license

2. Install x86 Home-brew:

arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

This will install homebrew at /usr/local/Homebrew/bin/brew

3. Add this short script in ~./zshrc:

archcheck () {
   if [ "$(uname -p)" = "i386" ]
   then
     echo "Running in i386 mode (Rosetta)"
     eval "$(/usr/local/Homebrew/bin/brew shellenv)"
     alias brew='/usr/local/Homebrew/bin/brew'
   elif [ "$(uname -p)" = "arm" ]
   then
     echo "Running in ARM mode (M1)"
     eval "$(/opt/homebrew/bin/brew shellenv)"
     alias brew='/opt/homebrew/bin/brew'
   else
     echo "Unknown architecture detected"
   fi
}
eval "archcheck"

This script will evaluate on start up of each terminal session and set the appropriate homebrew path.

4. Install pyenv:

brew install pyenv

5. Add pyenv configs in ~/.zshrc

export PYENV_ROOT="$HOME/.pyenv"
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(pyenv init -)"' >> ~/.zshrc

6. Restart the shell:

exec "$SHELL"

7. Install python build dependencies and poetry:

brew install openssl readline sqlite3 xz zlib tcl-tk poetry

8. Add poetry alias to ~/.zshrc:

echo 'alias poetry="/usr/local/Cellar/poetry/1.2.0_1/libexec/bin/poetry"' >> ~/.zshrc

9. Restart shell:

exec "$SHELL"

10. Install python 3.8.13 using pyenv:

pyenv install 3.8.13

[WiktorMadejski]

+1

Developing Azure ML from M1 requires additional set up, both for local and remote development. Makes it harder to advocate for Azure ML to clients working with M1 local workspaces.

[WiktorMadejski]

Install x86_64 python packages on M1 MacBook Apple Silicon

Found a temporary workaround that will allow M1 MacBook Pros to install x86 binaries.

The steps here will install x86_64 binarys of homebrew, poetry and python. I am using pyenv and poetry to manage python environments and dependencies respectively.

Configure new rosetta terminal

1. Duplicate a new terminal by going to applications>utilities>Terminal.app
2. Rename the duplicated terminal app to Rosetta Terminal.
3. Right click on Rosetta Terminal and select Get Info.
4. Tick the box Open using Rosetta

Install x86 dependencies

1. First of all to run x86 executables, we'll need to install [Rosetta]. Open a new Rosetta Terminal:

softwareupdate --install-rosetta --agree-to-license

2. Install x86 Home-brew:

arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

This will install homebrew at /usr/local/Homebrew/bin/brew

3. Add this short script in ~./zshrc:

archcheck () {
   if [ "$(uname -p)" = "i386" ]
   then
     echo "Running in i386 mode (Rosetta)"
     eval "$(/usr/local/Homebrew/bin/brew shellenv)"
     alias brew='/usr/local/Homebrew/bin/brew'
   elif [ "$(uname -p)" = "arm" ]
   then
     echo "Running in ARM mode (M1)"
     eval "$(/opt/homebrew/bin/brew shellenv)"
     alias brew='/opt/homebrew/bin/brew'
   else
     echo "Unknown architecture detected"
   fi
}
eval "archcheck"

This script will evaluate on start up of each terminal session and set the appropriate homebrew path.

4. Install pyenv:

brew install pyenv

5. Add pyenv configs in ~/.zshrc

export PYENV_ROOT="$HOME/.pyenv"
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(pyenv init -)"' >> ~/.zshrc

6. Restart the shell:

exec "$SHELL"

7. Install python build dependencies and poetry:

brew install openssl readline sqlite3 xz zlib tcl-tk poetry

8. Add poetry alias to ~/.zshrc:

echo 'alias poetry="/usr/local/Cellar/poetry/1.2.0_1/libexec/bin/poetry"' >> ~/.zshrc

9. Restart shell:

exec "$SHELL"

10. Install python 3.8.13 using pyenv:

pyenv install 3.8.13

@lloydhamilton works for me, thanks for describing it so nicely.

My notes:

• ) pyenv/poetry are not a must - key points are

1. Install -x86_64 brew using Rosetta Terminal
2. Use same brew to instal build dependencies
3. When python installs dependency make sure correct build dependencies are used

-) Keep in mind the executions will be very slow.

[luigiw]

v1 SDK is in maintenance only mode in favor of the v2 SDK being GA. Please file an issue for v2 SDK if your requests stand for v2. https://learn.microsoft.com/en-us/azure/machine-learning/how-to-migrate-from-v1

[WiktorMadejski]

Folks you can also use conda:

1. Instal miniconda on your MacOS with Apple Silicon

2. create conda environment: CONDA_SUBDIR=osx-64 conda create -p conda python=3.8.13 -y conda activate conda config --env --set subdir osx-64

3. Enjoy Azure ML Python SDK v1