Making the code platform-agnostic would lead to more contributions

[jonathanagustin]

Multi-platform support would lead to more contributions. Instead of using Unix-specific methods to handle the filesystem, Python libraries can be used instead. Using a Python library would generally handle most of the OS-specific issues.

I do not have any specific problems, but I recognize that others may not have access to different operating systems. Perhaps they may not be technically proficient enough to use WSL or Docker.

I observed that the example Jupyter notebooks contain OS-specific code. For example, in evals/examples/lafand-mt.ipynb, there is an assumption of a Unix filesystem:

events = f"/tmp/evallogs/{log_name}"

Here, it seems better to use Python's tempfile module to handle temporary files and directories across different platforms.

In evals/examples/mmlu.ipynb, there are these commands:

!curl -O https://people.eecs.berkeley.edu/~hendrycks/data.tar
!tar -xf data.tar

Here, it seems better to use a Python library like urllib to download datasets because it is built into the language and is usable across different operating systems.

[mstooks]

+1

[Tshering-Penjor]

I think it supports all OS and more importantly all versions of windows.

[SinanAkkoyun]

Just install WSL if you're on windows, it should work flawlessly

[Toby-Mascot]

Installing WSL should do the trick

[jonathanagustin]

@Toby-Mascot @SinanAkkoyun

I understand that WSL is a solution. My concern is that WSL may be difficult to install for some. I also recognize that Unix-based OS is also difficult for some to learn as well. The aim is to make it more platform agnostic: Using Python libraries will achieve this aim and facilitate more contributions.

[Toby-Mascot]

Oh, yeah... I totally understand your view now. I hope it's implemented soon

On Thu, Mar 16, 2023, 12:06 PM jonathanagustin @.***> wrote:

@Toby-Mascot https://github.com/Toby-Mascot @SinanAkkoyun https://github.com/SinanAkkoyun

I understand that WSL is a solution. My concern is that WSL may be difficult to install for some. I also recognize that Unix-based OS is also difficult for some to learn as well. The aim is to make it more platform agnostic: Using Python libraries will achieve this aim and facilitate more contributions.

— Reply to this email directly, view it on GitHub https://github.com/openai/evals/issues/209#issuecomment-1471744743, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASXGAFIR7443YSNZGLCAWMLW4LX4LANCNFSM6AAAAAAV4S4BUU . You are receiving this because you were mentioned.Message ID: @.***>

[logankilpatrick]

Most of the dev team at OpenAI uses MacOS, but you are welcome to open a PR to add windows related info!