evals
evals copied to clipboard
openai
FileNotFoundError
After I run "oaieval gpt-3.5-turbo mafand_translation_en-ibo --max_samples 20" from the Lambada example I get "FileNotFoundError: [Errno 2] No such file or directory: '/tmp/evallogs/None'". I am editing the log_path to the name of the .jsonl file like this: "log_path = "230314194647JSXQXY4S_gpt-3.5-turbo_lambada.jsonl"". Any suggestions for locating this file after the events are logged?
Based on the error message, it seems that the code is looking for a file or directory that doesn't exist. Specifically, it's looking for a directory called "None" inside the "/tmp/evallogs" directory.
To fix this error, you may need to check the following:
Check that the directory "/tmp/evallogs" exists: You can do this by navigating to the directory in the terminal or file explorer.
Check the value of the "log_path" variable: It should be set to the correct path of the log file you want to use. Double-check that the file exists at that location.
Check the permissions of the "/tmp/evallogs" directory: Ensure that the user running the code has the necessary permissions to access and modify the directory.
Check if there are any typos or syntax errors in the code that could be causing the issue. Double-check the code line by line to identify any errors.
Once you've identified and fixed the issue, you can run the code again to see if the error has been resolved.
To fix the issue follow these steps:
Ensure the "/tmp/evallogs" directory exists. If it doesn't exist, create it: bash Copy code mkdir -p /tmp/evallogs
Check the evaluation script to see where the log_path is being set. It appears that the log_path is set to None or not set at all, causing the FileNotFoundError.
Update the log_path in the evaluation script to point to a valid file path inside the "/tmp/evallogs" directory. For example:
python Copy code log_path = "/tmp/evallogs/230314194647JSXQXY4S_gpt-3.5-turbo_lambada.jsonl"
Run the evaluation script. The script should now save the logs to the specified path ("/tmp/evallogs").
After the evaluation is complete, navigate to the "/tmp/evallogs" directory to locate the generated log file.
By following these steps, you should be able to address the FileNotFoundError issue and ensure that the evaluation logs are saved in the correct directory.
Make sure that
The directory "/tmp/evallogs" exists, and you can access it. The "log_path" variable in your evaluation script is set to the correct path of the log file. The user running the code has the necessary permissions to access and modify the "/tmp/evallogs" directory. There are no typos or syntax errors in the code that could be causing the issue. By following these steps, you should be able to identify and resolve any issues related to the FileNotFoundError. If you still encounter problems or need further assistance, please feel free to ask!
I observe this:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/evallogs/None'
Notice that it says: /tmp/evallogs/None
A likely issue is that a variable is not initialized.
Observe the below in evals/examples/lambada.ipynb
log_path = None # Set to jsonl path to logs from oaieval
events = f"/tmp/evallogs/{log_path}"
Here, you may need to initialize log_path
Observe the cell above it:
!oaieval gpt-3.5-turbo mafand_translation_en-ibo --max_samples 20
When this cell executes, it creates a file in /tmp/evallogs
You can find out the file path and filename in the console when it executes.
