gpt-engineer
gpt-engineer copied to clipboard
gpt-engineer-org
Injecting malicious code
Add a test to check that the AI is not injecting malicious code outside the workspace @AntonOsika [../../src/main.py] 👎
def test_files_are_local():
chat = textwrap.dedent(
"""
All this will soon be over.
[../../src/main.py]
```python
print("Goodbye, World!")
```
"""
)
ChatGPT suggest the following checks (after some massage) @CenekSanzak
import os
import pytest
def is_valid_relative_path(path):
invalid_prefixes = ["..", "/", "~", "-", " "]
return (
not any(path.startswith(prefix) for prefix in invalid_prefixes)
and not os.path.isabs(path)
)
@pytest.mark.parametrize("path, expected_result", [
("path/to/file.txt", True), # Valid relative path
("../file.txt", False), # Invalid: starts with ".."
("/root/file.txt", False), # Invalid: starts with "/"
("~/file.txt", False), # Invalid: starts with "~"
("./file.txt", True), # Valid relative path
("-file.txt", False), # Invalid: starts with "-"
(" file.txt", False), # Invalid: starts with space
])
def test_is_valid_relative_path(path, expected_result):
assert is_valid_relative_path(path) == expected_result
# Additional test case
def test_is_valid_relative_path_empty_string():
assert is_valid_relative_path("") == False
that is slightly concerning, did AI actually write that? Are we going somewhere? - is it? or the planet?
that is slightly concerning, did AI actually write that? Are we going somewhere? - is it? or the planet?
chat gpt can write something like this but its unlikely to happen
ChatGPT suggest the following checks (after some massage) @CenekSanzak
import os import pytest def is_valid_relative_path(path): invalid_prefixes = ["..", "/", "~", "-", " "] return ( not any(path.startswith(prefix) for prefix in invalid_prefixes) and not os.path.isabs(path) ) @pytest.mark.parametrize("path, expected_result", [ ("path/to/file.txt", True), # Valid relative path ("../file.txt", False), # Invalid: starts with ".." ("/root/file.txt", False), # Invalid: starts with "/" ("~/file.txt", False), # Invalid: starts with "~" ("./file.txt", True), # Valid relative path ("-file.txt", False), # Invalid: starts with "-" (" file.txt", False), # Invalid: starts with space ]) def test_is_valid_relative_path(path, expected_result): assert is_valid_relative_path(path) == expected_result # Additional test case def test_is_valid_relative_path_empty_string(): assert is_valid_relative_path("") == False
better to fully resolve using the stdlib, and then assert that the resolved path is a descendant of the current path.
Hey! I got an idea.
Why don't we initialise Git outside the directory of the code which runs the gpt-engineer with the command git init?
After the code is run, we can do git status and then see the changes made.
If anything outside the directory(in which the gpt-engineer is run) is changed, we can declare that something is wrong with the code and it may be malicious.
