clearing the workspace after completed session.

[gdssingh7]

trafficstars

I would like to understand if we can flush the workspace after a session. if I deploy this application now. the workspace keep on filling the memory. Is there a way we can empty the workspace periodically. Please let me know how can i do it. Thanks

[liqul]

@gdssingh7 Hi, could you clarify a bit on the 'workspace' in your question? is it the session data stored on disk under the project/workspace/ folder or you mean the memory spent for each new session?

[gdssingh7]

yeah @liqul it's about the session data stored in Disk.

[twtester2]

@gdssingh7

The files are located in directories inside project/workspace/sessions.

When TaskWeaver is not running, I was able to delete them with the following script:

session_dir = "project/workspace/sessions"

    for filename in os.listdir(session_dir):
        file_path = os.path.join(session_dir, filename)
        try:
            if os.path.isfile(file_path) or os.path.islink(file_path):
                os.unlink(file_path)
            elif os.path.isdir(file_path):
                shutil.rmtree(file_path)
        except Exception as e:
            print('Failed to delete %s. Reason: %s' % (file_path, e))

However, when TaskWeaver is running and the session is created, the files are unable to be deleted because the kernel continues to run while logging remains in usage. I found that making the following change to lines 74-79 of taskweaver/code_interpreter/code_executor.py will end the kernels:

result = self.exec_client.execute_code(exec_id, code)

if result.is_success:
    self.stop()

    for artifact in result.artifact:

which makes it possible to delete the session files using the above code. These changes don't appear to break TaskWeaver, so it seems to accomplish clearing the workspace.

@liqul, do you think it makes sense to add "self.stop()" above to code_executor.py?

Thanks, --TWT

[liqul]

@gdssingh7

The files are located in directories inside project/workspace/sessions.

When TaskWeaver is not running, I was able to delete them with the following script:

session_dir = "project/workspace/sessions"

    for filename in os.listdir(session_dir):
        file_path = os.path.join(session_dir, filename)
        try:
            if os.path.isfile(file_path) or os.path.islink(file_path):
                os.unlink(file_path)
            elif os.path.isdir(file_path):
                shutil.rmtree(file_path)
        except Exception as e:
            print('Failed to delete %s. Reason: %s' % (file_path, e))

However, when TaskWeaver is running and the session is created, the files are unable to be deleted because the kernel continues to run while logging remains in usage. I found that making the following change to lines 74-79 of taskweaver/code_interpreter/code_executor.py will end the kernels:

result = self.exec_client.execute_code(exec_id, code)

if result.is_success:
    self.stop()

    for artifact in result.artifact:

which makes it possible to delete the session files using the above code. These changes don't appear to break TaskWeaver, so it seems to accomplish clearing the workspace.

@liqul, do you think it makes sense to add "self.stop()" above to code_executor.py?

Thanks, --TWT

Thanks for providing a solution of deleting the files. However, I didn't quite understand the purpose of deleting a session folder of a running session.

If you don't need to review the session files, adding the deletion logic inside TaskWeaverApp.stop_session would be a better choice.