"Saving As" is problematic

[LucasSchelkes-BA]

Thank you for this amazing app!! ❤️

After reading through some other issues on the topic of saving notebooks, here are some oddities I've observed:

Opening the app for the first time brings the user to the "Carnets plus - history" file list. From there, the user intuitively clicks the "+" button at the top to create a new notebook. Saving this notebook with Menu->File->SaveAs gives a path "private/var/mobile/Library/Mobile Documents/iCloud~AsheKube~Carnets/Documents/", to which a name e.g. "test.ipynb" can be appended. This results in a Dead Kernel warning. After trying to restart the kernel, it's a mystery as to where the notebook is actually saved. When I search for it using the file manager all, I can't seem to find it "On my iPhone" or on "iCloud Drive". Also, it doesn't show up in the "Carnets plus - history" file list. When repeating this process with the same file name however, it says that the file already exists and asks whether it should overwrite it. So, there are two oddities here: where is the file saved and why is there always a dead kernel message when trying to "save as"?

The other option, when opening the app is to click on the "Browse" tab at the bottom and then either selecting "On my iPhone" or "iCloud Drive". In both cases there are 2 ways to open a new notebook: the "create document" button or the "+" button at the top, both seem to work interchangeably.

In the first case of the "iCloud Drive" the message appears "Carnets-sci does not have access to the directory around this notebook". However, the button to grant read and write access through the "unlock directory button" is not visible, which means that saving is not possible as it would result in an error. At least an "Untitled.ipynb" is created in the iCloud-Drive path and visible via the file manager, and it can be renamed from there, into e.g. "test2.ipynb". Saving any content seems to work, however when trying to save the notebook with "Save as" gives the relative path "private/var/mobile/Library/Mobile Documents/com~apple~CloudDocs/", but appending "test2.ipynb" to the end of it results in the error that writing is not permitted:

Traceback (most recent call last): File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/tornado/web.py", line 1713, in _execute result = await result ^^^^^^^^^^^^ File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/tornado/gen.py", line 776, in run yielded = self.gen.throw(*exc_info) # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/notebook/services/contents/handlers.py", line 240, in put yield maybe_future(self._upload(model, path)) File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/tornado/gen.py", line 769, in run value = future.result() ^^^^^^^^^^^^^^^ File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/tornado/gen.py", line 234, in wrapper yielded = ctx_run(next, result) ^^^^^^^^^^^^^^^^^^^^^ File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/notebook/services/contents/handlers.py", line 152, in _upload model = yield maybe_future(self.contents_manager.new(model, path)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/notebook/services/contents/manager.py", line 421, in new model = self.save(model, path) ^^^^^^^^^^^^^^^^^^^^^^ File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/notebook/services/contents/largefilemanager.py", line 44, in save return super().save(model, path) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/notebook/services/contents/filemanager.py", line 485, in save if is_hidden(os_path, self.root_dir) and not self.allow_hidden: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/notebook/utils.py", line 194, in is_hidden if is_file_hidden(abs_path): ^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/var/containers/Bundle/Application/323631AF-23B8-45A1-BCAE-3A301BAD4148/Carnets-sci.app/Library/lib/python3.11/site-packages/notebook/utils.py", line 148, in is_file_hidden_posix stat_res = os.stat(abs_path) ^^^^^^^^^^^^^^^^^ PermissionError: [Errno 1] Operation not permitted: '/private/var/mobile/Library/Mobile Documents/com~apple~CloudDocs/test2.ipynb'

Where is the button to grant access here? And why does it seem to work when simply saving the notebook and renaming it via the file manager?

In the second case of the "On my iPhone" option, when creating a new notebook the same read and write access warning appears as described above, with the same results, as the unlock button is not findable. However, when selecting the "Carnets plus" directory (btw strangely it says it contains 5 objects where there are just 2 notebook files in it.. are there maybe hidden objects?!) and then creating a new notebook inside of it, the "Save as" now gives the path "private/var/mobile/Containers/Data/Application/699C5A6D-5425-44ED-8FAE-6E17830ED75A/Documents/" and appending "test3.ipynb" to it let's the kernel die once again, but at least the file is written into the path without write access problems. So here the problem is the dying kernel. This would be my preferable way of working in notebooks (Is this also your recommendation of best-practice @holzschu?) because saving seems to work and the files are locatable. I can live with the dying kernel, even though it's annoying. Btw, the kernel also dies when selecting Menu->File->Rename. But this is not really a huge problem, as simply tapping the save button works flawlessly and does not let the kernel die - and this action is done much more often than "saving as".

Of course there are other options of "saving as" (e.g. Dropbox), but I didn't investigate these.

I'm curious if this helps anyone spot bugs or find a good and stable way to work. These are just my observations of using this fantastic app as a newbie. Thanks again for this great contribution @holzschu !!! 🚀

[LucasSchelkes-BA]

Device: iPhone 11, iOS 16.5 Latest App Version of Carnets plus: 1.7.9

[holzschu]

That's a lot of information, and I'm not sure I understand everything, so I'll start with the easy answers:

• the unlock directory button is in the top right corner, it has an icon with a folder and an unlocked lock, and it should always be visible.
• the part you see before you create a notebook or open one are managed by the system, for security reasons. Carnets does not know where you are, or what you are seeing, or what you are doing, until you create or open a notebook. That is good (in terms of privacy) but it has some drawbacks.
• for example, I would love for new users to not start with "Recents" and rather start with "On my iPad", but that is not an option available (that I know of, at least). "Recents" is a terrible idea in terms of user experience, as you describe.
• similarly, you can create a notebook at the "iCloud Drive" top level, but that is a bad idea (folder permissions are likely to be problematic, as you found out). The system expects you to place files inside one of the folders below "iCloud Drive" (I'm actually surprised that it let you create the file). Same with the top level of "On my iPhone".
• Jupyter creates configuration files, stored in hidden folders in the main directory, hence the fact that this directory has 5 objects by 2 notebooks (theres is .jupyter, .ipython and .ipynb_checkpoints, I think) (anything that begins with a "." is hidden in the Unix world, and iOS is a Unix under the hood).
• I'll check the "Rename" part (the kernel should not die, I agree with you).

[LucasSchelkes-BA]

Thanks for your quick reply and thorough thoughts @holzschu ! 🙌 that really cleared things up and helped me find my way around!

• Okay, I found the "unlock access" button as you described and learned how to use it, thanks!

• Right. So I will use the "on my iPhone" drive to store the files.

• Yes, would be nice if the dying kernel is fixed at "saving as" or "rename"

• 👍 to all the other points :)

[holzschu]

You use "iCloud Drive", but it's better if you go down one level (inside "Documents", or inside "Carnets", or basically anywhere except the top level). Same with "On my iPhone": it's better if you go down one level (inside "Carnets").

Another key point is that we only support file provides that use the standard API (so Dropbox, Google Drive, etc. won't work, but iCloud Drive, Working Copy and ShellFish App will work).

[LucasSchelkes-BA]

Thank you!

Everything seems clarified in my view. Only the scenarios when the kernel dies is still an open question - how shall we proceed? Do you want me to close this issue and open a new one, or leave it open for the time being, WDYT?

[holzschu]

We can leave the issue open for now; many users have similar issues, and they might appreciate the input.

For the "Rename" issue, I'm in the process of updating all packages to the latest version; it's a tedious task, which requires concentration and should take at least 2-3 weeks. I'll get back to the Rename issue after that (with a possibility that it has been fixed as a side-effect of updating packages).

[LucasSchelkes-BA]

Great, thank you! Bless you for your great work 🙏

[holzschu]

Hi, I've been testing "Rename" with the latest TestFlight version of Carnets (https://testflight.apple.com/join/yevwlUs1), and it seems to work (the kernel does not die). If you could test it on your end and check whether the problem is resolved?

[LucasSchelkes-BA]

Sorry for my late reply @holzschu ! Thanks for working on this, I will test it and will report back 👍

[LucasSchelkes-BA]

The "rename" issue seems to be solved in the test app! 🙌🥳 There is still a warning message when renaming from Test.ipynb into Test2.ipynb and then going "back" from the notebook into the file structure (see below) but at least the kernel doesn't die!

[LucasSchelkes-BA]

I think though that this may not be a problem actually, so I guess this issue can be solved?

[holzschu]

I experienced the same issue. On one hand, it's annoying when you try to go back. On the other, it makes sense, from the computer point of view: the file does not exist anymore. Ideally, I'd like to remove the former file from the list of pages in the history, but I don't see how to do it.

[LucasSchelkes-BA]

Agree, from computer view it makes sense. Perhaps one day a revelation will come how to solve it, but in my opinion the initial bug is fixed, so great work @holzschu 🚀

[holzschu]

I think I'm going to add a "fast back" button, that takes you straight back to the file browser instead of going back page by page. There are many cases where I've wanted this button, especially when tutorials link into each other and you want to switch to another tutorial or folder.

[LucasSchelkes-BA]

Alright, so I'll leave this issue open for you to close at your disposal