ZODB
ZODB copied to clipboard
zodb install on termux linux for Android doesn't work due to no os.link
Here is the traceback error resulting from an attempt to import ZODB into any python script running on termux for Android.
import ZODB Traceback (most recent call last) File "data/data/com.termux/files/usr/lib/python3.7/site-packages/ZODB/init.py" line 28, in
from ZODB.DB import DB, connection File "data/data/com.termux/files/usr/lib/python3.7/site-packages/ZODB/init.py" line 27, in from ZODB.Connection import Connection, TransactionMetaData, noop File "data/data/com.termux/files/usr/lib/python3.7/site-packages/ZODB/Connection.py" line 31, in from ZODB.blob import Blob, rename_or_copy_blob, remove_commited_dir File "data/data/com.termux/files/usr/lib/python3.7/site-packages/ZODB/blob.py" line 950, in link_or_copy = os.link AttributeError: module os has no attribute 'link'
While a pip install zodb within the termux distro for Android completes without incident, it fails to run since any attempt to import ZODB within termux's python fails. This is because their python's os.link has been disabled for some cross platform reason related to windows. I have found two issues related to this problem that other apps experience. pip was the first one and it seems that pip was modified to test to see if os.link is available for the platform and substituting another call if it is not
https://github.com/termux/termux-packages/issues/29 https://github.com/pimutils/vdirsyncer/issues/744
Is there a similar fix that can be made available for zodb? I want to be able to use zodb not only within my laptop's scripts but also for these same scripts to run within the termux distro on Android.
@zennobjects The only ZODB library code which uses os.link
is in ZODB.blob
, where it is selected as the value for the link_or_copy
name on Posix systems: on Windows, link_or_copy
is mapped to shutil.copy
. If you can figure out how to detect that you are on termux
, then you can patch ZODB.blob
use shutil.copy
as well in that case.
The migrateblobs
script does a similar dance.
hasattr(os, 'link') might be a reasonable check.
In order to identify that I'm on termux, I use the following code in my scripts. It's based upon identifying certain features from within python - my platform (aka operating system) and - my device (hardware).
Plus one more additional piece of information. Termux modifies the python that their "pkg install python" provides for a number of specific reasons, one being that their file system layout is different (because, because). Since termux is running within Google's android services, the app relies upon a specifc NDK in order to be built. Ergo there is an attribute patched into their python that tells us, we're running on top of Android system #21 as an example. Here is my code:
import` platform, sys, os
# 1. platform_type
try:
NDK = sys.getandroidapilevel()
platform_type = 'android'
except AttributeError as e:
platform_type = platform.system().lower()
# 2. device_type
device_type = 'phone' if platform.machine() == 'armv7l' else 'laptop'
# 3. device_platform
device_platform = '_'.join((device_type, platform_type))
# ---- printout----
print(f" platform_type = {platform_type}, (which is {platform.system().lower()} )")
print(f" device_type = {device_type} (which is {platform.machine()})")
print(f" device_platform = {device_platform}")
So for my purposes, putting zodb's os.link calls between
try:
link_or_copy = os.link
except AttributeError as e:
if platform_type == 'android':
sink_or_copy = shutil.copy
should suffice to run the necessary substitution of the shutil.copy command for os.link.
Looking at the code within blob.py, the change might be something like this:
try:
hasattr(os, link)
os_link = True
except AttributeError as e:
os_link is False
try:
NDK = sys.getandroidapilevel()
platform_type = 'android'
except AttributeError as e:
platform_type = platform.system().lower()
if sys.platform == 'win32':
# On Windows, you can't remove read-only files, so make the
# file writable first.
...
link_or_copy = shutil.copy
elif platform_type == 'android' and os_link is False:
# On Android platforms, the linux distros like termux may have
# removed os.link
...
link_or_copy = shutil.copy
else:
...
link_or_copy = os.link
In order to test the changes, I simply used vim to change blob.py residing in the user python's site-packages/ZODB, and insert the line
link_or_copy = os.link
between a
try:
link_or_copy = os.link
except AttributeError:
link_or_copy = shutil.copy
rather than go through the whole recompiling and wheel creation of zodb for my termux version of linux running on an Android platform. It works like a charm. So I can confirm that there are no other issues preventing zodb from working as a result of this change.
There is no need to identify that zodb is running within termux, that it's on an Android platform, whatever. It's a simple change to the blob.py script that can be easily added to zodb.
Shall I submit a patch?
@zennobjects Please do submit a PR. You can add a #pragma: no cover
to the except AttributeError:
line.
@tseaver @zennobjects Was the issue solved? I don't see a linked PR.
Wow, a long time ago. Seems like I didn’t create the pull requests. I can’t even find my clone of this app where I changed the line within the blob.py file.
What I can say is this. The solution that I wrote, the one where the value for link_or_copy gets changed in the try:except clauses, works. The quickest solution is to use an editor and change that line in the blob.py file under ZODB within the termux ZODB.
I just submitted a pull request for my proposed changes.
There are two test failures, 1 failure for Ubuntu.lint and another for each python version on Windows.
[1] I think the failure for Ubuntu.lint is due to a lint check wrt the use of '#'. I've made minor changes to those comments.
[2] I am somewhat at a loss to see why the following error occurs for each python version on windows.
ImportError: cannot import name 'link_or_copy' from 'ZODB.blob' (D:\a\ZODB\ZODB\src\ZODB\blob.py)
It could be that the following check is no longer valid, possibly due to the check for sys.platform == win32 failing
if sys.platform == 'win32': ... link_or_copy = shutil.copy
in which case link_or_copy is never set. If this is the case, then the fix that I proposed is not what breaks the code for Windows Nevertheless, the following might fix this
It could be that we should remove the setting of link_or_copy completely from within the 'if' block.
if sys.platform == 'win32' ....
and instead have a stand alone setting of link_or_copy which is at the same level as the various functions within blob.py Ie:
def rename_or_copy_blob(f1, f2, chmod=True): """Try to rename f1 to f2, fallback to copy. ... ...
try: link_or_copy = os.link except AttributeError: # pragma: no cover # FBO termux on Android. # See https://github.com/zopefoundation/ZODB/issues/257 link_or_copy = shutil.copy
I'm resubmitting the pull request with these two changes.
Man, it doesn't pay to be fixing code just after surgery. The littlest changes result in errors due to the discomfort. Hopefully, the link errors are fixed. Plus there should not be any more failures on Windows given the latest indentation change.