drive
drive copied to clipboard
odeke-em
help with terminal encoding problems on Linux
Hi,
When push an folder which has accents character on it's name all of it's contents is transferred, I can see it on Google Drive but drive can not list it. Please see examples bellow.
Local folder name: CONSTRUÇÃO ( Upper case accents characters cause error ) Local folder name: C___ção ( Lower case accents characters are OK ! )
drive push -verbose -ignore-name-clashes -destination TST_ACCENT
Resolving...
+ /TST_ACCENT/CONSTRUÇÃO
+ /TST_ACCENT/CONSTRUÇÃO/ACENTUAÇÃO.txt
+ /TST_ACCENT/CONSTRUÇÃO/Acent__ção.txt
+ /TST_ACCENT/C___ção
+ /TST_ACCENT/C___ção/ACENTUAÇÃO.txt
+ /TST_ACCENT/C___ção/Acent__ção.txt
Addition count 6 src: 730.00B
Proceed with the changes? [Y/n]:y
0 / 730 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 0.00%Push::Started /TST_ACCENT/CONSTRUÇÃO/Acent__ção.txt
Push::Started /TST_ACCENT/CONSTRUÇÃO/ACENTUAÇÃO.txt
Push::Started /TST_ACCENT/CONSTRUÇÃO
Push::Started /TST_ACCENT/C___ção/Acent__ção.txt
Push::Started /TST_ACCENT/C___ção/ACENTUAÇÃO.txt
Push::Started /TST_ACCENT/C___ção
215 / 730 [==================================================>--------------------------------------------------------------------------------------------------------------------------] 29.45% 7sPush::Done /TST_ACCENT/CONSTRUÇÃO/ACENTUAÇÃO.txt
215 / 730 [==================================================>--------------------------------------------------------------------------------------------------------------------------] 29.45% 7sPush::Done /TST_ACCENT/CONSTRUÇÃO/Acent__ção.txt
730 / 730 [=============================================================================================================================================================================] 100.00% 0sPush::Done /TST_ACCENT/CONSTRUÇÃO
730 / 730 [=============================================================================================================================================================================] 100.00% 0sPush::Done /TST_ACCENT/C___ção
730 / 730 [=============================================================================================================================================================================] 100.00% 0sPush::Done /TST_ACCENT/C___ção/ACENTUAÇÃO.txt
730 / 730 [=============================================================================================================================================================================] 100.00% 0sPush::Done /TST_ACCENT/C___ção/Acent__ção.txt
730 / 730 [=============================================================================================================================================================================] 100.00% 6s
Transfers was OK but can not list
drive list TST_ACCENT/C___ção
/TST_ACCENT/C___ção/ACENTUAÇÃO.txt
/TST_ACCENT/C___ção/Acent__ção.txt
drive list TST_ACCENT/CONSTRUÇÃO
"/TST_ACCENT/CONSTRUÇÃO" cannot be found remotely
When push again with changed files a mess starts on the "CONSTRUÇÃO" folder. The "C___ção" folder works perfectly
date;drive push -verbose -ignore-name-clashes -destination TST_ACCENT ;date
Sex Set 29 11:55:45 BRT 2017
Resolving...
M /TST_ACCENT/CONSTRUÇÃO/Acent__ção.txt
M /TST_ACCENT/CONSTRUÇÃO/ACENTUAÇÃO.txt
M /TST_ACCENT/C___ção/ACENTUAÇÃO.txt
M /TST_ACCENT/C___ção/Acent__ção.txt
Modification count 4 src: 850.00B dest: 730.00B
Proceed with the changes? [Y/n]:y
0 / 850 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 0.00%Push::Started /TST_ACCENT/C___ção/ACENTUAÇÃO.txt
Push::Started /TST_ACCENT/CONSTRUÇÃO/Acent__ção.txt
Push::Started /TST_ACCENT/CONSTRUÇÃO/ACENTUAÇÃO.txt
Push::Started /TST_ACCENT/C___ção/Acent__ção.txt
293 / 850 [===========================================================>-----------------------------------------------------------------------------------------------------------------] 34.47% 2sPush::Done /TST_ACCENT/C___ção/ACENTUAÇÃO.txt
850 / 850 [=============================================================================================================================================================================] 100.00% 0sPush::Done /TST_ACCENT/CONSTRUÇÃO/Acent__ção.txt
850 / 850 [=============================================================================================================================================================================] 100.00% 0sPush::Done /TST_ACCENT/C___ção/Acent__ção.txt
850 / 850 [=============================================================================================================================================================================] 100.00% 0sPush::Done /TST_ACCENT/CONSTRUÇÃO/ACENTUAÇÃO.txt
850 / 850 [=============================================================================================================================================================================] 100.00% 4s
Sex Set 29 11:55:59 BRT 2017
Please see that there are now 2 "CONSTRUÇÃO" folder !
drive list TST_ACCENT
/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/C___ção
drive list TST_ACCENT/C___ção
/TST_ACCENT/C___ção/ACENTUAÇÃO.txt
/TST_ACCENT/C___ção/Acent__ção.txt
drive list TST_ACCENT/CONSTRUÇÃO
"/TST_ACCENT/CONSTRUÇÃO" cannot be found remotely
As more push are performed more confusion is done on "CONSTRUÇÃO" folder
drive list TST_ACCENT
/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/C___ção
rive list TST_ACCENT/CONSTRUÇÃO
"/TST_ACCENT/CONSTRUÇÃO" cannot be found remotely
Just folders are affected. Files worked as expected, even if it has upper case accent characters on name.
This problem also appear on rclone and on the Google Drive API V2
https://github.com/ncw/rclone/issues/1675
https://issuetracker.google.com/issues/64468406
Do you think that Google could fix it on the API or drive and rclone will have to work around this strange bug?
Thank you
Roberto
Thank you for the bug report and welcome to drive!
So for starters:
- What's your version of drive?
- What's your OS? Environment?
I can't reproduce this on
$ drive version
drive version: 0.3.9.1
Commit Hash: '611b523a468baba5da9e70485c18332632cb10c0'
Go Version: devel +0e9d293 Sun Jun 11 04:25:26 2017 +0000
OS: darwin/amd64
BuildTime: 2017-07-23 23:43:19.27559449 -0600 MDT m=+0.012289696
Please update your drive and let me know what that produces, and also if you fill in the requested information, that'd be great.
Hello,
Sorry, I should have provided this info in the first post.
drive version
drive version: 0.3.9.1
Commit Hash: '1cb01ece65b6d9d39060da1ecf2a440a90b24954'
Go Version: go1.8.3
OS: linux/amd64
BuildTime: 2017-05-29 14:38:38.77322112 -0300 BRT
System info:
uname -a
Linux spitfire.mha.com.br. 2.6.32-573.22.1.el6.centos.plus.x86_64 #1 SMP Wed Mar 23 17:51:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
env | grep -i lang
LANG=pt_BR.UTF-8
drive list TST_ACCENT/
/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/C___ção
drive list TST_ACCENT/CONSTRUÇÃO
"/TST_ACCENT/CONSTRUÇÃO" cannot be found remotely
drive list TST_ACCENT/CONSTRUÇÃO/
"/TST_ACCENT/CONSTRUÇÃO" cannot be found remotely
In your tests, could you try change the file ls.txt and push again to see if it works ?
Please, let me know if you want more info
Thank you
Roberto
Thanks for the update @robjlg, am on OSX and I can't reproduce your issue
However, I think perhaps this wiki note might be helpful https://github.com/odeke-em/drive/wiki#linux-terminal-character-encoding although your terminal is UTF-8 enabled already.
Let me ping our terminal encoding guru @colinkeenan for help.
I will start looking into this, but might take awhile. What interests me is that at some point I suspect many of my folders were duplicated even though I don't use "unusual" characters, but I haven't been motivated to look into it. I wasn't using drive list before, and will look into what it says about my various folders. It may be that after clearing up the confusion, it won't happen again which is why @odeke-em can't duplicate the problem.
Before trying to find if any of my directories were duplicated (I know something funny happened a month or so ago), I just tried to duplicate this issue. I duplicated it:
[colin] ~/Google Drive % mkdir CONSTRUÇÃO
[colin] ~/Google Drive % touch CONSTRUÇÃO/ACENTUAÇÃO.txt
[colin] ~/Google Drive % touch CONSTRUÇÃO/Acent__ção.txt
[colin] ~/Google Drive % drive push
Resolving...
+ /CONSTRUÇÃO
+ /CONSTRUÇÃO/ACENTUAÇÃO.txt
+ /CONSTRUÇÃO/Acent__ção.txt
Addition count 3
Proceed with the changes? [Y/n]:
[colin] ~/Google Drive % drive ls CONSTRUÇÃO
"/CONSTRUÇÃO" cannot be found remotely
By the way, the problem is only with drive ls, not with the actual upload. Here is a screenshot showing CONSTRUÇÃO was uploaded:
https://drive.google.com/open?id=0B1X0bdl44fGjVlNFY1ExUlVDWEU
But, please try to push it again...
It will duplicate the folder on drive
Em 1 de out de 2017 10:07 PM, "Colin Keenan" [email protected] escreveu:
By the way, the problem is only with drive ls, not with the actual upload. Here is a screenshot showing CONSTRUÇÃO was uploaded:
https://drive.google.com/open?id=0B1X0bdl44fGjVlNFY1ExUlVDWEU
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/odeke-em/drive/issues/955#issuecomment-333420131, or mute the thread https://github.com/notifications/unsubscribe-auth/ASZRvrc8EDHlio0gFdLsoNPNoVkPxiidks5soDc-gaJpZM4PpI7_ .
Immediately after my previous comment, my computer died and wouldn't even go to the BIOS screen. I've been cleaning all the dust out since then, but it's still not working. I'm using an old laptop that I luckily got working a couple of weeks ago. I will wait to work on this until I've figured out what's going on with my main computer.
I had left my main computer on while typing the previous comment 2 hours ago, and then it finished booting up just a couple of minutes later. I was busy with other stuff and I'm worried about my main computer still, but it's working.
I've verified that if I modify the files in CONSTRUÇÃO then drive push creates a second CONSTRUÇÃO. The first one ends up with no files in it, and the second one has the updated files.
Next, locally, I did rm -r CONSTRUÇÃO and drive push --ignore-name-clashes. That worked! Both CONSTRUÇÃO folders were removed.
By the way, there actually is an issue with regular files. When I published the screenshot shown 2 comments ago, I first tried using drive pub CONSTRUÇÃO.png which gave the same error that we've been getting when trying to use drive list CONSTRUÇÃO.
I've verified that if I modify the files in CONSTRUÇÃO then drive push creates a second CONSTRUÇÃO. The first one ends up with no files in it, and the second one has the updated files.
Also for me. After modifying the files in CONSTRUÇÂO again and then drive push, a third CONSTRUÇÂO is created. One empty, the second contain one of my test files, an the third contain another test file. That is, the 2 edited files in CONSTRUÇÂO end up in DRIVE each of them on one distinct CONTRUÇÃO folders.
The push command only worked with --ignore-name-clashes.
After another edit files and drive push, DRIVE end up with two CONSTRUÇÃO folders, one empty and the other with both correct edited files.
I confirm that files with upper case accents can not be listed.
drive list -depth 10 TST_ACCENT/TST_ACCENT
/TST_ACCENT/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/TST_ACCENT/CONSTRUÇÃO
/TST_ACCENT/TST_ACCENT/C___ção
---More---
/TST_ACCENT/TST_ACCENT/CONSTRUÇÃO/Acent__ção.txt
/TST_ACCENT/TST_ACCENT/CONSTRUÇÃO/ACENTUAÇÃO.txt
/TST_ACCENT/TST_ACCENT/C___ção/Acent__ção.txt
/TST_ACCENT/TST_ACCENT/C___ção/ACENTUAÇÃO.txt
drive list TST_ACCENT/TST_ACCENT/C___ção/Acent__ção.txt
/TST_ACCENT/TST_ACCENT/C___ção/Acent__ção.txt
drive list TST_ACCENT/TST_ACCENT/C___ção/ACENTUAÇÃO.txt
"/TST_ACCENT/TST_ACCENT/C___ção/ACENTUAÇÃO.txt" cannot be found remotely
drive list TST_ACCENT/TST_ACCENT/CONSTRUÇÃO/Acent__ção.txt
"/TST_ACCENT/TST_ACCENT/CONSTRUÇÃO/Acent__ção.txt" cannot be found remotely
There is nothing to be done with the local on my machine and I'm sure it's fine for @robjlg too.
@odeke-em, is there anyway to get Google Drive to report the Unicode in decimal or hex for a given file name? I want to know what the difference is between what we see in our terminal and what Google Drive sees after uploading it. Simply using a one character file would be easiest.
[colin] ~/Google Drive % touch Ã
[colin] ~/Google Drive % drive push Ã
Resolving...
+ /Ã
Addition count 1
Proceed with the changes? [Y/n]:Y
[colin] ~/Google Drive % drive ls Ã
"/Ã" cannot be found remotely
[colin] ~/Google Drive % printf à | uniname
character byte UTF-32 encoded as glyph name
0 0 0000C3 C3 83 Ã LATIN CAPITAL LETTER A WITH TILDE
Above, I use uniname from the uniutils package to show that à is encoded as C3 83.
Is there anyway to get Google Drive to report the same information?
If not, is there anyway to get Google Drive to list files that match a pattern? I mean it would be helpful to be able to replace troublesome characters with a ? and let Google Drive match whatever it wants for the ?.
has this ever been resolved ? I get problems with 'drive list' command : files containing french accents cannot be listed, renamed nor pulled using the command line.
