ZIP does not support timestamps before 1980

[clach04]

Workaround

Starting a shell session in the container and using a linux file system avoids problems, see the notes in the readme for starting a session (ideally with a name, -name=pebbleDev for persistence):

docker run --rm -it -v ~/pebble-dev/:/pebble/ bboehmke/pebble-dev

---

Thanks for this Docker image - its the one I've had the most success with so far.

Platform information (which I'm not sure is relevant but included for completeness).

Windows 10 64-bit with Docker 64-bit:

C:\code\docker\pebble\bboehmke>docker --version
Docker version 19.03.5, build 633a0ea

C:\code\docker\pebble\bboehmke>ver

Microsoft Windows [Version 10.0.17134.1184]

My simple demo batch script:

set IMAGE=bboehmke/pebble-dev

mkdir build

REM Works :-) interactive shell
REM docker run --rm -it -v %CD%\build:/pebble/ bboehmke/pebble-dev


docker run --rm -it -v %CD%\build:/pebble %IMAGE% pebble --version
docker run --rm -it -v %CD%\build:/pebble %IMAGE% pebble new-project test
docker run --rm -it -v %CD%\build\test:/pebble %IMAGE% pebble clean
docker run --rm -it -v %CD%\build\test:/pebble %IMAGE% pebble build

The compile part of the build stage is successful, the bundling into PBW is not:

.
..
...
[55/64] aplite | memory_usage_report: build/aplite/pebble-app.elf build/aplite/app_resources.pbpack
-------------------------------------------------------
APLITE APP MEMORY USAGE
Total size of resources:        4092 bytes / 128KB
Total footprint in RAM:         846 bytes / 24KB
Free RAM available (heap):      23730 bytes
-------------------------------------------------------
[56/64] diorite | pebble-app.raw.bin: build/diorite/pebble-app.elf -> build/diorite/pebble-app.raw.bin
[57/64] chalk | pebble-app.raw.bin: build/chalk/pebble-app.elf -> build/chalk/pebble-app.raw.bin
[58/64] basalt | pebble-app.raw.bin: build/basalt/pebble-app.elf -> build/basalt/pebble-app.raw.bin
[59/64] aplite | pebble-app.raw.bin: build/aplite/pebble-app.elf -> build/aplite/pebble-app.raw.bin
[60/64] diorite | inject-metadata: build/diorite/pebble-app.raw.bin build/diorite/pebble-app.elf build/diorite/app_resources.pbpack -> build/diorite/pebble-app.bin
[61/64] chalk | inject-metadata: build/chalk/pebble-app.raw.bin build/chalk/pebble-app.elf build/chalk/app_resources.pbpack -> build/chalk/pebble-app.bin
[62/64] basalt | inject-metadata: build/basalt/pebble-app.raw.bin build/basalt/pebble-app.elf build/basalt/app_resources.pbpack -> build/basalt/pebble-app.bin
[63/64] aplite | inject-metadata: build/aplite/pebble-app.raw.bin build/aplite/pebble-app.elf build/aplite/app_resources.pbpack -> build/aplite/pebble-app.bin
[64/64] app_bundle:  -> build/pebble.pbw
Waf: Leaving directory `/pebble/build'
Build failed
Traceback (most recent call last):
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 123, in process
    ret=self.run()
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 47, in run
    return m1(self)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/extras/process_bundle.py", line 42, in run
    mkbundle.make_watchapp_bundle(timestamp=self.generator.bld.env.TIMESTAMP,appinfo=self.generator.bld.path.get_bld().find_node('appinfo.json').abspath(),binaries=binaries,js=[js_file.abspath()for js_file in js_files],outfile=outfile)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 397, in make_watchapp_bundle
    bundle.write(outfile, verbose)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 286, in write
    z.write(bf, os.path.join(f.subfolder, os.path.basename(bf)))
  File "/usr/local/lib/python2.7/zipfile.py", line 1158, in write
    zinfo = ZipInfo(arcname, date_time)
  File "/usr/local/lib/python2.7/zipfile.py", line 328, in __init__
    raise ValueError('ZIP does not support timestamps before 1980')
ValueError: ZIP does not support timestamps before 1980

Build failed.

Timestamps in host are today (2020-01-25). I've not dug into this any further. Logging what I have at this point.

[clach04]

Hacking zipfile:

    if date_time[0] < 1980:
        raise ValueError('ZIP does not support timestamps before 1980 - %r' % (date_time,))

and dumping into my path

env PYTHONPATH=../ pebble build

Seeing:

Waf: Leaving directory `/pebble/test/build'
Build failed
Traceback (most recent call last):
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 123, in process
    ret=self.run()
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 47, in run
    return m1(self)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/extras/process_bundle.py", line 42, in run
    mkbundle.make_watchapp_bundle(timestamp=self.generator.bld.env.TIMESTAMP,appinfo=self.generator.bld.path.get_bld().find_node('appinfo.json').abspath(),binaries=binaries,js=[js_file.abspath()for js_file in js_files],outfile=outfile)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 397, in make_watchapp_bundle
    bundle.write(outfile, verbose)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 286, in write
    z.write(bf, os.path.join(f.subfolder, os.path.basename(bf)))
  File "/pebble/zipfile.py", line 1158, in write
    zinfo = ZipInfo(arcname, date_time)
  File "/pebble/zipfile.py", line 328, in __init__
    raise ValueError('ZIP does not support timestamps before 1980 - %r' % (date_time,))
ValueError: ZIP does not support timestamps before 1980 - (1970, 1, 1, 0, 0, 0)

Build failed.

[clach04]

I'm going to guess this is a weird docker thing:

    raise ValueError('ZIP does not support timestamps before 1980 - %r' % ((date_time, filename),))
ValueError: ZIP does not support timestamps before 1980 - ((1970, 1, 1, 0, 0, 0), 'diorite/pebble-app.bin')

Build failed.
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/pebble-app.bin
-rwxr-xr-x 1 root root 840 Jan  1  1970 build/diorite/pebble-app.bin
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/
total 0
drwxrwxrwx 1 root root  4096 Jan 25 19:16 .
drwxrwxrwx 1 root root  4096 Jan 25 19:16 ..
-rwxr-xr-x 1 root root  4092 Jan 25 19:16 app_resources.pbpack
-rwxr-xr-x 1 root root   826 Jan 25 19:16 appinfo.auto.c
-rwxr-xr-x 1 root root  4260 Jan 25 19:16 appinfo.auto.c.30.o
-rwxr-xr-x 1 root root   840 Jan 25 19:16 pebble-app.bin
-rwxr-xr-x 1 root root 48650 Jan 25 19:16 pebble-app.elf
-rwxr-xr-x 1 root root   840 Jan 25 19:16 pebble-app.raw.bin
-rwxr-xr-x 1 root root   799 Jan 25 19:16 pebble_app.ld.auto
drwxrwxrwx 1 root root     0 Jan 25 19:16 src
-rwxr-xr-x 1 root root   110 Jan 25 19:16 system_resources.resball
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/pebble-a*
-rwxr-xr-x 1 root root   840 Jan 25 19:16 build/diorite/pebble-app.bin
-rwxr-xr-x 1 root root 48650 Jan 25 19:16 build/diorite/pebble-app.elf
-rwxr-xr-x 1 root root   840 Jan 25 19:16 build/diorite/pebble-app.raw.bin
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/pebble-app.bin
-rwxr-xr-x 1 root root 840 Jan 25 19:16 build/diorite/pebble-app.bin
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/pebble-app.bin
-rwxr-xr-x 1 root root 840 Jan 25 19:16 build/diorite/pebble-app.bin

NOTE one of the ls -al shows jan 1s (no year shown) and then later Jan 25th!? These were entered immediately after each other. There is a virus check running in the background in my host OS (I have no permissions to disable).

I also saw odd issues with the .pyc file having a later date than a modied py file for the zipfile hacks I was making so resorted to removing the pyc file each time.

Current hack:

    if date_time[0] < 1980:
        date_time = (2020, 1, 1, 0, 0, 0)  # clach04 hack, maybe use now/today instead?
        #raise ValueError('ZIP does not support timestamps before 1980 - %r' % ((date_time, filename),))  # clach04 diag hack

This lead to one successful build. Later (and so far consistently) getting errors (note includes debug):

[64/64] app_bundle:  -> build/test.pbw
<type 'tuple'>
"((2020, 1, 25, 19, 20, 31), 'appinfo.json')"
len(filename), len(extra) '(12, 0)'
len(filename), len(extra) '(12, 0)'
<type 'tuple'>
"((1970, 1, 1, 0, 0, 0), 'diorite/pebble-app.bin')"
len(filename), len(extra) '(22, 0)'
Waf: Leaving directory `/pebble/test/build'
Build failed
Traceback (most recent call last):
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 123, in process
    ret=self.run()
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 47, in run
    return m1(self)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/extras/process_bundle.py", line 42, in run
    mkbundle.make_watchapp_bundle(timestamp=self.generator.bld.env.TIMESTAMP,appinfo=self.generator.bld.path.get_bld().find_node('appinfo.json').abspath(),binaries=binaries,js=[js_file.abspath()for js_file in js_files],outfile=outfile)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 397, in make_watchapp_bundle
    bundle.write(outfile, verbose)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 286, in write
    z.write(bf, os.path.join(f.subfolder, os.path.basename(bf)))
  File "/pebble/zipfile.py", line 1202, in write
    self.fp.write(zinfo.FileHeader(zip64))
  File "/pebble/zipfile.py", line 399, in FileHeader
    len(filename), len(extra))
error: ushort format requires 0 <= number <= USHRT_MAX

Moving into /tmp (vm) rather than native host drive avoided all problems.

Any ideas on this with Windows?

[bboehmke]

I have never used this image on windows. I also never really used docker on windows at all.

The only thing I can imagine is that the time of the docker VM is maybe not set correctly. A simple date call gives maybe more information. This should show the current time inside the VM and the container.

[bboehmke]

Otherwise the

Moving into /tmp (vm) rather than native host drive avoided all problems.

does not really match this theory.

[clach04]

Yeah super odd - current theory is either something in Windows (Symantec) or some sync issue between host/container.

I documented the current (reliable) workaround in the original description. Having git in the container helps but I now need a reliable way to edit/build. I opened issued #7 to discuss possible options.

Again thanks for this, I played with a couple of pebble containers and all had some major problems, this really has been the smoothest option since moving away from a VM (previously I was using a 32-bit VM but my machine has HyperV enabled and Virtual Box won't host 32-bit images because of that).

[bboehmke]

It seems that there is a base issue in docker for windows.

I tried a simple touch -m src/c/test.c which caused a

touch: setting times of 'src/c/test.c': Software caused connection abort

followed by a complete destruction of the volume mount which requires a restart of the docker daemon.

I dont think we can fix this problem in the image. (At least I currently dont have an idea how)