luarocks icon indicating copy to clipboard operation
luarocks copied to clipboard
verified publisher icon luarocks

Broken package: lsqlite3

Open dustyhorizon opened this issue 4 months ago • 6 comments

  • Platform:

    • Linux

  • LuaRocks version:

    • luarocks --version /usr/bin/luarocks 3.8.0

  • LuaRocks output from when the issue occurred:

# luarocks install lsqlite3 --verbose
...
fs.absolute_name("/tmp/luarocks_luarocks-rockspec-lsqlite3-0.9.6-1-554432/lsqlite3-0.9.6-1.rockspec")
fs.current_dir()
fs.make_temp_dir("luarocks-source-lsqlite3-0.9.6-1")
fs.system_temp_dir()
fs.change_dir("/tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330")
fs.download("https://lua.sqlite.org/home/zip/lsqlite3_v096.zip?uuid=v0.9.6", "lsqlite3_v096.zip", nil)
fs.absolute_name("lsqlite3_v096.zip")
fs.current_dir()
GET     https://lua.sqlite.org/home/zip/lsqlite3_v096.zip?uuid=v0.9.6
fs.pop_dir()
fs.change_dir("/tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330")
fs.unpack_archive("lsqlite3_v096.zip")
fs.absolute_name("lsqlite3_v096.zip")
fs.current_dir()
fs.unzip("/tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip")
fs.is_tool_available("unzip -n", "unzip", "--help")
fs.execute_quiet("unzip -n", "--help")
fs.Q("--help")
fs.execute_string("unzip -n '--help'")

os.execute:     unzip -n '--help'
UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.

Usage: unzip [-Z] [-opts[modifiers]] file[.zip] [list] [-x xlist] [-d exdir]
  Default action is to extract files in list, except those in xlist, to exdir;
  file[.zip] may be a wildcard.  -Z => ZipInfo mode ("unzip -Z" for usage).

  -p  extract files to pipe, no messages     -l  list files (short format)
  -f  freshen existing files, create none    -t  test compressed archive data
  -u  update files, create if necessary      -z  display archive comment only
  -v  list verbosely/show version info       -T  timestamp archive to latest
  -x  exclude files that follow (in xlist)   -d  extract files into exdir
modifiers:
  -n  never overwrite existing files         -q  quiet mode (-qq => quieter)
  -o  overwrite files WITHOUT prompting      -a  auto-convert any text files
  -j  junk paths (do not make directories)   -aa treat ALL files as text
  -U  use escapes for all non-ASCII Unicode  -UU ignore any Unicode fields
  -C  match filenames case-insensitively     -L  make (some) names lowercase
  -X  restore UID/GID info                   -V  retain VMS version numbers
  -K  keep setuid/setgid/tacky permissions   -M  pipe through "more" pager
See "unzip -hh" or unzip.txt for more help.  Examples:
  unzip data1 -x joe   => extract all files except joe from zipfile data1.zip
  unzip -p foo | more  => send contents of foo.zip via pipe into program more
  unzip -fo foo ReadMe => quietly replace existing ReadMe if archive file newer
Results: 1
  1 (number): 0
fs.execute_quiet("unzip -n", "/tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip")
fs.Q("/tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip")
fs.execute_string("unzip -n '/tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip'")

os.execute:     unzip -n '/tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip'
Archive:  /tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of /tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip or
        /tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip.zip, and cannot find /tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip.ZIP, period.
Results: 1
  1 (number): 2304

Error: Failed extracting /tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip: failed extracting /tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330/lsqlite3_v096.zip
fs.change_dir_to_root()
fs.delete("/tmp/luarocks_luarocks-source-lsqlite3-0.9.6-1-5420330")
fs.delete("/tmp/luarocks_luarocks-rockspec-lsqlite3-0.9.6-1-554432")
  • Description: luarocks installs lsqlite3 from source per the rockspec. However, it seems that upstream has added some form of browser/origin check which is breaking downloads via CLI

Attached is a screengrab of the browser check

Image

dustyhorizon avatar Aug 19 '25 08:08 dustyhorizon

I'm also having this problem fwiw

ricomariani avatar Aug 28 '25 02:08 ricomariani

ticket on the lib itself: https://lua.sqlite.org/home/info/4a118ffd0cea06f91fee5754b8a1b5ce4ba6699f

@javierguerragiraldez I noticed on the LuaRocks site that the previous version is available in both a rockspec and a src version. Whilst the current version is only available in rockspec format. If it is only available in rockspec format, it requires LuaRocks to download the source/zip, which will then fail. If you can upload a src rock, then no download will be necessary.

Tieske avatar Aug 29 '25 17:08 Tieske

Hello.

I had a look on this issue, and learned a lot of interesting things (completely offtopic) related to SQLite and LSQLite3.

Background context

SQLite website (https://www.sqlite.org) is suffering some sort of bot-net attack (read at https://www2.sqlite.org/forum/info/c9d2f11f163be1d1421411e8885be6b0e6ff5aff16a98fd72123c9033fab071b). In short, for this reason, they implemented a JavaScript-like browser bot detection, which is the root cause for the zip file download issue (LSQLite3 is hosted on a subdomain).

After a read, I've found that both SQLite and LSQLite3 use fossil (https://fossil-scm.org) for their development (a Git-like SCM tool).

How to build LSQLite3 from source code

So, to workaround the issue, you have to

  1. download and install fossil;

    • Debian-based distros: sudo apt install -y fossil
    • macOS: brew install fossil
    • Windows: visit (https://fossil-scm.org/home/uv/download.html) and choose the correct zip file, then unzip it and place the directory of fossil.exe on your PATH environment variable;

  2. fetch LSQLite3 source code:

    mkdir LSQLite3-build
cd LSQLite3-build
fossil open "https://lua.sqlite.org/home/timeline"

  3. change the tree for the current tag (v0.9.6):

    fossil checkout "v0.9.6"

  4. after all these steps, then run the command

    luarocks make lsqlite3complete-0.9.6-1.rockspec

    to build LSQLite3 (bundled with SQLite) from the local tree.

Reproduction on CI

I wrote a small GitHub Actions workflow to demonstrate these steps (see https://github.com/luau-project/ci-tests/actions/runs/17714139404):

name: LuaRocks issue 1822

on: push

jobs:
  test:
    runs-on: ${{ matrix.os }}

    strategy:

      fail-fast: false

      matrix:

        lua-version:
          - 5.1
          - 5.2
          - 5.3
          - 5.4
          - luajit
          - luajit-openresty

        os:
          - ubuntu-latest
          - macos-latest
          - windows-latest

    steps:

      - uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1.13.0
        if: ${{ runner.os == 'Windows' && !startsWith(matrix.lua-version, 'luajit') }}

      - name: Install Fossil (Unix)
        if: ${{ runner.os != 'Windows' }}
        run: |
          if [ "${{ runner.os }}" = "Linux" ]; then
            sudo apt install -y fossil
          elif [ "${{ runner.os }}" = "macOS" ]; then
            brew install fossil
          else
            echo "Unknown runner OS"
            exit 1
          fi

      - name: Install Fossil 2.26 (Windows)
        if: ${{ runner.os == 'Windows' }}
        shell: cmd
        env:
          FOSSIL_ZIP: ${{ runner.temp }}\fossil.zip
          FOSSIL_APP_DIR: ${{ runner.temp }}\fossil-app
        run: |
          IF EXIST "${{ env.FOSSIL_ZIP }}" DEL "${{ env.FOSSIL_ZIP }}"
          IF "${{ runner.arch }}"=="X86" curl --output "${{ env.FOSSIL_ZIP }}" -kL "https://fossil-scm.org/home/uv/fossil-w32-2.26.zip"
          IF "${{ runner.arch }}"=="X64" curl --output "${{ env.FOSSIL_ZIP }}" -kL "https://fossil-scm.org/home/uv/fossil-w64-2.26.zip"
          IF "${{ runner.arch }}"=="ARM64" curl --output "${{ env.FOSSIL_ZIP }}" -kL "https://fossil-scm.org/home/uv/fossil-win-arm-2.26.zip"
          IF NOT EXIST "${{ env.FOSSIL_ZIP }}" (
            ECHO Failed to download fossil
            EXIT /B 1
          )
          IF EXIST "${{ env.FOSSIL_APP_DIR }}\" RMDIR /S /Q "${{ env.FOSSIL_APP_DIR }}"
          MKDIR "${{ env.FOSSIL_APP_DIR }}"
          "C:\Program Files\7-zip\7z.exe" "-o${{ env.FOSSIL_APP_DIR }}" x "${{ env.FOSSIL_ZIP }}"
          ECHO ${{ env.FOSSIL_APP_DIR }}>>${{ github.path }}

      - name: Checkout LSQLite3 v0.9.6
        run: |
          fossil open "https://lua.sqlite.org/home/timeline"
          fossil checkout "v0.9.6"

      - uses: luarocks/gh-actions-lua@989f8e6ffba55ce1817e236478c98558e598776c # v11
        with:
          luaVersion: ${{ matrix.lua-version }}
          buildCache: false

      - uses: luarocks/gh-actions-luarocks@7c85eeff60655651b444126f2a78be784e836a0a # v6
        with:
          luaRocksVersion: 3.12.2

      - name: Build and Install LSQLite3 v0.9.6 (bundled with SQLite)
        run: luarocks make lsqlite3complete-0.9.6-1.rockspec

luau-project avatar Sep 14 '25 17:09 luau-project

@luau-project Thank you for sharing this!

Personally I decided to take an easier path: download the zip file manually and commit it in my repository. Updates aren’t frequent so I don’t think it’s a problem. Then I just need to:

unzip /build/lsqlite3_v096.zip; \
  cd /build/lsqlite3_v096; \
  luarocks-5.4 make lsqlite3-0.9.6-1.rockspec

The full fix can be found at https://github.com/prose-im/prose-pod-server/commit/583599878c55d9b90d429bff4f37c5a370f295bc.

RemiBardon avatar Sep 14 '25 18:09 RemiBardon

Just here to link two related tickets:

  • https://github.com/luarocks/luarocks-site/issues/228
  • https://github.com/lumen-oss/lux/issues/1119

@javierguerragiraldez I noticed on the LuaRocks site that the previous version is available in both a rockspec and a src version. Whilst the current version is only available in rockspec format. If it is only available in rockspec format, it requires LuaRocks to download the source/zip, which will then fail. If you can upload a src rock, then no download will be necessary.

This is the easiest fix for now I would say. I can't make head or tails of the fossil ticket tracker (seems like I wont be notified of updates as I had to edit it anonymously) so please comment here once it is fixed :)

teto avatar Oct 19 '25 17:10 teto

After a read, I've found that both SQLite and LSQLite3 use fossil (https://fossil-scm.org) for their development (a Git-like SCM tool).

No comments on Fossil (but I have plenty of opinions) I created a mirror of the lsqlite3 repo on https://git.sr.ht/~mcepl/lsqlite3/ (no guarantees that I will ever update it).

mcepl avatar Oct 19 '25 21:10 mcepl