tiup icon indicating copy to clipboard operation
tiup copied to clipboard
verified publisher icon pingcap

DEB Package

Open dveeden opened this issue 10 months ago • 25 comments

dveeden avatar Feb 06 '25 16:02 dveeden

This is a subtask of #2506.

I am currently a bit baffled about these module dependencies that point to a subdirectory inside a specific branch in a module:

src/github.com/pingcap/tiup/components/client/main.go:27:2: cannot find package "github.com/gizak/termui/v3" in any of:
	/tmp/test/tiup/debian/build/src/github.com/pingcap/tiup/vendor/github.com/gizak/termui/v3 (vendor tree)
	/usr/lib/go-1.24/src/github.com/gizak/termui/v3 (from $GOROOT)
	/tmp/test/tiup/debian/build/src/github.com/gizak/termui/v3 (from $GOPATH)
src/github.com/pingcap/tiup/components/client/main.go:28:2: cannot find package "github.com/gizak/termui/v3/widgets" in any of:
	/tmp/test/tiup/debian/build/src/github.com/pingcap/tiup/vendor/github.com/gizak/termui/v3/widgets (vendor tree)
	/usr/lib/go-1.24/src/github.com/gizak/termui/v3/widgets (from $GOROOT)
	/tmp/test/tiup/debian/build/src/github.com/gizak/termui/v3/widgets (from $GOPATH)
src/github.com/pingcap/tiup/components/client/main.go:32:2: cannot find package "github.com/xo/usql/drivers/mysql" in any of:
	/tmp/test/tiup/debian/build/src/github.com/pingcap/tiup/vendor/github.com/xo/usql/drivers/mysql (vendor tree)
	/usr/lib/go-1.24/src/github.com/xo/usql/drivers/mysql (from $GOROOT)
	/tmp/test/tiup/debian/build/src/github.com/xo/usql/drivers/mysql (from $GOPATH)
src/github.com/pingcap/tiup/components/client/main.go:33:2: cannot find package "github.com/xo/usql/env" in any of:
	/tmp/test/tiup/debian/build/src/github.com/pingcap/tiup/vendor/github.com/xo/usql/env (vendor tree)
	/usr/lib/go-1.24/src/github.com/xo/usql/env (from $GOROOT)
	/tmp/test/tiup/debian/build/src/github.com/xo/usql/env (from $GOPATH)
src/github.com/pingcap/tiup/components/client/main.go:34:2: cannot find package "github.com/xo/usql/handler" in any of:
	/tmp/test/tiup/debian/build/src/github.com/pingcap/tiup/vendor/github.com/xo/usql/handler (vendor tree)
	/usr/lib/go-1.24/src/github.com/xo/usql/handler (from $GOROOT)
	/tmp/test/tiup/debian/build/src/github.com/xo/usql/handler (from $GOPATH)
src/github.com/pingcap/tiup/components/client/main.go:35:2: cannot find package "github.com/xo/usql/rline" in any of:
	/tmp/test/tiup/debian/build/src/github.com/pingcap/tiup/vendor/github.com/xo/usql/rline (vendor tree)
	/usr/lib/go-1.24/src/github.com/xo/usql/rline (from $GOROOT)
	/tmp/test/tiup/debian/build/src/github.com/xo/usql/rline (from $GOPATH)
src/github.com/pingcap/tiup/embed/embed.go:15:12: pattern examples: no matching files found

These all stem from https://github.com/pingcap/tiup/blob/master/components/client/main.go#L27-L35

What is the point of this code, why isn't it just importing the module in a normal fashion?

ottok avatar Mar 20 '25 05:03 ottok

I faced the same in #2507. I just created #2517 for this.

dveeden avatar Mar 20 '25 06:03 dveeden

Seems TiUp has a bunch of Go dependencies that are not yet in Debian at all. One major one is xo/usql, for which I started packaging at https://salsa.debian.org/go-team/packages/usql/-/merge_requests/1

ottok avatar Mar 20 '25 17:03 ottok

For the record, dh-make-golang estimate github.com/pingcap/tiup reports on latest v.1.16.2:

Bringing github.com/pingcap/tiup to Debian requires packaging the following Go modules:
github.com/pingcap/tiup
  github.com/AstroProfundis/sysinfo
  github.com/ScaleFT/sshkeys
    github.com/dchest/bcrypt_pbkdf
  github.com/appleboy/easyssh-proxy
  github.com/asaskevich/EventBus
  github.com/cavaliergopher/grab/v3	(github.com/cavaliergopher/grab in Debian)
  github.com/cheggaaa/pb/v3	(github.com/cheggaaa/pb in Debian)
  github.com/gibson042/canonicaljson-go
  github.com/gizak/termui/v3	(github.com/gizak/termui in Debian)
  github.com/jedib0t/go-pretty/v6	(github.com/jedib0t/go-pretty in Debian)
  github.com/jeremywohl/flatten
  github.com/joomcode/errorx
  github.com/pingcap/errors
    github.com/pingcap/log
  github.com/pingcap/failpoint
  github.com/pingcap/fn
  github.com/pingcap/kvproto
  github.com/pingcap/tidb-insight/collector
    github.com/lorenzosaino/go-sysctl
      golang.org/x/lint
    github.com/yusufpapurcu/wmi
      github.com/go-ole/go-ole
  github.com/r3labs/diff/v2
  github.com/relex/aini
  github.com/sethvargo/go-password
  github.com/xo/usql
    github.com/ClickHouse/clickhouse-go
    github.com/IBM/nzgo
    github.com/MichaelS11/go-cql-driver
    github.com/VoltDB/voltdb-client-go
    github.com/alexbrainman/odbc
    github.com/amsokol/ignite-go-client
    github.com/apache/arrow/go/arrow
    github.com/apache/calcite-avatica-go/v5
    github.com/apache/thrift
    github.com/beltran/gohive
    github.com/beltran/gosasl
    github.com/bippio/go-impala
    github.com/btnguyen2k/consu/semita
    github.com/btnguyen2k/gocosmos
    github.com/cloudspannerecosystem/go-sql-spanner
    github.com/couchbase/go-couchbase
    github.com/couchbase/go_n1ql
    github.com/couchbase/gomemcached
    github.com/couchbase/goutils
    github.com/exasol/exasol-driver-go
    github.com/genjidb/genji
    github.com/godror/godror
    github.com/gohxs/readline
    github.com/google/goexpect
    github.com/googleapis/gax-go/v2	(github.com/googleapis/gax-go in Debian)
    github.com/jmrobles/h2go
    github.com/mattn/go-adodb
    github.com/mithrandie/csvq
    github.com/mithrandie/csvq-driver
    github.com/mithrandie/go-text
    github.com/nakagami/firebirdsql
    github.com/ory/dockertest/v3
    github.com/prestodb/presto-go-client
    github.com/sijms/go-ora/v2
    github.com/snowflakedb/gosnowflake
    github.com/thda/tds
    github.com/trinodb/trino-go-client
    github.com/uber/athenadriver
    github.com/vertica/vertica-sql-go
    github.com/xo/dburl
    github.com/xo/tblfmt
      github.com/nathan-fiscaletti/consolesize-go
    github.com/yookoala/realpath
    github.com/zaf/temp
    gorm.io/driver/bigquery
    modernc.org/ccgo/v3
    modernc.org/libc
    modernc.org/ql
    modernc.org/sqlite
    sqlflow.org/gohive
    sqlflow.org/gomaxcompute
    github.com/Microsoft/go-winio
    github.com/Microsoft/hcsshim
    github.com/Nvveen/Gotty
    github.com/alecthomas/template
    github.com/beltran/gssapi
    github.com/btnguyen2k/consu/gjrc
    github.com/btnguyen2k/consu/olaf
    github.com/btnguyen2k/consu/reddo
    github.com/cenkalti/backoff/v3	(github.com/cenkalti/backoff in Debian)
    github.com/cespare/xxhash/v2	(github.com/cespare/xxhash in Debian)
    github.com/cloudflare/golz4
    github.com/cncf/udpa/go
    github.com/cncf/xds/go
    github.com/envoyproxy/go-control-plane
    github.com/envoyproxy/protoc-gen-validate
    github.com/exasol/error-reporting-go
    github.com/golang-sql/civil
    github.com/jcmturner/rpc/v2	(github.com/jcmturner/rpc in Debian)
    github.com/mithrandie/go-file/v2
    github.com/mithrandie/readline-csvq
    github.com/mithrandie/ternary
    github.com/pierrec/lz4/v4	(github.com/pierrec/lz4 in Debian)
    github.com/unchartedsoftware/witch
    github.com/xinsnake/go-http-digest-auth-client
    github.com/xwb1989/sqlparser
    gitlab.com/nyarla/go-crypt
    gopkg.in/jcmturner/gokrb5.v6	(gopkg.in/jcmturner/gokrb5.v5 in Debian)
    gopkg.in/jcmturner/rpc.v1
    lukechampine.com/uint128
    modernc.org/b
    modernc.org/cc/v3
    modernc.org/db
    modernc.org/file
    modernc.org/golex
    modernc.org/lldb
    modernc.org/memory
    modernc.org/opt
    modernc.org/sortutil
    modernc.org/strutil
    modernc.org/token
    modernc.org/zappy
  github.com/apache/arrow/go/v14
  github.com/coreos/go-systemd/v22	(github.com/coreos/go-systemd in Debian)
    github.com/godbus/dbus/v5	(github.com/godbus/dbus in Debian)

The usql is one of the largest dependencies with a large number of dependencies of its own.

ottok avatar Apr 10 '25 15:04 ottok

I have the build working on multiple platforms now, but tests not passing. Still requires quite a lot of work.

Image

ottok avatar Apr 11 '25 16:04 ottok

I am seriously struggling with this code base and design of TiUP. It assumes there is always a ~/.tiup directory, a root.json file and a mirror defined. I am not able to override the root.json lookup by simply loading an existing system-wide /usr/share/tiup/root.json as loading the profile, home directory and having the root.json specifically inside the home directory is mandatory through multiple levels of abstractions and simply injecting a custom path in local.go or local_manifests.go won't work.

Also, by default it is not possible to run any commands unless the repository object is loaded. By default, running any command requires network connection even though the command does not do anything that would actually need network, e.g.:

./bin/tiup tiup-client --help
The component `tiup-client` version  is not installed; downloading from repository.
Error: fetch /timestamp.json from mirror(https://tiup-mirrors.pingcap.com/) failed: download from https://tiup-mirrors.pingcap.com/timestamp.json failed: Get "https://tiup-mirrors.pingcap.com/timestamp.json": dial tcp: lookup tiup-mirrors.pingcap.com on 172.20.91.201:53: dial udp 172.20.91.201:53: connect: network is unreachable

The PR 2538 fixes the above issue, but the code base is full of assumptions that are inherently incompatible with installing the binaries at /usr/bin/tiup, /usr/bin/tiup-client etc and allowing users to run them as-is without additional mandatory setup and creating new .tiup directories etc. This makes it very challenging to package this for any Linux distribution.

ottok avatar May 31 '25 10:05 ottok

Generic requirements for a Go program in Linux:

  • Doing a regular Go build should produce the binary using regular Go source code structure and dependencies.
  • The name of the binary needs to be specific for the project, e.g. not just client or server or a.
  • The binary should run as-is without requiring that user has any extra ENV set or extra magic files. Extra stuff is allowed when it enables more functionality, but basic functionality should work as-is (e.g. program --help should produce help text and not an error).
  • It should be possible to run the binary on a read-only filesystem without network. Functionality can be limited, but it should still run.

Currently, tiup can be built, but tiup-client is produced only by make build that has a custom rename step.

Currently, tiup can't run at all without extra magic file (root.json) available, a writeable .tiup directory and live network connectivity. For example tiup --help won't produce any help text unless the listed prerequisited are satisfied. I assumed I could hack around these with a few Go patches, but this structure seems quite deep and a fundamental part of how TiUP is designed.

This will also block #2507 and #2509 as they have the same generic requirements of being able to run a binary without magic ENV or extra files.

I am currently iterating on #2538 to maybe get past these issues, but not sure if I will make progress.

ottok avatar May 31 '25 10:05 ottok

While going through the dependencies I noticed that https://github.com/AstroProfundis/sysinfo is a fork of https://github.com/zcalusic/sysinfo that seems to now be maintained in a PingCAP employee personal GitHub account. As PingCAP relies on this repository, it should probably be in PingCAP's namespace. Alternatively all improvements could be sent upstream to not need to have a fork at all.

ottok avatar Jun 09 '25 14:06 ottok

More notes on dependencies:

  • TiUP depends on old github.com/r3labs/diff/v2 but Debian golang-github-r3labs-diff-dev has only v3, which does not build on TiUP. Would it be feasible to port TiUP to use this newer library version?

    • I tried doing this in https://github.com/ottok/tiup/tree/r3labs-diff-v3 but v3 requires some of the functions to be rewritten.

  • TiUP depends on old github.com/grpc-ecosystem/grpc-gateway v1.16.0 but the Debian golang-github-grpc-ecosystem-grpc-gateway-dev is of version 2.x, and does not build with TiUP. Would it be feasible to port TiUP to use this newer library version?

    • I tried doing this myself in https://github.com/ottok/tiup/tree/grpc-gateway-v2 but quickly got too complicated as the protobuffer definitions need to be regenerated with protoc.
    • Additionally the code needst to be updated to use new functions or build fails with:
# github.com/pingcap/tiup/pkg/cluster/api/dmpb
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1152:141: too many arguments in call to runtime.NewPattern
	have (number, []int, []string, string, unknown type)
	want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1152:149: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1154:169: too many arguments in call to runtime.NewPattern
	have (number, []int, []string, string, unknown type)
	want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1154:177: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1156:170: too many arguments in call to runtime.NewPattern
	have (number, []int, []string, string, unknown type)
	want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1156:178: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1158:147: too many arguments in call to runtime.NewPattern
	have (number, []int, []string, string, unknown type)
	want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1158:155: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1160:199: too many arguments in call to runtime.NewPattern
	have (number, []int, []string, string, unknown type)
	want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1160:207: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1160:199: too many errors

ottok avatar Jun 11 '25 18:06 ottok

Other PingCAP/TiDB specific issues when trying to build dependencies as independent packages for Debian:

golang-github-pingcap-tidb-insight

dh_auto_build -O--builddirectory=debian/.build/upstream -O--buildsystem=golang
	cd debian/.build/upstream && go install -trimpath -v -p 4 github.com/pingcap/tidb-insight/collector github.com/pingcap/tidb-insight/collector/insight github.com/pingcap/tidb-insight/collector/kmsg github.com/pingcap/tidb-insight/docs github.com/pingcap/tidb-insight/tools
src/github.com/pingcap/tidb-insight/collector/insight/insight.go:22:2: cannot find package "github.com/AstroProfundis/sysinfo" in any of:
	/usr/lib/go-1.24/src/github.com/AstroProfundis/sysinfo (from $GOROOT)
	/tmp/build/source/debian/.build/upstream/src/github.com/AstroProfundis/sysinfo (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/syscfg.go:21:2: cannot find package "github.com/lorenzosaino/go-sysctl" in any of:
	/usr/lib/go-1.24/src/github.com/lorenzosaino/go-sysctl (from $GOROOT)
	/tmp/build/source/debian/.build/upstream/src/github.com/lorenzosaino/go-sysctl (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/process.go:10:2: cannot find package "github.com/shirou/gopsutil/cpu" in any of:
	/usr/lib/go-1.24/src/github.com/shirou/gopsutil/cpu (from $GOROOT)
	/tmp/build/source/debian/.build/upstream/src/github.com/shirou/gopsutil/cpu (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/pd.go:11:2: cannot find package "github.com/shirou/gopsutil/process" in any of:
	/usr/lib/go-1.24/src/github.com/shirou/gopsutil/process (from $GOROOT)
	/tmp/build/source/debian/.build/upstream/src/github.com/shirou/gopsutil/process (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/socket.go:16:2: cannot find package "github.com/vishvananda/netlink" in any of:
	/usr/lib/go-1.24/src/github.com/vishvananda/netlink (from $GOROOT)
	/tmp/build/source/debian/.build/upstream/src/github.com/vishvananda/netlink (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/epoll.go:16:2: cannot find package "golang.org/x/sys/unix" in any of:
	/usr/lib/go-1.24/src/golang.org/x/sys/unix (from $GOROOT)
	/tmp/build/source/debian/.build/upstream/src/golang.org/x/sys/unix (from $GOPATH)
src/github.com/pingcap/tidb-insight/tools/prom2influx.go:13:2: cannot find package "github.com/influxdata/influxdb/client/v2" in any of:
	/usr/lib/go-1.24/src/github.com/influxdata/influxdb/client/v2 (from $GOROOT)
	/tmp/build/source/debian/.build/upstream/src/github.com/influxdata/influxdb/client/v2 (from $GOPATH)
src/github.com/pingcap/tidb-insight/tools/prom2influx.go:14:2: cannot find package "github.com/prometheus/common/model" in any of:
	/usr/lib/go-1.24/src/github.com/prometheus/common/model (from $GOROOT)
	/tmp/build/source/debian/.build/upstream/src/github.com/prometheus/common/model (from $GOPATH)

golang-github-pingcap-kvproto

Investigating (0) golang-google-genproto-dev:amd64 < none -> 0.0~git20210726.e7812ac-4 @un uN Ib >
Broken golang-google-genproto-dev:amd64 Depends on golang-github-golang-protobuf-1-5-dev:amd64 < none | 1.5.4-1 @un uH >
  Considering golang-github-golang-protobuf-1-5-dev:amd64 5 as a solution to golang-google-genproto-dev:amd64 11
  Holding Back golang-google-genproto-dev:amd64 rather than change golang-github-golang-protobuf-1-5-dev:amd64
Investigating (0) golang-go.opencensus-dev:amd64 < none -> 0.24.0-2 @un uN Ib >
Broken golang-go.opencensus-dev:amd64 Depends on golang-github-golang-protobuf-1-5-dev:amd64 < none | 1.5.4-1 @un uH >
  Considering golang-github-golang-protobuf-1-5-dev:amd64 5 as a solution to golang-go.opencensus-dev:amd64 8
  Holding Back golang-go.opencensus-dev:amd64 rather than change golang-github-golang-protobuf-1-5-dev:amd64
Investigating (0) golang-google-cloud-dev:amd64 < none -> 0.56.0-6 @un uN Ib >
Broken golang-google-cloud-dev:amd64 Depends on golang-github-golang-protobuf-1-5-dev:amd64 < none | 1.5.4-1 @un uH >
  Considering golang-github-golang-protobuf-1-5-dev:amd64 5 as a solution to golang-google-cloud-dev:amd64 6
  Holding Back golang-google-cloud-dev:amd64 rather than change golang-github-golang-protobuf-1-5-dev:amd64
Investigating (0) golang-github-google-cel-go-dev:amd64 < none -> 0.18.2+ds-5 @un uN Ib >
Broken golang-github-google-cel-go-dev:amd64 Depends on golang-github-golang-protobuf-1-5-dev:amd64 < none | 1.5.4-1 @un uH >
  Considering golang-github-golang-protobuf-1-5-dev:amd64 5 as a solution to golang-github-google-cel-go-dev:amd64 6
  Holding Back golang-github-google-cel-go-dev:amd64 rather than change golang-github-golang-protobuf-1-5-dev:amd64
Investigating (0) golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 < none -> 2.20.0-2 @un uN Ib >
Broken golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 Depends on golang-google-genproto-dev:amd64 < none | 0.0~git20210726.e7812ac-4 @un uH > (>= 0.0~git20210726.e7812ac~)
  Considering golang-google-genproto-dev:amd64 11 as a solution to golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 0
  Holding Back golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 rather than change golang-google-genproto-dev:amd64
Investigating (0) golang-google-api-dev:amd64 < none -> 0.61.0-6 @un uN Ib >
Broken golang-google-api-dev:amd64 Depends on golang-go.opencensus-dev:amd64 < none | 0.24.0-2 @un uH > (>= 0.23.0)
  Considering golang-go.opencensus-dev:amd64 8 as a solution to golang-google-api-dev:amd64 0
  Holding Back golang-google-api-dev:amd64 rather than change golang-go.opencensus-dev:amd64
Investigating (0) golang-github-pingcap-kvproto-build-deps:amd64 < 1.0 @iU K Nb Ib >
Broken golang-github-pingcap-kvproto-build-deps:amd64 Depends on golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 < none | 2.20.0-2 @un uH >
  Considering golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 0 as a solution to golang-github-pingcap-kvproto-build-deps:amd64 -2
  Removing golang-github-pingcap-kvproto-build-deps:amd64 rather than change golang-github-grpc-ecosystem-grpc-gateway-dev:amd64
Investigating (1) golang-google-grpc-dev:amd64 < none -> 1.64.0-7 @un uN Ib >
Broken golang-google-grpc-dev:amd64 Depends on golang-github-google-cel-go-dev:amd64 < none | 0.18.2+ds-5 @un uH >
  Considering golang-github-google-cel-go-dev:amd64 6 as a solution to golang-google-grpc-dev:amd64 14
  Holding Back golang-google-grpc-dev:amd64 rather than change golang-github-google-cel-go-dev:amd64
Investigating (1) golang-github-googleapis-gax-go-dev:amd64 < none -> 2.14.1-1 @un uN Ib >
Broken golang-github-googleapis-gax-go-dev:amd64 Depends on golang-google-grpc-dev:amd64 < none | 1.64.0-7 @un uH >
  Considering golang-google-grpc-dev:amd64 14 as a solution to golang-github-googleapis-gax-go-dev:amd64 1
  Holding Back golang-github-googleapis-gax-go-dev:amd64 rather than change golang-google-grpc-dev:amd64

golang-github-pingcap-errors

=== RUN   TestFormatNew
    format_test.go:38: test 3: line 2: fmt.Sprintf("%+v", err):
         got: "error\ngithub.com/pingcap/errors.TestFormatNew\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:26\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want: "error\ngithub.com/pkg/errors.TestFormatNew\n\t.+/github.com/pkg/errors/format_test.go:26"
    format_test.go:38: test 3: line 3: fmt.Sprintf("%+v", err):
         got: "error\ngithub.com/pingcap/errors.TestFormatNew\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:26\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want: "error\ngithub.com/pkg/errors.TestFormatNew\n\t.+/github.com/pkg/errors/format_test.go:26"
--- FAIL: TestFormatNew (0.00s)
=== RUN   TestFormatErrorf
    format_test.go:64: test 3: line 2: fmt.Sprintf("%+v", err):
         got: "error\ngithub.com/pingcap/errors.TestFormatErrorf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:56\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want: "error\ngithub.com/pkg/errors.TestFormatErrorf\n\t.+/github.com/pkg/errors/format_test.go:56"
    format_test.go:64: test 3: line 3: fmt.Sprintf("%+v", err):
         got: "error\ngithub.com/pingcap/errors.TestFormatErrorf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:56\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want: "error\ngithub.com/pkg/errors.TestFormatErrorf\n\t.+/github.com/pkg/errors/format_test.go:56"
--- FAIL: TestFormatErrorf (0.00s)
=== RUN   TestFormatWrap
    format_test.go:116: test 3: line 2: fmt.Sprintf("%+v", err):
         got: "error\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:82\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
        want: "error\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:82"
    format_test.go:116: test 3: line 3: fmt.Sprintf("%+v", err):
         got: "error\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:82\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
        want: "error\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:82"
    format_test.go:116: test 6: line 3: fmt.Sprintf("%+v", err):
         got: "EOF\nerror\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:96\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want: "EOF\nerror\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:96"
    format_test.go:116: test 6: line 4: fmt.Sprintf("%+v", err):
         got: "EOF\nerror\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:96\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want: "EOF\nerror\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:96"
    format_test.go:116: test 7: line 3: fmt.Sprintf("%+v", err):
         got: "EOF\nerror1\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:103\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
        want: "EOF\nerror1\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:103\n"
    format_test.go:116: test 7: line 4: fmt.Sprintf("%+v", err):
         got: "EOF\nerror1\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:103\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
        want: "EOF\nerror1\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:103\n"
--- FAIL: TestFormatWrap (0.00s)
=== RUN   TestFormatWrapf
    format_test.go:157: test 3: line 3: fmt.Sprintf("%+v", err):
         got: "EOF\nerror2\ngithub.com/pingcap/errors.TestFormatWrapf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:134\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want: "EOF\nerror2\ngithub.com/pkg/errors.TestFormatWrapf\n\t.+/github.com/pkg/errors/format_test.go:134"
    format_test.go:157: test 3: line 4: fmt.Sprintf("%+v", err):
         got: "EOF\nerror2\ngithub.com/pingcap/errors.TestFormatWrapf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:134\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want: "EOF\nerror2\ngithub.com/pkg/errors.TestFormatWrapf\n\t.+/github.com/pkg/errors/format_test.go:134"
    format_test.go:157: test 6: line 2: fmt.Sprintf("%+v", err):
         got: "error\ngithub.com/pingcap/errors.TestFormatWrapf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:149\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
        want: "error\ngithub.com/pkg/errors.TestFormatWrapf\n\t.+/github.com/pkg/errors/format_test.go:149"
    format_test.go:157: test 6: line 3: fmt.Sprintf("%+v", err):
         got: "error\ngithub.com/pingcap/errors.TestFormatWrapf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:149\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
        want: "error\ngithub.com/pkg/errors.TestFormatWrapf\n\t.+/github.com/pkg/errors/format_test.go:149"
--- FAIL: TestFormatWrapf (0.00s)
=== RUN   TestFormatWithStack
    format_test.go:476: test 3: block 2: fmt.Sprintf("%+v", err):
        got:
        "github.com/pingcap/errors.TestFormatWithStack\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:175\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want:
        "github.com/pkg/errors.TestFormatWithStack\n\t.+/github.com/pkg/errors/format_test.go:175"
        all-got:
           EOF
           github.com/pingcap/errors.TestFormatWithStack
        	/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:175
        testing.tRunner
        	/usr/lib/go-1.24/src/testing/testing.go:1792
        runtime.goexit
        	/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700
        all-want:
           EOF
           github.com/pkg/errors.TestFormatWithStack
        	.+/github.com/pkg/errors/format_test.go:175
--- FAIL: TestFormatWithStack (0.00s)
=== RUN   TestFormatWithMessage
    format_test.go:476: test 3: block 2: fmt.Sprintf("%+v", err):
        got:
        "github.com/pingcap/errors.TestFormatWithMessage\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:244\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
        want:
        "github.com/pkg/errors.TestFormatWithMessage\n\t.+/github.com/pkg/errors/format_test.go:244"
        all-got:
           error
           github.com/pingcap/errors.TestFormatWithMessage
        	/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:244
        testing.tRunner
        	/usr/lib/go-1.24/src/testing/testing.go:1792
        runtime.goexit
        	/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700
           error2
        all-want:
           error
           github.com/pkg/errors.TestFormatWithMessage
        	.+/github.com/pkg/errors/format_test.go:244
           error2
--- FAIL: TestFormatWithMessage (0.00s)
=== RUN   TestFrameLine
    stack_test.go:40: Frame(5391080): want: 9, got: 10
    stack_test.go:40: Frame(5433480): want: 20, got: 21
    stack_test.go:40: Frame(5433563): want: 28, got: 24
--- FAIL: TestFrameLine (0.00s)
=== RUN   TestFrameFormat
    stack_test.go:125: test 2: line 1: fmt.Sprintf("%+s", err):
         got: "github.com/pingcap/errors.init\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go"
        want: "github.com/pkg/errors.init\n\t.+/github.com/pkg/errors/stack_test.go"
    stack_test.go:125: test 2: line 2: fmt.Sprintf("%+s", err):
         got: "github.com/pingcap/errors.init\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go"
        want: "github.com/pkg/errors.init\n\t.+/github.com/pkg/errors/stack_test.go"
    stack_test.go:125: test 5: line 1: fmt.Sprintf("%d", err):
         got: "10"
        want: "9"
    stack_test.go:125: test 11: line 1: fmt.Sprintf("%v", err):
         got: "stack_test.go:10"
        want: "stack_test.go:9"
    stack_test.go:125: test 12: line 1: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.init\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:10"
        want: "github.com/pkg/errors.init\n\t.+/github.com/pkg/errors/stack_test.go:9"
    stack_test.go:125: test 12: line 2: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.init\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:10"
        want: "github.com/pkg/errors.init\n\t.+/github.com/pkg/errors/stack_test.go:9"
--- FAIL: TestFrameFormat (0.00s)
=== RUN   TestFuncname
--- PASS: TestFuncname (0.00s)
=== RUN   TestStackTrace
    stack_test.go:203: test 1: line 1: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:155"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:154"
    stack_test.go:203: test 1: line 2: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:155"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:154"
    stack_test.go:203: test 2: line 1: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:160"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:159"
    stack_test.go:203: test 2: line 2: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:160"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:159"
    stack_test.go:203: test 3: line 1: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:165"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:164"
    stack_test.go:203: test 3: line 2: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:165"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:164"
    stack_test.go:203: test 4: line 1: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace.func1\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:170"
        want: "github.com/pkg/errors.(func·009|TestStackTrace.func1)\n\t.+/github.com/pkg/errors/stack_test.go:169"
    stack_test.go:203: test 4: line 2: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace.func1\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:170"
        want: "github.com/pkg/errors.(func·009|TestStackTrace.func1)\n\t.+/github.com/pkg/errors/stack_test.go:169"
    stack_test.go:203: test 4: line 1: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:170"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:169"
    stack_test.go:203: test 4: line 2: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:170"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:169"
    stack_test.go:203: test 5: line 1: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace.TestStackTrace.func2.func3\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:179"
        want: "github.com/pkg/errors.(func·010|TestStackTrace.func2.1)\n\t.+/github.com/pkg/errors/stack_test.go:178"
    stack_test.go:203: test 5: line 2: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace.TestStackTrace.func2.func3\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:179"
        want: "github.com/pkg/errors.(func·010|TestStackTrace.func2.1)\n\t.+/github.com/pkg/errors/stack_test.go:178"
    stack_test.go:203: test 5: line 1: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace.func2\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:180"
        want: "github.com/pkg/errors.(func·011|TestStackTrace.func2)\n\t.+/github.com/pkg/errors/stack_test.go:179"
    stack_test.go:203: test 5: line 2: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace.func2\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:180"
        want: "github.com/pkg/errors.(func·011|TestStackTrace.func2)\n\t.+/github.com/pkg/errors/stack_test.go:179"
    stack_test.go:203: test 5: line 1: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:181"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:180"
    stack_test.go:203: test 5: line 2: fmt.Sprintf("%+v", err):
         got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:181"
        want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:180"
--- FAIL: TestStackTrace (0.00s)
=== RUN   TestStackTraceFormat
    stack_test.go:276: test 11: line 2: fmt.Sprintf("%+v", err):
         got: "\ngithub.com/pingcap/errors.stackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:211\ngithub.com/pingcap/errors.TestStackTraceFormat\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:262"
        want: "\ngithub.com/pkg/errors.stackTrace\n\t.+/github.com/pkg/errors/stack_test.go:210\ngithub.com/pkg/errors.TestStackTraceFormat\n\t.+/github.com/pkg/errors/stack_test.go:261"
    stack_test.go:276: test 11: line 3: fmt.Sprintf("%+v", err):
         got: "\ngithub.com/pingcap/errors.stackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:211\ngithub.com/pingcap/errors.TestStackTraceFormat\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:262"
        want: "\ngithub.com/pkg/errors.stackTrace\n\t.+/github.com/pkg/errors/stack_test.go:210\ngithub.com/pkg/errors.TestStackTraceFormat\n\t.+/github.com/pkg/errors/stack_test.go:261"
    stack_test.go:276: test 11: line 4: fmt.Sprintf("%+v", err):
         got: "\ngithub.com/pingcap/errors.stackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:211\ngithub.com/pingcap/errors.TestStackTraceFormat\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:262"
        want: "\ngithub.com/pkg/errors.stackTrace\n\t.+/github.com/pkg/errors/stack_test.go:210\ngithub.com/pkg/errors.TestStackTraceFormat\n\t.+/github.com/pkg/errors/stack_test.go:261"
    stack_test.go:276: test 11: line 5: fmt.Sprintf("%+v", err):
         got: "\ngithub.com/pingcap/errors.stackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:211\ngithub.com/pingcap/errors.TestStackTraceFormat\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:262"
        want: "\ngithub.com/pkg/errors.stackTrace\n\t.+/github.com/pkg/errors/stack_test.go:210\ngithub.com/pkg/errors.TestStackTraceFormat\n\t.+/github.com/pkg/errors/stack_test.go:261"
    stack_test.go:276: test 12: line 1: fmt.Sprintf("%#v", err):
         got: "[]errors.Frame{stack_test.go:211, stack_test.go:270}"
        want: "\\[\\]errors.Frame{stack_test.go:210, stack_test.go:269}"
--- FAIL: TestStackTraceFormat (0.00s)

ottok avatar Jun 25 '25 07:06 ottok

Ttest PPA is now available:

sudo add-apt-repository ppa:otto/tidb
sudo apt update

ottok avatar Jun 25 '25 07:06 ottok

Notes from chat with @dveeden

  • Daniel will take a look into updating r3labs/diff v2 -> v3 and grpc-gateway v1 -> v2.
  • TiUP has had a policy of not updating dependencies proactively, but only if there are specific errors that absolutely require dependencies to be updates. This explains why many dependencies are outdated, and the Debian packaging may serve as the reason to update some of the dependencies now.
  • Daniel will also check what is the status of these dependencies: github.com/pingcap/check v0.0.0-20211026125417-57bd13f7b5f0 github.com/pingcap/errors v0.11.5-0.20201126102027-b0a155152ca3 github.com/pingcap/failpoint v0.0.0-20240528011301-b51a646c7c86 github.com/pingcap/fn v1.0.0 github.com/pingcap/kvproto v0.0.0-20230331024443-349815129e6d github.com/pingcap/tidb-insight/collector v0.0.0-20220902034607-fb5ae0ddc8c1 github.com/pkg/errors v0.9.1
    • Maybe they should be updated to latest to ensure TiUP benefits from fixes done in them, and also to avoid risk of anybody re-doing fixes on old versions that actually already existed in latest version?
    • Maybe the PingCAP maintained modules themselves should have some updates, or at least clarified if the latest tagged version or latest mainline commit should be used?
    • For the pingcap/errors library, consider removing references to original pkg/errors, and in TiUP stop depending direcly on pkg/errors to not have two libraries that are basically the same?

ottok avatar Jun 25 '25 08:06 ottok

Ttest PPA is now available:

sudo add-apt-repository ppa:otto/tidb
sudo apt update

This doesn't seem to work with a debian:latest or ubuntu:latest container.

It does work with a ubuntu:questing container with these steps:

apt update
apt install software-properties-common
add-apt-repository ppa:otto/tidb
apt update
apt install tiup

Some things I noticed:

  • Having a marker that this is OS installed in tiup --version might be useful
  • The tiup --version output doesn't show the git hash etc
  • Running tiup client doesn't seem to be ale to find/execute tiup-client
  • tiup-client is directly in the path, but probably this shouldn't be the case as it should not be called directly.
  • Maybe tiup completion bash things could be shipped as a file in /etc/bash_completion.d/ ?

dveeden avatar Jun 25 '25 08:06 dveeden

For golang-github-pingcap-errors:

  • I get a clean go test on current master
  • The latest release/tag is from a long time ago
  • The README needs some attention as too much of it is copied from upstream
  • Upstream seems to have stopped, the repo is archived.
  • Some (or all?) of this could be replaced with newer error functionality in Go

dveeden avatar Jun 26 '25 05:06 dveeden

The usage for this:

dvaneeden@dve-carbon:~/dev/pingcap/tiup$ git grep -E '(errors|perrs)\..*\(' | sed -E 's/^.*(errors|perrs)\.([A-Z][a-zA-Z]*).*$/errors.\2/' | sort | uniq -c | sort -rn
    129 errors.Annotatef
    123 errors.Errorf
    120 errors.Trace
    107 errors.New
     83 errors.AddStack
     68 errors.Is
     65 errors.Annotate
     60 errors.Cause
      7 errors.WithStack
      1 errors.As

This summary mixes pingcap/errors with errors from the standard library.

dveeden avatar Jun 26 '25 05:06 dveeden

Some updates / notes:

  • I updated https://github.com/pingcap/tiup/pull/2548 to fix a merge conflict
  • I created https://github.com/pingcap/tiup/pull/2552 (not directly related to OS packaging)
  • Looks like replace rule in go.mod for go-pretty is needed for https://github.com/jedib0t/go-pretty/pull/318 which didn't make it upstream. We should see if we can switch to upstream or move this to the pingcap org
  • Looks like the replace rule for yaml.v2 is needed because the upstream is abandoned/unmaintained. Not sure what the best way forward is. #2558
  • We should move away from pingcap/check to something else. I've created https://github.com/pingcap/tiup/issues/2554 for this
  • For pingcap/tidb-insight/collector: I think we should move this to the tiup repo as it isn't actively used elsewhere afaik and it also doensn't seem to be really maintained. I've created https://github.com/pingcap/tiup/pull/2553 for this
  • Created #2560

dveeden avatar Jul 11 '25 09:07 dveeden

https://github.com/pingcap/tiup/pull/2569 should help to get rid of grpc-gateway

dveeden avatar Aug 01 '25 13:08 dveeden

I pushed now tiup_1.16.2+20250802~f57cbe37 to my test PPA at https://launchpad.net/~otto/+archive/ubuntu/tidb. The package is only buildable for Ubuntu devel (questing) as it depends on new dependencies not yet in any actual release.

The package can be tested in e.g. a Ubuntu devel container with:

add-apt-repository ppa:otto/tidb
apt update
apt install tiup

Unfortunately the tiup commands that depend on the real-time update check are segfaulting due to a bug I have in #2538, but the point here is now to demonstrate the build and dependency management.

Thanks to the latest changes on master, I was able to remove these vendored dependencies:

  • "github.com/AstroProfundis/sysinfo"
  • "github.com/gibson042/canonicaljson-go"
  • "github.com/joomcode/errorx"
  • "github.com/pingcap/fn"
  • "github.com/pingcap/tidb-insight"
  • "github.com/r3labs/diff"
  • "github.com/relex/aini"
  • "github.com/grpc-ecosystem/grpc-gateway"

ottok avatar Aug 09 '25 01:08 ottok

@ottok so these dependencies changed from vendored to non-vendored?

dveeden avatar Aug 20 '25 05:08 dveeden

@ottok so these dependencies changed from vendored to non-vendored?

No, they were removed completely.

Thanks for https://github.com/pingcap/tiup/commit/f6aa0ac09b9d6405929c2cea3c0207880e53d199 too, now we can rely on yaml v3 alone.

ottok avatar Aug 20 '25 05:08 ottok

Are you sure the list is correct? Some of these are still in the go.mod on the master branch

dveeden avatar Aug 20 '25 05:08 dveeden

Are you sure the list is correct? Some of these are still in the go.mod on the master branch

You are right, sorry for getting mixed in up in dependency tracking. The only dependencies we got completely rid of was pingcap/tidb-insight and grpc-ecosystem/grpc-gateway.

While working on the latest git head of TiUP I came across a new finding about very outdated transitive dependency zaf/temp in #2588

ottok avatar Aug 22 '25 19:08 ottok

Didn't we also get rid of pingcap/check and maybe some others?

dveeden avatar Aug 25 '25 07:08 dveeden

Alright, so recent changes on TiUP git master branch helped get rid of: golang-github-alecthomas-assert-dev, golang-github-grpc-ecosystem-grpc-gateway-dev, golang-github-otiai10-copy-dev and golang-github-pingcap-check-dev.

Names are Debian package names and screenshot below shows diff between earlier Debian packaging prototype and latest version, which also has new dependencies after I've packaged them for Debian.

Image

Latest proto version available in the TiUP PPA. Build logs at https://launchpad.net/~otto/+archive/ubuntu/tidb/+builds?build_text=&build_state=all show what went into it recently.

ottok avatar Aug 26 '25 08:08 ottok

FYI @dveeden I posted now #2602 and also non-native official packaging draft at https://salsa.debian.org/go-team/packages/tiup/-/merge_requests/1

ottok avatar Sep 10 '25 21:09 ottok