ch-go icon indicating copy to clipboard operation
ch-go copied to clipboard

feat: support Time/Time64 data types

Open ernado opened this issue 6 months ago • 1 comments

Types are introduced in 25.6.

https://github.com/ClickHouse/ClickHouse/pull/81217

ernado avatar Jun 26 '25 14:06 ernado

Hi Aleksandr, I'd like to give this a try.

I've reviewed some of the older type-related commits, and I think I understand the pattern.

  1. Create the type
  2. Create the column type (satisfy interfaces)
  3. Write the tests
  4. Generate _gen files using the cmd/ch-gen-col/main.go
  5. Run the tests

Let me know if there's anything specific you'd like me to keep in mind before I start, or if you’re already working on this.

Thanks!

tosolveit avatar Jun 28 '25 16:06 tosolveit

Hey @tosolveit, I’m not working on this so you can pick this issue. Will be happy to help and review.

Looks like implementation will be similar to DateTime/DateTime64 in some way.

Your plan looks ok, go ahead.

ernado avatar Jun 28 '25 23:06 ernado

Hey, I did couple of things to make it work, I'm having a problem when generating the _gen files:

Here is the summary:

  1. Add new type variants to proto/cmd/ch-gen-col/main.go
		{ // Time32
			Bits:   32,
			Signed: true,
			Kind:   KindDateTime,
		},
		{ // Time64
			Bits:   64,
			Signed: false,
			Kind:   KindDateTime,
		}
  1. Implement time32.go and time64.go types (time32 is clickhouse's Time type, I can rename it later)
  2. Implement col_time32.go and col_time64.go
  • Add new constants to the column.go
	ColumnTypeTime64   ColumnType = "Time64"
	ColumnTypeTime32   ColumnType = "Time32"

In the proto/cmd/ch-gen-col/main.go

  • unsafe_gen will not be created because it is false for both types
  • _gen will be skipped
  • _safe_gen has to be created
  • _gen_test has to be created

based on the logic below

	for _, v := range variants {
		if !v.Byte() {
			v.GenerateUnsafe = true
		}
		base := "col_" + v.ElemLower()
		if v.Kind == KindFixedStr {
			base = "col_fixedstr" + strconv.Itoa(v.Bytes())
		}
		if !v.DateTime() {
			if err := write(base+"_gen", v, tpl); err != nil {
				return errors.Wrap(err, "write")
			}
		}
		if err := write(base+"_safe_gen", v, tplSafe); err != nil {
			return errors.Wrap(err, "write")
		}
		if v.GenerateUnsafe {
			if err := write(base+"_unsafe_gen", v, tplUnsafe); err != nil {
				return errors.Wrap(err, "write")
			}
		}
		if err := write(base+"_gen_test", v, tplTest); err != nil {
			return errors.Wrap(err, "write test")
		}
	}

unfortunately when I run go run cmd/ch-gen-col/main.go I don't see the expected files, so maybe I'm missing something obvious, do you have any suggestions, thanks

tosolveit avatar Jul 02 '25 20:07 tosolveit

I think you can try adding another kind like KindTime instead of KindDateTime and updating templates.

Also you can create WIP draft PR so we can already iterate over it.

ernado avatar Jul 03 '25 10:07 ernado

@ernado I took some progress, I had to change the main.go and some tmpl files and while introducing Time64 & Time32 I noticed new bugs related to templates, I only committed the files that are related to Time64 & Time32 not the tmpl file changes, I need a few more days to fix those and finalize the PR.

tosolveit avatar Jul 08 '25 22:07 tosolveit

@ernado kindly correct me if I am wrong,

to get gen files working for Time/Time64, the generator would need to support custom encode/decode logic for types that require conversion to/from Go’s time.Time.

and hence, I suggest we stick with a manual implementation for these types, just like we do for DateTime.

shivanshuraj1333 avatar Jul 11 '25 15:07 shivanshuraj1333

Hi @ernado Generators support custom encode/decode for these types and I didn't push those changes yet, I made tmpl modifications that works for time64 and time32 , but I noticed that there are a few bugs, tomorrow I have time to sit and get rid of these bugs, once I'm confident with the templates I will finalize and push them.

Are we on the same page? Please correct me if my direction is not correct.

tosolveit avatar Jul 11 '25 16:07 tosolveit

I wanted to wrap up https://github.com/ClickHouse/clickhouse-go/pull/1596 and hence I started working on https://github.com/ClickHouse/ch-go/pull/1074 both the changes are complete and are open for reviews. Apologies @tosolveit but I wanted to wrap it up, if any of the changes gets merged in ch-go to support time and time64, it should be okay because then the support on the actual client can be tested.

@ernado please help reviewing my changes whenever you've time, this is my first time contributing to ch-go so I may have missed something.

thanks!

shivanshuraj1333 avatar Jul 12 '25 08:07 shivanshuraj1333