hive-testbench icon indicating copy to clipboard operation
hive-testbench copied to clipboard

Published 20 hours ago • verified publisher icon hortonworks

hive-testbench doesn't generate database

Open vivian5-git opened this issue 4 years ago • 9 comments

Hello,

I'm using hive-testbench-hdp3 on HDP3.1.4. A bunch of files were generated in target HDFS directory after running tpcds-setup.sh. However no database was generated. Any advice to address the issue?

My steps are like:

  1. run tpcds-build.sh

  2. run 'FORMAT=parquet ./tpcds-setup.sh 10 /benchmarks/tpcds'. Note it reported an error as below

TPC-DS text data generation complete. Loading text data into external tables. make: *** [date_dim] Error 1 Loading constraints Data loaded into database tpcds_bin_partitioned_parquet_10.

  1. check databses in hive - there is no new databse generated.

vivian5-git avatar Dec 02 '20 06:12 vivian5-git

Most likely beeline command needs to be modified in line no 73 of tpcds-setup.sh as per your setup. Add DEBUG_SCRIPT=ON to get verbose logging, that should help you to resolve your issue.

amitagarwal06 avatar Dec 02 '20 08:12 amitagarwal06

Thanks for the advice. I ran the commands in tpcds-setup.sh manually. It seems the issue is about "make -j 1 -f $LOAD_FILE". This line returns error as below. Do you know what the problem is about?

No implicit rule found for all'. Considering target file date_dim'. File date_dim' does not exist. Finished prerequisites of target file date_dim'. Must remake target date_dim'. Invoking recipe from load_parquet_2.mk:3 to update target date_dim'. Putting child 0x24cc530 (date_dim) PID 21140 on the chain. Live child 0x24cc530 (date_dim) PID 21140 Reaping losing child 0x24cc530 PID 21140 make: *** [date_dim] Error 1 Removing child 0x24cc530 PID 21140 from chain.

vivian5-git avatar Dec 03 '20 04:12 vivian5-git

Finally I got all commands run successfully manually. Thanks.

vivian5-git avatar Dec 03 '20 07:12 vivian5-git

Hi @vivian5-git - how did you resolve the issue? I am facing similar issue.

ssainz avatar Mar 09 '21 14:03 ssainz

Finally I got all commands run successfully manually. Thanks.

Where did you modify it

ergouy avatar Mar 24 '21 03:03 ergouy

how did you resolve the issue? I am facing similar issue.

nuclearwu avatar Jul 20 '21 07:07 nuclearwu

About this issue, I think you can refer to: https://community.cloudera.com/t5/Support-Questions/Not-able-to-run-hive-benchmark-test/m-p/137781

lyzGithub avatar Jul 23 '21 02:07 lyzGithub

The process of loading is text file table->orc table. The default table type (hive.default.fileformat) of my hive environment is orc, then the text file table ddl may become orc type by mistake, and then the same error will occur. This problem can be solved by changing the default type setting in /settings/load-flat.sql: Set hive.default.fileformat=TextFile

huanghuazhang avatar Dec 06 '23 07:12 huanghuazhang

you can find the error message in hive server2 log.

bazingagain avatar Jul 09 '24 03:07 bazingagain