zhparser icon indicating copy to clipboard operation
zhparser copied to clipboard

在高版本的PostgreSQL上执行可能会出现字典创建失败的异常

Open chasoneil opened this issue 1 year ago • 3 comments

replay condition

  1. scws v 1.2.3 & zhparser-master & postgresql v15.2
  2. install zhparser
[postgres@localhost bin]$ ./psql 
psql (15.2)
Type "help" for help.

postgres=# \c test_zhpr;
You are now connected to database "test_zhpr" as user "postgres".
test_zhpr=# create extension zhparser;
CREATE EXTENSION
test_zhpr=# SELECT * FROM ts_parse('zhparser', '保障房资金压力');
NOTICE:  zhparser add dict : "/chason/postgres/code/pg15/data/base/zhprs_dict_test_zhpr.txt" failed!
 tokid | token 
-------+-------
   118 | 保障
   110 | 房
   110 | 资金
   110 | 压力
(4 rows)

There is a error print after execute SQL.

Reason

postgreSQL 15.2 and other versions base directory path changed!

[postgres@localhost base]$ ls 
1  16394  4  5
[postgres@localhost base]$ pwd
/chason/postgres/code/pg15/data/base

there is more level by database id. so we need to change code from zhparser.c

/**
	 * %s datadir is the data directory of the current database
	 * %s get_database_name(MyDatabaseId) is the name of the current database
	*  add mydatabaseId to path
	 */
	snprintf(dict_path, MAXPGPATH, "%s/base/%d/zhprs_dict_%s.txt",
			DataDir, MyDatabaseId, get_database_name(MyDatabaseId));

and then, there is no error after execute the SQL.

[postgres@localhost bin]$ ./psql 
psql (15.2)
Type "help" for help.

postgres=# \c test_zhpr;
You are now connected to database "test_zhpr" as user "postgres".
test_zhpr=# create extension zhparser;
CREATE EXTENSION
test_zhpr=# SELECT * FROM ts_parse('zhparser', '保障房资金压力');
 tokid | token 
-------+-------
   118 | 保障
   110 | 房
   110 | 资金
   110 | 压力
(4 rows)

chasoneil avatar Dec 18 '24 07:12 chasoneil

我也遇到了这个问题,我尝试在对应路径下手动创建一份文件,也无法解决

SharkSyl avatar Jan 24 '25 03:01 SharkSyl

postgres=# create extension zhparser
postgres-# ;
CREATE EXTENSION
postgres=# select sync_zhprs_custom_word();
 sync_zhprs_custom_word
------------------------

(1 row)

postgres=# select to_tsvector('zhparsercfg', '你是AI助手云计算泽阳');
ERROR:  text search configuration "zhparsercfg" does not exist
LINE 1: select to_tsvector('zhparsercfg', '你是AI助手云计算泽阳');
                           ^
postgres=# CREATE TEXT SEARCH CONFIGURATION zhparsercfg (PARSER = zhparser);
CREATE TEXT SEARCH CONFIGURATION
postgres=# ALTER TEXT SEARCH CONFIGURATION zhparsercfg ADD MAPPING FOR n,v,a,i,e,l WITH simple;
ALTER TEXT SEARCH CONFIGURATION
postgres=# select to_tsvector('zhparsercfg', '你是AI助手云计算泽阳');
                  to_tsvector
-----------------------------------------------
 'ai':2 '云':4 '助手':3 '是':1 '泽':6 '计算':5
(1 row)

Image

SharkSyl avatar Jan 24 '25 05:01 SharkSyl

最新的2.3版本应该解决了这个问题: https://github.com/amutu/zhparser/releases/tag/v2.3

amutu avatar Jan 24 '25 09:01 amutu