service_test脚本检测失败，疑似没有生成txt文件

[anyuan95]

服务启动后，执行service_test脚本失败。 手动请求healthcheck接口，检查目录下也并没有生成${timestamp}.txt文件。 但是检查日志发现，能够正常输出[AnalysisServerHealthCheck]日志。

[company@set-xr-ed-wca-server-test01 codeanalysis_server]$ sh -x deploy/service_test.sh 
+ echo '[TCAServerHealthCheck] *start detect status of every service*'
[TCAServerHealthCheck] *start detect status of every service*
+ main_server_detect
++++ dirname deploy/service_test.sh
+++ cd deploy
+++ pwd
++ dirname /opt/company/codeanalysis_server/deploy
+ current_path=/opt/company/codeanalysis_server
+ file_path=/opt/company/codeanalysis_server/projects/main
+ delete_txt_file /opt/company/codeanalysis_server/projects/main
+ rm -rf '/opt/company/codeanalysis_server/projects/main/*.txt'
++ date +%s
+ current_timestamp=1650360653
+ file_path=/opt/company/codeanalysis_server/projects/main/1650360653.txt
+ target='http://0.0.0.0:8000/main/healthcheck/?file_name=1650360653'
++ curl -I -s --connect-timeout 5 'http://0.0.0.0:8000/main/healthcheck/?file_name=1650360653' -w '%{http_code}'
++ tail -n1
+ ret_code=200
+ [[ x200 == \x\2\0\0 ]]
+ echo '[TCAServerHealthCheck] *service main db check pass*'
[TCAServerHealthCheck] *service main db check pass*
+ (( i=0 ))
+ (( i<3 ))
+ sleep 5
+ '[' -f /opt/company/codeanalysis_server/projects/main/1650360653.txt ']'
+ (( i++ ))
+ (( i<3 ))
+ sleep 5
+ '[' -f /opt/company/codeanalysis_server/projects/main/1650360653.txt ']'
+ (( i++ ))
+ (( i<3 ))
+ sleep 5
+ '[' -f /opt/company/codeanalysis_server/projects/main/1650360653.txt ']'
+ (( i++ ))
+ (( i<3 ))
+ echo -e '\e[31m❌ service main failed, reason might be celery has not started\e[0m'
❌ service main failed, reason might be celery has not started
+ exit -1
[company@set-xr-ed-wca-server-test01 codeanalysis_server]$ curl -I -s --connect-timeout 5 'http://0.0.0.0:8000/main/healthcheck/?file_name=99999999' -w '%{http_code}'
HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Tue, 19 Apr 2022 09:32:10 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 32
Connection: keep-alive
Vary: Accept, Origin
Allow: GET, HEAD, OPTIONS
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin

200[company@set-xr-ed-wca-server-test01 codeanalysis_server]$ ll projects/main/
total 44
drwxr-xr-x 9 company company 4096 Apr 19 16:51 apps
drwxr-xr-x 4 company company 4096 Apr 19 16:51 codedog
drwxr-xr-x 2 company company 4096 Apr 19 17:04 log
-rw-r--r-- 1 company company  975 Mar 20 14:30 main.gunicorn.conf.py
-rw-r--r-- 1 company company    5 Apr 19 17:04 main-master.pid
-rw-r--r-- 1 company company  629 Apr  1 17:31 manage.py
-rw-r--r-- 1 company company    0 Apr 19 17:04 nohup_beat.out
-rw-r--r-- 1 company company 1376 Apr 19 17:04 nohup_worker.out
drwxr-xr-x 2 company company 4096 Apr 19 16:51 __pycache__
-rw-r--r-- 1 company company  182 Mar 20 14:30 README.md
drwxr-xr-x 2 company company 4096 Mar 20 14:30 static
drwxr-xr-x 6 company company 4096 Apr 19 16:51 util
[company@set-xr-ed-wca-server-test01 codeanalysis_server]$ find . -name "*.txt"
./configs/requirements.txt
./projects/scmproxy/requirements.txt
./projects/file/requirements.txt
./projects/login/requirements.txt

main目录下 tail -f *.log日志输出：

==> codedog.log <==
-2022-04-19 17:35:50,595-INFO-apps.base.apis: [MainServerHealthCheck] Step 1: check db connection and orm operation

==> codedog_info.log <==
-2022-04-19 17:35:50,595-INFO-apps.base.apis: [MainServerHealthCheck] Step 1: check db connection and orm operation

==> codedog.log <==
-2022-04-19 17:35:50,597-INFO-apps.base.apis: [MainServerHealthCheck] Step 2: check celery status and asynchronous tasks

==> codedog_info.log <==
-2022-04-19 17:35:50,597-INFO-apps.base.apis: [MainServerHealthCheck] Step 2: check celery status and asynchronous tasks

==> gunicorn_access.log <==
[19/Apr/2022:17:35:50 +0800] <4114> 127.0.0.1 'HEAD /healthcheck/?file_name=99999999 HTTP/1.0' 200 0.014728 32 -' 'curl/7.29.0'

此时请求main服务的其他api接口，也是正常的，例如：

curl -X GET --location "http://127.0.0.1:8000/main/api/orgs/LqWw4PxurQU/teams/COMMON/repos/" \
    -H "Authorization: Token 0712b895f30c5e958ec71a7c22e1b1a2ad1d5c6b" 

代码版本：07f6cf3d

[anyuan95]

BTW，main_celery.log里有以下报错信息，不知道是否有关？

[2022-04-19 17:17:50,027: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x7f936e1e11e0> (args:('apps.codeproj.tasks.base.handle_scheduled_projects', '70f1ea21-5644-4603-a34a-48b1e4b    fe770', {'lang': 'py', 'task': 'apps.codeproj.tasks.base.handle_scheduled_projects', 'id': '70f1ea21-5644-4603-a34a-48b1e4bfe770', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_    index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '70f1ea21-5644-4603-a34a-48b1e4bfe770', 'parent_id': None, 'argsrepr': '()', 'kwargsrepr': '{}', 'origin': 'gen792@set-gh-ed-wca-serv    er-test06.mt', 'ignore_result': False, 'properties': {'correlation_id': '70f1ea21-5644-4603-a34a-48b1e4bfe770', 'reply_to': '7672141e-46ae-36f9-8a43-0bec8a606dc2', 'delivery_mode': 2, 'delivery_info'    : {'exchange': '', 'routing_key': 'celery'}, 'priority': 0, 'body_encoding': 'base64', 'delivery_tag': '410bfaf9-8cb9-441e-b143-f3d60a1be01e'}, 'reply_to': '7672141e-46ae-36f9-8a43-0bec8a606dc2', 'co    rrelation_id': '70f1ea21-5644-4603-a34a-48b1e4bfe770', 'hostname': '[email protected]', 'delivery_info': {'exchange':... kwargs:{})
[2022-04-19 17:17:50,053: ERROR/ForkPoolWorker-13] Task apps.codeproj.tasks.base.handle_scheduled_projects[70f1ea21-5644-4603-a34a-48b1e4bfe770] raised unexpected: AttributeError("module 'apps.codepr    oj.models' has no attribute 'ScanSchedule'")
Traceback (most recent call last):
  File "/home/company/.local/lib/python3.7/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/company/.local/lib/python3.7/site-packages/celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
  File "/docker/opt/company/codeanalysis_server/projects/main/apps/codeproj/tasks/base.py", line 123, in handle_scheduled_projects
    project_schedules = models.ScanSchedule.objects.filter(
AttributeError: module 'apps.codeproj.models' has no attribute 'ScanSchedule'

[RussGan]

麻烦执行ps -aux |grep celery命令，检查一下各服务的celery是否正常启动

[RussGan]

脚本监测及服务已更新，麻烦拉去最新代码后重试哈