when project which has a retention policy deleted, the policy is still in db

[j4ckstraw]

If you are reporting a problem, please make sure the following information are provided:

Expected behavior and actual behavior: A clear and concise description of what you expected to happen and what's the actual behavior. If applicable, add screenshots to help explain your problem.

when project which has a retention policy deleted, the policy is still in db. does this need clean up?

Steps to reproduce the problem: Please provide the steps to reproduce this problem.

1. create a project
2. create a retention policy with given project
3. delete the project
4. get retention by /retentions/{id} api

Versions: Please specify the versions of following systems.

• harbor version: [2.3.1]
• docker engine version: [20.10.7]
• docker-compose version: [1.24.1]

Additional context:

• Harbor config files: You can get them by packaging harbor.yml and files in the same directory, including subdirectory.
• Log files: You can get them by package the /var/log/harbor/ .

[wy65701436]

Because we don't use the project id as the forign key for retention, and I agree this should be an issue.

[IwanSE]

Hi all! 🐈

Up to this problem.

Harbor version: v2.7.1-6015b3ef Installation: Kubernetes v1.25.6 Helm Chart version: 1.11.1

I'm trying to create retentions via API.

1. create a project
2. create a retention policy with given project

My case - Problem:

After curl -X 'DELETE' https://harbor.x.x/api/v2.0/retentions/9

I can't create any policy in project.

API return: 500 http code.

{
  "errors": [
    {
      "code": "UNKNOWN",
      "message": "internal server error"
    }
  ]
}

harbor-core:

2023-04-19T20:08:26Z [DEBUG] [/server/middleware/security/session.go:47][requestID="cfe786a440324bd3e9271c71a10e6e23"]: a session security context generated for request POST /api/v2.0/retentions
2023-04-19T20:08:26Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:35]: system administrator xxx require create action for resource /project/143/tag-retention
2023-04-19T20:08:26Z [DEBUG] [/pkg/cached/project_metadata/redis/manager.go:85]: get project 143 metadata from cache error: key not found:redis: nil, will query from database.
2023-04-19T20:08:26Z [DEBUG] [/lib/http/error.go:61]: {"errors":[{"code":"BAD_REQUEST","message":"bad request: project 143 already has retention policy 9"}]}

Total:

1. If delete policy in UI - API request simple json:

{
  "id": 0,
  "algorithm": "or",
  "rules": [],
  "trigger": {
    "kind": "Schedule",
    "settings": {
      "cron": ""
    }
  },
  "scope": {
    "level": "project",
    "ref": 143
  }
}

2. If delete via API - deleted from database and can't create any policy in project. 2.1 fixed copy database row and give:

9,project,143,Schedule,"{""id"":0,""algorithm"":""or"",""rules"":[],""trigger"":{""kind"":""Schedule"",""settings"":{""cron"":""""}},""scope"":{""level"":""project"",""ref"":143}}",2023-04-19 22:04:20.790783,2023-04-19 22:32:58.246354

3. After 2.1 - 500 errors are gone - and new policies can be created.

Dear authors @wy65701436, @Vad1mo, @OrlinVasilev - is it possible to fix this in 2.8.1? or other versions. we will be very happy, it will work for us.

I'll check this later on version 2.8.0 - if the problem remains, I'll write.

[david-caro]

I think this might be causing issues on our instance by spamming the scheduler job task.

We had many (>3k) projects created, each with their own retention policy, that we deleted, but the retentions are still in the schedule and the schedule runner seems to go over all of them even if the project is deleted.

harbor=> select count(*) from project as p join project_metadata as pm on p.project_id=pm.project_id where p.deleted='t' and pm.name='retention_id';
 count 
-------
  3189
(1 row)

Is there a way to list all the retention policies without the project id? (as they are deleted, now the API /project will not return those projects, and /retentions does not list the retentions, just allows creating them).

[Vad1mo]

@david-caro what Harbor version are you using?

[david-caro]

@david-caro what Harbor version are you using?

Currently v2.9.0-6d1ad65c