vulnerablecode icon indicating copy to clipboard operation
vulnerablecode copied to clipboard

Add initial support for the VulnerableCode agent

Open ziadhany opened this issue 11 months ago • 2 comments

The VulnerableCode agent currently focuses on one main task: extracting the correct version range from the vulnerability summary.

image

Screenshot from 2025-02-10 07-07-49

Screenshot from 2025-02-10 06-57-58

ziadhany avatar Feb 10 '25 06:02 ziadhany

@pombredanne, this is an initial base for the AI summary improver:

Right now, we have two prompts—one to extract the purl and another to get the affected_versions and fixed_versions—without using RAG.

I think I should also feed the model with agent/purl_db/PURL.rst so it can generate more accurate results. I have already implemented the basics of this step.

However, I encountered a small issue related to testing and evaluating our improver because the model sometimes returns a different output each time.

How should we approach testing it?

There’s just a little work left, and I think this improver will be ready soon.

Input Summary:

Off-by-one error in the apr_brigade_vprintf function in Apache APR-util before 1.3.5
              on big-endian platforms allows remote attackers to obtain sensitive information or cause a
              denial of service (application crash) via crafted input.

Output:

purl: pkg:apache/apr-util@<1.3.5
{
    "affected_versions": ["< 1.3.5"],
    "fixed_versions": [">= 1.3.5"]
}

ziadhany avatar Feb 23 '25 11:02 ziadhany

@pombredanne This is a small document for the budget you requested. I used some sources like https://llm-stats.com/, and I think the best option is to avoid running the model locally or in the cloud and instead use an API.

Please let me know if you have any comments on this. https://docs.google.com/document/d/1JZ49FqjessEyMhdKlp1HmfheITr3qKA8xMbNZIZW7UA/edit?usp=sharing

ziadhany avatar Mar 04 '25 16:03 ziadhany