cyclonedx-python icon indicating copy to clipboard operation
cyclonedx-python copied to clipboard
verified publisher icon CycloneDX

Support for the dependency graph extension

Open r614 opened this issue 5 years ago • 7 comments

Will there be added support for the dependency graph extension, or will that be a separate library?

r614 avatar Jan 08 '20 19:01 r614

Support for dependency graphs should be included in every official CycloneDX implementation, including this one. Currently, only the Maven plugin supports it.

I'm relying heavily on the community for these types of enhancements. PR's are highly encouraged.

stevespringett avatar Jan 08 '20 19:01 stevespringett

How does this look

<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.0" version="1">
    <components>
        <component type="library">
            <publisher>the purl authors</publisher>
            <name>packageurl_python</name>
            <version>0.9.3</version>
            <description>A "purl" aka. Package URL parser and builder</description>
            <hashes>
                <hash alg="MD5">d051230d016990f856c14ceb6ec7836c</hash>
                <hash alg="SHA-256">0682b2eddab16151da5bd4ef38081e9b27f8eb33cd29baf41f4996d4e88e6e70</hash>
            </hashes>
            <licenses>
                <license>
                    <name>MIT</name>
                </license>
            </licenses>
            <purl>pkg:pypi/[email protected]</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Kenneth Reitz</publisher>
            <name>requests</name>
            <version>2.25.0</version>
            <description>Python HTTP for Humans.</description>
            <hashes>
                <hash alg="MD5">2966d68a5a4e6832d967763d41f48d04</hash>
                <hash alg="SHA-256">e786fa28d8c9154e6a4de5d46a1d921b8749f8b74e28bde23768e5e16eece998</hash>
            </hashes>
            <licenses>
                <license>
                    <name>Apache 2.0</name>
                </license>
            </licenses>
            <purl>pkg:pypi/[email protected]</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Davide Brunato</publisher>
            <name>xmlschema</name>
            <version>1.2.5</version>
            <description>An XML Schema validator and decoder</description>
            <hashes>
                <hash alg="MD5">7a5623bbe80f43d96b1a77a8cdd95619</hash>
                <hash alg="SHA-256">7c528e0ec3eac97276491e7657d843f6090cbc2ea9216eb4398553623859a23f</hash>
            </hashes>
            <licenses>
                <license>
                    <name>MIT</name>
                </license>
            </licenses>
            <purl>pkg:pypi/[email protected]</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Python Packaging Authority</publisher>
            <name>setuptools</name>
            <version>50.3.2</version>
            <description>Easily download, build, install, upgrade, and uninstall Python packages</description>
            <hashes>
                <hash alg="MD5">079395a567856392c1445a76a2833370</hash>
                <hash alg="SHA-256">2c242a0856fbad7efbe560df4a7add9324f340cf48df43651e9604924466794a</hash>
            </hashes>
            <purl>pkg:pypi/[email protected]</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>David Fischer</publisher>
            <name>requirements_parser</name>
            <version>0.2.0</version>
            <description>Parses Pip requirement files</description>
            <hashes>
                <hash alg="MD5">611b0cab139e9a35363ec4ffa1fe6c8c</hash>
                <hash alg="SHA-256">76650b4a9d98fc65edf008a7920c076bb2a76c08eaae230ce4cfc6f51ea6a773</hash>
            </hashes>
            <licenses>
                <license>
                    <name>BSD</name>
                </license>
            </licenses>
            <purl>pkg:pypi/[email protected]</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Donald Stufft and individual contributors</publisher>
            <name>packaging</name>
            <version>20.7</version>
            <description>Core utilities for Python packages</description>
            <hashes>
                <hash alg="MD5">da81732f29c8f3d3bd3ff16f85c42b7c</hash>
                <hash alg="SHA-256">eb41423378682dadb7166144a4926e443093863024de508ca5c9737d6bc08376</hash>
            </hashes>
            <purl>pkg:pypi/[email protected]</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Daniel Blanchard</publisher>
            <name>chardet</name>
            <version>3.0.4</version>
            <description>Universal encoding detector for Python 2 and 3</description>
            <hashes>
                <hash alg="MD5">0004b00caff7bb543a1d0d0bd0185a03</hash>
                <hash alg="SHA-256">fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691</hash>
            </hashes>
            <licenses>
                <license>
                    <name>LGPL</name>
                </license>
            </licenses>
            <purl>pkg:pypi/[email protected]</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others</publisher>
            <name>pytest</name>
            <version>6.1.2</version>
            <description>pytest: simple powerful testing with Python</description>
            <hashes>
                <hash alg="MD5">4b715c5f2f17acc462c992839e1811af</hash>
                <hash alg="SHA-256">4288fed0d9153d9646bfcdf0c0428197dba1ecb27a33bb6e031d002fa88653fe</hash>
            </hashes>
            <licenses>
                <license>
                    <name>MIT</name>
                </license>
            </licenses>
            <purl>pkg:pypi/[email protected]</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Julian Berman</publisher>
            <name>jsonschema</name>
            <version>3.2.0</version>
            <description>An implementation of JSON Schema validation for Python</description>
            <hashes>
                <hash alg="MD5">7617cd8e4a79ba49cfd602eb921b08d8</hash>
                <hash alg="SHA-256">4e5b3cf8216f577bee9ce139cbe72eca3ea4f292ec60928ff24758ce626cd163</hash>
            </hashes>
            <purl>pkg:pypi/[email protected]</purl>
            <modified>false</modified>
        </component>
    </components>
    <dg:dependencies>
        <dg:dependency ref="pkg:pypi/[email protected]" />
        <dg:dependency ref="pkg:pypi/[email protected]">
            <dg:dependency ref="pkg:pypi/[email protected]" />
            <dg:dependency ref="pkg:pypi/[email protected]" />
            <dg:dependency ref="pkg:pypi/[email protected]" />
            <dg:dependency ref="pkg:pypi/[email protected]" />
        </dg:dependency>
        <dg:dependency ref="pkg:pypi/[email protected]">
            <dg:dependency ref="pkg:pypi/[email protected]" />
        </dg:dependency>
        <dg:dependency ref="pkg:pypi/[email protected]" />
        <dg:dependency ref="pkg:pypi/[email protected]" />
        <dg:dependency ref="pkg:pypi/[email protected]">
            <dg:dependency ref="pkg:pypi/[email protected]" />
        </dg:dependency>
        <dg:dependency ref="pkg:pypi/[email protected]" />
        <dg:dependency ref="pkg:pypi/[email protected]" />
        <dg:dependency ref="pkg:pypi/[email protected]">
            <dg:dependency ref="pkg:pypi/[email protected]" />
            <dg:dependency ref="pkg:pypi/[email protected]" />
            <dg:dependency ref="pkg:pypi/[email protected]" />
            <dg:dependency ref="pkg:pypi/[email protected]" />
        </dg:dependency>
    </dg:dependencies>
</bom>

I have rough implementation of this, will open a PR soon.

This is in reference to https://cyclonedx.org/ext/dependency-graph/ . That page says "It has been incorporated (with minor changes) into CycloneDX v1.2 and higher. #"

Could someone point me to the "minor changes" ?

sbs2001 avatar Mar 08 '21 16:03 sbs2001

The dependency graph extension should not be used. Rather, the built-in dependency graph elements should be used instead. This is going to require #9 to be implemented.

The 'minor change' is actually with regard to #9 - the metadata section. The dependency graph example provided in https://github.com/CycloneDX/cyclonedx-python/issues/40#issuecomment-792871084 is not capable of describing direct vs transitive relationships. The 'minor change' in v1.2 is that the dependency graph can now make that distinction. Refer to https://cyclonedx.org/use-cases/#dependency-graph

stevespringett avatar Mar 08 '21 16:03 stevespringett

@stevespringett thanks for the links . Correct me if I am wrong: to translate the v1.2 spec to python world, the setup.py or something top level would need to be parsed. That would be enough to make the metadata node. All the things in the requirements.txt would be treated as it's direct dependencies. And their subsequent (2-degree dependency) would be the transitive deps.

Also could you elaborate

The dependency graph extension should not be used. Rather, the built-in dependency graph elements should be used instead

does that simply mean <dg:dependency> gets changed to <dependency> ?

sbs2001 avatar Mar 10 '21 10:03 sbs2001

does that simply mean dg:dependency gets changed to ?

Correct

stevespringett avatar Mar 10 '21 15:03 stevespringett

@sbs2001 , were you able to raise PR for above? Thanks

mgrajesh1 avatar Jul 11 '21 19:07 mgrajesh1

Leaving this open as some work may be required in this application for outputting dependency graphs once cyclonedx-python-lib successfully supports bom.dependencies - see https://github.com/CycloneDX/cyclonedx-python-lib/issues/7.

madpah avatar Sep 16 '21 12:09 madpah

I'm struggling to understand the current state of this feature. It seems like the library supports this now, but there's still some work left on this side of things, correct?

If so, would it be a good idea to work on #303 and go from there, or does it need a new attempt? I'd love for this feature to finally make it into the tool and am willing to help out.

KramNamez avatar Nov 22 '22 08:11 KramNamez

re: https://github.com/CycloneDX/cyclonedx-python/issues/40#issuecomment-1323303400 looks the same to me. CycloneDX lib got support, but there is nothing done in the actual detection of dependencies.

jkowalleck avatar Dec 22 '22 12:12 jkowalleck

I have had reason and time to look at this again, and now I understand how... thorny this can be.

I think I can port the work from #303 onto a newer version of the source, so that the poetry and environment parsers will support it, and I have a PoC for how to do it from a requirements.txt that has been generated by pip-compile.

Should I just open a branch with all of these changes, as a proposal? Or should I leave the pip-compile-based one out, since that is a specific variant of the regular requirements.txt?

KramNamez avatar Feb 07 '23 10:02 KramNamez

related: https://github.com/CycloneDX/cyclonedx-python/discussions/487#discussioncomment-5623022

this shows an idea how to create a dep tree from requirements.txt

@madpah FYI

jkowalleck avatar Apr 27 '23 07:04 jkowalleck

will be part of upcoming v4 at least for some sources

jkowalleck avatar Nov 14 '23 18:11 jkowalleck

fixed by #605

jkowalleck avatar Dec 25 '23 15:12 jkowalleck

This feature will be part of the next/upcoming major release. Changelog: see https://github.com/CycloneDX/cyclonedx-python/pull/605 Install via: pip install cyclonedx-bom==4.0.0rc1

jkowalleck avatar Dec 25 '23 18:12 jkowalleck