Feature: install dependencies alongside Qt libraries on demand

[pzhlkj6612]

Hi, there.

Background

I'm using Qt for MinGW. From my observations, the current official Qt online installer (4.1.1-0-202105251051) will automatically install the corresponding version of MinGW tool when the user wants to install Qt for MinGW, even if that MinGW tool is not checked (selected) in the tree of "Developer and Designer Tools".

The dependency relationship is described in the Updates.xml file (Qt 6.1.2 on Windows):

<Updates>
  <PackageUpdate>
    <Name>qt.qt6.612.win64_mingw81</Name>
    ...
    <Dependencies>qt.tools.qtcreator, qt.tools.win64_mingw810, qt.qt6.612.doc, qt.qt6.612.examples</Dependencies>
    ...

Requested features

I think it would be great if aqtinstall had the following abilities (sorted by importance):

1. Installing Qt libraries with dependencies.
2. Configuring related files (such as the qtenv2.bat file for MinGW on Windows. See https://github.com/miurahr/aqtinstall/pull/279#issuecomment-884687570).
3. Filtering dependencies:
   • Filtering names by using pattern matching or regular expressions.
   • Filtering versions with semantic versioning.

---

Based on the post in https://github.com/miurahr/aqtinstall/issues/299#issuecomment-884694093 .

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[ddalcino]

I think this is a graph traversal problem. Each <PackageUpdate> that you install contains a list of dependencies in the <Dependencies> xml tag. You need to find the Updates.xml file that contains each of the PackageUpdates in this list and recursively install each PackageUpdate for your architecture.

If you imagine the dependencies as a graph, where each <PackageUpdate> is a node, and each item in the <Dependencies> tag is an edge to another node, you could visit every node that the starting node is connected to and install every item in <DownloadableArchives> that you encounter along the way.

I think that the hard part here is following the edges. If you start at qt.qt6.620.clang_64 from https://download.qt.io/online/qtsdkrepository/mac_x64/desktop/qt6_620/Updates.xml, your dependencies are "qt.tools.qtcreator, qt.qt6.620.doc, qt.qt6.620.examples". None of these PackageUpdates exist in the same Updates.xml file as qt.qt6.620.clang_64; you need some way to translate the name qt.tools.qtcreator into the url https://download.qt.io/online/qtsdkrepository/mac_x64/desktop/tools_qtcreator/Updates.xml.

I have written some code that attempts to solve the edge problem here: https://gist.github.com/ddalcino/0998f62fc87b4265a410dbf0555bbc99. It depends on some code in ci/generate_combinations.py, so I have been running it within the ci folder. The download_all_xmls() function downloads every Updates.xml file in the repository; this prevents a fresh download every time you want to read an Updates.xml file. The pkg_name_to_folder() function attempts to figure out the path to an Updates.xml folder, based on a PackageUpdate name (like qt.qt6.620.clang_64) and a couple of contextual clues that do not exist in the name, like the folder extension. I think that some of these clues are necessary. The check_folder_names() function attempts to test the pkg_name_to_folder() function by running it on every PackageUpdate name in the Updates.xml files. For this to work, pkg_name_to_folder() must return the right path every time.

My pkg_name_to_folder() function is fairly ugly, but it works for the subset of Updates.xml files I have tested it on. There are a couple of outlier PackageUpdate names in the preview and early access folders that I couldn't figure out; hopefully we will not need these.

Hope this helps.

[pzhlkj6612]

Whoa, I think aqtinstall will eventually become a package manager. XD

[miurahr]

Whoa, I think aqtinstall will eventually become a package manager. XD

Do you know why aqt was given its name with reverence of apt - another package tool？

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[miurahr]

Here is article of Graph traversal

• wikiepdia - graph traversal
• medium - graph traversal in python

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[miurahr]

Here is an experimental implementation of graph parsing of set of Update.xml database. https://github.com/miurahr/aqtxmlparser

Please see a test case to traverse dependency.

@pytest.mark.parametrize(
    ("target,in_file,expect_depend"),
    [
        ("qt.qt5.5140.qtcharts.win32_mingw73", "windows-5140-update.xml", "qt.tools.win32_mingw730"),
    ],
)
def test_get_module_to_package(target: str, in_file: str, expect_depend: str):
    xml = (Path(__file__).parent / "data" / in_file).read_text("utf-8")
    update_xml = xmlparser.Updates.fromstring(xml)
    module = target.split(".")[-2]
    #
    packages = update_xml.dfs(target)
    module_to_package = xmlparser.ModuleToPackage({module: packages})
    assert module_to_package.has_package(target)
    assert module_to_package.has_package(expect_depend)

[miurahr]

Now start integration on #533 #533 will improve Updates.xml parsing and store into its data class.

[miurahr]

Now feature is merged.