Donya

Donya copied to clipboard

Hi,

The DonyaOS is not yet ready and complete and we need the help of others.

Here's a chat room.

We can talk about the tasks we want to work on here. And set a development path for ourselves.

Study and research can be one of the things that will always be needed in this project.

Please state if something goes wrong. I appreciate you.

At this point, we have a script that can generate a new Linux distribution and produce .ISO image file. https://github.com/DonyaOS/Donya/blob/815216b91e7df81c9d3aebc1217cd9a2b3d698b3/donya.sh

But this is not enough.

We need to separate the scripts and have such a separate file. (Not a single .sh file)

For example, Scripts:

• Downloading dependency files (Maybe someone has these file in their PC)
• Compile kernel, base
• Compile OS/BusyBox and other packages
• Generate .ISO image file

Also Makefile can help to compile quickly.

With Donya's package manager, I'd suggest a packaging system like KISS Linux (here's an overview: https://k1ss.org/package-system). What did you have in mind?

OK, with package management, this is what I had in mind—just a suggestion, I'm not entirely sure what your plans are:

We make a repository in the DonyaOS GitHub organization, and we call it packages or something to that effect. This will be the default source of packages in the package manager. Then, the package manager (which I'll call dpak until we come up with a proper name) will look for packages within that GitHub repository, at the root, by filename; so if we were to do something like: dpak install ExamplePackage, then dpak would read the data in the ExamplePackage.yml file at the root of the packages repository. This is similar to how the Homebrew package manager on macOS works.

Each package file in the repository will have several standard data fields, which I suggest we represent in YAML. These fields will be description, license, homepage, dependencies, binary (optional), and install. The package manager will show the package metadata (including the list of dependencies) before prompting the user if they want to install (like with APT). The install variable is a series of commands which will be ran by the system to get the package installed, useful for if the package is compiled (like how Gentoo handles packages), not a binary. In the event a package is binary-only, then the binary variable will point to the URL of the binary release, and then dpak will simply install it into the system. Here's an example of a package called dlib; stored in DonyaOS/packages/dlib.yml:

description: "C++ library for machine learning"
homepage: "http://dlib.net/"
binary: "http://dlib.net/files/dlib-19.21.tar.bz2"
license: "BSL-1.0"
dependencies:
- cmake
- jpeg
- libpng
- openblas
install: |
  git clone https://github.com/davisking/dlib.git
  cd dlib-master
  mkdir build
  cd build
  cmake ..
  cmake --build . --config Release

This package manager model has several key differences over currently existing package managers:

1. It's simple—package repositories are easy to create, because they're just GitHub repositories
   • APT and Snapcraft struggle greatly in this department. Packaging software, and then publishing it into a repository, is a difficult process for developers who want to get their work installed via those package managers. Moving everything to a simple, GitHub-centric system will make publishing easier.
2. It allows for compiling packages rather than just getting binaries
   • This is only really seen in Gentoo nowadays. Making a simpler alternative to Portage is a good idea.
3. Greater GitHub integration; by making each software source a GitHub repository, it will force software publishers to become familiar with the open-source ecosystem, particularly if they wish to publish in the primary DonyaOS package repository. This promotes open-source development.

Thanks for your suggestion. I did create packages repository in this organization.

https://github.com/DonyaOS/Packages

We need to design the package manager software. Something similar to apt in ubuntu, pacman in Arch, pkg in FreeBSD.

I think GoLang is a good choice for this purpose. C is also good.

Easy command line is very important.

We can use an alias:

• i for install.
• u or r for uninstall and remove
• s for search

For example:

d i php ; install php version 7.1
d i php7.4 ; install php version 7.2
d i gcc ; install gcc

d s php ; search all package with php perfix

d r php ; remove php package
d r php* ; remove all php prefix package

d i php* ; install all php prefix package

Display loading while downloading and package information before installing is a good feature. Centos has this feature as well.

Ubuntu apt has a bad feature, this forces you to remove lock files if you stop this using ctrl+x, ctrl+c etc.

Hi,

Does anyone want to work on the package manager or package service? @sudo-corvus @limeschool Or others We welcome this help. I think GoLang is a good choice for this purpose.

Please tell me your comment. Thanks,

PackageManager: It's a client software, this should be installed at DonyaOS in default. https://github.com/DonyaOS/PackageManager

Packages: This is an API. This is a web service, PackageManager will get his data from this repository.

https://github.com/DonyaOS/Packages

---

For example: If PackageManager wants to search in packages list, should get a list of packages from the Packages repository. and save this as a cache.

I think we can host packages files in GitHub. Otherwise, we can create a server and host on that. No problem with this.

Hey @BaseMax i'm familiar with GoLang and i can help you guys with this! Just let me know where to begin...

Hey Amir, @amir-shiati Thanks for letting me know you agree to work on it. I gave you full access. Therefore, you can submit and send changes in the desired repository without any permission. Thanks,

I'm not familiar with Go or C. I'll try to make my own package manager in D, and then we can compare and pick which is best.

EDIT: D was not a good choice. Writing it in C# right now. It should perform rather well, with the time to complete an operation being almost entirely package-dependent and not noticeably impacted by any overhead.

I will take this up in C I have been going through RPM to understand the process

Hi, @amir-shiati is working to create Package Manager. Can you please start working on the Packages repository? @sudo-corvus, and @limeschool

all right on it @BaseMax

Just as a side project:

I've made a package manager in C# that works rather fast and does not require installing any dependencies to use it.

It's called gany and does not use the KISS structure currently in the DonyaOS/Packages repository, but rather a simpler one where each package is a single file. For an example of this, see my own package repository which has a package definition for gany itself, allowing for self-updating. gany can build packages, and it can download binary versions as well. Pretty neat, though a little incomplete right now.

---

Since you seem to be going with the KISS package structure, however, what packages should @sudo-corvus and I put in DonyaOS/Packages? Should we just copy over the default KISS software repository?

Hi Emil,

I was excited to see your activity. Where is package definition of the gany? (Your link is broken)

Anyway, you has full-access to this organization and can push your code/commits directly.

I've made a package manager in C#...

I not sure about this, Usually, C# is not a good choice for Linux commands And this is not a common thing.

This is a good language though. But C# is compiled into IL, not binary.

To install gany, you need 'libkrb5-dev' & '.NET Core 3.1' to build...

However, if this is necessary, We can think of rewriting it in another language. @amir-shiati, Have you done anything? This can give him a good view of how to develop a package manager.

KISS package structure...

I just use this as the first version for a starter package.

We can think about and develop a new structure. Anyway, I like your idea about the KISS structure. And this is great for most purposes.

For example ffmpeg: https://github.com/kisslinux/repo/tree/master/extra/ffmpeg

build
checksums
depends
sources
version 

Please show me your current packages repository.

---

Update: https://github.com/limeschool/packages

You can add some sample packages just for testing: https://github.com/kisslinux/repo/tree/master/core/curl

Thanks,

Please push your repositories to the Donya organization. @limeschool

Please explain where the name gany comes from? donya and d was not a short and better name?

---

Some feature needed for package manager:

• Checksum
• Ability to display the size of all files (before start to downloading)

In your structure file size is not calculable. (This is missing side by side with bash commands)

• Display loading x% while downloading files...

A sample: https://github.com/DonyaOS/Packages/blob/master/core/curl/package.donya

Direct HTTP/S request: https://donyaos.github.io/Packages/core/curl/package.donya

In this case, But we need to know the name of the group. (core at this sample) Otherwise, we have to not group the packages.

If a package need multi-file to download. How we can control run make/configure in different downloaded files?

I no mean dependency. I mean multi required file to download.

https://github.com/DonyaOS/Packages/commit/9947725b08c32e8a63161259806b0ae2bd5bf785

We can do unrar x, unzip, and tar xvf automatically in the package manager.

But I think we have to write cd manually... What do you think?

About the package manager:

Consider that before the installation step (install in the package file), All required packages must be installed.

This must be done in a recursive manner. Because the required packages may also depend on other packages.

Hey everyone! this is a simple diagram showing how the package manager is going to work, please take a look at it and let me know if any other blocks needs to be added to the diagram...

And also this is an example of how a package would be installed: lets say the user wants to install curl: $ donya install curl or $ donya i curl

the package manager would send a request to the package manager api, the package manager api would search for curl and then return (if exists) the url of curls "package.donya" file which is for example this, the package manager would download the file and open it, then after that: 1- it would install all the required packages in a recursive manner. 2- it would download the package file. 3- it would check the checksum to make sure that the downloaded file is not corrupted. 4- it would run the given instructions to install package.

Let me know what you guys think...

Thanks, Amir, That's fine. @amir-shiati

Who is currently responsible for designing the API? @sudo-corvus, and @limeschool We have already assigned two people.

One of our problems is to host files at GitHub. Do we will can to develop API and web server based on GitHub?

We can set up files on own server, and connect the GitHub repository to server using an SSH Action.

• https://github.com/marketplace/actions/ssh-remote-commands
• https://github.com/marketplace/actions/run-ssh-command
• https://github.com/marketplace/actions/remote-ssh-commands
• ...

But I think we can follow a better way.

Update:

Also, Using the GitHub API might be another way to do this. https://developer.github.com/v3/ https://developer.github.com/v3/search/#search-code

Hey everyone, For hosting the API oh github i came across this, However the api is static.

Thanks for sharing this. It's awesome. @MatMoore

https://github.com/MatMoore/jargon

---

By using github pages you are accepting their usage limits which means that your API could still go down if it gets too much traffic, and they don’t recommend using it for commercial purposes:

GitHub Pages is not intended for or allowed to be used as a free web hosting service to run your online business, e-commerce site, or any other website that is primarily directed at either facilitating commercial transactions or providing commercial software as a service (SaaS).

https://docs.github.com/en/github/working-with-github-pages/about-github-pages

• Published GitHub Pages sites may be no larger than 1 GB.
• GitHub Pages sites have a soft bandwidth limit of 100GB per month.
• GitHub Pages sites have a soft limit of 10 builds per hour.

We cannot implement a powerful search API using GitHub pages. GitHub pages execute a script file(such as ruby) and create folder/files.

You can watch the gh-pages branch of the mentioned example: https://github.com/MatMoore/jargon/tree/gh-pages

We can cache a list of packages and store this in local system, finally search between packages using package manager client software.

Hi @limeschool, Thanks for your commits and message.

I do not think we can find an algorithm that finds the exact file-size of compiled files from the source code. However, we can specify the approximate size of the data in the package information.

Can you combine the test branch with the main branch in Packages repository? We need to go to the next steps.

The figure of pkg in FreeBSD:

root@base:~ # pkg install php74

Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 4 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	libargon2: 20190702
	libxml2: 2.9.10
	pcre2: 10.35
	php74: 7.4.8

Number of packages to be installed: 4

The process will require 40 MiB more space.
6 MiB to be downloaded.

Proceed with this action? [y/N]: wrong_key
Please type 'Y[es]' or 'N[o]' to make a selection

Proceed with this action? [y/N]: y
[1/4] Fetching php74-7.4.8.txz: 100%    4 MiB   4.2MB/s    00:01    
[2/4] Fetching libxml2-2.9.10.txz: 100%  828 KiB 848.0kB/s    00:01    
[3/4] Fetching libargon2-20190702.txz: 100%   64 KiB  65.5kB/s    00:01    
[4/4] Fetching pcre2-10.35.txz: 100%    1 MiB   1.3MB/s    00:01    
Checking integrity... done (0 conflicting)
[1/4] Installing libxml2-2.9.10...
[1/4] Extracting libxml2-2.9.10: 100%
[2/4] Installing libargon2-20190702...
[2/4] Extracting libargon2-20190702: 100%
[3/4] Installing pcre2-10.35...
[3/4] Extracting pcre2-10.35: 100%
[4/4] Installing php74-7.4.8...
[4/4] Extracting php74-7.4.8: 100%

The figure of pkg search in FreeBSD:

root@base:~ # pkg search mariadb | grep 10.3

mariadb103-client-10.3.23      Multithreaded SQL database (client)
mariadb103-server-10.3.23      Multithreaded SQL database (server)

root@base:~ # pkg search mariadb

mariadb-connector-c-3.1.9      MariaDB database connector for C
mariadb-connector-odbc-3.1.7_1 MariaDB database connector for odbc
mariadb103-client-10.3.23      Multithreaded SQL database (client)
mariadb103-server-10.3.23      Multithreaded SQL database (server)
mariadb104-client-10.4.13_4    Multithreaded SQL database (client)
mariadb104-server-10.4.13_4    Multithreaded SQL database (server)
rubygem-azure_mgmt_mariadb-0.17.3 Microsoft Azure Microsoft Azure MariaDB Library for Ruby Client Library for Ruby

Hi. Where did the package manager get to and how did it happen? @amir-shiati https://github.com/DonyaOS/PackageManager