Outreachy Summer 2023

[moazzammoriani]

Some info about Outreachy summer 2023

Hi all,

here's some information about the Outreachy application period. First of all: welcome! :partying_face:

For non-applicants confused by this see this section.

For applicants

Thanks for contributing to the MirageOS ecosystem! This is the right place to start, if you're considering applying to any of the following three Outreachy projects:

• Persistent Storage in MirageOS
• Extend the network testing tool conntest with visualizations
• MIDI with MirageOS

Installing OCaml on Your System

Quick Installation

The quickest way to install OCaml currently is using the OCaml Platform Installer.

You'll need to install some dependencies if they're not already installed. This can be done easily using a package manager.

If you're running an Ubuntu/Debian system you can run the following command in your terminal.

sudo apt install bzip2 make gcc bubblewrap rsync patch curl unzip

After your dependencies are installed run the below install script.

bash < <(curl -sL https://ocaml.org/install-platform.sh)

Install the Platform tools.

ocaml-platform

We also need to install an opam switch that uses a version of OCaml 4 that Mirage currently supports.

opam switch create mirage 4.14.1

Finally, we simply update the new opam environment.

eval $(opam env --switch=mirage) 

Other Installation Methods

If the OCaml Platform installation method does not suit you then you can take a look at the variety of other installation methods described at ocaml.org.

What is MirageOS?

You see OS in the name so you're probably guessing that MirageOS is some sort of operating system. That's certainly true but perhaps that distracts from the problem MirageOS tries to solve.

MirageOS is indeed an operating system, but it is a special kind of operating system. It is a library operating system that constructs this thing called a unikernel. A unikernel is a stripped-down, bare-bones operating system that has only the parts needed to run a single application.

When it comes to containing applications, if the container--e.g Docker--was the successor to virtual machines then the next step after the container is the unikernel.

You can learn more about MirageOS from its overview page and its documentation page.

Contributions

Let's get right at it! By the end of the day, you're here to contribute to the MirageOS ecosystem and to apply to one of the three projects mentioned above.

First step: the MirageOS hello-world

If you want to apply to any of the those projects, the first step for you will be to compile and run the MirageOS hello-world unikernel: https://mirage.io/docs/hello-world. Please, read through Step 0 and Step 1 of that document to build a Unix binary containing your hello-world unikernel. When running ./hello, you should see hello (no need to build it for the hvt backend as well - although, of course, you're welcome to do so, if you're interested and on a Linux machine).

Once you've achieved that, please let us know here on the issue that you have. And if you have troubles with it, don't hesitate to ask!

Second step: a good-first-issue

Once got the hello-world unikernel to run, you're ready to start contributing. Where to find good issues to work on depends on the project you want to apply to.

Persistent Storage in MirageOS

Please find a list of relevant issues to contribute towards here: https://github.com/mirage/ocaml-mbr/issues/15

Once you've tackled one good-first-issue issue, you can try a medium issue as well (optional):

<We're going to post an issue tracker with medium issues for the Persistent Storage in MirageOS soon.>

Extend the network testing tool conntest with visualizations

Please find a list of relevant issues to contribute towards here: https://github.com/rand00/conntest/issues

Once you've tackled one good-first-issue issue, you can try a medium issue as well (optional):

<We're going to post an issue tracker with medium issues for the Extend the network testing tool conntest with visualizations soon.>

MIDI with MirageOS

Good first issues:

• https://github.com/pitag-ha/cardio-crumble/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22
• https://github.com/rand00/fry

Once you've tackled one good-first-issue issue, you can try a medium issue as well (optional):

<We're going to post an issue tracker with medium issues for the MIDI with MirageOS soon.>

Note: when working with cardio-crumble make sure that your opam switch is using OCaml 5

Ask Questions!

You are bound to get stuck somewhere and that's perfectly fine because that means you're moving towards progress!

If you don't know how to move forward with the problem you're stuck with then don't hesitate to ask .

For general questions, you can ask on discord, which you can access via this invitation link. Once you've joined discord, if you have a question about PPX that's not related to some issue, you can ask on the MirageOS channel. If you have a general question about something else, e.g. how to get OCaml up and running, you can ask on another discord channel, for example on the Outreachy channel.

Happy hacking :)

Mentors

You have five mentors for the application period: Reynir (github: @reynir), Claes (@rand00), Oskar (@musha68k), Moazzam (github: @moazzammoriani, discord: mwzm) and Sonja (github: @pitag-ha, discord: pitag). Reynir and Claes are both experienced MirageOS hackers (including OCaml experience). Oskar, long time OCaml and MirageOS enthusiast is very familiar with networks and general systems programming. Moazzam has completed his own Outreachy internship about a year ago and has since been hacking on RISC-V. Sonja is very familiar with OCaml, has some experience with MirageOS and MIDI, and has already mentored during three previous Outreachy rounds.

We're all very excited about having Outreachy applicants and later Outreachy interns contributing to the MirageOS context! On weekends we tend to be non- or less available.

For Non-applicants

Outreachy is an initiative to give folks from under-represented groups in open-source the opportunity to do a remote internship in an open-source project. So for open-source communities, Outreachy is a great opportunity to work together with folks from lots of different backgrounds and maybe even grow the community and become more diverse! Outreachy provides the infrastructure for the internships and application process, and the open-source communities need to provide projects and mentoring. Three of the projects we -i.e. the OCaml community- is participating with in this Outreachy round are MirageOS related. So we might get some Outreachy applicants contributing to some of our MirageOS repos from (...) till (...). as part of the application process to those projects :)

A Note on Using ChatGPT and Generative AI for Contributions

We, the mentors for MirageOS-related projects (@moazzammoriani, @pitag-ha, @rand00, @reynir) have discussed the role of ChatGPT and generative AI during this contribution period and have identified some of the following--non-exhaustive list of--problems in using this technology to contribute.

1. We don't want these tools to be substitutes for your judgement and understanding.
2. We don't want your responsibility to be deferred to these tools since they are often wrong and hence unreliable.
3. We want to make sure mentors spend their time and efforts helping you improve and learn.

Your judgement and understanding

We don't want contributors to be using generated contributions in situations where we want you to make your judgement. Submitting generated code with little or no understanding as a contribution would be an extreme case of this.

We think if you make a contribution you should be exercising your own judgement and trying your best to solve some problem. Your effort and hard-work are the true essence of any contribution.

Borrowing code has almost always been fine, but even when you do that you are expected to make the judgement to borrow it in the first place. It also is still your responsibility to test that code and abstractly know why that code is there to begin with.

Responsibility

A large part of open-source involves is about taking on responsibility. We think that use of these tools should not take away from that.

Mentor time

With all the contributions, the mentor would be trying to read through this contribution and provide any feedback so that the contributor can learn from this and improve. This takes time and effort on the mentor's part. This would be wasted if they realize half-way that the code was AI-generated.

Mentors want to help you learn and improve. This is difficult or even impossible to do if you don't show them your own work.

Concluding remarks

We only request you to be mindful of all this.

If you happened to use ChatGPT for something like brain-storming ideas for a contribution, that's ok. We ask you in that case to let your mentor know. Giving them the heads up might help them save time and even put you in a better position of trust.

[AryanGodara]

Hello, I was on step 0, make depends, and I got the following errors, and I'm not yet able to figure them out yet!

make depends                                                                                                                                 ─╯
using overlay repository mirage: [opam-overlays, mirage-overlays] 
[opam-overlays] Initialised
[NOTE] Repository opam-overlays has been added to the selections of switch default only.
       Run `opam repository add opam-overlays --all-switches|--set-default' to use it in all existing switches, or in newly created switches,
       respectively.

[mirage-overlays] Initialised
[NOTE] Repository mirage-overlays has been added to the selections of switch default only.
       Run `opam repository add mirage-overlays --all-switches|--set-default' to use it in all existing switches, or in newly created switches,
       respectively.

 ↳ generate lockfile for monorepo dependencies
==> Using 1 locally scanned package as the target.
[WARNING] Unknown variable "ocaml-system:version"
[WARNING] Unknown variable "ocaml-base-compiler:version"
[WARNING] Unknown variable "ocaml-variants:version"
[WARNING] Unknown variable "ocaml-system:version"
[WARNING] Unknown variable "ocaml-base-compiler:version"
[WARNING] Unknown variable "ocaml-variants:version"
[WARNING] Unknown variable "ocaml-system:version"
[WARNING] Unknown variable "ocaml-base-compiler:version"
[WARNING] Unknown variable "ocaml-variants:version"
[WARNING] Unknown variable "ocaml-system:version"
[WARNING] Unknown variable "ocaml-base-compiler:version"
[WARNING] Unknown variable "ocaml-variants:version"
[WARNING] Unknown variable "ocaml-system:version"
[WARNING] Unknown variable "ocaml-base-compiler:version"
[WARNING] Unknown variable "ocaml-variants:version"
[WARNING] Unknown variable "ocaml-system:version"
[WARNING] Unknown variable "ocaml-base-compiler:version"
[WARNING] Unknown variable "ocaml-variants:version"
[WARNING] Unknown variable "ocaml-system:version"
[WARNING] Unknown variable "ocaml-base-compiler:version"
[WARNING] Unknown variable "ocaml-variants:version"
opam-monorepo: [ERROR] Can't find all required versions.
Selected: astring.0.8.5+dune base.v0.15.1 base-bigarray.base
          base-bytes.base+dune base-domains.base base-nnp.base
          base-threads.base base-unix.base cmdliner.1.1.1+dune cppo.1.6.9
          csexp.1.5.1 cstruct.6.1.1 domain-name.0.4.0
          dune.3.7.0-ocaml5.0.0+platform dune-configurator.3.7.0
          duration.0.2.1 fmt.0.9.0+dune functoria-runtime.4.3.4 ipaddr.5.3.1
          logs.0.7.0+dune2 lwt.5.6.1 macaddr.5.3.1 mirage-bootvar-unix.0.1.0
          mirage-clock.4.2.0 mirage-clock-unix.4.2.0 mirage-logs.1.2.0
          mirage-profile.0.9.1 mirage-runtime.4.3.4 mirage-unix.5.0.1
          noop-unix.zdev num.1.4+dune2 ocaml-base-compiler.5.0.0
          ocaml-compiler-libs.v0.12.4 ocaml-options-vanilla.1
          ocplib-endian.1.2 parse-argv.0.2.0 parsexp.v0.15.0
          ppx_cstruct.6.1.1 ppx_derivers.1.2.1 ppxlib.0.29.1
          ptime.1.0.0+dune2 sexplib.v0.15.1 sexplib0.v0.15.1
          stdlib-shims.0.3.0 ocaml-base-compiler&noop-unix base-domains
          ocaml-system ocaml-base-compiler
- ocaml-system -> (problem)
    Rejected candidates:
      ocaml-system.5.0.0: In same conflict class (ocaml-core-compiler) as ocaml-base-compiler
- ocaml-variants -> (problem)
    Rejected candidates:
      ocaml-variants.5.1.0+trunk: In same conflict class (ocaml-core-compiler) as ocaml-base-compiler
      ocaml-variants.5.0.1+trunk: In same conflict class (ocaml-core-compiler) as ocaml-base-compiler
      ocaml-variants.5.0.0+tsan: In same conflict class (ocaml-core-compiler) as ocaml-base-compiler
      ocaml-variants.5.0.0+options: In same conflict class (ocaml-core-compiler) as ocaml-base-compiler
      ocaml-variants.5.0.0~rc1+options: In same conflict class (ocaml-core-compiler) as ocaml-base-compiler
      ...
removing overlay repository [opam-overlays, mirage-overlays]
Repositories removed from the selections of switch default. Use '--all' to forget about them altogether.
Repositories removed from the selections of switch default. Use '--all' to forget about them altogether.
make[2]: *** [Makefile:36: mirage/noop-unix.opam.locked] Error 1
make[1]: *** [Makefile:39: lock] Error 2
make: *** [Makefile:51: depends] Error 2

I had followed all the previous steps carefully, and encountered no errors before this.

Then, I tried, mirage configure --extra-repo https://github.com/mirage/opam-overlays.git -t unix -> make depends (Fails) and mirage configure --no-extra-repo -t unix -> make depends

make build                                                                                                                                   ─╯
mirage build -f config.ml
File "dune.build", line 14, characters 18-29:
14 |    mirage-runtime mirage-unix)
                       ^^^^^^^^^^^
Error: Library "mirage-unix" not found.
-> required by _build/default/main.exe
-> required by alias all
-> required by alias default
make: *** [Makefile:43: build] Error 1  

In the end, I removed all changes to the repo, and ran make (default command), but since no dist/noop was created, I couldn't complete step 0. (Sorry for the verbose error)

[marrious11]

I suppose this issue needs a senior hand of advice. I am still setting up my whole working environment/ trying out step 0, if should incase i succeed i will let you know what went wrong okay! Perhaps before the end of tomorrow.

[moazzammoriani]

@AryanGodara what version of OCaml is your current opam switch using? you can check by running the following command.

$ opam switch 

[0xrotense]

Completed the first step successfully ✅ I also played with the ocaml and Lwt by adding custom texts and increasing the loop. 😃

[AryanGodara]

@AryanGodara what version of OCaml is your current opam switch using? you can check by running the following command.

$ opam switch 

@moazzammoriani I have OCaml version 5.0.0 mirage version 4.3.4 ocaml-platform version 0.8.0

[moazzammoriani]

I would suggest switching over to a recent version of OCaml 4. You can do this by creating a new opam switch.

[AryanGodara]

@moazzammoriani , it took some tries to get the versioning right, but it finally worked!

==> Found 61 opam dependencies for the target package.
==> Querying opam database for their metadata and Dune compatibility.
==> Calculating exact pins for each of them.
==> Wrote lockfile with 32 entries to mirage/hello-unix.opam.locked. You can now run opam monorepo pull to fetch their sources.
removing overlay repository [opam-overlays, mirage-overlays]
Repositories removed from the selections of switch 4.13.1. Use '--all' to forget about them altogether.
Repositories removed from the selections of switch 4.13.1. Use '--all' to forget about them altogether.
 ↳ opam install switch dependencies
Nothing to do.
 ↳ install external dependencies for monorepo
==> Using lockfile mirage/hello-unix.opam.locked
 ↳ fetch monorepo rependencies in the duniverse folder
==> Using lockfile mirage/hello-unix.opam.locked
Successfully pulled 32/32 repositories

╭─    ~/mirage-skeleton/tutorial/hello  on   main ?16 ──────────────────────────────────────────────── ✔  took 59s   at 03:13:56  ─╮
╰─ make build                                                                                                                                   ─╯
mirage build -f config.ml

╭─    ~/mirage-skeleton/tutorial/hello  on   main ?16 ──────────────────────────────────────────────── ✔  took 31s   at 03:14:40  ─╮
╰─ dist/hello                                                                                                                                   ─╯
2023-03-09 03:15:01 +05:30: INF [application] hello
2023-03-09 03:15:02 +05:30: INF [application] hello
2023-03-09 03:15:03 +05:30: INF [application] hello
2023-03-09 03:15:04 +05:30: INF [application] hello

[PizieDust]

@reynir I completed the Hello world tutorial. I ran into the same problems earlier mentioned by @AryanGodara but after switching from ocaml v5.0.0 to ocaml v4.14.0 the make command worked. Here is my output:

[Mojoeffect]

Hi everyone. Just joined in...aaand I'm super excited to get this experience. I've successfully ran those commands and set up my environment :)

So I tried opening the overview page, documentation, and MirageOS hello-world unikernel by clicking directly on the link, but I've been unsuccessful. I am met with a "site cannot be reached...time out error" I want to inquire if the server is down.

[Mojoeffect]

Regarding my issues earlier on, the site is back on.

[fikayo-sanni]

Hello Everyone. Just joining in on the project. Looking forward to learning and growing through this experience.

I want to confirm that if is the public channel for conversations on this project and there is no chatroom elsewhere?

[PizieDust]

Hi @fikkyJ The public channel for conversations is on discord. You can join here: Ocaml on Discord

[Mojoeffect]

Hiii, successfully completed step one :)

./dist/hello
2023-03-13 13:53:56 +01:00: INF [application] hello
2023-03-13 13:53:57 +01:00: INF [application] hello
2023-03-13 13:53:58 +01:00: INF [application] hello
2023-03-13 13:53:59 +01:00: INF [application] hello

[Mojoeffect]

I encountered similar issues like @AryanGodara but after switching the version as suggested by @moazzammoriani , it was successful. Thanks everyone :)

[AryanGodara]

@moazzammoriani , will we get good-first-issues for MIDI over ethernet with MirageOS soon ? (I'm still learning, so not in any hurry😅) Just wanted to know if they've been posted, as I can't find them here!

[moazzammoriani]

@AryanGodara you can checkout some first-issues to work on for the MIDI over Ethernet project over here: https://github.com/pitag-ha/cardio-crumble/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22

[AryanGodara]

@moazzammoriani A little late to ask this😅 But why do we use ocaml (functional language) when building mirageOS? I am having fun learning it, but was wondering, the benefits over rust of C. Maybe I missed some docs, where this was mentioned😅 (Just curious)

[0xrotense]

@AryanGodara I believe we're using OCaml because that's the default language for MirageOs for several reasons, including:

• Type safety: OCaml's strong type system ensures that the code is less prone to errors and is easier to reason about. This is particularly important for an operating system where even a small mistake can have catastrophic consequences.

• Concurrency: OCaml has built-in support for lightweight concurrency, making it easy to write concurrent code that can take advantage of multiple cores.

• Modularity: OCaml's module system allows for clean separation of concerns, making it easier to write reusable and composable code.

• Performance: OCaml is a compiled language and has a reputation for being fast. In addition, the MirageOS team has put a lot of effort into optimizing the runtime system to make it even faster.

• Maintainability: Because OCaml code tends to be concise and expressive, it can be easier to maintain than code written in more verbose languages.

While Rust and C are also popular languages for systems programming, they have different design goals and trade-offs compared to OCaml. Rust, for example, places a strong emphasis on memory safety and has a unique ownership model to prevent common programming errors. C, on the other hand, is a low-level language that gives the programmer fine-grained control over memory and hardware resources. Each language has its own strengths and weaknesses, and the choice ultimately depends on the specific requirements of the project.

I hope this helps :)

[AryanGodara]

@AryanGodara I believe we're using OCaml because that's the default language for MirageOs for several reasons, including:

• Type safety: OCaml's strong type system ensures that the code is less prone to errors and is easier to reason about. This is particularly important for an operating system where even a small mistake can have catastrophic consequences.
• Concurrency: OCaml has built-in support for lightweight concurrency, making it easy to write concurrent code that can take advantage of multiple cores.
• Modularity: OCaml's module system allows for clean separation of concerns, making it easier to write reusable and composable code.
• Performance: OCaml is a compiled language and has a reputation for being fast. In addition, the MirageOS team has put a lot of effort into optimizing the runtime system to make it even faster.
• Maintainability: Because OCaml code tends to be concise and expressive, it can be easier to maintain than code written in more verbose languages.

While Rust and C are also popular languages for systems programming, they have different design goals and trade-offs compared to OCaml. Rust, for example, places a strong emphasis on memory safety and has a unique ownership model to prevent common programming errors. C, on the other hand, is a low-level language that gives the programmer fine-grained control over memory and hardware resources. Each language has its own strengths and weaknesses, and the choice ultimately depends on the specific requirements of the project.

I hope this helps :)

Thanks for the info @0x0god , this was really helpful. But my main doubt was basically, why we prefer functional paradigm over OOP

[rand00]

@AryanGodara : There is a strong emphasis on correctness and safety in MirageOS, this is both in terms of the separation of the unikernel from the OS and other applications, and via the language OCaml.

As a statically typed functional language (with all that brings), OCaml is inherently more typesafe and easier to reason about than what is possible in OO languages.

Concerning the module system of OCaml which @0x0god mentioned, this is specifically also a nice fit for abstracting over different devices and backends with MirageOS. It is determined statically (when running mirage configure) what backend code will be included in the final unikernel, and with modules there is no performance-loss in contrast with dynamic dispatch of OO languages.

[0xrotense]

@AryanGodara : There is a strong emphasis on correctness and safety in MirageOS, this is both in terms of the separation of the unikernel from the OS and other applications, and via the language OCaml.

As a statically typed functional language (with all that brings), OCaml is inherently more typesafe and easier to reason about than what is possible in OO languages.

Concerning the module system of OCaml which @0x0god mentioned, this is specifically also a nice fit for abstracting over different devices and backends with MirageOS. It is determined statically (when running mirage configure) what backend code will be included in the final unikernel, and with modules there is no performance-loss in contrast with dynamic dispatch of OO languages.

Thanks for this...it helps :)

[AryanGodara]

@AryanGodara : There is a strong emphasis on correctness and safety in MirageOS, this is both in terms of the separation of the unikernel from the OS and other applications, and via the language OCaml.

As a statically typed functional language (with all that brings), OCaml is inherently more typesafe and easier to reason about than what is possible in OO languages.

Concerning the module system of OCaml which @0x0god mentioned, this is specifically also a nice fit for abstracting over different devices and backends with MirageOS. It is determined statically (when running mirage configure) what backend code will be included in the final unikernel, and with modules there is no performance-loss in contrast with dynamic dispatch of OO languages.

Thank you @rand00 and @0x0god , this really cleared up my doubts!

[reynir]

While OCaml in my opinion has clear advantages for writing unikernels there are unikernel projects that are written in C, Rust, and other languages. There is a list here though I don't know what the state is of the projects listed (besides Mirage) http://unikernel.org/projects/

[dianaoigo]

So far really enjoying the challenge. I've managed to run hello world and build for another backend. yaay. Loving every bit

[dinosaure]

It is clear that this response was generated by ChatGPT and that it conveys incorrect information (and that the reader should be made aware of this) with regard to MirageOS and OCaml. In particular, MirageOS is not compatible with OCaml 5 yet and we do not intend to use multiple cores (yet). The MirageOS team has also not been significantly involved in optimizing the OCaml runtime (other people/companies/groups are doing this work that we are discussing with but the MirageOS team has a more conservative approach to the OCaml runtime).

More generally, it is unfortunate to see this type of response especially in the context of Outreachy where mutual help, knowledge transfer and learning are central to the interactions between apprentices and mentors.

However, I hope that @reynir and @rand's answers (from their experiences) have clarified the questions originally asked.

[0xrotense]

It is clear that this response was generated by ChatGPT and that it conveys incorrect information (and that the reader should be made aware of this) with regard to MirageOS and OCaml. In particular, MirageOS is not compatible with OCaml 5 yet and we do not intend to use multiple cores (yet). The MirageOS team has also not been significantly involved in optimizing the OCaml runtime (other people/companies/groups are doing this work that we are discussing with but the MirageOS team has a more conservative approach to the OCaml runtime).

More generally, it is unfortunate to see this type of response especially in the context of Outreachy where mutual help, knowledge transfer and learning are central to the interactions between apprentices and mentors.

However, I hope that @reynir and @rand's answers (from their experiences) have clarified the questions originally asked.

Thanks for that,

I have a question; can you recommend any resources to learn/practice MBR/file system with OCaml?

[kamtoeddy]

Hi I have completed the setup and the First step (MirageOS hello-world)

My only issue is that I have not been able to setup the appropriate VS Code extensions for OCaml 😥 This is the extension I tried as recommended in the docs

[AryanGodara]

@kamtoeddy Just OCaml Platform extension should be enough, in the left bar (below ur docker, version control, etc) there is OCaml, where you can manage your Sandbox, Switches, and their installed dependencies, plus support for separate terminals.

[kamtoeddy]

Hi @AryanGodara, Upon selecting the sandbox, I get these errors

Thanks

[AryanGodara]

@kamtoeddy , Did you follow all the steps provided by @moazzammoriani for installing ocaml-platform? (And did you get the desired results there?) Maybe, you can show the dependencies installed in you currently applied opam switch. (The options is in GUI inside vsCode)

Next, can you check your ocaml --version opam --version ocaml-platform --version mirage --version (Atleast these should be installed correctly)

Then, there's this command you have to run

eval `opam config env`

In this maybe you can see everything that I have installed, and their versions. [Switch 4.13.1 is for ocaml-mbr repo, and switch default (ocaml 5.0.0) is for cardio-crumble]

I'm sorry for late reply, my mid-semester exams are going on. I'll update faster this time @kamtoeddy 😅