Implement Multithreading

[valentinbreiz]

This pull request is not to merge in master yet, this is more an opening for discussion on the implementation of multithreading in Cosmos.

All the original code of the first commit comes from @Og-Rok's work from Aura's fork of Cosmos. (the original pull request can be found here if needed: https://github.com/aura-systems/Cosmos/pull/40)

Here is a short demonstration video: https://www.youtube.com/watch?v=rea5sAkYyec

To do:

• [x] fix PC Speaker and process scheduler conflict due to PIT
• [x] alloc from new heap system
• [x] add test kernel

[madelynwith5ns]

This would be very beneficial.

[Nik300]

Guys have a huge problem with while loops... this is the code I am threading... and it just stops after some seconds

public void TaskRun()
{
            while (true)
                Sys.Emulation.Threading.TaskManager.Next();
}

public static void Next()
{
    foreach (Thread thread in LoadedThreads)
    {
       DateTime startTime = DateTime.Now;
       DateTime lastTime = startTime;
       while (lastTime.Second < startTime.Second + 1)
       {
            thread.exec.NextInstruction();
            lastTime = DateTime.Now;
        }
    }
}

[Nik300]

UPDATE: Ok so I just found out that the problem isn't the while loop but the keyboard instead... Here's the thing: When I load a task as long as I don't press any key it works... but as soon as I click a key the entire system shuts down

🤔

I'll dig more into that

EDIT: I think i may know the problem

The Console.ReadLine is being split across the threads needing it... let's say that i have 'Thread1' and 'Thread2' and both use 'Console.ReadLine()': the first thread starts and uses the keyboard to read the first 2 chars, then passes control to the second thread that takes the next two chars, and this happens until the user clicks the ENTER key and one of the two processes handles the result from the keyboard...

Now let's imagine a scenario where one of the two threads doesn't use 'Console.ReadLine()': as soon as thread1 passes control to thread2 the system crashes, because the thread finds something in its stack that shouldn't be there!

The only way to fix this is by modifying the behaviour of Console.ReadLine

EDIT: here's the proof:

[charlesbetros]

Is it any interrupt or just keyboard?

On Jan 4, 2021, at 12:21 PM, F4lc0131 [email protected] wrote:

 UPDATE: Ok so I just found out that the problem isn't the while loop but the keyboard instead... Here's the thing: When I load a task as long as I don't press any key it works... but as soon as I click a key the entire system shuts down

🤔

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[Og-Rok]

From what i remember with this, a lot of the core kernel systems (interrupts, heap, device management) are not thread safe, meaning that you will get a lot of situations like this where threads are tripping each other up. A fix for things like the heap would be to implement something like spinlocks

[Nik300]

Is it any interrupt or just keyboard? … On Jan 4, 2021, at 12:21 PM, F4lc0131 @.***> wrote:  UPDATE: Ok so I just found out that the problem isn't the while loop but the keyboard instead... Here's the thing: When I load a task as long as I don't press any key it works... but as soon as I click a key the entire system shuts down 🤔 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

as of now just the keyboard... I'll try to bust and fix those errors before releasing this to real cosmos

[Nik300]

Actually guys I had an idea on how we could handle this but I'll have to work on a "lock" system, so that a thread can acquire the lock over something (in this case the Console.ReadLine) tell me what you guys think (especially @valentinbreiz)

[Nik300]

For now this should do the job... but i'd like to do something more flexible

UPDATE: Good news: it doesn't crash! Bad news: after flushing the first thread, the input just hangs... 😢 have to work on this

UPDATE: Got it working guys! I'm gonna pull request the code to @valentinbreiz 's fork so that he can port it to this pull request

[Nik300]

UPDATE: Lol I just realized that there's something weirder going on... ok so with the Console.ReadLine() fix I still have issues with loading more than one thread that use a while loop... have to track that down too and fix that :(

UPDATE: No, i was just being an idiot :D, just forgot to fix the main problem... going to compile and see if it works, but i'll go to bed either if it does/doesn't work

EDIT: Nope doesn't work, ok i'll work on that tomorrow (i want to point out that as long as the second thread uses Console.ReadLine too, it works... just crashes when the second thread doesn't use it)

UPDATE: Ok so I think I got another problem like that, and surprisingly the problem is with Console.WriteLine().

Sys.Thread(() => { while (true) Console.WriteLine("test"); });

that's the thread, and happens the same exact thing... shell I add a locksystem to WriteLine too? Or just use the same lock in common with ReadLine?? lemme know

[Nik300]

UPDATE: Fixed! I think that Cosmos.Clear() needs the same treatment... because as soon as I use the command "cls" the system shuts down

[madelynwith5ns]

With the trend we're seeing with methods like Console.ReadLine() Console.WriteLine() and Console.Clear() I'm guessing that there is other methods with this issue.

[valentinbreiz]

You're right @TFTWPhoenix, some parts of Cosmos are not thread safe and maybe needs some locks too But WriteLine or Clear are working fine

[github-actions[bot]]

Stale pull request message

[ThomasBeHappy]

Will this ever be finished? Kinda really waiting for this.

[selkij]

Any news?

[AsertCreator]

are SMP and hyper-threading real on Cosmos? this multithreading uses only one core/logic processor

[valentinbreiz]

Not yet but it's on our plan once multithreading using APIC will be done

[zarlo]

then we just need multi core

[selkij]

Heya ! Yall got news ? I see that the todo is complete

[AsertCreator]

i think no. todo is complete, but all test kernels are failing, indicating that the processor scheduler doesnt work. (why does x86 hardware switch exists?)

[zarlo]

we are waiting for APIC https://github.com/CosmosOS/Cosmos/pull/2288

[AsertCreator]

so these are now connected?

[selkij]

are SMP and hyper-threading real on Cosmos? this multithreading uses only one core/logic processor

as I did a PR to move to limine its going to be easier for I saw that there was something with SMP

[AsertCreator]

limine smp api is available only if you're using limine boot protocol. since its almost impossible to port cosmos to limine boot protocol, we need to use apic

[selkij]

limine smp api is available only if you're using limine boot protocol. since its almost impossible to port cosmos to limine boot protocol, we need to use apic

Its not impossible at all, IL2CPU and Cosmos.Core just needs rework

[AsertCreator]

~i understood why tests are failing. only threading test is failing, it contains comment that BeforeRun, Run and AfterRun are not thread, they are just code. that means scheduler doesnt return back to them and test controller reports a timeout, because kernel didn't report success or failure~ no, i got it work another way

[AsertCreator]

i got this to work (maybe). currently there are two problems: stack corruption detection and Thread.Sleep method. first one thinks that stack was corrupted, although it just got replaced. second one is not implemented and makes thread sleep indefinitely, in other words, it makes thread die. when these are not used, multithreading somehow and somewhat works. do i need to submit PR?

[zarlo]

i would make a PR as the owner of the RP does not have time to work on this

[AsertCreator]

ok

[Neil-Ciciglycerin]

i found a bug with how multithreading interacts with the AC97 driver here