New dotnet libraries and mechanisms

[yoav-melamed]

Hello Ami,
A chance to thank you for your great job!
I'm using STP a lot and this is defiantly my favorite library!

Do you consider for future releases some of the new dotnet libraries and mechanisms?

• async/await?
• ValueTasks?
• System.Threading.Channels as a Base queue?

Thanks!

[amibar]

Hi Yoav,

Can you please give examples of how you wish to use these features with STP?

Regards,
Ami

[yoav-melamed]

Hi Ami,
Just thought maybe it will be a nice improvement to replace some of the blocking code and having some async methods (such as 'await enqueue' or 'await WaitForIdle')

The idea to combine it with ValueTask is just for having a better allocation than just using a regular Task.

Regarding the 'System.Threading.Channels', one thing I like is the 'bounded queue' backpressure mechanism that automatically waits the queue will have a room for additional items when the queue is full (instead of throwing a 'queue is full' exception).

Yoav

[amibar]

Hi @yoav-melamed,

By async/await I thought you meant that when you enqueue an async method it will continue to work on the STP.

For example let's take this async method:

public async Task DoIt()
{ 
   // Do first thing
   await Task.Delay(10);
   // Do second thing
}

Today if you enqueue it into to the STP, the work item will complete after "first thing" is done, and "second thing" will be done on the .net Thread Pool (out of the STP threads).

With an async/await feature the STP can do the second thing on the STP and only then will the work item complete.

Is this a required feature? Or you just need the equeue and WaitForIdle to be awaitable ?

Ami

[yoav-melamed]

Hi @amibar
Sorry for the super-late response - I missed your comment :/
Yes, having an awaitable task in the queue will defiantly be a big step for STP
It will allow each thread to be more efficient and will defiantly help with writing an async code

In addition, since the WaitForIdle is an expensive blocking method, having it async would be great.

[MichelZ]

Oh yeah, async/await feature so that you can do QueueWorkItem(async () => await xxx())

[amibar]

Hi @yoav-melamed & @MichelZ,

I started to work on the async/await a while ago and I am still figuring out how to solve some scenarios.
However, why do you want the async/await feature instead of just using it directly ?
I just want to know that my work will be useful for you.

Regards,
Ami

[yoav-melamed]

Hi @amibar
The reasons from my end is to make every thread of STP the most efficient it could be:

• Having an awaitable task so each thread of the STP could do several async operations
• Having awaitable 'WaitForIdle' so while the main thread waits for the STP to complete, we can render the UI/do other things (this operation could take some time and in the meanwhile the tread could do other stuff while waiting for the STP to complete)
• Having the STP to stop or cancel in an async way (this operation could take some time and in the meanwhile the tread could do other stuff while waiting for the STP to complete)

That is what I had in mind
Thank you for your hard work!

[amibar]

Hi @yoav-melamed,

Thanks.

The threads of the STP suppose to be worker threads, hence they should use the CPU.
When your work item on the STP awaits, it just delegate the work to another arbitrary thread which the STP has no control on.

One of the ideas behind STP is to limit concurrency.
If, for example, you limited the concurrency to 1 (max threads) and send 10 work items that each one awaits for a file to download. You'll end up with 10 concurrent downloads that executes on the threads of the .net ThreadPool, while the STP thread is idle.

As for WaitForIdleAsync() and GetResultAsync(), it's quite reasonable tom implement.

What do you think?

Regards,
Ami

[MichelZ]

I often use it to limit concurrency, yes.
But I also need to use other api's in the worker threads sometimes - and they happen to be more and more async. So it's more a matter of convenience (.GetAwaiter(false).GetResult() anyone?)

[amibar]

Did you mean .ConfigureAwait(false) ?

[MichelZ]

Yeah, that one :)

[amibar]

Do you use the WorkItemGroups to limit concurrency or use the STP directly ?

[MichelZ]

Do you use the WorkItemGroups to limit concurrency or use the STP directly ?

Both. We use STP in a few projects

[amibar]

I have dilemas regarding async methods and the WorkItemGroup.

1. Lets say you enqueue 2 async work items into a WorkItemGroup with concurrency of 1. The 1st starts executing and then awaits, so the thread is released. Should the 2nd work item starts executing or should it wait until the 1st work item completes?
   (by "complete" I mean the async method returns rather than awaits)

2. Another case is when several work items are enqued to a WorkItemGroup and after a while they are all in awaiting state. Does it mean that the WorkItemGroup is idle ? Or only when all these work items complete?

[yoav-melamed]

Hi Ami,
For you question

Do you use the WorkItemGroups to limit concurrency or use the STP directly ?

Same as @MichelZ , I'm using STP in several projects and it depends on the project I'm working on. Mainly I'm using the STP directly, but in some project, I need to have 4 workers that collect data each from a different source, and add it to a centralized queue to be processed (and each worker have a different priority) so - I'm using the WIG there

When your work item on the STP awaits, it just delegate the work to another arbitrary thread which the STP has no control on

I thought a lot about what you said, and you are right - executing an async task will make the dotnet ThreadPool control the thread life cycle, which is the opposite of what we want.
For calling async API's I think it's better to use the old fashion way (and NOT using ConfigureAwait(false) because we do want the results to returned to the caller thread [which is the STP workitem thread]).

A Sample I wrote to demonstrate what I mean:

using System;
using System.Threading.Tasks;
using Amib.Threading;

const int NUM_OF_ITEM = 10;

SmartThreadPool stp = new();
stp.Concurrency = 1;

for (int i = 1; i <= NUM_OF_ITEM; i++)
    stp.QueueWorkItem(Executor, i);

stp.WaitForIdle(); // would like to have `await stp.WaitForIdleAsync()` here

Console.WriteLine("All Done.");

void Executor(int ItemNumber)
{
    Console.WriteLine($"Executing item number {ItemNumber}...");

    // executing some async API
    var task = Task<int>.Run(async () =>
    {
        await Task.Delay(1000);
        return new Random().Next(1, 1000);
    });
    var results = task.Result;

    Console.WriteLine($"Item {ItemNumber} has done with results of {results}.");
}