How to run powershell 2 and 3 concurrently in windows 8
Taylor Gibb is a professional software developer with nearly a decade of experience. He served as Microsoft Regional Director in South Africa for two years and has received multiple Microsoft MVP (Most Valued Professional) awards. He currently works in R&D at Derivco International. Read more.
Windows 8 comes with a shiny new version of PowerShell, version 3. But while playing around with it, I have noticed a lot of scripts that I had written for version 2 are now throwing errors, so here’s how to get version 2 back while not losing version 3.
Launching PowerShell 3
Because it’s installed by default we don’t have to do anything special, just switch to the Start Screen and type PowerShell and hit enter. We can see our version by using the $PSVersionTable special variable.
Launching PowerShell 2
The awesome thing about Windows 8 is that Microsoft left the PowerShell 2 engine intact, that means we can easily access it. To get started create a new shortcut on your desktop.
The PowerShell directory has been pulled into our Path at system startup so we can reference the exe by name and pass it a version parameter, then click next.
Name the shortcut PowerShell 2 and click finish.
If you double-click on your new shortcut and check the PowerShell version with $PSVersionTable, you will see that its now on version 2.
Power Shell is actually the task automation framework of Microsoft. It consists of a command-line shell along with a scripting language. You might have been using Power Shell 2 or Power Shell 3 previously, but have you ever used both frameworks concurrently? No, then Windows 8 will help you in experiencing this functionality as well.
With a shiny version of Power Shell version 3, Windows 8 has upped their game. But while using version 3 people have been encountering errors. If you want to get the second version of Power Shell without losing the third then follow the steps given.
Others are Reading
- How to Set up Yahoo Mail on Mac
- IPhone Hotmail Email Settings Overview
Instructions
- font-size: 13px !important;color: #474747;text-align: justify;line-height: 21px;” >
The third version is already available so you do not have to launch it.
Go to the start screen and type Power Shell.
Using the variable $PSVersionTable, you can view the version by tapping Enter.
In order to launch Power Shell 2, you will first have to create a shortcut on your desktop.
Right click on the desktop and select New from the popup menu, from the drop down menu choose shortcut.
In the Type Name for this shortcut enter PowerShell 2 and click Finish.
Now you’ve both PowerShell 2 and 3 running in parallel.
If you have difficulties with these instructions then start again from the beginning and take your time to make sure that you do not skip any step. You can also check out some of the many forums and websites that deal specifically with Windows 8. You will find many expert users giving out some excellent tricks, tips and advice. Be sure to check out all of the previous posts to see if your questions have already been answered. You can also ask your own query on any forum to get more personalised assistance. Also, there are many excellent video tutorials that you can use to help with any type of situation or issue that you might be facing in Windows 8. These video tutorials are always a good idea as you can see each step being applied in front of you.
Take a quick trip to your nearest bookstore or library and check out the many books that discuss various issues and tips for using Windows 8. If you are a beginner then it might be a good idea that you learn and do as much research as possible to understand all the technicalities of Windows 8. This will help you with any future issues that you might face with Windows 8.
so I have two .ps1 scripts that check for new Tasks every 60 seconds and I want to run them at the same time. I use them every day and starting them gets a little annoying. Ideally, I want to write a script that I can run that runs those two for me.
The Problem is that as they don’t stop at some point, I can’t just start them one after the other and I can’t find a command to open a new PS ISE instance. However, one instance can’t run both scripts at the same time. The Ctrl+T Option would be perfect, but I can’t find the equivalent command.
Does anyone have an idea on how to solve this? Thanks!
3 Answers 3
I think what you want is something like
In addition to that: Use Import-Module $Modulepath inside the scripts to ensure the availability of the modules.
If you have Powershell 3.0+ you could use workflows . they are similar to functions, but have the advantage that you can run commands parallel.
so your workflow would be something like this:
keep in mind that a workflow behaves like a function. so it needs to be loaded into the cache first, and then called by running the RunScripts command.
Use CTRL+T to create a new powershell instance (a tab is created, which is called powershell 2, I believe) inside Powershell ISE.
From the new Powershell tab you can now open a second powershell script and run it aside the script running in other powershell tabs.
Not the answer you’re looking for? Browse other questions tagged powershell or ask your own question.
Related
Hot Network Questions
To subscribe to this RSS feed, copy and paste this URL into your RSS reader.
- Blog
Site design / logo © 2022 Stack Exchange Inc; user contributions licensed under cc by-sa. rev 2022.5.25.42222
By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.
I have a powershell script to do some batch processing on a bunch of images and I’d like to do some parallel processing. Powershell seems to have some background processing options such as start-job, wait-job, etc, but the only good resource I found for doing parallel work was writing the text of a script out and running those (PowerShell Multithreading)
Ideally, I’d like something akin to parallel foreach in .net 4.
Something pretty seemless like:
Maybe I’d be better off just dropping down to c#.
10 Answers 10
You can execute parallel jobs in Powershell 2 using Background Jobs. Check out Start-Job and the other job cmdlets.
The answer from Steve Townsend is correct in theory but not in practice as @likwid pointed out. My revised code takes into account the job-context barrier–nothing crosses that barrier by default! The automatic $_ variable can thus be used in the loop but cannot be used directly within the script block because it is inside a separate context created by the job.
To pass variables from the parent context to the child context, use the -ArgumentList parameter on Start-Job to send it and use param inside the script block to receive it.
(I generally like to provide a reference to the PowerShell documentation as supporting evidence but, alas, my search has been fruitless. If you happen to know where context separation is documented, post a comment here to let me know!)
Windows 8 มาพร้อมกับ PowerShell เวอร์ชั่นใหม่ที่แวววาวรุ่นที่ 3 แต่ในขณะที่เล่นกับมันฉันได้สังเกตเห็นสคริปต์มากมายที่ฉันเขียนไว้สำหรับเวอร์ชั่น 2 ตอนนี้กำลังโยนข้อผิดพลาดดังนั้นนี่คือวิธีรับเวอร์ชัน 2 กลับมาในขณะที่ไม่ สูญเสียรุ่น 3.
เปิดตัว PowerShell 3
เนื่องจากการติดตั้งเป็นค่าเริ่มต้นเราไม่จำเป็นต้องทำอะไรเป็นพิเศษเพียงสลับไปที่หน้าจอเริ่มและประเภท PowerShell และกด Enter เราสามารถดูเวอร์ชั่นของเราโดยใช้ตัวแปรพิเศษ $ PSVersionTable.
เปิดตัว PowerShell 2
สิ่งที่ยอดเยี่ยมเกี่ยวกับ Windows 8 คือ Microsoft ปล่อยให้โปรแกรม PowerShell 2 ครบถ้วนซึ่งหมายความว่าเราสามารถเข้าถึงได้อย่างง่ายดาย ในการเริ่มต้นสร้างทางลัดใหม่บนเดสก์ท็อปของคุณ.
ไดเรกทอรี PowerShell ถูกดึงเข้าสู่เส้นทางของเราที่เริ่มต้นระบบเพื่อให้เราสามารถอ้างอิง exe ตามชื่อและส่งผ่านพารามิเตอร์รุ่นแล้วคลิกถัดไป.
ตั้งชื่อช็อตคัท PowerShell 2 และคลิกเสร็จสิ้น.
หากคุณดับเบิลคลิกที่ทางลัดใหม่ของคุณและตรวจสอบรุ่น PowerShell ด้วย $ PSVersionTable คุณจะเห็นว่ามันเป็นเวอร์ชัน 2 ในขณะนี้.
PowerShell one-liners sometimes are an easy way to achieve something quickly. There are cases where you might want to run multiple separated PowerShell commands on one line. This can be handy if you do copy pasting of commands or if you want to make it easy for the end-user when you need to troubleshoot something. In this blog post, we will look at how you can chain and run multiple PowerShell commands on one line using pipelines and chaining commands.
📺 Agenda
0:00 Intro
0:27 Chain PowerShell commands
1:30 Pipe PowerShell
3:01 Single line
5:01 Chain Operators
8:41 Recap
⭐ About
Thomas works as a Senior Cloud Advocate at Microsoft. He engages with the community and customers around the world to share his knowledge and collect feedback to improve the Azure cloud platform. Prior to joining the Azure engineering team (Cloud + AI), Thomas was a Lead Architect and Microsoft MVP, to help architect, implement and promote Microsoft cloud technology.
If you want to know more about Thomas, check out:
PowerShell one-liners sometimes are an easy way to achieve something quickly. There are cases where you might want to run multiple separated PowerShell commands on one line. This can be handy if you do copy pasting of commands or if you want to make it easy for the end-user when you need to troubleshoot something. In this blog post, we will look at how you can chain and run multiple PowerShell commands on one line using pipelines and chaining commands.
📺 Agenda
0:00 Intro
0:27 Chain PowerShell commands
1:30 Pipe PowerShell
3:01 Single line
5:01 Chain Operators
8:41 Recap
⭐ About
Thomas works as a Senior Cloud Advocate at Microsoft. He engages with the community and customers around the world to share his knowledge and collect feedback to improve the Azure cloud platform. Prior to joining the Azure engineering team (Cloud + AI), Thomas was a Lead Architect and Microsoft MVP, to help architect, implement and promote Microsoft cloud technology.
If you want to know more about Thomas, check out:
I am trying to get a master powershell script to do the following:
- Run commands in 3 other powershell sessions (they all go for about
1h – I’d like them to run concurrently, so that the jobs they do can all get done at the same time)
Extremely simple example
My real use case is similar to the following, except the times always vary, ECHO “hi” should happen only once all the other (3) commands have finished (in this case we know they’ll take 10000 seconds, but in my actual use case this varies a lot). Also note, it’s not clear which of the 3 commands will take the longest each time.
I can see (here) that I can put an & in front of the command in order to tell powershell to wait until it’s complete before progressing to subsequent commands. However, I do not know how to do so with 3 simultaneous commands
2 Answers 2
You are indeed looking for Powershell background jobs, as Lee Daily advises.
However, jobs are heavy-handed, because each job runs in its own process, which introduces significant overhead, and can also result in loss of type fidelity (due to PowerShell’s XML-based serialization infrastructure being involved – see this answer).
The ThreadJob module offers a lightweight alternative based on threads. It comes with PowerShell [Core] v6+ and in Windows PowerShell can be installed on demand with, e.g.,
Install-Module ThreadJob -Scope CurrentUser . [1]
You simply call Start-ThreadJob instead of Start-Job , and use the standard *-Job cmdlets to manage such thread jobs – the same way you’d manage a regular background job.
Here’s an example:
The above yields something like the following; note that the individual commands’ output is reported as it becomes available, but execution of the calling script doesn’t continue until all commands have completed:
Note: In this simple case, it’s obvious which output came from which command, but more typically the output from the various jobs will run unpredictably interleaved, which makes it difficult to interpret the output – see the next section for a solution.
As you can see, the overhead introduced for the thread-based parallel execution in the background is minimal – overall execution took only a little longer than 10 seconds, the runtime of the longest-running of the 3 commands.
If you were to use the process-based Start-Job instead, the overall execution time might look something like this, showing the significant overhead introduced, especially the first time you run a background job in a session:
That is, at least on the first invocation in a session, the benefits of parallel execution in the background were negated – execution took longer than sequential execution would have taken in this case.
While subsequent process-based background jobs in the same session run faster, the overhead is still significantly higher than it is for thread-based jobs.
Synchronizing the job output streams
If you want show output from the background commands per command, you need to collect output separately.
Note: In a console window (terminal), this requires you to wait until all commands have completed before you can show the output (because there is no way to show multiple output streams simultaneously via in-place updating, at least with the regular output commands).
The above will print something like this:
Using Start-Process to run the commands in separate windows
On Windows, you can use Start-Process (whose alias is start ) to run commands in a new window, which is also asynchronous by default, i.e., serially launched commands do run in parallel.
In a limited form, this allows you to monitor command-specific output in real time, but it comes with the following caveats:
You’ll have to manually activate the new windows individually to see the output being generated.
The output is only visible while a command is running; on completion, its window closes automatically, so you can’t inspect the output after the fact.
To work around that you’d have to use something like Tee-Object in your PowerShell cmdlet in order to also capture output in a file, which the caller could later inspect.
This is also the only way to make the output available programmatically, albeit only as text.
Passing PowerShell commands to powershell.exe via Start-Process requires you to pass your commands as strings (rather than script blocks) and has annoying parsing requirements, such as the need to escape ” chars. as \” (sic) – see below.
Last and not least, using Start-Process also introduces significant processing overhead (though with very long-running commands that may not matter).
[1] In Windows PowerShell v3 and v4, Install-Module isn’t available by default, because these versions do not come with the PowerShellGet module. However, this module can be installed on demand, as detailed in Installing PowerShellGet
Geoffrey_Carr
“Windows 8” turi naują “PowerShell” versijos 3 versiją. Tačiau žaidžiant su juo, pastebėjau, kad daugelis scenarijų, kuriuos parašiau už 2 versiją, dabar klaidina, taigi, kaip gauti versiją 2, o ne praranda 3 versiją.
“PowerShell” paleidimas 3
Kadangi jis yra įdiegtas pagal nutylėjimą, mes neturime daryti nieko ypatingo, tiesiog pereikite į Pradžios ekraną ir įveskite “PowerShell” ir paspauskite “Enter”. Mes galime pamatyti mūsų versiją naudodami specialų kintamąjį $ PSVersionTable.
Paleidimas PowerShell 2
Puikus dalykas, susijęs su “Windows 8”, yra tai, kad “Microsoft” paliko “PowerShell 2” variklį nepažeistą, tai reiškia, kad mes galime lengvai jį pasiekti. Norėdami pradėti, sukurkite naują nuorodą darbalaukyje.
“PowerShell” aplankas buvo įtrauktas į mūsų kelią sistemos paleidimo metu, todėl galime nurodyti exe pavadinimu ir perduoti jo versijos parametrą, tada spustelėkite sekantį.
PowerShell –Version 2
Pavadinkite santrumpą PowerShell 2 ir spustelėkite baigti.
Jei du kartus spustelėsite savo naują nuorodą ir patikrinsite “PowerShell” versiją su “$ PSVersionTable”, pamatysite, kad ji dabar yra versija 2.
Geoffrey_Carr
Windows 8 идва с лъскава нова версия на PowerShell версия 3. Но докато играя с нея, забелязах, че много скриптове, които бях написал за версия 2, сега хвърлят грешки, така че ето как да получите версия 2, докато не губи версия 3.
Стартиране на PowerShell 3
Тъй като е инсталирана по подразбиране, не е нужно да правите нищо специално, просто преминете към началния екран и въведете PowerShell и натиснете клавиша enter. Можем да видим нашата версия, като използваме специалната променлива $ PSVersionTable.
Стартиране на PowerShell 2
Удивителното нещо за Windows 8 е, че Microsoft остави двигателя PowerShell 2 непокътнат, което означава, че можем лесно да го осъществим. За да започнете, създайте нов пряк път на работния си плот.
Директорията PowerShell е била изтеглена в нашата Path при стартиране на системата, за да можем да препратим exe по име и да й предадем параметър за версията, след което да кликнем на следващия.
PowerShell –Version 2
Наименувайте пряк път PowerShell 2 и кликнете върху края.
Ако кликнете два пъти върху новия ви пряк път и проверите версията PowerShell с $ PSVersionTable, ще видите, че сега е на версия 2.
Hello everyone! Doctor Scripto is elated to present some more great content from Joel Vickery, PFE , today he discusses using jobs in PowerShell. Stay tuned in the upcoming weeks for some amazing regular content. Take it away Joel!
I have to be honest, back in 2010 I was firmly entrenched in VBScript and had no interest in learning anything new. Then fate threw me a curve ball when I was presented with a task to fix over 9,000 workstations that had lost their “parent” antivirus server. Each machine needed to have the exact same set of commands executed on it, a new file copied and service restarted, to fix them so I created a quick VBScript to connect to each machine and perform the fix actions. The only bad part about this script was that it took days to run against the machines, processing each one sequentially in a For loop before moving on to the next. I did some research on multi-threading with VBScript and kept getting results that referred to PowerShell “jobs”, which gave me the motivation to start learning PowerShell.
For those of you who come from a Unix or Linux background, jobs will be a familiar concept since you can background any command by placing an “&” at the end of the command. PowerShell Cmdlets sometimes include the “-AsJob” parameter that will allow the command to submitted in the background, releasing the console to be used for other commands. Alternatively, you can use the Start-Job CmdLet to run commands as Jobs. This also means that you can submit multiple jobs in the background and continue processing without waiting for each job to complete. Keep in mind that there are limits to everything so keep your machine’s resource consumption in mind as you test this concept out. Submitting a large number of background commands could be resource intensive.
Below are two different ways to do a WMI Query as a job:
These jobs do all of their processing in the background and store the output until you receive them. You can check on the status of a job by running Get-Job, the status is in the “State” column, which will show if the command is Running, Completed, or Failed. Also notice the HasMoreData column. This indicates that there is output to be retrieved from the job. In the example output below, notice that the job is still running.
After the job has been kicked off, you can check on the status of the job by running the Get-Job command, noting the State and HasMoreData values. The State will change to Completed when the job has finished and the HasMoreData value will indicate if there is output.
Once the job has completed, you can use the Receive-Job cmdLet to get the data from the command.
The output of the command is now delivered to the console:
Note that the HasMoreData value has now changed to False after running the Receive-Job command:
The important thing to remember here is that you have one chance to get the information from the job so make sure that you capture it in a variable if the output needs to be evaluated.
Once you are done getting the job’s output, the job will basically hang out there until you remove it by running the Remove-Job Cmdlet.
As you can tell, there are a lot of moving parts to this.
The PowerShell script that I created to address the antivirus client remediation task had to have a mechanism to control how many jobs I could submit at a time, as well as constant monitoring of the queue to receive input from the completed jobs, remove them, and refill the queue with new jobs.
To top it off, once I finished processing the queue of items, I had to write some code to wait for the last batch of jobs was completed to make sure that I received the output from all of them.
That is a lot of overhead. Fast forward to today and we have PowerShell WorkFlows that make this process much easier to manage which I’ll cover in another post.
So that is all there is to with basic PowerShell jobs! Pop by next week as we look into some introductory work with PowerShell Workflows!
I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at [email protected], or post your questions on the Official Scripting Forum. See you tomorrow. Until then, Keep on Scripting!
Your good friend, Doctor Scripto
PowerShell, Doctor Scripto, Joel Vickery, Jobs
Recently, I received a task which required me to run a particular command on a several thousands of servers. Since execution of this command takes some time, it is just logical to run them in a parallel mode. And PowerShell Background Jobs are the first thing comes in mind.
But resources of my PC are limited — it cannot run more than a 100 jobs simultaneously. Unfortunately, PowerShell doesn’t have a built-in functionality for limiting background jobs yet.
Though, I’m not the first one who stuck with the same problem: official “Hey, Scripting Guy!” blog has introduced us a queue based on .NET Framework objects. But I couldn’t manage this solution to work and needed something simpler. After all, all we need are:
- a loop
- a counter, for how much jobs are active and running
- a variable, allowing next job in queue to start
Eventually, I came up with a piece of code like this:
Whereas my version is similar to a solution proposed at StackOverflow (which I only found AFTER completing my own version, of course), the SO version suffers from a bug where some items in queue may be skipped.
While PS Jobs are so easy to play with, Boe Prox claims that runspaces work much and much faster. Go, check it out, how you can use them to accelerate your job queue in his blog.
Some other queueing technics in PowerShell:
7 thoughts on “How to limit a number of PowerShell Jobs running simultaneously”
Thank you very much for the code snippet. Exactly what I was looking for.
Well, finally somebody who knows how to keep it simple. Nice and clean and works like a charm
Your version suffers from a while loop excessively checking/polling for the running status count to change
A better solution would be to use the built-in Wait-Job -Any command-let to check for threads finishing.
The problem with the stack overflow version is that it was doing an if else and it only waited in the else then moved on to the next iteration item without starting a job for it.
My suggestion is to check for the conditions to wait on first.
If the condition is true wait
then start the job – always happens.
foreach ($object in $objects) $running = @(Get-Job -State Running)
$freeMemory = (Get-WmiObject win32_OperatingSystem).freephysicalmemory * 1KB
if ($freeMemory -lt 1.5GB -or $running.Count -ge $maxConcurrentJobs) $null = $running | Wait-Job -Any
>
$scriptBlock = #Insert the code of your workload here
Write-Output “Do your work here…”
>
Start-Job -ScriptBlock $ScriptBlock
Start-Sleep -Seconds 3 # not needed but gives some breathing room.
>
I added the memory check to avoid throwing System.OutOfMemoryException but that value is dependent on your system
Nice catch, man! Didn’t pay enough attention to Wait-Job at the time. Thank you!
@Kirill and @Mackenson
thanks both so much. It’s so simple but effective and exactly what I needed.
I had been asked to supply a script with a list of objects, could be 10, could be 200, whatever the DB spits out, but only do 10 at a time, and NOT break up the script into multiple runs of 10.
I used the “Wait-Job -Any” code and I just added in front of it … write-host ” Waiting … ”
Great stuff.
All great examples, thank you. Simplified it a bit by creating a reusable function for this:
“`powershell
Function Wait-MaxRunningJobsHC
[CmdletBinding()]
Param (
[Parameter(Mandatory)]
[System.Management.Automation.Job[]]$Name,
[Parameter(Mandatory)]
[Int]$MaxThreads,
[Int]$FreeMemory = 1GB
)
Begin Function Get-FreeMemoryHC (Get-WmiObject win32_OperatingSystem).FreePhysicalMemory * 1KB
>
Function Get-RunningJobsHC @($Name).Where( )
>
>
Process while (
((Get-FreeMemoryHC) -lt $FreeMemory) -or
((Get-RunningJobsHC).Count -ge $MaxThreads)
) $null = Wait-Job -Job $Name -Any
>
>
>
“`
Short description
Provides information about PowerShell thread-based jobs. A thread job is a type of background job that runs a command or expression in a separate thread within the current session process.
Long description
PowerShell concurrently runs commands and scripts through jobs. There are three jobs types provided by PowerShell to support concurrency.
- RemoteJob – Commands and scripts run in a remote session. For information, see about_Remote_Jobs.
- BackgroundJob – Commands and scripts run in a separate process on the local machine. For more information, see about_Jobs.
- PSTaskJob or ThreadJob – Commands and scripts run in a separate thread within the same process on the local machine.
Thread-based jobs are not as robust as remote and background jobs, because they run in the same process on different threads. If one job has a critical error that crashes the process, then all other jobs in the process are terminated.
However, thread-based jobs require less overhead. They don’t use the remoting layer or serialization. The result objects are returned as references to live objects in the current session. Without this overhead, thread-based jobs run faster and use fewer resources than the other job types.
The parent session that created the job also monitors the job status and collects pipeline data. The job child process is terminated by the parent process once the job reaches a finished state. If the parent session is terminated, all running child jobs are terminated along with their child processes.
There are two ways work around this situation:
- Use Invoke-Command to create jobs that run in disconnected sessions. For more information, see about_Remote_Jobs.
- Use Start-Process to create a new process rather than a job. For more information, see Start-Process.
How to start and manage thread-based jobs
There are two ways to start thread-based jobs:
- Start-ThreadJob – from the ThreadJob module
- ForEach-Object -Parallel -AsJob – the parallel feature was added in PowerShell 7.0
Use the same Job cmdlets described in about_Jobs to manage thread-based jobs.
Using Start-ThreadJob
The ThreadJob module first shipped with PowerShell 6. It can also be installed from the PowerShell Gallery for Windows PowerShell 5.1.
To start a thread job on the local computer, use the Start-ThreadJob cmdlet with a command or script enclosed in curly braces ( ).
The following example starts a thread job that runs a Get-Process command on the local computer.
The Start-ThreadJob command returns a ThreadJob object that represents the running job. The job object contains useful information about the job including its current running status. It collects the results of the job as the results are being generated.
Using ForEach-Object -Parallel -AsJob
PowerShell 7.0 added a new parameter set to the ForEach-Object cmdlet. The new parameters allow you to run script blocks in parallel threads as PowerShell jobs.
You can pipe data to ForEach-Object -Parallel . The data is passed to the script block that is run in parallel. The -AsJob parameter creates jobs objects for each of the parallel threads.
The following command starts a job that contains child jobs for each input value piped to the command. Each child job runs the Write-Output command with a piped input value as the argument.
The ForEach-Object -Parallel command returns a PSTaskJob object that contains child jobs for each piped input value. The job object contains useful information about the child jobs running status. It collects the results of the child jobs as the results are being generated.
How to wait for a job to complete and retrieve job results
You can use PowerShell job cmdlets, such as Wait-Job and Receive-Job to wait for a job to complete and then return all results generated by the job.
The following command starts a thread job that runs a Get-Process command, then waits for the command to complete, and finally returns all data results generated by the command.
The following command starts a job that runs a Write-Output command for each piped input, then waits for all child jobs to complete, and finally returns all data results generated by the child jobs.
The Receive-Job cmdlet returns the results of the child jobs.
Because each child job runs parallel, the order of the generated results is not guaranteed.
Thread job performance
Thread jobs are faster and lighter weight than other types of jobs. But they still have overhead that can be large when compared to work the job is doing.
PowerShell runs commands and script in a session. Only one command or script can run at a time in a session. So when running multiple jobs, each job runs in a separate session. Each session contributes to the overhead.
Thread jobs provide the best performance when the work they perform is greater than the overhead of the session used to run the job. There are two cases for that meet this criteria.
Work is compute intensive – Running a script on multiple thread jobs can take advantage of multiple processor cores and complete faster.
Work consists of significant waiting – A script that spends time waiting for I/O or remote call results. Running in parallel usually completes quicker than if run sequentially.
The first example above shows a foreach loop that creates 1000 thread jobs to do a simple string write. Due to job overhead, it takes over 36 seconds to complete.
The second example runs the ForEach cmdlet to do the same 1000 operations. This time, ForEach-Object runs sequentially, on a single thread, without any job overhead. It completes in a mere 7 milliseconds.
In the following example, up to 5000 entries are collected for 10 separate system logs. Since the script involves reading a number of logs, it makes sense to do the operations in parallel.
The script completes in half the time when the jobs are run in parallel.
Thread jobs and variables
There are multiple ways to pass values into the thread-based jobs.
Start-ThreadJob can accept variables that are piped to the cmdlet, passed in to the script block via the $using keyword, or passed in via the ArgumentList parameter.
ForEach-Object -Parallel accepts piped in variables, and variables passed directly to the script block via the $using keyword.
Since thread jobs run in the same process, any variable reference type passed into the job has to be treated carefully. If it is not a thread safe object, then it should never be assigned to, and method and properties should never be invoked on it.
The following example passes a thread-safe .NET ConcurrentDictionary object to all child jobs to collect uniquely named process objects. Since it is a thread safe object, it can be safely used while the jobs run concurrently in the process.
If I have a script that I need to run against multiple computers, or with multiple different arguments, how can I execute it in parallel, without having to incur the overhead of spawning a new PSJob with Start-Job ?
But I don’t want to wait for each PSSession to connect and invoke the command. How can this be done in parallel, without Jobs?
4 Answers 4
Update – While this answer explains the process and mechanics of PowerShell runspaces and how they can help you multi-thread non-sequential workloads, fellow PowerShell aficionado Warren ‘Cookie Monster’ F has gone the extra mile and incorporated these same concepts into a single tool called Invoke-Parallel – it does what I describe below, and he has since expanded it with optional switches for logging and prepared session state including imported modules, really cool stuff – I strongly recommend you check it out before building you own shiny solution!
Reducing inescapable waiting time
In the original specific case, the executable invoked has a /nowait option which prevents blocking the invoking thread while the job (in this case, time re-synchronization) finishes on its own.
This greatly reduces the overall execution time from the issuers perspective, but connecting to each machine is still done in sequential order. Connecting to thousands of clients in sequence may take a long time depending on the number of machines that are for one reason or another inaccessible, due to an accumulation of timeout waits.
To get around having to queue up all subsequent connections in case of a single or a few consecutive timeouts, we can dispatch the job of connecting and invoking commands to separate PowerShell Runspaces, executing in parallel.
What is a Runspace?
A Runspace is the virtual container in which your powershell code executes, and represents/holds the Environment from the perspective of a PowerShell statement/command.
In broad terms, 1 Runspace = 1 thread of execution, so all we need to “multi-thread” our PowerShell script is a collection of Runspaces that can then in turn execute in parallel.
Like the original problem, the job of invoking commands multiple runspaces can be broken down into:
- Creating a RunspacePool
- Assigning a PowerShell script or an equivalent piece of executable code to the RunspacePool
- Invoke the code asynchronously (ie. not having to wait for the code to return)
RunspacePool template
PowerShell has a type accelerator called [RunspaceFactory] that will assist us in the creation of runspace components – let’s put it to work
1. Create a RunspacePool and Open() it:
The two arguments passed to CreateRunspacePool() , 1 and 8 is the minimum and maximum number of runspaces allowed to execute at any given time, giving us an effective maximum degree of parallelism of 8.
2. Create an instance of PowerShell, attach some executable code to it and assign it to our RunspacePool:
An instance of PowerShell is not the same as the powershell.exe process (which is really a Host application), but an internal runtime object representing the PowerShell code to execute. We can use the [powershell] type accelerator to create a new PowerShell instance within PowerShell:
3. Invoke the PowerShell instance asynchronously using APM:
Using what is known in .NET development terminology as the Asynchronous Programming Model, we can split the invocation of a command into a Begin method, for giving a “green light” to execute the code, and an End method to collect the results. Since we in this case are not really interested in any feedback (we don’t wait for the output from w32tm anyways), we can make due by simply calling the first method
Wrapping it up in a RunspacePool
Using the above technique, we can wrap the sequential iterations of creating new connections and invoking the remote command in a parallel execution flow:
Assuming that the CPU has the capacity to execute all 8 runspaces at once, we should be able to see that the execution time is greatly reduced, but at the cost of readability of the script due to the rather “advanced” methods used.
Determining the optimum degree of parallism:
We could easily create a RunspacePool that allows for the execution of a 100 runspaces at the same time:
But at the end of the day, it all comes down to how many units of execution our local CPU can handle. In other words, as long as your code is executing, it does not make sense to allow more runspaces than you have logical processors to dispatch execution of code to.
Thanks to WMI, this threshold is fairly easy to determine:
If, on the other hand, the code you are executing itself incurs a lot of wait time due to external factors like network latency, you can still benefit from running more simultanous runspaces than you have logical processors, so you’d probably want to test of range possible maximum runspaces to find break-even:
PowerShell one-liners sometimes are an easy way to achieve something quickly. There are cases where you might want to run multiple separated PowerShell commands on one line. This can be handy if you do copy pasting of commands or if you want to make it easy for the end-user when you need to troubleshoot something. In this blog post, we will look at how you can chain and run multiple PowerShell commands on one line using pipelines and chaining commands.
PowerShell Pipelines
In many cases, you will be happy to use pipelines in PowerShell. A pipeline is a series of commands connected by pipeline operators (|). Each pipeline operator sends the results of the preceding command to the next command.
For example, you can do things like get a process and then stop that process.
Get-Process “notepad” | Stop-Process
This will get the notepad process and pipe it to the next command, which will then stop that process. This is a great way to get the result from one PowerShell command to the other if these commands are related.
Chaining multiple PowerShell commands in one line
But what about if these commands are not related and have nothing to do with each other? In bash for example we can use && to chain commands and put two commands together. In PowerShell, we can achieve a similar thing using firstcommand; secondcommand . Expect that in this case, it will run the second command even if the first command fails. So for example in PowerShell you can use chaining:
Get-Process “notepad”; start “
In this case, we still do the first task, which is getting the notepad process, but we also want to open a website. If you can, it really makes sense to use PowerShell pipelines. However, if you want to do run multiple unrelated commands on the same line, you can use the firstcommand; secondcommand method. In this case, the second command gets executed even if the first command fails.
Chain PowerShell commands
This is also handy if one of the commands, for example, is a PowerShell command and the other might just a command-line tool.
This is nothing new and is working with Windows PowerShell 5.1 (and earlier) as well as PowerShell 7.
Pipeline Chain Operators
One of the new features in PowerShell 7 are the && and || operators to chain pipelines based on conditions. These operators are known in PowerShell as pipeline chain operators. They work very simply. The && operator would execute the right-hand pipeline if the left-hand pipeline succeeded. The || operator would run the right-hand pipeline if the left-hand pipeline failed.
Get-Process “notepad” && start “
If the notepad process was running, it would open the website. If notepad isn’t running, this command would just provide an error that the process “notepad” could not be found.
Get-Process “notepad” || Start-Process notepad
With this command, we would see if the notepad process is running. If it is running, the second command doesn’t get executed. If the notepad process is not running, this would now run the second command, which would start the notepad process.
PowerShell Pipeline Chain Operators
Make sure you check out the other new PowerShell 7 features. There is some cool stuff in there, like PowerShell Predictive IntelliSense, and much more.
Conclusion
You can see there are many different ways you can chain commands in PowerShell depending on your needs. If you have any questions, feel free to leave a comment.
Because Az PowerShell modules now have all the capabilities of AzureRM PowerShell modules and more, we’ll retire AzureRM PowerShell modules on 29 February 2024.
To avoid service interruptions, update your scripts that use AzureRM PowerShell modules to use Az PowerShell modules by 29 February 2024. To automatically update your scripts, follow the quickstart guide.
PowerShell supports asynchronous action with PowerShell Jobs. Azure PowerShell is heavily dependent on making, and waiting for, network calls to Azure. You may often find yourself needing to make non-blocking calls. To address this need, Azure PowerShell provides first-class PSJob support.
Context Persistence and PSJobs
Since PSJobs are run as separate processes, your Azure connection must be shared with them. After signing in to your Azure account with Connect-AzureRmAccount , pass the context to a job.
However, if you have chosen to have your context automatically saved with Enable-AzureRmContextAutosave , the context is automatically shared with any jobs you create.
Automatic Jobs with -AsJob
As a convenience, Azure PowerShell also provides an -AsJob switch on some long-running cmdlets. The -AsJob switch makes creating PSJobs even easier.
You can inspect the job and progress at any time with Get-Job and Get-AzureRmVM .
When the job completes, get the result of the job with Receive-Job .
Receive-Job returns the result from the cmdlet as if the -AsJob flag were not present. For example, the Receive-Job result of Do-Action -AsJob is of the same type as the result of Do-Action .
Example Scenarios
Create several VMs at once:
In this example, the Wait-Job cmdlet causes the script to pause while jobs run. The script continues executing once all of the jobs have completed. Several jobs run in parallel then the script waits for completion before continuing.
One of the challenges of working with so many Office 365/Windows Live ID’s/Microsoft accounts is that you are constantly logging in and out of them. You can run multiple browsers or run in incognito/private modes, but that can be a pain, and sometimes you want to be in multiple sessions concurrently. For example, you might have a personal Microsoft Account, a work account, and a Dev/Test/MSDN Tenant.
Another option is to run browser instances as different windows users using “Run As”. This runs the browser as a whole different user keeping the session, cookies etc. isolated.
However creating a user account, and then running the browser with the shift right click and “Run as a different user” and then entering the account details is a pain. Fortunately PowerShell to the Rescue. I’ve created a PowerShell module that sets this up for you. It creates a new windows account and an application shortcut on the desktop. The Shortcut actually runs a PowerShell script which then runs the target application as the given user account, meaning you don’t need to enter a password every time you launch the application.
The end result is a shortcut that runs as the user, making it easy to have a browser instance for each of your accounts.
Username and Password are for the local account that will be created
Module Path defines the module path, as this module is loaded at shortcut run
Here is what the desktop shortcut is actually running:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -command “& Import-Module ‘C:\Dropbox\PowerShell\Published\PowerShellRunAs’ ; PowerShell-RunAs -UserName msft -Password Password123 -Program ‘C:\Program Files\Internet Explorer\iexplore.exe’ -Verbose”
It’s actually running PowerShell, loading the module, then running the PowerShell-RunAs Function with the username, password and the application as parameters. It’s not the most secure thing in the world having the password in plain text I know.
I’ve not tested it extensively, but it works ok for me on Windows 8.1/10. If you hit an issue, the PowerShell window will likely launch and close too quickly to read, but you can paste that command into another PowerShell window to see what’s going on.
My view on IT from the trenches
Set Max Concurrent Tasks in Veeam with PowerShell
In this blog post, I’ll look at how to set the Max Concurrent Tasks in Veeam with PowerShell. When configuring your Veeam backup environment for the best possible backup performance there are a lot of settings to tweak. The defaults do a good job to get you going fast and well. But when you have more resources it pays to optimize. One of the things to optimize is Max Concurrent Tasks.
NOTE: all PowerShell here was tested against VBR v10a
Where to set max concurrent tasks or task limits
There are actually 4 places (2 specific for Hyper-V) where you can set the this in Veeam for a Hyper-V environment.
- Off-host proxy
- On-host proxy
- File Share Proxy (NEW in V10)
- Repository or SOBR extent
Max Concurrent Tasks on an off-host proxy
Task limit on the on-host Hyper-V proxy
Max Concurrent tasks on a file proxy (V10)
Limit maximum concurrent tasks on a repository or SOBR extent
Now, let’s dive into those a bit and show the PowerShell to get it configured.
Configuring the proxies
When configuring the on-host or off-host proxies, the max concurrent tasks are based on virtual disks. Let’s look at some examples. 4 virtual machines with a single virtual disk consume 4 concurrent tasks. A single virtual machine with 4 virtual disks also consumes 4 concurrent tasks. 2 virtual machines with 2 virtual disks each consumes, you guessed it, 4 concurrent tasks.
Note that it doesn’t matter if these VMs are in a single job or multiple jobs. The limits are set at the proxy level. So it is the sum of all virtual disks in the VMs of all concurrently running backup jobs. Once you hit the limit, as a result, the remainder of virtual disks (which might translate into complete VMs) will be pending.
set the max concurrent tasks for on-host proxies
set THE MAX CONCURRENT TASKS for off-host proxies
PowerShell code to set THE MAX CONCURRENT TASKS for file proxies
Last but not least, note that VBR v10 PowerShell also has the Get-VBRNASProxyServer and Set-VBRNASProxyServer commands to work with. However, initially, it seemed not to be reporting the name of the proxies which is annoying. But after asking around I learned it can be found as a property of the Server object it returns. While I was expecting $FileProxy. to exist (based on other Veeam proxy commands) I need to use Name$FileProxy.Server.Name
Configuring the repositories/SOBR extents
First of all, for Backup Repositories, the max concurrent tasks are not based on virtual disks but on backup files (.vbk, .vib & .vrb).
Secondly, you can use either per VM backup files or non-per VM backup files. In the per VM backup files every VM in the job will have its own backup file. So this consumes more concurrent talks in a single job than the non-per VM backup files mode where a single job will have a single file. Let’s again look at some examples to help clarify this. A single backup job in non-per VM mode will use a single backup file and as such one concurrent task regardless of the number of VMs in the job. A single backup job using per VM backup mode will use a single backup file per VM in the job.
What you need to consider with repositories is that synthetic tasks (merges, transformations, synthetic fulls) also consume tasks and count towards the concurrent task limit on a repository/etxent. So when setting it, don’t think is only related to running active backups.
Finally, when you combine roles, please beware the same resources (cores, memory) will have to be used towards those task limits. That also means you have to consider other subsystems like the storage. If that can’t keep up, your performance will suffer.
PowerShell code to set the task limit for a repository/extent
For a standard backup repositories this will do the job
For the extends of a SOBR you need to use something like this
I you put the output of Get-VBRBackupRepository in a foreach next you can also configuret/report on individual Backup repositories when requiered.
Conclusion
So I have shown you ways to automate. Similar settings for different purposes. The way off automating differs a bit depending on the type of proxy or if it is a repository. I hope it helps some of you out there.
Summary: Microsoft Scripting Guy, Ed Wilson, shows how to use background jobs to run a Windows PowerShell script that produces a server uptime report.
Hey, Scripting Guy! I like your script to get uptime and to get free disk space, but I cannot be responsible for running it on a regular basis. In addition, keep in mind we have a very large network, and the script will take a long time to run. Is there a command to make this run in the background?
Hello RE,
Microsoft Scripting Guy, Ed Wilson, is here. PowerShell Saturday in Charlotte, North Carolina is a little more than a month away (September 15, 2012), and we have all the sponsors and all the speakers lined up, the agendas posted, and nearly half of the tickets are already sold. If you want to attend, you need to get your reservations placed before the tickets are all gone. There will be 200 people attending, and it will be a tremendous day of learning delivered by some “rock star” trainers! There are three different tracks (Beginner, Advanced, and Applied), so there will be something for anyone who uses or anticipates using Windows PowerShell.
Use Start-Job to run the report
Note This is the fourth blog in a series about using the Convertto-HTML cmdlet and creating a HTML server uptime report. The first blog talked about creating an HTML uptime report, the second one used Windows PowerShell to create a report that displays disk space in addition to uptime, the third discusses adding color and stuff to the uptime report. You should read the three previous blogs prior to reading today’s.
If you have a large network, it can take a very long time to reach out and touch every server on the network and to bring back the information to a central location. One thing that can help is to run the script as a background job. Using jobs, Windows PowerShell will run the script in the background. You then have the option of using the Wait-Job cmdlet to pause the Windows PowerShell console until the script completes, or you can allow the job to complete. At any rate, it is not really necessary to use the Receive-Job cmdlet to receive the job because the HTML page creates it as part of the script, and not as part of the job itself. The command shown here creates a job and runs the script.
Start-Job -FilePath C:\fso\HTML_Uptime_FreespaceReport.ps1
To receive the job, use the following commands.
When you have done this, you may want to remove the job. This command accomplishes that task.
Use Invoke-Command to multithread the job
Using Start-Job on the local computer does not really do much in the way of speeding up things—it simply hides how long the script takes by placing it in the background. If you have a large network, use the Invoke-Command to run the commands against remote machines. Windows PowerShell automatically determines the best number of concurrent connections (by default 32). To use the Invoke-Command cmdlet effectively requires a few changes to the HTML_UptimeReport.ps1 script. The changes involve removing the hard-coded list of servers in the param portion of the script. This first change is shown here.
The next change involves removing the servers parameter from the call to the Get-UpTime function as shown here.
The next change I make changes the servers parameter on the Get-UpTime function. This change is shown here.
The last change removes the Invoke-Item command that displayed the completed HTML report. This will not work remotely anyway, and it is therefore unnecessary.
I leave the foreach command because I do not want to make more changes to the function than required. The revised HTML_UptimeReport.ps1 is shown here as Invoke_HTML_UptimeReport.ps1.
Invoke_HTML_UptimeReport.ps1
Foreach ($s in $servers)
$os = Get-WmiObject -class win32_OperatingSystem -cn $s
New-Object psobject -Property @ uptime = (get-date) – $os.converttodatetime($os.lastbootuptime)>>> ConvertTo-Html -As Table -body “ The following report was run on $(get-date)” >> $path I now open my Windows PowerShell console, and I retrieve and store a credential object to be used when making the remote connections. Next, I provide an array of server names. Finally, I call the Invoke-Command cmdlet and specify the computer names and the credentials. I use the FilePath parameter to point to a script that exists on my local computer. Next I use the AsJob parameter to run the command as a job. Finally, I specify a friendly name for the job. This command permits up to 32 simultaneous remote connections to obtain the information. The commands I type are shown here. $credential = Get-Credential -Credential iammred\administrator Invoke-Command -ComputerName $servers -Credential $credential -FilePath C:\fso\Invoke_HTML_UptimeReport.ps1 -AsJob -JobName uptime There is one problem with this command. The c:\fso\uptime.html command refers to a folder (c:\fso), which must exist on the remote server. Actually, several of my remote servers have a c:\fso folder, but the ones that do not will generate an error when I receive the job. The commands and associated output are shown in the following image. To override the default value that is contained in the script, I supply a value for the args parameter of the Invoke-Command cmdlet. The values for the args parameter must appear in the same order that the script declares the parameters. If I run the command and do not make provisions for multiple file access, file locking issues arise. There are two easy ways to deal with this issue. The first is to dial down the ThrottleLimit value to 1. Doing this, however, does no more than allow for a single connection at a time. This is the same thing the original script accomplished; and therefore, it is not a great solution. A better approach is to modify the output file name, to include the server name. Doing this permits simultaneous file access. Here is a modification to the code. >> (“_Uptime.html” -f $path, $env:COMPUTERNAME) When I use Invoke-Command now, I do not include the file name, but only the \\dc1\share path. This command is shown here. Invoke-Command -ComputerName $servers -Credential $credential -FilePath C:\fso\Invoke_HTML_UptimeReport.ps1 -AsJob -JobName uptime -args “\\dc1\share\” RE, that is all there is to using jobs to run a Windows PowerShell script and produce an uptime report. Join me tomorrow for more Windows PowerShell cool stuff. My name is Vimal Dweep Chauhan, I have around 15 years of experience in web development using Sitecore, Sharepoint and .Net, currently I am working in HCL Technologies, Noida, India. I have been working in Sitecore since 8+ years. My job profile primarily includes designing solutions, preparing, and implementing solutions for Sitecore applications. Technologies: .Net, C#, SQL, Sitecore, PXM, TDS, Unicorn, Helix & Habitat, Solr, InDesign Server CC 2015, Sharepoint, MVC, Asp.Net, Javascript, Entity Framework View all posts by Vimal Chauhan […] If you have not installed Solr 7.2.1 in your machine yet, you can refer to this quick guide. […] Some PowerShell cmdlets and Windows commands such as REG ADD and SUBINACL have to be run from an elevated prompt, there are several ways of doing this. It is possible to right click Powershell.exe (or it’s Start menu shortcut) and run it ‘As Admin’. To elevate a script from a (non-elevated) PowerShell command line: PS C:\> Start-Process powershell -ArgumentList ‘-noprofile -file MyScript.ps1’ -verb RunAs To run (and optionally elevate) a PowerShell script from a CMD shell, see the PowerShell.exe page. A set of commands can also be saved in a scriptblock variable, and then passed to a new (elevated) PowerShell session: To run an entire PowerShell session ‘As Admin’ from an existing PowerShell (non-elevated) session: PS> Start-Process powershell.exe -Verb runAs If you use Invoke-Command to run a script or command on a remote computer, then it will not run elevated even if the local session is. This is because any prompt for elevation will happen on the remote machine in a non-interactive session and so will fail. Using Enter-PSSession to start a whole new session will support elevation if you specify CredSSP, which enables the delegation of user credentials: New-PSSession ss64dom.com -Auth CredSSP -cred ss64dom\user64 A function that will return $True if the current session is running elevated or $False if not: To ensure a script is only run As Admin, add this to the beginning In PowerShell v4.0 the above can be simplified by using a #Requires statement: If a script needs to be run elevated, then you can ensure it will only ever be run elevated by including the logic within the script. If a scheduled task invokes a UAC prompt, then the task may fail to run unattended, to prevent this make sure that you select the ‘Run With Highest Privileges’ check box, (or run Set-ScheduledJobOption -RunElevated ) When a script is run with elevated permissions several aspects of the user environment will change: The current directory, the current TEMP folder and any mapped drives will be disconnected. To add a “Run as Administrator” context menu for .ps1 files, run this from an elevated PowerShell prompt: “Winners make a habit of manufacturing their own positive expectations in advance of the event” Windows PowerShell has developed into a great tool to manage and execute scripts on Windows machines. Its built into every install of Windows 10. In this post, we go into how to run plotters with PowerShell. There are a few advantages to running plotters in PowerShell: There are a few house keeping rules first before using PowerShell so you don’t hit the pitfalls that I have. Perform these steps: Now that PowerShell is setup to prevent you from making my horrible mistake, we can proceed with how to get a plotter started. In your PS window, enter the following command: In the command above, note the two bolded locations. The “ ” needs to be replaced with YOUR username on your machine. Next the “app-1.0.5” is going to be replaced with the version of chia that you are running. Currently the latest is 1.0.5. But it won’t be like this always. After you Change Directories (cd) to that folder, enter the following command to begin a plotter: There is a lot of information here so I’ll break it up: That’s pretty much it for the PowerShell command. Executing this command will start the plotter on its journey and it will print its progress in the window. Next is the scripting part. Below I have placed the script I use on my machine. Copy and paste this script into notepad and save the file as plotterscript.ps1, while selecting “All Files” for the file type: Here is the script. Copy/paste the invoke-expression line for each plotter you wish to start : Lets breakdown the script above. You will see semicolons in the script above. These separate each command of the new PowerShell window. There you have it, this should provide you with enough information in order to create and tweak your own scripts in order to maximize your plotters on your machine without having to worry about Chia Client issues. With the file saved, all you need to execute it is to right click the file and select “Run with PowerShell”. Limiting the number concurrently running Tasks is not that often requirement in a code, but eventually you’ll find yourself in the situation that for the most likely performance issues you’ll have to limit number of concurrently running Tasks. One of these situations is when you have long running operations in the Task. If you spin up to many Tasks you will seriously degrade your application performance. This can be easily done with System.Threading.Semaphore class which is introduced in .NET Framework 2. It is a robust class which can also provide concurrency limitation for the multiple processes as well as inside the process. Since we are only going to deal with Task concurrency inside the single process we’ll use System.Threading.SemaphoreSlim . It is a light-weight class for managing number of concurrently executing methods which is introduced in .NET Framework 4. The following a simple example of System.Threading.SemaphoreSlim usage. The maximum number of executing tasks is set to 3 and we’ll try to run 4 tasks at the same time. To simulate long running processing I used simple Thread.Sleep method. When you run the code above you will see only first three tasks executed right away, but the last 4th task will be executed with a delay of 3 seconds which is the delay value of the Thread.Sleep method parameter. Method SemaphoreSlim.Wait is a void method which will block the executing thread until all Tasks are finished. If one of your Tasks is “stuck” your invoking thread will be waiting for it to end. The nice thing is that Wait method has several overloads and one of them is a void method which accepts CancellationToken class instance. Once CancellationToken is canceled, in our case when timeout expires, it will throw OpeartionCancelledException which means Semaphore did not get released before the CancellationToken expired. Another overload of Wait method returns Boolean value and have expiration parameter which can be either System.TimeSpan or System.Threading.CancellaonToken . After the expiration condition is reached, code will continue running without waiting for semaphore to release. The following example will wait for semaphore to be released for 3 seconds. Since out task code will finish in 5 seconds, we will reach the Console message code which will inform that wait time expired before at least one task finished. If you need to limit Task executing in at the same time, one of these usages of System.Threading.SemaphoreSlim should be working for you. Purpose of the code contained in snippets or available for download in this article is solely for learning and demo purposes. Author will not be held responsible for any failure or damages caused due to any other usage. In Windows 8 and 8.1, like in previous Microsoft client OS versions, only one simultaneous incoming RDP connection is supported. It means that only one user (one session) can simultaneously connect to a Windows computer using the Remote Desktop. In most cases it is enough, but sometimes it would be useful if multiple remote users could work simultaneously in their own sessions. A good example is when a computer is used as a Media Center, when video is played in the system console session and you need simultaneously to work remotely with the system without interrupting the video on TV. When you trying to start the second RDP session to a computer with Windows 8.1 / 8, a message appears that another user is already signed in locally and this session should be closed. For example, in the server versions of Windows two simultaneous concurrent administrative connections with the individual sessions are supported (in case of the terminal RDS server this number may be even higher). Nevertheless, you can find a special patch in the web that allows to ignore this restriction. Due to this patch, multiple users can simultaneously connect to Windows 8 / Windows 8.1 computer over RDP. So, the patch replaces the original system file %SystemRoot%\System32\termsrv.dll (a library used by Remote Desktop Services). Here are the links to the modified library versions: Before you replace the library, back up original termsrv.dll using the command: Now, if something goes wrong, you can always roll back to the original configuration by replacing the current file with the original termsrv.dll_old. Download the archive with the patched termsrv library for your Windows version. In Windows 8 at first you have to change the values of the following keys in HKLM\System\CurrentControlSet\Control\Terminal Server\: The same operation can be performed from the command prompt: Then go to C:\Windows\System32, find the file termsrv.dll and open its properties. By default, the owner of this file is TrustedInstaller and even the administrator doesn’t have any right to replace it. Go to the Security tab and click Edit button. In the access list, find the local administrators group and give it full control over this file and save the changes. The next step before replacing the library file is to open the service management console (services.msc) and stop Remote Desktop Services. Copy termsrv.dll from the downloaded archive for your Windows version to %SystemRoot%\System32\ (with replacement). After replacing the file, run Remote Desktop Services and try to create two RDP sessions with the patched computer from different accounts. If you did everything correctly, two separate Remote Desktop session should open. Running PowerShell commands in Office 365 based environment Resource Mailbox is a special type or a Mailbox that uses for the management of Meeting Rooms and Equipment. The term “resource Mailbox” includes two types of Mailboxes – Room Mailbox and Equipment Mailbox. There is a small difference between the two types of the resource mailbox but the use, and the purpose is quite similar (most of the examples will relate to room Mailbox). A little confession – I must admit that it wasn’t easy to give up the strong need for naming these articles: Get a Room! 4sysops – The online community for SysAdmins and DevOps The Call Quality Dashboard (CQD) available in Office 365 for Microsoft Teams and Skype for Business is not a new tool. It was born from its on-premises server counterpart and translated for cloud usage. CQD is a collection of performance and diagnostic data gathered by the Teams or Skype client during an audio call, video call, or screen-sharing session. At the end of each session, the client packages and uploads this quality-of-experience data to a central reporting database. CQD provides a portal for analyzing this data and providing additional insights into the quality of these services. For example, you can build custom reports to see where on the corporate network audio calls are performing poorly. CQD allows for building reports and tables of data and provides a mechanism for extracting the data in the portal. However, if you have multiple reports used to extract data on a regular basis, this can be a tedious process to export the data from the portal manually. To assist with this process, Microsoft has built an API that allows extraction of this data, and on top of that, they have provided a PowerShell module that uses simple commands to extract the data. If you are unfamiliar with CQD, I would suggest checking out the following resources first: First, let’s discuss how to install this PowerShell module. Microsoft provides this module via the PowerShell Gallery repository. Hosting the module here allows for installing it directly from a PowerShell prompt using the Install-Module command. If you have never installed a module from the PowerShell Gallery, you may receive prompts about installing additional software as well as trusting the remote repository. Installing the CQD PowerShell module The CQDPowerShell module has three commands: While Get-CQDData is the workhorse in this module, we will not get far without the dimensions and measures. Dimensions and measures determine what kind of data we want to extract and how to group the data. Running the Get commands will display the property names we need later. These are some examples of the dimensions and measures returned from these commands: Examples of available dimensions and measures Before we get into extracting data using these properties, let’s look at a built-in report in CQD to understand why they are important. This is a typical report with the number of good, unclassified, and poor audio streams as well as the percent of poor audio streams. Given this is a small demo tenant, there’s not much in the way the chart looks, but that’s not what we’re focusing on. CQD example report If we edit this report, we will see that these chart properties are all measurements, and they are grouped by month and year or a dimension: Report details showing dimensions and measurements Looking at the left side of the edit window, we see the full names of the dimensions and measures used for building the report. Using the Get-CQDDimensions and Get-CQDMeasures commands and some filtering, we can find the equivalent names of these properties for our PowerShell command: Searching for matching dimensions and measures These are translations from the CQD report to the PowerShell equivalents: We now have all the pieces to get the CQD through our PowerShell command. The Get-CQDData command has several parameters, but the ones we are interested in for now are Dimensions, Measures, and OutPutType. We will build the command using the PowerShell names of the same dimensions and measures used in our report. Since there are spaces in them, we need to enclose each one in quotes and separate multiple ones with commas, and we will output the data into a DataTable: Get CQDData command with dimensions and measures By running this command, we now have the same data from our report listed in a PowerShell window, and we can save it to a variable or export it to a CSV file using other parameters of Get-CQDData. From here, I would recommend continuing to build reports in the CQD portal, and at the same time, find the PowerShell names of the dimensions and measures to build the equivalent command. This will allow you to automate the extraction of the data to ingest it into other tools for analysis. This can be a big time-saver if you are constantly going into the CQD portal to download this data manually. In Windows 8 and 8.1, like in previous Microsoft client OS versions, only one simultaneous incoming RDP connection is supported. It means that only one user (one session) can simultaneously connect to a Windows computer using the Remote Desktop. In most cases it is enough, but sometimes it would be useful if multiple remote users could work simultaneously in their own sessions. A good example is when a computer is used as a Media Center, when video is played in the system console session and you need simultaneously to work remotely with the system without interrupting the video on TV. When you trying to start the second RDP session to a computer with Windows 8.1 / 8, a message appears that another user is already signed in locally and this session should be closed. For example, in the server versions of Windows two simultaneous concurrent administrative connections with the individual sessions are supported (in case of the terminal RDS server this number may be even higher). Nevertheless, you can find a special patch in the web that allows to ignore this restriction. Due to this patch, multiple users can simultaneously connect to Windows 8 / Windows 8.1 computer over RDP. So, the patch replaces the original system file %SystemRoot%\System32\termsrv.dll (a library used by Remote Desktop Services). Here are the links to the modified library versions: Before you replace the library, back up original termsrv.dll using the command: Now, if something goes wrong, you can always roll back to the original configuration by replacing the current file with the original termsrv.dll_old. Download the archive with the patched termsrv library for your Windows version. In Windows 8 at first you have to change the values of the following keys in HKLM\System\CurrentControlSet\Control\Terminal Server\: The same operation can be performed from the command prompt: Then go to C:\Windows\System32, find the file termsrv.dll and open its properties. By default, the owner of this file is TrustedInstaller and even the administrator doesn’t have any right to replace it. Go to the Security tab and click Edit button. In the access list, find the local administrators group and give it full control over this file and save the changes. The next step before replacing the library file is to open the service management console (services.msc) and stop Remote Desktop Services. Copy termsrv.dll from the downloaded archive for your Windows version to %SystemRoot%\System32\ (with replacement). After replacing the file, run Remote Desktop Services and try to create two RDP sessions with the patched computer from different accounts. If you did everything correctly, two separate Remote Desktop session should open. PowerShell’s built-in BackgroundJob jobs (Start-Job) are run in separate processes on the local machine. This module extends the existing PowerShell BackgroundJob to include a new thread based ThreadJob job. This is a PowerShell’s built-in BackgroundJob jobs (Start-Job) are run in separate processes on the local machine. This module extends the existing PowerShell BackgroundJob to include a new thread based ThreadJob job. This is a ThreadJob jobs will tend to run quicker because there is lower overhead and they do not use the remoting serialization This module exports a single cmdlet, Start-ThreadJob, which works similarly to the existing Start-Job cmdlet. One difference is that ThreadJob jobs support a ThrottleLimit parameter to limit the number of running jobs, Source for this module is at GitHub. Please submit any issues there. Added Runspace cleanup. Last Updated: May 4th, 2021 by When you are working on the Windows OS, it is an important part of any system administrator to know how to a kill process using PowerShell. When Windows operating system starts, it will create a process for each system service. Windows assigns a unique PID “Process Identifier” for each process. For example, when you open the firefox application, the operating system creates a process for that application. There are several ways to kill a process in Windows operating system. In this tutorial, we’ll show you how to kill a process using PowerShell in Windows environment. Learn other useful PowerShell commands with this cheat sheet. To open a PowerShell interface, press Windows + R keys together on the keyboard (or click the Windows button and type in “run” and hit Enter button to bring up the run windows) and type the following in the Run box as shown below: Next, press OK button to open a PowerShell interface as shown below: The basic syntax of Taskkill command to kill a process is shown below: taskkill /F /PID process-number taskkill /IM process-name /F Next, open the PowerShell interface and run the following command to list all running processes and their PIDs: You should see the following screen: If you want to terminate the application WordPad, you will need to kill the process of wordpad.exe and cmd.exe application by its PID or by its name. To kill a wordpress.exe process by its PID (1180), run the following command: taskkill /F /PID 1180 To kill a cmd.exe process by its name, run the following command: taskkill /IM CMD.EXE /F Note : It is necessary to open PowerShell as Administrator to kill the process. The basic syntax to kill a procees with Stop-Process command in PowerShell as shown below: Stop-Process -Name process-name -Force Stop-Process -ID process-id -Force To kill a process with PID 1856 run the following command: Stop-Process -ID 1856 -Force To kill a process with process name WordPad run the following command: Stop-Process -Name wordpad -Force Similarly, you can kill the process of Notepad and Firefox with the following command: Stop-Process -Name notepad -Force In the above tutorial, we’ve learned how to kill a process using a PowerShell in Windows. Feel free to ask me if you have any questions below in the comments section! There are several ways to get a list of currently logged on users on a system, but only a few return the things that I like to know. In case of my servers, I’d like to know which users are connected to which session. One way to do so is by: but this doesn’t give a consistent return (on some of my 2012 R2 servers it doesn’t return anything, even though users are logged on to the system and RDS is setup) and it doesn’t include the 0 session user(s) WMI can also do the job, but by default will give a lot of unnecessary information (It’ll return DWM-1, DWM-2, DWM-3, IUSR, SYSTEM and more things I don’t need to know or work with) To filter most unwanted items, you’d get a big command like: This’ll return a list of all users (including administrator), but no sessions: So until now the first command would be the best; but since it’s pretty inconsistent wether it indeed returns a list of users, I thought of ‘misusing’ another command for this purpose. The Get-process command. Since every logged in user has several processes in their name, this might be the best approach: This will return a list of all users, but users that are connected to multiple sessions, will only show in one this way. To get the list of users based on the session and filter out the accounts like DWM-1 to 3 etc, the following command The Where-Object part will make sure the UserName field is filled when walking through it, otherwise you’d get lots of errors on not being able to call a method on a mull-valued expression. Thus to rule these out, this part is added. This will return an object with UserName and SessionID for that user. Once multiple users are connected to the same session (like session 0), the filter for the domain name or computer name is needed to get the list of names you’d be interested in. Here is a screenshot of the result of the last command: The IncludeUserName switch has been added since Powershell 4.0, thus older versions will not be able to use this switch. Also the switch requires this command to be ran elevated; otherwise you’d get an error. In the end I will definitely use the Get-Process command to get a consistent list of logged on users on my servers. Edit: By the way, if you’re about to run this on a remote computer, wrap an Invoke-Command around it. Because if you include the -ComputerName switch, the -IncludeUserName switch can’t be combined. Next to that, the Get-Process command with -ComputerName will return all processes as running on session 0. Thus if you are to use it on a remote computer, use a command like: Note that in the end there’s an extra Select-Object, because the Invoke-Command will add 2 extra properties to the result; PSComputerName and RunspaceId. You can also add credentials to this command and other parameters, depending on your needs. PowerShell 7.0 Preview 3 is now available with a new ForEach-Object Parallel Experimental feature. This feature is a great new tool for parallelizing work, but like any tool, it has its uses and drawbacks. This article describes this new feature, how it works, when to use it and when not to. ForEach-Object -Parallel is a new parameter set added to the existing PowerShell ForEach cmdlet. Don’t confuse ForEach-Object cmdlet with PowerShell’s foreach keyword. The foreach keyword does not handle piped input but instead iterates over an enumerable object. There is currently no parallel support for the foreach keyword. The new ForEach-Object -Parallel parameter set uses existing PowerShell Application Programming Interface [API] PowerShell currently supports parallelism in three main categories. This new feature uses the third method for running scripts in parallel. It has the least overhead of the other two methods and does not use the PowerShell remoting system. So it is generally much faster than the other two methods. However, there is still quite a bit of overhead to run script blocks in parallel. Script blocks run in a context called a PowerShell runspace . The runspace context contains all of the defined variables, functions and loaded modules. So initializing a runspace for script to run in takes time and resources. When scripts are run in parallel they must be run within their own runspace. And each runspace must load whatever module is needed and have any variable be explicitly passed in from the calling script. The only variable that automatically appears in the parallel script block is the piped in object. Other variables are passed in using the $using: keyword. Given the overhead required to run scripts in parallel, the -ThrottleLimit becomes very useful to prevent the system from being overwhelmed. There are some cases where running a lot of script blocks in parallel makes sense, but also many cases where it does not. There are two primary reasons to run script blocks in parallel with the ForEach-Object -Parallel feature (keeping in mind that this feature runs the script on separate system threads).Manual Installation
Usage Path All users (global path) C:\Program Files\WindowsPowerShell\Modules Specific user C:\Users\\Documents\WindowsPowerShell\Modules Custom location Any path Share this:
Like this:
Related
Published by Vimal Chauhan
Leave a Reply Cancel reply
Blog Stats
Recent Post
Arhives
Streamlined Sitecore 9.1.1 Installation – Bien Angelo Relucano on SOLR 7.2.1 Installation Guide with SSL for Sitecore August 2, 2021
Shortcuts can be edited to always run as Admin – Properties | Shortcut | Advanced then tick “Run as administrator”.Testing for Elevation
#Requires -RunAsAdministratorSelf-Elevating script
Scheduled Tasks
Windows Explorer Context Menu
How to throttle number of running tasks to a specific number in C#
References
Disclaimer
Article Table of content | Click to Expand
PowerShell | Help & additional information
To be able to run the PowerShell commands specified in the current article, you will need to create a remote PowerShell with Azure Active Directory or Exchange Online. In case that you need help with the process of creating a Remote PowerShell session, you can use the links on the bottom of the Article.
The Resource Mailbox doesn’t require a license.
Management of Resource Mailbox could implement by using one of the following options:
(But I know that this is a “serious” WebSite and, for this reason, I use a formal name).Subscribe to 4sysops newsletter!
They provide excellent isolation but are resource heavy. Running hundreds of BackgroundJob jobs can quickly
absorb system resources.
lighter
They provide excellent isolation but are resource heavy. Running hundreds of BackgroundJob jobs can quickly
absorb system resources.
lighter weight solution for running concurrent PowerShell scripts that works within the existing PowerShell job
infrastructure.
system. And they will use up fewer system resources. In addition output objects returned from the job will be
‘live’ since they are not re-hydrated from the serialization system. However, there is less isolation. If one
ThreadJob job crashes the process then all ThreadJob jobs running in that process will be terminated.
The main difference is that the jobs which are created run in separate threads within the local process.
and thus active threads, at a time. If more jobs are started then they go into a queue and wait until the current
number of jobs drops below the throttle limit.
Added Using variable expression support.
Added StreamingHost parameter to stream host data writes to a provided host object.
Added Information stream handling.
Bumped version to 2.0.0, and now only support PowerShell version 5.1 and higher.
Fixed using keyword bug with PowerShell preview version, and removed unneeded version check.
Added setting current working directory to running jobs, when available.
Added help URI to module. Hitesh J in Windows
Open a PowerShell Interface
Kill a Process with Taskkill
Kill a process using PowerShell
Stop-Process -Name firefox -Force Conclusion
What is ForEach-Object -Parallel?
ForEach-Object -Parallel is not the same as the foreach language keyword
How does it work?
When should it be used?
Highly compute intensive script. If your script is crunching a lot of data over a significant period of time and the scripts can be run independently, then it is worthwhile to run them in parallel. But only if the machine you are running on has multiple cores that can host the script block threads. In this case the -ThrottleLimit parameter should be set approximately to the number of available cores. If you are running on a Virtual Machine [VM]
The script above collects 50,000 log entries on the local machine from 10 system log names. Running this in parallel is almost twice as fast as running sequentially, because it involves some relatively slow disk access and can also take advantage of the machine multiple cores as it processes the log entries.
When should it be avoided?
ForEach-Object -Parallel should not be thought as something that will always speed up script execution. And in fact it can significantly slow down script execution if used heedlessly. For example, if your script block is executing trivial script then running in parallel adds a huge amount of overhead and will run much slower.
The above example, a trivial script block is run 1000 times. The ThrottleLimit is 5 by default so only 5 runspace/threads are created at a time, but still a runspace and thread is created 1000 times to do a simple string evaluation. Consequently, it takes over 10 seconds to complete. But removing the -Parallel parameter and running the ForEach-Object cmdlet normally, results in completion in about 18 milliseconds.
So, it is important to use this feature wisely.
Implementation details
As previously mentioned, the new ForEach-Object -Parallel feature uses existing PowerShell functionality to run script blocks concurrently. The primary addition is the ability to limit the number of concurrent scripts running at a given time with the -ThrottleLimit parameter. Throttling is accomplished by a PSTaskPool class that holds running tasks (running scripts), and has a settable size limit which is set to the throttle limit value. An Add method allows tasks to be added to the pool, but if it is full then the method blocks until a new slot becomes available. Adding tasks to the task pool was initially performed on the ForEach-Object cmdlet piped input processing thread. But that turned out to be a performance bottleneck, and now a dedicated thread is used to add tasks to the pool.
PowerShell itself imposes conditions on how scripts run concurrently, based on its design and history. Scripts have to run in runspace contexts and only one script thread can run at a time within a runspace. So in order to run multiple scripts simultaneously multiple runspaces must be created. The current implementation of ForEach-Object -Parallel creates a new runspace for each script block execution instance. It may be possible to optimize this by re-using runspaces from a pool, but one concern in doing this is leaking state from one script execution to another.
Runspace contexts are an isolation unit for running scripts, and generally do not allow sharing state between themselves. However, variables can be passed at the beginning of script execution through the $using: keyword, from the calling script to the parallel script block. This was borrowed from the remoting layer which uses the keyword for the same purpose but over a remote connection. But there is a big difference when using the $using: keyword in ForEach-Object -Parallel . And that is for remoting, the variable being passed is a copy sent over the remoting connection. But with ForEach-Object -Parallel , the actual object reference is being passed from one script to another, violating normal isolation restrictions. So it is possible to have a non thread-safe variable used in two scripts running on different threads, which can lead to unpredictable behavior.
Problem:
Due to various PowerShell errors, you cannot configure a server connection to the target on-premises Exchange environment in your CodeTwo software. You may encounter one of the following errors:
Solution:
The connection error can be caused by many different reasons. The problem is usually related to an invalid or incomplete configuration of either the source or target environment. The most common causes for those errors include:
You can also follow these instructions to simulate a remote PowerShell session to your server to exclude a possibility that this issue is caused by your CodeTwo software.
The target server is not accepting incoming PowerShell connections
Freshly installed Exchange servers may not be configured to accept incoming PowerShell connections. To fix this and let your server configure all the necessary services, execute the following command in PowerShell:
Make sure to run this command with local administrator permissions.
You use an old PowerShell version in either the source or the target environment
CodeTwo software works best if the latest available Windows PowerShell version is installed on the machine that sends (migrates) the data as well as on the target Exchange server which receives the data. To check which version of PowerShell is currently installed on either of your machines, run the following cmdlet in PowerShell:
and check the value for PSVersion (Fig. 1.).
Fig. 1. Checking the PowerShell version in Windows 10.
If there is no output visible after running this command, it means that you are using PowerShell 1.0 and the update is necessary.
It is strongly recommended to run the latest version of PowerShell on both machines. If necessary, update PowerShell to the latest available version for your operating system by following the steps from this Microsoft article.
Furthermore, if have installed the software on a machine running one of previous iterations of Windows (including Windows 7, Windows Server 2008 R2, and older), it is a good idea to test the software on another machine connected to the source environment where a newer version of Windows operating system is installed. If the connection succeeds on that machine, you should consider moving your migration tool installation there.
The source server cannot properly authenticate with the target environment
It may be that your machine cannot resolve the FQDN or DNS name of the server to which you are connecting. To fix this, you can try one of the following solutions:
- If your source environment has a physical connection with an entire target environment (including its Domain Controllers and DNS servers), you can try configuring the source DNS server so that it has the necessary information to authenticate PowerShell connections.
- If your source environment is only able to access the target Exchange server and not the other machines in the target environment, you need to configure a connection to that Exchange Server via an IP address. However, you need to make sure that:
- the machine where the software is installed allows using basic authentication for WinRM protocol. To verify this, execute the following command in the Command Prompt: If this returns Basic = false , run the following command to enable basic authentication:
- the target server has Basic authentication for PowerShell connections enabled.
The WinRM configuration prevents the connection
Another possible reason for these errors to occur is when the WinRM (Windows Remote Management) service is not configured to accept a remote PowerShell connection that the program is trying to make. You can troubleshoot this problem by:
Verify that your source server can connect to the target one
- Open PowerShell on your target server and execute the following cmdlet to view all trusted hosts that can connect to that server. By default, the list of trusted hosts is empty.
- If your source machine is not listed as a trusted host, you can add it to the list by running this cmdlet:
where ‘ ‘ is the FQDN of the host. Using the asterisk (*) wildcard character is permitted, e.g. *.domain.com. You can also allow all machines to connect by inserting the asterisk only:
Disable the SSL requirement
If the SSL connection cannot be established, you can consider disabling SSL requirement for PowerShell connections. To do so, follow these steps:
- Open Internet Information Services (IIS) Manager.
- Navigate to >Sites >Default Web Site >PowerShell.
- Double-click SSL Settings (Fig. 2).
Fig. 2. Accessing the SSL settings in IIS Manager.
- Clear the Require SSL checkbox and select to Ignore client certificates (Fig. 3). Apply the changes.
Fig. 3. Disabling the SSL requirement for PowerShell.
Keep in mind that disabling the SSL requirement should only be done in an isolated environment where there is no threat of intercepting the migrated data over a wire.
The network configuration prevents the connection
If you have followed all the steps above and are still unable to configure the connection, double check your network configuration, including all physical/software firewalls. By default, PowerShell uses the following TCP ports communication:
- 5985 (or 80) for HTTP
- 5986 (or 443) for HTTPS.
Simulating a remote PowerShell session
You may attempt to manually simulate a remote PowerShell session in order to connect to your target server in the same way this is done by the software. This will help you verify whether the problem is related to the CodeTwo software or a specific configuration of your environment.
To do so, follow these steps:
- when connecting via FQDN, refer to this article;
- when connecting via IP, use the following commands in Windows PowerShell:
