PowerShell Job Controller


Do you juggle? I shall always remember a wedding reception at which some of my friends broke out their clubs and balls and began team juggling in the banquet hall. Luckily, they were amazingly good jugglers and not too many in attendance were injured. Now about that lump on the back of my head…

Where was I?

Ah yes, juggling.

Does your shell juggle? Computing environments in the 1960s and 1970s included some beautiful job control languages for juggling computing workloads in timesharing and multiprocessing environments on mainframes and minicomputers. Circa 1978, Bill Joy developed the C shell as a part of the Berkeley UNIX distributions. It is worth noting that /bin/csh and its derivative /bin/tcsh have some fundamental job control features for running some work in the foreground, other work in the background, and switching between jobs. Also intrinsic in nearly any UNIX shell, even shells older than csh, was the ability to easily launch tasks into the background and pick up their output later.

Fast forward to the late 1990s. In PowerShell version 2.0, Microsoft introduced the ability to launch (background) jobs in PowerShell. With PowerShell version 3.0, the need arose to clarify the distinction between these Background Jobs and another new feature called Scheduled Jobs (different than scheduled tasks). PowerShell version 4.0 still supports both types of jobs, background and scheduled.

Although the ability to spawn one pipeline as a background job interactively in PowerShell, and then either launch some more background jobs or do some foreground work may seem natural with some practice, using background jobs inside a script can seem a bit more baroque.

Students in the “Automating Administration with Windows PowerShell” course (GK6457 / M10961A) which includes topics on both kinds of PowerShell jobs often have a question about how useful these features are. In addition, students in dozens of other courses which touch on use of PowerShell, as well as people looking for advice on Windows, System Center, SharePoint, Exchange, SQL, Lync, and other technologies often have a case of perplexion, scratching their heads wondering how jobs really fit into scripting and automation.

Consider the following script, which is one of at least thirty-one flavors of the type of examples I like to present to make the baroque seem more palatable. The goal is to be able to juggle several tasks at once. Juggling pins and balls are optional.

$dirJob = start-job { get-childitem c:,d:,e: }
while( $dirJob.State -eq “Running” ){
Start-Sleep -Milliseconds 500
Write-Warning “Waiting… $(get-date)”
# perhaps start another job, or check another job’s status
if( $dirJob.State -eq “Completed” ){
Write-Warning “Directory Listing job completed”
$dirJob | Receive-Job | ConverTo-Html |
out-file C:dirjob-$(get-random).htm
$dirJob | Remove-Job

Although this rudimentary example only launches one job, the while loop illustrates that something else can be done while this job is running, even if it is to do something as trivial as waiting, and noting that we are waiting; the comment is the key: “perhaps start another job, or check another job’s status.” The subsequent if statement receives the job output, converts it into HTML format and saves it to a file. Finally, the script removes the record within this shell instance that the job had been running, although the side-effects such as the resultant HTML file are still certainly in force. In reality, this sort of “job controller” script could evolve into something much more elaborate, juggling several balls and clubs at one time, catching each when the time is right. I find it best to introduce the job controller script concept with a simple example.

I will leave you with a slight variation on this theme which is not inordinately more complicated. This script takes a list of paths and searches all subfolders beneath those folders or file systems (drives) for PowerShell scripts. In the end, the output is shown in a PowerShell GridView window rather than sent to an HTML file. Neither of these are production scripts, just educational examples. For instance, in this second one there are several assumptions which are made, such as if the job is not running that it has completed and not failed, and furthermore not actually looking at how the user has answered the “Want output?” question but blindly unconditionally receiving the directory listing of the PowerShell scripts regardless of the answer (although typing Control-C to cancel further execution is an option).

param( $paths )
$j = start-job -ArgumentList $paths {
param( $pathsInside )
get-childitem -Recurse -Path $pathsInside -Filter *.ps1
while( $j.State -eq “Running” ){
write-output “Running…”
# launch other jobs, check other job status
start-sleep 5
Write-output “Job done”
$answer = read-host “Want output?”
Receive-Job -Job $j | Out-GridView

Windows PowerShell provides a rich set of capabilities for working with background jobs, which are certainly not limited to applicability to interactive shell use, but can be utilized quite powerfully to juggle simultaneous multiprocessing work loads with ease.