These are partial answers, as of 11/18/2010.  Will finish them by
11/20/2010.

***Definitions:

-Latency
-Throughput
-Scalability
-Deadlock (what are the necessary conditions)
-Livelock
-Starvation
-Lock contention
-Open call
-Spinlock

Answer: Get these from the book

***Short answer:

- What are some of the costs that are incurred by using threads?  What
  are ways to avoid some of these costs?  

  Answer: Thread creation, context switching, memory synchronization,
  and blocking.  See Chapter 11.3 in the book for an elaboration of
  each.  We can reduce these costs by reducing communication among
  threads, reducing lock contention (discussed more below), and
  reducing the number of threads.

- It is recommended that when using Executors the tasks be
  independent.  Why is this?  The ForkJoin framework does allow
  dependencies among tasks --- describe more precisely how this is so.
  Explain why such dependencies are OK in this context.

  Answer: if you have a fixed number of threads in your thread pool,
  it's possible you could deadlock; cf. one of the questions from the
  first midterm.  In a similar way, in a regular Executor if you had
  one task create two new tasks and then wait for those tasks to
  complete, you could end up having all available threads blocking on
  child tasks that are never allocated a thread.  The ForkJoin
  framework avoids this problem by allowing the thread running a task
  that waits for its children to start running another task instead,
  rather than blocking.

- How does the example program we gave in class, involving bank
  account transfers, prevent deadlock.  The solution appears to be
  pretty complicated --- the root of the complication is that the
  program does not know how many bank accounts there are.  If you did
  know that there was a fixed number of bank accounts (say, 2), how
  could your solution be made simpler?

  Answer: the solution ensures that most of the time all pairs of bank
  accounts are totally ordered, and thus the locks can be acquired in
  that order, avoiding the possibility of a circular waiting pattern
  (where one thread holds a lock and waits for a lock held by another,
  which is waiting for the lock held by the first).  If two accounts
  happen to have the same hashcode, a tie is broken by them having to
  acquire a third (global) lock before acquring their own locks which,
  again, prevents a circularity.

  If you only had two bank accounts, e.g., each stored in a
  differently named field, you could just pick the order in advance
  and always acquire the locks in that order.  No dynamic test would
  be needed.

- Improving performance by parallelization can actually increase the
  total work performed by all threads while reducing the latency of
  performing the work by all threads in parallel.  More specifically,
  some optimizations that could make a single-threaded program go
  faster could actually slow down a parallel program.  Can you give an
  example of such a situation?

  Answer: object pooling is one example.  This is a practice of not
  using new Foo(...) to create Foo objects, but rather to have a
  custom allocator with static methods 

    public static Foo newFoo();
    public static void freeFoo(Foo x);

  Calling freeFoo() sticks the object on a list, and newFoo() pulls an
  object off the list, if the list is non-empty, or else creates a new
  Foo object on the fly.

  The problem is that in a multithreaded setting, you need to
  synchronize newFoo() and freeFoo() to protect the list of Foo
  objects from data races.  But then this becomes a serialization
  bottleneck, slowing down your program.  You would be better off just
  allocating objects fresh, and avoiding the extra synchronization.
  (This is because in the JVM, the built-in allocator pre-allocates
  some memory to each thread, and since this memory is thread-local,
  no synchronization is required to use it.)

- Name three ways to reduce lock contention

  Answer: (1) Reduce duration that locks are held, e.g., using smaller
  synchronized blocks; (2) Reduce frequency of lock requests, e.g.,
  via lock splitting or striping; (3) Replace exclusive locks with
  coordination mechanisms that permit greater concurrency, e.g.,
  Reader/Writer locks, non-blocking data structures etc.

- Nonblocking synchronization are often referred to as "optimistic"
  compared to blocking (i.e., lock-based) synchronization, which is
  referred to as "pessimistic".  Explain this.

  Answer: it is optimistic in the sense that it assumes the best case:
  it performs an operation hoping the shared data it is working from
  is still up to date; i.e., the data is not contended.  If it's
  right, then the operation will proceed correctly without having to
  acquire any locks.  If it's wrong, it's answer will be incorrect and
  it will have to retry.  Using locks is pessimistic in that it
  assumes the worst: that contention could/will happen, and holding
  locks will prevent the possibility of it.

- Under what circumstances would you expect a non-blocking algorithm
  to outperform a blocking algorithm, e.g., a counter implemented with
  non-blocking synchronization vs. one implemented with blocking
  synchronization?  When would you expect you expect the reverse
  situation?

  Answer: a non-blocking algorithm will win when there is little to
  moderate contention, thus it will not pay to acquire locks and will
  rarely have to retry.  On the other hand, the blocking algorithm
  will win when contention is high, since it will completely avoid
  retries and only pay to acquire locks once.

- Explain what the CAS (compare-and-swap) operator does.  Provide
  pseudocode, if you wish.

  Answer: CAS(x,y) updates memory location y only if it currently
  contains x, otherwise it leaves it alone.  It returns the current
  contents of y.

- Give the definition of Amdahl's Law, explaining the parts of the
  equation you show.  When/how do you use this law?

  Answer:

                  1
  Speedup <= -----------
                  (1-F)
              F + -----
	            N

  where F is the fraction of work that must be executed serially, and
  N is the number of processors.  You can use this law in two ways.
  First, you can use it to judge, at a high level, how much more
  speedup you could hope to get out of the *problem* you are trying to
  solve.  Some amount of work is inherently serial: if operation A
  must occur after operation B, then the two cannot be parallelized
  under any circumstances and contribute to F.  So if you think of
  your problem at a high-level you can see the limits of the best-case
  solution.  Second, you can use Amdahl's law to analyze your current
  *implementation*.  We did this in class, inferring the F parameter
  for the WordCountParallelScale class by repeated runs.  We can then
  contemplate optimizations to our code and estimate whether they will
  reduce the F parameter or not, and thus improve performance.  One
  last note: synchronization operations impose serial execution; e.g.,
  using a blocking queue to coordinate two tasks requires each
  submission and removal from the queue to be serial, if the queue is
  using the monitor pattern.  These sorts of things must be accounted
  for.

- Erlang uses message passing to coordinate actions among concurrent
  tasks, while Java uses shared memory.  How do these two styles
  relate?  What are the advantages of one vs. the other?

  Answer: we can encode a shared memory location in a message passing
  system with a process with a state value that can be "read" (sending
  the "read" message and getting the current state back) and "written"
  (sending the "write" message overwrites the current state).  Thus,
  we can encode all the same problems as shared memory has.  However,
  we can not provide read/write messages for our shared "objects" in
  Erlang, but rather only the messages we want to be atomic, as in the
  sequence.erl example.  Thus the style or programming encourages
  atomicity, and helps to avoid bugs.  On the other hand, having lots
  of processes and shared communications between them is potentially
  slower than native access to shared memory.

- Erlang variables are "assign once".  Explain what this means.  Give
  a code example that shows how "assign once" is useful for verifying
  results.

  Answer: once I assign a value to a variable X, I can never assign
  any other value (that is not identical to the first) to X; Erlang
  will complain that the second value does not match the first.  As
  such, we can write code like

  X = mysort(L),
  X = quicksort(L)

  to confirm that mysort() and quicksort() return the same answer when
  given the same list.

- Can Erlang programs have atomicity violations?  If so, give an
  example.

  Answer: yes.  See example with read/write messages above, and in
  badsequence.erl linked from the course homepage.

***Java Programming:

- Write a parallel mergesort using the fork/join framework.  You can
  start from the MaxSolver.java and Maxproblem.java classes that we
  provided.  Use Arrays.sort() to perform the sequential work of
  sorting the array; you will have to write the merge() routine
  yourself (you can find code that does this on the Internet).  You
  could also make the problem a little simpler by merging the two
  classes and eschewing the separate notion of a SortProblem akin to
  MaxProblem.

  Answer:

//Copy into file SortProblem.java
import java.util.Arrays;

public class SortProblem {
    private final int[] array;
    private final int start, end;
    public SortProblem(int[] array, int start, int end) {
        int sz = array.length;
        if (start < 0 || start > sz || end <= start || end > sz)
            throw new IllegalArgumentException("Illegal problem");
        this.array = array;
        this.start = start;
        this.end = end;
    }
    public void solveSequentially() {
        Arrays.sort(array,start,end);
    }
    public int size() {
        return (end - start);
    }
    public SortProblem subproblem(int newStart, int newEnd) {
        newStart += start;
        newEnd += start;
        if (newStart > end || newEnd <= newStart || newEnd > end)
            throw new IllegalArgumentException("Illegal subproblem");
        return new SortProblem(array,newStart,newEnd);
    }
    public void merge(SortProblem right, int[] tmpArray) {
        if ((right.start != end) || (right.array != array) || (tmpArray.length < right.end))
            throw new IllegalArgumentException("Illegal merge");

        int leftPos = start;
        int rightPos = right.start;
        int rightEnd = right.end - 1;
        int leftEnd = rightPos - 1;
        int tmpPos = leftPos;
        int numElements = rightEnd - leftPos;
        
        // Main loop
        while( leftPos <= leftEnd && rightPos <= rightEnd )
            if( array[ leftPos ] <= ( array[ rightPos ] ) )
                tmpArray[ tmpPos++ ] = array[ leftPos++ ];
            else
                tmpArray[ tmpPos++ ] = array[ rightPos++ ];
        
        while( leftPos <= leftEnd )    // Copy rest of first half
            tmpArray[ tmpPos++ ] = array[ leftPos++ ];
        
        while( rightPos <= rightEnd )  // Copy rest of right half
            tmpArray[ tmpPos++ ] = array[ rightPos++ ];
        
        // Copy tmpArray back
        for( int i = 0; i < numElements; i++, rightEnd-- )
            array[ rightEnd ] = tmpArray[ rightEnd ];
    }
}

// Copy into file MergeSort.java
import java.util.Random;
import jsr166y.RecursiveAction;
import jsr166y.ForkJoinPool;

public class MergeSort extends RecursiveAction {
	private final static int THRESHOLD = 1000;
	private final SortProblem problem;
        private static int[] tmpArray;
	
	public MergeSort(SortProblem prob) {
		this.problem = prob;
	}
	
	@Override
	protected void compute() {
		if (problem.size() < THRESHOLD) {
			problem.solveSequentially();
		}
		else {
			int m = problem.size() / 2;
                        SortProblem left = problem.subproblem(0, m),
                            right = problem.subproblem(m,problem.size());
			invokeAll(new MergeSort(left),new MergeSort(right));
                        left.merge(right,tmpArray);
		}
	}
	
    /**
     * Creates an array of the given size with random contents.
     */
    static private Random rand = new Random();
    public static int[] createRand(int num) {
        int[] result = new int[num];
        for(int i = 0 ; i < num; i++)
            result[i] = rand.nextInt(1000);
        return result;
    }

	public static void main(String args[]) {
		int nThreads = 9;
		int arrSz = 100000000;
		int[] array = createRand(arrSz);
                tmpArray = new int[arrSz];
		SortProblem prob = new SortProblem(array,0,arrSz);
		long startTm = System.currentTimeMillis();
		prob.solveSequentially();
		long endTm = System.currentTimeMillis();
		System.out.println("SEQ: time ="+(endTm-startTm)+" ms");
		startTm = System.currentTimeMillis();
		ForkJoinPool pool = new ForkJoinPool(nThreads);
		MergeSort solver = new MergeSort(prob);
		pool.invoke(solver);
		endTm = System.currentTimeMillis();
		System.out.println("FJ: time ="+(endTm-startTm)+" ms");
	}
}

- If you have not done so already, implement the Maze solving
  algorithm using the puzzlesolving framework from Chapter 8.  This
  boilds down to changing the Maze class to implement the Puzzle
  interface (p. 183) and then using the code already provided with
  project 3 to execute the solving process.  Implement it as part of
  your project so you can confirm that the solution is correct.

- Recall the basic formula for implementing a non-blocking algorithm:
  read the current value of the state, make a change, and then try to
  write the change using CAS to make sure that the original value has
  not changed.  Using this idea, implement the add() and remove() below.

public class NonBlockingLinkedList { 
  private static class Node { 
    public final int item; 
    public Node next; 
    public Node(int item) { 
      this.item = item; 
    } 
  } 
  public void add(int item) { 
    // FILL IN
  }

  public void remove() {
    // FILL IN
  }
}

  Answer: look at SyncStack.java example we provided

- Suppose your application needs to keep an accurate count of the
  number of threads using a resource. You decide to use an
  AtomicInteger to maintain this count.  Fill in the code below to
  implement this strategy when a thread starts and stops using the
  resource.  You may not add any synchronized blocks or other locking
  mechanisms beyond that provided by AtomicInteger. For operations on
  AtomicIntegers you may only use the get() and compareAndSet(int
  expect, int update) methods.

public class CustomerTracking { 
  public AtomicInteger numCust = new AtomicInteger(0); // the count 
  public void enter() { // FILL IN 
  } 
  public void exit() { // FILL IN 
  } 
} 

  Answer:

public class CustomerTracking { 
  public AtomicInteger numCust = new AtomicInteger(0); // the count 
  public void enter() {
    boolean success = false; 
    do { 
      Integer old = numCust.get(); 
      Integer desired = new Integer(old + 1); 
      success = numCust.compareAndSet(old, desired); 
    } while (!success); 
  } 
  public void exit() {
    boolean success = false; 
    do { 
      Integer old = numCust.get(); 
      Integer desired = new Integer(old - 1); 
      success = numCust.compareAndSet(old, desired); 
    } while (!success); 
  } 
} 

- In class we went over the program processes.erl, which created N
  processes in Erlang, and measured the time to do so.  Write a
  similar program in Java and run both it and the Erlang program on
  your machine.  How much does the task creation time differ?  What
  are the ramifications of this difference on how you do programming
  in each language?

***ERLANG Programming

- Write a ring benchmark in Erlang.  Create N processes in a ring.
Send a message round the ring M times so that a total of N * M
messages get sent.  Time how long this takes for differernt values of
N and M.  Look at the example processes.erl file to see how timing is
done.  Otherwise, you should be able to use the standard spawn, send
(using !) and receive operations.

-module(process_send).
-export([max/2]).

max(N,M) ->
    Max = erlang:system_info(process_limit),
    io:format("Maximum allowed processes:~p~n" ,[Max]),
    First = create_process_list(N, self()),       
    io:format("Starting...~n", []),
    statistics(runtime),
    statistics(wall_clock),
    iter_n(M-1,fun(_) ->
		       First ! {"Hello World", 0},
		       receive
			   X -> X = {"Hello World", N}
		       end
	       end),
    {_, Time1} = statistics(runtime),
    {_, Time2} = statistics(wall_clock),
    U1 = Time1 * 1000 / (N * M),
    U2 = Time2 * 1000 / (N * M),
    io:format("Message send time=~p (~p) microseconds.~n" , [U1, U2]).

iter_n(0,F) -> F(0), ok;
iter_n(N,F) when N > 0 ->
    F(N),
    iter_n(N-1,F).

wait(Next) ->
    receive
        {X, N} -> Next ! {X, N+1}
    end,
    wait(Next).

create_process_list(0, Next) -> Next;                                
create_process_list(N, Next) ->
    Last = spawn(fun() -> wait(Next) end),
    create_process_list(N-1, Last).

- Implement a countdown latch in Erlang in the same style as the
  semaphore.erl implementation.  Recall: a countdown latch starts with
  a particular number, and has two functions: decrement() and
  await().  The latter blocks until the count reaches 0, while the
  former decrements the count (no blocking).

Answer:

-module(latch). 
-export([make_latch/1, await/1, decrement/1]). 

% Create a latch
make_latch(N) -> 
    spawn(fun() -> latch_loop(N,[]) end). 

% Server function
latch_loop(N,Pids) -> 
    receive 
        decrement -> 
	    if N > 1 ->
		    latch_loop(N-1,Pids);
	       true ->
		    lists:foreach(fun(Pid) -> Pid ! ok end,Pids),
		    latch_loop(0,[])
	       end;
	{From, await} ->
	    if N > 0 ->
		    latch_loop(N,[From|Pids]);
	       true ->
		    From ! ok,
		    latch_loop(0,[])
	    end
    end.

% Wait for latch to fire
await(Latch) -> 
    Latch ! {self(), await},
    receive 
        ok -> ok
    end.

% Decrement a latch 
decrement(Latch) ->
    Latch ! decrement,
    ok.

- Extend the implementation of blockingqueue.erl provided with project
  4 to implement a priority queue.  The API should now be

  take(Q) returns E where E is the highest priority item in Q.
    The highest priority item is the "smallest" according to Erlang's
    standard < ordering
  put(Q,E) enqueues E into Q

  (I changed the problem a bit from the original question, to make it
  a bit simpler.)

Answer:

-module(prioqueue).
-compile({no_auto_import,[put/2]}).
-export([make_queue/0, put/2, take/1]).
-export([test/0]).

% starts a process that handles put/take requests for the Q
make_queue() -> spawn(fun() -> queue_loop([]) end).

% used by queue_loop to find the least element of those queued up
least([H|T]) -> least(H,[],T).
least(Least,Rest,[]) -> {Least,Rest};
least(Least,Rest,[H|T]) -> 
    if H < Least ->
	    least(H,[Least|Rest],T);
       true ->
	    least(Least,[H|Rest],T)
    end.

% queue loop, for handling requests.  Its argument is the list of
% previously queued items
queue_loop(Elems) ->
    receive 
	% received request to dequeue
	{From, Id, take} ->
	    case Elems of
		[] -> % queue is empty
		    receive % wait for put
			{put, Element} -> 
			    From ! {Id, ok, Element},
			    queue_loop([])
			end;
		_ -> % queue non-empty; find the least item
		    {Least,Rest} = least(Elems),
		    From ! {Id, ok, Least},
		    queue_loop(Rest)
	    end;
	% received elem, but no one waiting for one; store it
	{put,Element} -> 
	    queue_loop([Element|Elems])
    end.

take(Q) ->
    Id = make_ref(),
    Q ! {self(), Id, take},
    receive
	{Id, ok, Element} -> Element
    end.

put(Q,Element) ->
    Q ! {put, Element}, 
    ok.

% Unit test
test() ->
    Q = make_queue(),
    L = mkList(10), % [10,9,8,...]
    lists:foreach(fun(M) -> put(Q,M) end, L),
    % dequeues least to greatest: 1,2,3,...
    M = lists:foldl(fun(_,Acc) -> X = take(Q), [X|Acc] end, [], L),
    lists:reverse(M),
    % reversed list should match the original
    M = L,
    spawn(fun() -> put(Q,10) end),
    timer:sleep(50),
    take(Q),
    ok.
		  
mkList(K) -> mkList(K,[]).
mkList(0,L) -> lists:reverse(L);
mkList(N,L) when N > 0 -> 
    mkList(N-1,[N|L]).


--The following ask you to write some standard functional programs in
Erlang.  Write them using recursion, and also try to use lists:map()
and lists:foldr().

-Write a function iter_n(N,F) that executes F(N), F(N-1), ..., F(0),
 and then returns ok.  If N < 0 the result is undefined (pattern match
 error).

-Write a function prod(list(integer())) -> integer() that returns the
 product of the elements of l. The function prod should return 1 if
 the list is empty.

-Write a function add_tail(list(X),Y) -> list(X|Y) that takes a list l
 and a single element e and returns a new list where e is appended to
 the back of l. For example, add_tail([1,2],3) = [1,2,3] (having type
 list(integer())), and add_tail([1,2],4.0] = [1,2,4.0] (having type
 list(integer()|float())).

-Write a function index(list(any()),any()) -> integer() that takes a
 list and a single element and returns the position of the last
 occurrence of that element in the list (indexed by zero). You should
 return -1 if the element is not in the list.

-Write a function app_int(fun(integer())->X,integer(),integer()) ->
 list(X) that returns the list [f(m); (f (m+1)); ...; f(n)] where f is
 the function passed to app_int, and m and n are the second and third
 arguments, respectively. It should return the empty list if n < m.

Answers:

These Erlang programs use recursion, not map/foldr.

iter_n(0,F) -> F(0), ok;
iter_n(N,F) when N > 0 ->
    F(N),
    iter_n(N-1,F).

prod([]) -> 1;
prod([H|T]) -> H * prod(T).

add_tail([],X) -> [X];
add_tail([H|T],X) -> [H|add_tail(T,X)].

% alternative to the above, which uses case:
add_tail2(L,X) ->
    case L of
        [] -> [X];
        [H|T] -> [H|add_tail2(T,X)]
    end.

index([],_X) -> -1;
index([H|T],X) -> 
  Y = index(T,X),
  if 
    Y == -1 -> 
      (if 
         X == H -> 0;
         true -> -1
      end);
    true -> 1+Y
  end.

app_int(F,M,N) ->
    if M > N -> [];
       true -> [F(M)|app_int(F,M+1,N)]
    end.

***Analysis:

- Here are three possible strategies for solving the maze in project
  3: one based on breadth-first search, one based on depth-first
  search (as in Sam Blitzstein's project solution, posted, that we
  discussed in class), and one based on bidirectional search, which
  starts from the entry and exit of the maze in parallel and tries to
  "meet in the middle".  Explain the basic idea of how each strategy
  might work, and describe why you think the strategy will produce a
  speedup compared to the sequential case for BFS, and why it might
  not.  Explain why your own project had a speedup, or didn't, as the
  case may be.

- The following program acquires locks on a set of resources. The
  acquire() method recursively acquires locks on each of the resources
  in the list, and then executes a Runnable command.  The main()
  method gives an example.  However, the ResourceMgr class could be
  used in a way that could lead to deadlock.  Produce an alternative
  main() method that will induce a deadlock (the main() method will
  have to spawn/use multiple threads).  Give an execution trace
  showing how it can lead to deadlock.

public class ResourceMgr { 
  Runnable comm; 
  ResourceMgr (Runnable comm) { 
    this.comm = comm; 
  } 
  public class Resource { 
    public synchronized void acquire(List<Resource> rest) { 
      if (rest.size() > 0) { 
        (rest.remove(0)).acquire(rest); 
      } else 
        comm.run(); 
    } 
  } 
  public static void main(String[] args) { 
    ResourceMgr bb = new ResourceMgr(new Runnable () {public void run () {}}); 
    // bb.new Resource() makes a new Resource inside bb 
    final ResourceMgr.Resource r1 = bb.newResource(); 
    final ResourceMgr.Resource r2 = bb.newResource(); 
    final ResourceMgr.Resource r3 = bb.newResource(); 
    List <Resource> resList = new ArrayList<Resource>(); 
    resList.add(r2); 
    resList.add(r3); 
    r1.acquire(resList); 
  } 
} 

Answer:

public static void main(String[] args) { 
  ResourceMgr bb = new ResourceMgr(new Runnable() {public void run() {}}); 
  final ResourceMgr.Resource r1 = bb.new Resource(); 
  final ResourceMgr.Resource r2 = bb.new Resource(); 
  final List<Resource> resList1 = new ArrayList<Resource>(); 
  resList1.add(r2); 
  final List<Resource> resList2 = new ArrayList<Resource>(); 
  resList2.add(r1); 
  Executor executor = Executors.newCachedThreadPool(); 
  executor.execute(new Runnable() {public void run() {r1.acquire(resList1);}}); 
  executor.execute(new Runnable() {public void run() {r3.acquire(resList2);}}); 
} 

- The following program can deadlock. Give an execution trace showing
how this can occur.  

public class Deadlock { 
  static class Friend { 
    public synchronized void bow(Friend bower) { 
      System.out.println("I bow to you"); 
        bower.bowBack(this); 
    } 
    public synchronized void bowBack(Friend bower) { 
      System.out.println("I bow back to you"); 
    } 
  } 
  public static void main(String[] args) { 
    final Friend f1 = new Friend(), f2 = new Friend(); 
    ExecutorService exec = Executors.newCachedThreadPool(); 
    exec.execute(new Runnable() {
      public void run() { while (true) {f1.bow(f2);}}}); 
    exec.execute(new Runnable() {
      public void run() { while (true) {f2.bow(f1);}}}); 
  } 
} 

Answer:
T1 enters bow - holds lock on f1 
T2 eners bow - holds lock on f2 
T1 cal ls f2.bowback (f1) - waits to acquire lock on f2 
T2 cal ls f1.bowback (f2) - waits to acquire lock on f1 


- Look at WordCountParallelScale.java and WordCountParallel.java and
  examine the differences.  In both cases, identify the sequential
  bottlenecks of the program (i.e., the parts that, no matter how many
  threads we use, will always take the same amount of total time), and
  the parallel parts.  Which has a greater degree of parallelism?  

Answer: 

The only difference between the two is in the main() methods.
WordCountParallel has greater parallelism because it will read from
the disk to get chunks of the file to count, and then immediately
creates and executes a task to count them.  So these tasks will
operate in parallel with other tasks, and with the main thread reading
from the disk.  WordCountParallelScale creates a list of counting
tasks while it reads in the chunks, and only when it's done reading
the file will it start the counting processes in parallel.

One sequential bottleneck is the reading of data from the disk.  This
cannot be effectively parallelized because the disk head can only read
one chunk at a time (some disk configurations might be able to do
better).  So the length to do the counting will always be at least as
long as the time to read from disk.  Another sequential bottleneck is
the task to identify the longest word.  While words can be discovered
in different file chunks in parallel, all executions will take at
least the time to find the longest word, e.g., if we chunked things
just right to find one word per task.  Finally, there is the
sequential cost of creating all of the threads and waiting for them to
complete.