Answer 1

In a neural network every layer needs it's predecessor layer to compute it's output so for multithreading one is limited to only thread all nodes in one layer and waiting for all threads to finish and then move on to the next... I bet there are some tricks where you could partially compute the value although not all nods are finished but that's the basic idea. So as long as your computations per node/layer aren't very complex multithreading a neural network in a efficient way is hardly possible