Understanding MultiThreading In Java

The semantics of multithreaded programs and the rules that says how the values may be seen by a read of shared memory that is updated by multiple threads are known as the Java programming language memory model. In this discussion we will see how Java threads are scheduled on different environments, how the java object wait set and thread wait works, difference between wait, sleep and yield in java. Also we will see how Thread.join example and how it actually works and how yield() is different from join(). We will also discuss about Preemptive Scheduling and Time Slicing and Concurrency and Parallelism in Java. Finally we will see Green Threads and Native Threads, how to scale Threads According to CPU Cores and checking Available processors and impact of Hyperthreading.

Java Synchronization, Monitor, Wait Sets and Notification Examples

Each Java object is associated with a monitor and a wait set. Java provides different mechanisms for communicating between threads and synchronization is one among them and is implemented using monitors. A thread can lock or unlock java object monitor but only one thread at a time may hold a lock on a monitor. Any other threads attempting to lock that monitor are blocked until the the monitor holding thread unlock it.
The synchronized statement first checks the reference to an object and then perform a lock action on that object’s monitor. If the object monitor is not already locked, the thread aquires the lock and then then executes the body of the synchronized block. Once completed either normally or abruptly, unlock action is triggered on the same monitor.
Another point to note here is that, if the method is an instance method, instance’s (“this”) monitor or if the method is static, it locks the monitor associated with the Class object that represents the class.

Object obj = new TestObject();
synchronized (obj) {
	if(safeToExecute()==false){
		obj.wait();
	}
} 

The statement “synchronized (obj)” will result if locking the monitor of Object obj if monitor is not already locked.

Java Object Wait Set and thread wait

As mentioned above each Java object is associated with a monitor and a wait set. When an object is first created, its wait set is empty. Wait sets are manipulated through methods Object.wait(), Object.notify(), and Object.notifyAll().
When a thread t be the thread executing the wait() method on object obj, then thread t is added to the wait set of object obj, and performs unlock actions on object obj’s monitor. Thread t does not execute any further instructions until it has been removed from object O’s wait set. To remove the thread from wait set, a notify(), notifyAll() action should be performed on object obj. Other scenario, when the thread t gets removed from obj’s wait list is because of any interrupt action(Thread.interrupt or ThreadGroup.interrupt) on thread t or if a timed wait (if wait() is called with time parameters) is elapsed.

Object obj = new TestObject();
synchronized (obj) {
	if(safeToExecute()==false){
		//This action adds the thread to obj's wait set and stops further execution.
		obj.wait();
	}
} 
makeSureSafety();
setSafeToExecute(true);
synchronized(obj) {
	//This statement will remove thread t from obj's wait set and resume execution of first synchronized block.
    obj.notifyAll();
} 

Mutex (Mutual exclusion locks) and Race Condition

In a multi-threaded system thread scheduling algorithm can swap between threads at any time, and a race condition or deadlock can happen when two or more threads can access shared data and they try to change it at the same time. Race condition usually causes unexpected results.In Java this can be avoided by putting a lock around the shared data. This is implemented using “Synchronized” block or functions. A Mutex (a mutually exclusive flag) acts as a key to a synchronized section of code allowing one thread in and blocking access to all others. The thread holding the mutex can execute the block and will be released only once it return from the code block.This ensures that the code being controled will only be executed by a single thread at a time.

Java Thread wait(), sleep() Methods and CPU Cycles

Difference between Wait and Sleep, Yield in Java

Waiting releases the lock on the object that wait() is called on but a call on sleep() on a thread does not release the locks (if any) it holds. Usually we dont call sleep() inside within a lock because that will block monitor from other threads. The waiting thread will not use any CPU cycle and OS won’t even try to schedule the task unless other thread calls notify() or notifyAll(). Also in case of sleep(..) OS doesn’t schedule the sleeping thread until requested time has passed. Sleep() causes the currently executing thread to sleep for a specified period of time.
yield() method pauses the currently executing thread temporarily. Yield gives a chance to the remaining waiting threads of the same priority to execute. If there is no waiting thread or all the waiting threads have a lower priority then the same thread will continue its execution.

Thread join() Java example and Difference between Yield and Join

See the eblow java code:

Thread t1 = new Thread(new Runnable() {
    public void run()
    {
         // Do some work
    }});  
t1.start();
Thread t2 = new Thread(new Runnable() {
    public void run()
    {
         // Do some work
    }});  
t2.start();

while (true) {
   try {
      t1.join();
      t2.join();
      break;
   } catch (InterruptedException e) {
      e.printStackTrace();
   }
}

So what does Thread.join() do? In one work Thread.join() is a blocker method, that means the main thread (the thread that makes t1.join() and t2.join()) will wait on the t1.join() until t1 finishes its work and retrun to main thread and then main thread call t2.join() and wait for t2 to complete its work and return to main thread. The main thread has to wait till both threads to finish to continue even though both threads have been running in parallel.
By now you clearly knows how Yield and Join are different in terms of functionality and behaviour.

Concurrency and Parallelism in Java

The JVM uses operating system native threads, and on mainstream platforms, threading behaviour essentially depends on underlying OS threading and the OS does the scheduling of threads.
Concurrency is a condition exists when a minimum of two threads making progress but they both never runs at the same instant of time. For example in a single core machine multiple threads are managed by time-slice scheme. In dual core or mult-core systems Parallelism is possible because two threads are executing simultaneously at the same instance of time. One core can execute 1 thread at a time. In a 4 core machine, a Java programme with 100 threads, all threads will share 4 CPU core by getting CPU time slice one by one or one thread per core at a time. Of those 100 threads, only 4 are running at a time, and the other 96 are waiting. The CPU will schedule threads and give each thread a slice of time to run on a core. One systems with Hyper-Threading CPU/processors CPU may have more than 2 threads per core.
Usually one thread per core appraoch is good for specialized CPU bound programs. For asynchronous computing usecases it is always good to have multiple threads utilizing limited CPU cores.

Java Thread Scheduling and Priority Levels. Preemptive Scheduling vs. Time Slicing

Preemptive Scheduling and Time Slicing in Java

Thread scheduling depends on the underlying OS policies. Preemptive scheduling ensures the highest priority thread to execute until it enters the waiting or completes its execution. Time slicing scheme is when thread is executed for a predefined slice of time and then re-enters into the pool of ready threads. Next thread will enter the execution phase for the time slice. Compared to time-slice schedulring, in preemptive scheduling, if a task with lower priority is executing, it can be preempted with a task with higher priority.
Java thread priority Levels set in the application are not always expected to be mapped to underlying OS “native” priority levels. Since java uses platform-dependent thread scheduling, it is not feasible to provide guarantees that apply across platforms with markedly different underlying threading models. So developers are well advised to limit their use of thread priorities to fine-tuning the responsiveness of a working application.
The priority levels are from Thread.MIN_PRIORITY to Thread.MAX_PRIORITY (values 1 and 10 respectively). The JVM defines a range of ten logical priorities for Java threads, including:

java.lang.Thread.MIN_PRIORITY  = 1
java.lang.Thread.NORM_PRIORITY = 5
java.lang.Thread.MAX_PRIORITY  = 10

In the Hotspot VM for Windows, setPriority() map to Windows relative priorities. This is set by the SetThreadPriority() API call.
Linux OS Thread Priorities and -XX:ThreadPriorityPolicy Option
The Linux® operating system supports various scheduling policies. The default universal time sharing scheduling policy is SCHED_OTHER, which is used by most threads. SCHED_RR and SCHED_FIFO can be used by threads in real-time applications.SCHED_OTHER uses time slicing, which means that each thread runs for a limited time period, after which the next thread is allowed to run. SCHED_FIFO can be used only with priorities greater than zero. This usage means that when a SCHED_FIFO process becomes available it preempts any normal SCHED_OTHER thread. SCHED_RR is an enhancement of SCHED_FIFO. The difference is that each thread is allowed to run only for a limited time period. If the thread exceeds that time, it is returned to the list for its priority.
Regular Java threads, that is, threads allocated as java.lang.Thread objects, use the default scheduling policy of SCHED_OTHER. Real-time threads, that is, threads allocated as java.realtime.RealtimeThread, and asynchronous event handlers use the SCHED_FIFO scheduling policy.
On Linux machine by switching to root and by using -XX:ThreadPriorityPolicy=1 will set the required thread priority levels as specified in application. See it here http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/87ee5ee27509/src/os/linux/vm/os_linux.cpp

How to Scale Java Threads According to CPU Cores?

Java :How To Get Number of Physical Core?

If you want to create number of threads spcifically based on number of cpu cores, you can create fixed thred pools using Executor service.

int processors = Runtime.getRuntime().availableProcessors();
ExecutorService e = Executors.newFixedThreadPool(processors);
//Ofcource, you can move the thread logic to some other class.
e.execute(new Runnable() {
        public void run() {
            // Your parallel task
        }
    });

Ensure to shutdown the ExecutorService once not required.

Java Availableprocessors and Hyperthreading

Runtime.getRuntime().availableProcessors() returns different number of cpu cores if the system is running on Hyper-Threading CPU/processors. Processors like Intel® Core™ i7 with 4 physical processor cores returns 8 as number of processors(threads). This will include four physical cores, plus something called “Hyper-Threading,” logic processors.

Green Threads and Native Threads.

Green threads are threads that are scheduled by a runtime library (such as VM) itself rather than natively by the underlying OS. Both are mechanisms to support multithreaded execution of Java programs. Current Java version use native threads though in Java 1.1, green threads were the only threading model used by the JVM. Green threads have some limitations such as inability to do pre-emptively switching between threads etc.

References

1. Java Platform Garbage Collection Tuning Guide
2. Real-time Threads
3. Defining Multithreading Terms