parallelize

parallelize ( list list , int nbThreads , closure method , int _verbose ) : void

Launch parallel threads which run method on some items of the list. All actions in methods must be independant with elements of the list.
Be careful to use local variables in your method.
To fully understand parallelisation, please read this article.
Returns true when all threads are finished. Note parallelize interrupt the execution of the code in the main tread (synchronous execution). So your code continue its execution when all parallelized threads have finished.

Example


parallelize( [1,2,3,4,5] , 2 , {def param->
print(param)
} )

Same than
parall={def param->
print(param)
}

parallelize([1,2,3,4,5],2, parall )


It is often appropriate to list the tasks to be accomplished by asking yourself the question "what are the variable elements of my tasks?" and then paralleling this list of tasks:

myTasks=[
["id_category":27, "page":1],
["id_category":27, "page":2],
["id_category":27, "page":3],
["id_category":580, "page":1],
["id_category":580, "page":2],
]
parallelize(myTasks,50, { def params->
def id_category = get(params, "id_category")
def page = get(params, "page")

process_category_page(id_category, page)
} )


Note on multi-thread operations

All Grimport variables are global by default, use "def" to make local variables in your method. Use synchronize to manage shared variables between thread (read and write operations).

See also

getOrCreateThreadObject
asynchronous
synchronize
waitDelay

Other solution : Gpars

You can use in another way the Gpars features, like that :

import groovyx.gpars.*

list = [ 1, 2, 3, 4, 5 ]
GParsPool.withPool{
result = list.eachParallel {param->
print(param)
}
}
With Gpars the number of optimal threads is calculated automatically. In a withPool block, you can use features each, collect and find like normally, adding the term Parallel at the end.
This functionality is taken from Groovy but we note however generally unsatisfactory results in the context of Web-Mining with this method.

Parameters

list

list where are arguments or array of arguments for the method

nbThreads

Number of parallel threads to launch

method

The function which is launched in threads

_verbose (optional)

Default 2. Conditions the number of messages returned by the function.
• 0 = none
• 1 = just 1 at the end
• 2 = more messages