Question :
I recently started with python’s threading module. After some trial and error I managed to get basic threading working using the following sample code given in most tutorials.
class SomeThread(threading.Thread):
def __init__(self, count):
threading.Thread.__init__(self)
def run(self):
print "Do something"
My problem is: I have a Class that has class variables and a function that I want to be run in a separate thread. However the function uses class variables and also writes to class variables. Like so:
class MyClass:
somevar = 'someval'
def func_to_be_threaded(self):
# Uses other class functions
# Do something with class variables
So how would I essentially ‘put the thread class in MyClass’. So that if MyClass().func_to_threaded() is called it would run in a thread.
Answer #1:
If I understand correctly you want to run a function in a separate thread? There are several ways to do that. But basically you wrap your function like this:
class MyClass:
somevar = 'someval'
def _func_to_be_threaded(self):
# main body
def func_to_be_threaded(self):
threading.Thread(target=self._func_to_be_threaded).start()
It can be shortened with a decorator:
def threaded(fn):
def wrapper(*args, **kwargs):
threading.Thread(target=fn, args=args, kwargs=kwargs).start()
return wrapper
class MyClass:
somevar = 'someval'
@threaded
def func_to_be_threaded(self):
# main body
Edit Updated version with a handle:
def threaded(fn):
def wrapper(*args, **kwargs):
thread = threading.Thread(target=fn, args=args, kwargs=kwargs)
thread.start()
return thread
return wrapper
class MyClass:
somevar = 'someval'
@threaded
def func_to_be_threaded(self):
print 'xyz'
This can be used as follows:
>>> my_obj = MyClass()
>>> handle = my_obj.func_to_be_threaded()
>>> handle.join()
Now it is possible to extend it even more if you wish to return a value from the function. Consider this:
from threading import Thread
from concurrent.futures import Future
def call_with_future(fn, future, args, kwargs):
try:
result = fn(*args, **kwargs)
future.set_result(result)
except Exception as exc:
future.set_exception(exc)
def threaded(fn):
def wrapper(*args, **kwargs):
future = Future()
Thread(target=call_with_future, args=(fn, future, args, kwargs)).start()
return future
return wrapper
class MyClass:
@threaded
def get_my_value(self):
return 1
>>> my_obj = MyClass()
>>> fut = my_obj.get_my_value() # this will run in a separate thread
>>> fut.result() # will block until result is computed
1
If you don’t have concurrent.futures.Future class (because for example you are using Python2.7 or older) then you can use this simplified implementation:
from threading import Event
class Future(object):
def __init__(self):
self._ev = Event()
def set_result(self, result):
self._result = result
self._ev.set()
def set_exception(self, exc):
self._exc = exc
self._ev.set()
def result(self):
self._ev.wait()
if hasattr(self, '_exc'):
raise self._exc
return self._result
I advice reading through concurrent.futures module since it has a lot of neat tools. For example Thread
class should be replaced with a ThreadPoolExecutor
instance to limit concurrency (e.g. you don’t want to spam 10k threads). Also with ThreadPoolExecutor
the code is even simplier (and less error prone):
from concurrent.futures import ThreadPoolExecutor
tp = ThreadPoolExecutor(10) # max 10 threads
def threaded(fn):
def wrapper(*args, **kwargs):
return tp.submit(fn, *args, **kwargs) # returns Future object
return wrapper
Just remember you have to tp.shutdown()
after you’re done with all parallel work.
Answer #2:
You can pass class instance to the thread:
class SomeThread(threading.Thread):
def __init__(self, count, instance):
threading.Thread.__init__(self)
self.instance = instance
def run(self):
print "Do something"
self.instance.some_param = data
self.instance.some_function()
Answer #3:
I’m fairly certain that you can’t make a single function threaded.
The whole class will be threaded (sort of). When you instantiate the object, its __init__
will be called on another thread, and then when you call start()
on that object, its run()
will be called once, on another thread.
So, if you have a TASK that needs to be on its own thread (disc IO, socket listening, etc), then you need a class to handle that task.
@ndpu’s answer solves your scope/access problems.