tag:blogger.com,1999:blog-3029438816324283001.post1210455087121295111..comments2018-02-17T10:48:35.354+00:00Comments on Coded Structure: Concurrent Queue.get() with timeouts eats CPUBen Basshttp://www.blogger.com/profile/02283998269313171389noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-3029438816324283001.post-75970346822857220802011-02-20T23:30:50.982+00:002011-02-20T23:30:50.982+00:00Yeah, sorry, I've been meaning to add native l...Yeah, sorry, I've been meaning to add native lock timeouts to threading2 to address this very problem, but just haven't got around to it...Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3029438816324283001.post-11661703293728557902011-02-20T21:36:10.110+00:002011-02-20T21:36:10.110+00:00Unfortunately my memory has served me wrong. threa...Unfortunately my memory has served me wrong. threading2 has no optimized timeout for locks. I guess I have to roll my own backport of Python 3.2's lock ...Christian Heimeshttps://www.blogger.com/profile/16043034511693193747noreply@blogger.comtag:blogger.com,1999:blog-3029438816324283001.post-6370625387581786132011-02-20T20:00:34.793+00:002011-02-20T20:00:34.793+00:00Thanks both, some useful info. I was aware the GIL...Thanks both, some useful info. I was aware the GIL code was changing in 3.2, but if this timeout behaviour is improved too then hopefully this issue will disappear in time.<br /><br />I'm content being aware of the issue and working round it till then...Ben Basshttps://www.blogger.com/profile/02283998269313171389noreply@blogger.comtag:blogger.com,1999:blog-3029438816324283001.post-66685059723530200222011-02-20T19:06:51.077+00:002011-02-20T19:06:51.077+00:00The code is fairly new and was added during the de...The code is fairly new and was added during the development phase of Python 3.2. All Python 2.x versions and 3.1 are using a busy loop with an increasing timeout. AFAIR the timeout starts at 1ms and goes up to 50ms. This means every lock with a timeout (conditions, events, queues ...) is waken up the CPU 20 times a second. <br /><br />The threading2 library may offer a solution for you. I haven't tested or reviewed it myself yet.Christian Heimeshttps://www.blogger.com/profile/16043034511693193747noreply@blogger.comtag:blogger.com,1999:blog-3029438816324283001.post-56092969266298389272011-02-20T01:38:58.599+00:002011-02-20T01:38:58.599+00:00CPython just uses whatever timed lock acquisition ...CPython just uses whatever timed lock acquisition primitives the platform has available, so exactly how the behaviour changes will vary by platform.<br /><br />I know you said you didn't want to dig into the source code, but the three most common examples can be seen at:<br />Posix threads: http://svn.python.org/view/python/branches/py3k/Python/thread_pthread.h?view=markup<br />Windows: http://svn.python.org/view/python/branches/py3k/Python/thread_nt.h?view=markup<br /><br />(There are two variants in the pthreads file - one that uses semaphores and another that uses condition variables when a working semaphore implementation isn't provided by the platform)Anonymousnoreply@blogger.com