| 1 |
|
|---|
| 2 | \section{\module{Queue} ---
|
|---|
| 3 | A synchronized queue class}
|
|---|
| 4 |
|
|---|
| 5 | \declaremodule{standard}{Queue}
|
|---|
| 6 | \modulesynopsis{A synchronized queue class.}
|
|---|
| 7 |
|
|---|
| 8 |
|
|---|
| 9 | The \module{Queue} module implements a multi-producer, multi-consumer
|
|---|
| 10 | FIFO queue. It is especially useful in threads programming when
|
|---|
| 11 | information must be exchanged safely between multiple threads. The
|
|---|
| 12 | \class{Queue} class in this module implements all the required locking
|
|---|
| 13 | semantics. It depends on the availability of thread support in
|
|---|
| 14 | Python.
|
|---|
| 15 |
|
|---|
| 16 | The \module{Queue} module defines the following class and exception:
|
|---|
| 17 |
|
|---|
| 18 |
|
|---|
| 19 | \begin{classdesc}{Queue}{maxsize}
|
|---|
| 20 | Constructor for the class. \var{maxsize} is an integer that sets the
|
|---|
| 21 | upperbound limit on the number of items that can be placed in the
|
|---|
| 22 | queue. Insertion will block once this size has been reached, until
|
|---|
| 23 | queue items are consumed. If \var{maxsize} is less than or equal to
|
|---|
| 24 | zero, the queue size is infinite.
|
|---|
| 25 | \end{classdesc}
|
|---|
| 26 |
|
|---|
| 27 | \begin{excdesc}{Empty}
|
|---|
| 28 | Exception raised when non-blocking \method{get()} (or
|
|---|
| 29 | \method{get_nowait()}) is called on a \class{Queue} object which is
|
|---|
| 30 | empty.
|
|---|
| 31 | \end{excdesc}
|
|---|
| 32 |
|
|---|
| 33 | \begin{excdesc}{Full}
|
|---|
| 34 | Exception raised when non-blocking \method{put()} (or
|
|---|
| 35 | \method{put_nowait()}) is called on a \class{Queue} object which is
|
|---|
| 36 | full.
|
|---|
| 37 | \end{excdesc}
|
|---|
| 38 |
|
|---|
| 39 | \subsection{Queue Objects}
|
|---|
| 40 | \label{QueueObjects}
|
|---|
| 41 |
|
|---|
| 42 | Class \class{Queue} implements queue objects and has the methods
|
|---|
| 43 | described below. This class can be derived from in order to implement
|
|---|
| 44 | other queue organizations (e.g. stack) but the inheritable interface
|
|---|
| 45 | is not described here. See the source code for details. The public
|
|---|
| 46 | methods are:
|
|---|
| 47 |
|
|---|
| 48 | \begin{methoddesc}{qsize}{}
|
|---|
| 49 | Return the approximate size of the queue. Because of multithreading
|
|---|
| 50 | semantics, this number is not reliable.
|
|---|
| 51 | \end{methoddesc}
|
|---|
| 52 |
|
|---|
| 53 | \begin{methoddesc}{empty}{}
|
|---|
| 54 | Return \code{True} if the queue is empty, \code{False} otherwise.
|
|---|
| 55 | Because of multithreading semantics, this is not reliable.
|
|---|
| 56 | \end{methoddesc}
|
|---|
| 57 |
|
|---|
| 58 | \begin{methoddesc}{full}{}
|
|---|
| 59 | Return \code{True} if the queue is full, \code{False} otherwise.
|
|---|
| 60 | Because of multithreading semantics, this is not reliable.
|
|---|
| 61 | \end{methoddesc}
|
|---|
| 62 |
|
|---|
| 63 | \begin{methoddesc}{put}{item\optional{, block\optional{, timeout}}}
|
|---|
| 64 | Put \var{item} into the queue. If optional args \var{block} is true
|
|---|
| 65 | and \var{timeout} is None (the default), block if necessary until a
|
|---|
| 66 | free slot is available. If \var{timeout} is a positive number, it
|
|---|
| 67 | blocks at most \var{timeout} seconds and raises the \exception{Full}
|
|---|
| 68 | exception if no free slot was available within that time.
|
|---|
| 69 | Otherwise (\var{block} is false), put an item on the queue if a free
|
|---|
| 70 | slot is immediately available, else raise the \exception{Full}
|
|---|
| 71 | exception (\var{timeout} is ignored in that case).
|
|---|
| 72 |
|
|---|
| 73 | \versionadded[the timeout parameter]{2.3}
|
|---|
| 74 |
|
|---|
| 75 | \end{methoddesc}
|
|---|
| 76 |
|
|---|
| 77 | \begin{methoddesc}{put_nowait}{item}
|
|---|
| 78 | Equivalent to \code{put(\var{item}, False)}.
|
|---|
| 79 | \end{methoddesc}
|
|---|
| 80 |
|
|---|
| 81 | \begin{methoddesc}{get}{\optional{block\optional{, timeout}}}
|
|---|
| 82 | Remove and return an item from the queue. If optional args
|
|---|
| 83 | \var{block} is true and \var{timeout} is None (the default),
|
|---|
| 84 | block if necessary until an item is available. If \var{timeout} is
|
|---|
| 85 | a positive number, it blocks at most \var{timeout} seconds and raises
|
|---|
| 86 | the \exception{Empty} exception if no item was available within that
|
|---|
| 87 | time. Otherwise (\var{block} is false), return an item if one is
|
|---|
| 88 | immediately available, else raise the \exception{Empty} exception
|
|---|
| 89 | (\var{timeout} is ignored in that case).
|
|---|
| 90 |
|
|---|
| 91 | \versionadded[the timeout parameter]{2.3}
|
|---|
| 92 |
|
|---|
| 93 | \end{methoddesc}
|
|---|
| 94 |
|
|---|
| 95 | \begin{methoddesc}{get_nowait}{}
|
|---|
| 96 | Equivalent to \code{get(False)}.
|
|---|
| 97 | \end{methoddesc}
|
|---|
| 98 |
|
|---|
| 99 | Two methods are offered to support tracking whether enqueued tasks have
|
|---|
| 100 | been fully processed by daemon consumer threads.
|
|---|
| 101 |
|
|---|
| 102 | \begin{methoddesc}{task_done}{}
|
|---|
| 103 | Indicate that a formerly enqueued task is complete. Used by queue consumer
|
|---|
| 104 | threads. For each \method{get()} used to fetch a task, a subsequent call to
|
|---|
| 105 | \method{task_done()} tells the queue that the processing on the task is complete.
|
|---|
| 106 |
|
|---|
| 107 | If a \method{join()} is currently blocking, it will resume when all items
|
|---|
| 108 | have been processed (meaning that a \method{task_done()} call was received
|
|---|
| 109 | for every item that had been \method{put()} into the queue).
|
|---|
| 110 |
|
|---|
| 111 | Raises a \exception{ValueError} if called more times than there were items
|
|---|
| 112 | placed in the queue.
|
|---|
| 113 | \versionadded{2.5}
|
|---|
| 114 | \end{methoddesc}
|
|---|
| 115 |
|
|---|
| 116 | \begin{methoddesc}{join}{}
|
|---|
| 117 | Blocks until all items in the queue have been gotten and processed.
|
|---|
| 118 |
|
|---|
| 119 | The count of unfinished tasks goes up whenever an item is added to the
|
|---|
| 120 | queue. The count goes down whenever a consumer thread calls \method{task_done()}
|
|---|
| 121 | to indicate that the item was retrieved and all work on it is complete.
|
|---|
| 122 | When the count of unfinished tasks drops to zero, join() unblocks.
|
|---|
| 123 | \versionadded{2.5}
|
|---|
| 124 | \end{methoddesc}
|
|---|
| 125 |
|
|---|
| 126 | Example of how to wait for enqueued tasks to be completed:
|
|---|
| 127 |
|
|---|
| 128 | \begin{verbatim}
|
|---|
| 129 | def worker():
|
|---|
| 130 | while True:
|
|---|
| 131 | item = q.get()
|
|---|
| 132 | do_work(item)
|
|---|
| 133 | q.task_done()
|
|---|
| 134 |
|
|---|
| 135 | q = Queue()
|
|---|
| 136 | for i in range(num_worker_threads):
|
|---|
| 137 | t = Thread(target=worker)
|
|---|
| 138 | t.setDaemon(True)
|
|---|
| 139 | t.start()
|
|---|
| 140 |
|
|---|
| 141 | for item in source():
|
|---|
| 142 | q.put(item)
|
|---|
| 143 |
|
|---|
| 144 | q.join() # block until all tasks are done
|
|---|
| 145 | \end{verbatim}
|
|---|