Python Multiprocessing: How To Modify A Dictionary Created In The Main Process From A Subprocess?

This question is related to: multiprocessing: How do I share a dict among multiple processes? I have multiple numpy arrays stored in a multiprocessing Dict. The multiprocessing di

Solution 1:

This is due to the way the Manager() manages the dict. Only the dict itself is managed, not its entries (unless the entries happen to be Proxy Objects). Thus when working with mutable dict entries, updates to the objects are not handled (or even registered) by the manager.

However, it is possible to update the dict itself, e.g. by replacing a full dict entry. Thus, a way to get around the above limitation, is to get a local reference to the entry, update it, and then re-assign the local reference to the managed dict.

In this particular case, it would mean that the appenddict function has to be modified slightly:

def apenddict(D, tik,gnr):
    tik_array = D[tik]  # Create local reference
    for i in range(8):
        print(tik)
        tik_array[:-1] = D[tik][1:]  # Update local reference
        tik_array[-1:, :] = gnr.next()  # Update local reference
    D[tik] = tik_array  # Assign local reference to managed dict

With these changes, your program ought to work more as you expect it to.