Skip to content Skip to sidebar Skip to footer

How Do I Pickle Pyephem Objects For Multiprocessing?

I am trying to calculate some values of satellites, the data-generation takes quite long so I want to implement this using multiprocessing. The problem is that I get this error fro

Solution 1:

The reason is something like this... when you try to pickle a function, it can attempt to pickle globals(), so whatever you have in your global namespace is also pickled (just in case your function has a reference to something in globals() -- yes, that's unexpected, but that's how it is). So, an easy fix is to isolate the function you want to pickle in another file -- in this case, put the multiprocessing stuff in one file and the other code in another file... so there's less in globals() for the pickler to struggle with. Another thing that might help is to use multiprocess instead of multiprocessing -- multiprocess uses the dill serializer instead of pickle, so you have a better chance of serializing objects that will be sent across the workers in the Pool.


Post a Comment for "How Do I Pickle Pyephem Objects For Multiprocessing?"