Ipython: How To Automagically Load Npz File And Assign Values To Variables?

I'm new to Python and I'm eagerly migrating from MATLAB to IPython as my preferred language for data analysis at the lab. In MATLAB, after a session of data crunching, I would do

Solution 1:

locals().update(npzfile)
a   # and/or b

In the Ipython session, locals() is a large dictionary with variables the you've defined, the input history lines, and various outputs. update adds the dictionary values of npzfile to that larger one.

By the way, you can also load and save MATLAB .mat files. Use scipy.io.loadmat and savemat. It handles v4 (Level 1.0), v6 and v7 to 7.2 files. But you have same issue - the result is a dictionary.

Octave has an expression form of the load command, that loads the data into a structure

S = load ("file", "options", "v1", "v2", ...)

In general numpy and ipython does not have the same sort of 'save a snapshot of the workspace' idea that MATLAB does. I mostly write scripts to generate (and load) the necessary data, and run them either as stand alone python sessions or run them from with in ipython. ipython does have a good history feature. You might also find the ipython notebook tool useful. I haven't used that.

---

Within a function, locals() is different from outside it. It is 'local' to each function. There is a globals() dictionary that might allow setting variables outside the function. But this style of programming is not encouraged.

The SO question that Anurag cited points to a save_ipython_variables package. https://pypi.python.org/pypi/save_ipython_variables/0.0.3 It saves variables with pickle, and loads them by doing something like

exec'__builtins__[name] = pickle.load(...)

With use of exec and __builtins__ it is pushing some safe programming boundaries. So use with caution. But a small test does work in my environment. The same technique might work with npz files.

Solution 2:

I was looking for the same thing again and again, getting tired of having to load dozens of arrays each time in the way:

data = np.load("file.npz")
var1 = data["my_var1"]
...

Since I let computations regularly run on the campus cluster and post-process the results (numpy arrays) on my laptop, I really appreciate the free Spyder IDE (which btw runs an IPython console interactively), which has a Matlab GUI-like variable explorer that lets you load a whole npz-archive (or selective variables if you so wish) or spydata archive with a mouse click. It greatly speeds up my workflow with numpy arrays.