Create Row, Column, Data Pandas Dataframe From Sparse Matrix

How can I create a sparse matrix in the format of COO and have the pandas dataframe not unnest to a dense layout but keep the COO format for row,column,data? import numpy as np imp

Solution 1:

The values you want to put in the dataframe are available as

a_coo.row, a_coo.col, a_coo.data

Solution 2:

one possible workaround could be to use mtx serialization and interpreting the data as a CSV.

from scipy import io
io.mmwrite('sparse_thing', a_csr)
!cat sparse_thing.mtx

sparse_mtx_mm_df = pd.read_csv('sparse_thing.mtx', sep=' ', skiprows=3, header=None)
sparse_mtx_mm_df.columns = ['row', 'column', 'data_value']
sparse_mtx_mm_df

Is there a better (native, non serialization-baased) solution?

re_sparsed = coo_matrix((sparse_mtx_mm_df['data_value'].values, (sparse_mtx_mm_df.numpy_row.values, sparse_mtx_mm_df.numpy_column.values)))
re_sparsed.todense()

would then give back the initial numpy array