dataframe_to_rows doesn't work correctly with categorical data type
dataframe_to_rows doesn't work correctly with category data type. The values of the corresponding categorically column are returned not as a list, but as one value from the existing ones. As a result, the code crashes with an error
Index error: string index out of range
for idx, v in enumerate(expanded):
row = [data[j][idx] for j in range(ncols)]
After brifly research the error occurs due to incorrect unpacking in this fragment, where the column is assigned one value:
for col_loc, col in zip(b.mgr_locs, result):
data[col_loc] = col
CategoricalBlock is stored for each column separately, so they cannot be unpacked in this way.
Simplest solution for me is convert category columns to string columns before dataframe_to_rows function.
I suppose that in the code you can add a check for the type of datablocks, since they are stored in a single copy for each column.