How to write multiple lists into a dataframe using loop

How to write multiple lists into a dataframe using loop

Problem Description:

I have several lists that are generated from a get_topic() function. That is,

list1 = get_topic(1)
list2 = get_topic(2)
and another dozens of lists.

# The list contains something like

[('A', 0.1),('B', 0.2),('C',0.3)]

I am trying to write a loop so that all different lists can be saved to different columns in a dataframe. The code I tried was:

for i in range(1,number) # number is the total number of lists + 1
    df_02 = pd.DataFrame(get_topic(i)

This only returns with list1, but no other lists. The result that I would like to get is something like:

Could anyone help me to correct the loop? Thank you.

Solution – 1

I reconstruct a hypothetical get_topic() function that simply fetches a list from a list of lists.

The idea is to use pd.concat() in order to concatenate dataframes at each iteration.

import pandas as pd

topics = [
    [('A', 0.1), ('B', 0.2), ('C', 0.3)],
    [('D', 0.3), ('E', 0.4), ('F', 0.5)]
]
number = len(topics)


def get_topic(index) -> []:
    return topics[index]


if __name__ == '__main__':
    df = pd.DataFrame()
    for i in range(0, number):  # number is the total number of lists
        curr_topic = get_topic(i)
        curr_columns = ['List ' + str(i+1), 'Number ' + str(i+1)]
        df = pd.concat([df, pd.DataFrame(data=curr_topic, columns=curr_columns)], axis=1)

print(df)

Output will be:

  List 1  Number 1 List 2  Number 2
0      A       0.1      D       0.3
1      B       0.2      E       0.4
2      C       0.3      F       0.5

Solution – 2

df = pd.DataFrame()
for i in range(1, number):
    df[f'List {i}'], df[f'Number {i}'] = zip(*get_topic(i))

Solution – 3

You are creating a new DataFrame at every iteration.

This will create a structure similar to what you want:

df = pd.DataFrame([get_topic(i) for i in range(1, number)])
df = df.apply(pd.Series.explode).reset_index(drop=True)
df = df.transpose()

Result:

   0    1  2    3  4    5
0  A  0.1  D  0.1  G  0.1
1  B  0.2  E  0.2  H  0.2
2  C  0.3  F  0.3  I  0.3

One-liner version:

df = pd.DataFrame([get_topic(i) for i in range(1, number)]).apply(pd.Series.explode).reset_index(drop=True).transpose()