Read multiple csv files with Pandas assign different names

Question

How do I read different csv files in folder without concatenating them but just assigning them with the original file name? For example, file with path ...\table1.csv will be named "table1"

I managed to get all file names how do I read each file now?

from os import listdir
from os.path import isfile, join

mypath= ...
onlyfiles = [f for f in listdir(mypath) if isfile(join(mypath, f))]

In other words, instead of reading multiple csv files in pandas like this:

table1 = pd.read_csv(r'C:\Users\username\Folder\Desktop\FolderA\FolderB\Sub_Folder\OneDrive_1_22-06-2022\table1.csv')

table2 = pd.read_csv(r'C:\Users\username\Folder\Desktop\FolderA\FolderB\Sub_Folder\OneDrive_1_22-06-2022\table2.csv')

table3 = pd.read_csv(r'C:\Users\username\Folder\Desktop\FolderA\FolderB\Sub_Folder\OneDrive_1_22-06-2022\table3.csv')
...

is there a better way?

Does this answer your question? How loop and store values in independent variable in python — Michael Delgado
– Michael Delgado, Commented Jun 22, 2022 at 5:14
I don't think declaring variables from external inputs is a good practice, sure you can do it with eval or exec. But this makes the program unpredictable and vulnerable, for almost most of the time. — Brandon
– Brandon, Commented Jun 22, 2022 at 5:18

Corralien · Accepted Answer · 2022-06-22 05:28:45Z

3

Use pathlib and dictionary:

import pandas as pd
import pathlib

dfs = {f.stem: pd.read_csv(f) for f in pathlib.Path().glob('*.csv')}

Strongly discouraged, prefer method above

If you want to create variables dynamically:

for name, df in dfs.items():
    locals()[name] = df
    # locals()[f"df_{name}"] = df

Output:

>>> data1
   0:00  0:30
0     1     5
1     2     6
2     3     7
3     4     8

edited Jun 22, 2022 at 5:28

answered Jun 22, 2022 at 5:17

Corralien

121k8 gold badges44 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

nilsinelabore Over a year ago

Hi Corralien, thanks for your solution, instead of getting data in a whole chunk, can I assign them with different names and pandas dataframe?

Corralien Over a year ago

What do you mean? Is it not already the case here: one file, one variable?

nilsinelabore Over a year ago

It's working now! Sorry I missed out the second bit - may I know it's discouraged?

Corralien Over a year ago

For example, your file names need to be valid python identifier. data 1.csv is not a valid python identifier. If you search files recursively, you can have 2 files with same name and override a variable previously defined (but this is the same problem for a dict :))

Wei Shan Lee Over a year ago

May I ask what f.stem: pd.read_csv(f) does?

|

BeRT2me · Accepted Answer · 2022-06-22 05:13:58Z

1

dfs = {file.split('.')[0]: pd.read_csv(file) for file in onlyfiles}
print(dfs['table1'])
...
<Your dataframe here>

answered Jun 22, 2022 at 5:13

BeRT2me

13.3k2 gold badges18 silver badges39 bronze badges

Comments

kelvt · Accepted Answer · 2022-06-22 05:15:41Z

1

let's try:

for file in onlyfiles:
    # get file name
    fname = file.split('.')[0]

    # read dataframe with file name as variable name
    exec('{} = pd.read_csv(file)'.format(fname))

answered Jun 22, 2022 at 5:15

kelvt

1,0588 silver badges18 bronze badges

Collectives™ on Stack Overflow

Read multiple csv files with Pandas assign different names

3 Answers 3

8 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

8 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related