Merging multiple csv files(unnamed colums) from a folder in python

Question

import pandas as pd
import os
import glob

path = r'C:\Users\avira\Desktop\CC\SAIL\Merging\CISF'

files = glob.glob(os.path.join(path, '*.csv'))

combined_data = pd.DataFrame()


for file in files :
    
    data = pd.read_csv(file)
    print(data)
    
    combined_data = pd.concat([combined_data,data],axis=0,ignore_index=True)
    
    
combined_data.to_csv(r'C:\Users\avira\Desktop\CC\SAIL\Merging\CISF\data2.csv')

The files are merging diagonally,ie-next to the last cell of the first file, is the beginning of second file. ALSO, it is taking the first entry of file as column names. All of my files are without column names. How do I vertically merge my files,and provide coluumn names to the merged csv.

You can read manual of pandas concat which is what you are looking for here: pandas.pydata.org/docs/reference/api/pandas.concat.html — PTQuoc
– PTQuoc, Commented Jun 12, 2022 at 9:38
Prefer to use concat once at the end instead of once per loop iteration - it will save time & memory. — ramslök
– ramslök, Commented Jun 12, 2022 at 9:50

AKHacks · Accepted Answer · 2022-06-12 09:52:51Z

1

For the header problem while reading csv , u can do this:

pd.read_csv(file, header=None)

While dumping the result u can pass list containing the header names

df.to_csv(file_name,header=['col1','col2'])

edited Jun 12, 2022 at 9:52

answered Jun 12, 2022 at 9:42

AKHacks

2071 silver badge5 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

gtomer · Accepted Answer · 2022-06-12 10:32:11Z

0

You need to read the csv with no headers and concat:

data = pd.read_csv(file, header=None)
combined_data = pd.concat([combined_data, data], ignore_index=True)

If you want to give the columns meaningful names:

combined_data.columns = ['name1', 'name2', 'name3']

edited Jun 12, 2022 at 10:32

answered Jun 12, 2022 at 9:50

gtomer

6,6041 gold badge14 silver badges29 bronze badges

Collectives™ on Stack Overflow

Merging multiple csv files(unnamed colums) from a folder in python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related