Get count of rows and group by subfolder using python

Get count of rows and group by subfolder using python

Problem Description:

I have 3 sub folders which contain multiple csv files.I want to show record count of all the files grouped by sub folders

I have a code which gives me count of a single file only

with open (r"c:usersdocfile.csv",'r') as fp:
   print('total lines:', lines)`

Expected output:

Foldername Count

Product1 -53000321

Product2- 1245

Product3- 162

Solution – 1

My approach would be first to get list of subfolders, then traverse each to find files and add to list/touple.

import os
d  = r'.' #set to your path where subfolders are
result =[] #this will be used to store results

#get list of subfolders
subdirs = [os.path.join(d, o) for o in os.listdir(d) if os.path.isdir(os.path.join(d,o))]

for fld in subdirs: #iterate subfolders
    files = os.listdir(fld)
    files = [f for f in files if os.path.isfile(fld+'/'+f)] #Filtering only the files.
    for f in files: #iterate files
        #if needed to filter by filename or extension, add here
        with open(fld+'/'+f,'r') as fp:
            result.append({'Foldername': os.path.basename(fld), 'Count':lines})
            #basename gives last folder name from path

Output #in result

Rate this post
We use cookies in order to give you the best possible experience on our website. By continuing to use this site, you agree to our use of cookies.