Skip to main content

Convert complex txt to csv with python

I want to convert text to csv. Input file contains 10000K lines. Sample of input file is as below:-

Item=a
Price=10
colour=pink


Item=b
Price=20
colour=blue Pattern=checks

My output should look like this

Item Price Colour Pattern

a    10    pink
b    20    blue   checks

I am getting output if there is only single '=' in 1 line, if there are more than 1 like 2/3 '=' then I am not sure how to apply for loop. Can someone check my for loop part? Am I going wrong somewhere?

import csv
import glob
import os


def dat_to_csv(filename, table_name):
    with open(filename, 'r') as reader:
        list_of_columns = []
        table_values = []
        master_table = []
        counter = 0
        for line in reader:
            #stripped_line = line.strip()
            if line == "\n":
                #copy all elements which have values else paste a null
                if (table_values):
                    #master_table.append(table_values)
                    master_table.append([])
                    master_table[counter] = table_values.copy()
                    counter=counter+1
                    length = len(table_values)
                    for element in range(length):
                        table_values[element] = []
                continue
            if line == "\n":
                continue

            extra_stripped_line = ' '.join(line.split())
            data = extra_stripped_line.split("=",1)
            column_name = data[0].strip()
            if "=" in data[1]:            
                data1 = data[1].split(" ",1)
                value = data1[0].strip()
                data2 = data1[1].split("=",1)
                column_name = data2[0].strip()
                value = data2[1].strip()
                continue
            value = data[1].strip()

            if column_name not in list_of_columns:
                list_of_columns.append(column_name)
                table_values.append([])

            index = list_of_columns.index(column_name)

            #table_values[index].append(value)
            table_values[index] = value
             
        #with open("output\\{}.csv".format(table_name), 'w', newline='') as csvfile:
        with open("folderpath\\{}.csv".format(table_name), 'w', newline='') as csvfile:
            writer = csv.writer(csvfile, delimiter=',', quotechar='"', quoting=csv.QUOTE_ALL)

            writer.writerow(list_of_columns)

            #t_table_values = zip(*table_values)
            
            max_elements = len((master_table))
            
            master_table_transp = []
            for cntr in range(max_elements):
                master_table_transp.append([])
            num_objects = len(master_table)

            for cntr_obj in range(num_objects):
                for cntr_row in range(max_elements):
                    if (cntr_row<len(master_table[cntr_obj])):
                        master_table_transp[cntr_row].append(master_table[cntr_obj][cntr_row])
                    else:
                        master_table_transp[cntr_row].append([])




            t_table_values = zip(*master_table_transp)

            for values in t_table_values:
                writer.writerow(values)

if __name__ == '__main__':
    path = "folderpath"

    for filename in glob.glob((os.path.join(path, '*.txt'))):
        name_only = os.path.basename(filename).replace(".txt", "")
        dat_to_csv(filename, name_only)

Output which I am getting is as below: enter image description here



source https://stackoverflow.com/questions/71270425/convert-complex-txt-to-csv-with-python

Comments

Popular posts from this blog

Why is my reports service not connecting?

I am trying to pull some data from a Postgres database using Node.js and node-postures but I can't figure out why my service isn't connecting. my routes/index.js file: const express = require('express'); const router = express.Router(); const ordersCountController = require('../controllers/ordersCountController'); const ordersController = require('../controllers/ordersController'); const weeklyReportsController = require('../controllers/weeklyReportsController'); router.get('/orders_count', ordersCountController); router.get('/orders', ordersController); router.get('/weekly_reports', weeklyReportsController); module.exports = router; My controllers/weeklyReportsController.js file: const weeklyReportsService = require('../services/weeklyReportsService'); const weeklyReportsController = async (req, res) => { try { const data = await weeklyReportsService; res.json({data}) console...

How to show number of registered users in Laravel based on usertype?

i'm trying to display data from the database in the admin dashboard i used this: <?php use Illuminate\Support\Facades\DB; $users = DB::table('users')->count(); echo $users; ?> and i have successfully get the correct data from the database but what if i want to display a specific data for example in this user table there is "usertype" that specify if the user is normal user or admin i want to user the same code above but to display a specific usertype i tried this: <?php use Illuminate\Support\Facades\DB; $users = DB::table('users')->count()->WHERE usertype =admin; echo $users; ?> but it didn't work, what am i doing wrong? source https://stackoverflow.com/questions/68199726/how-to-show-number-of-registered-users-in-laravel-based-on-usertype

The number of nodes and edges I calculated manually and by a method provided by networkx library dont match

I am trying to analyse a graph using networkx library in Python. I have implemented a function that returns the nodes and edges of that graph. After that I counted the number of nodes and edges by implementing the len() function for nodes and edges, which gave me 17 for both nodes and edges. However, when I used the method graph.number_of_nodes() of networkx library it gave me 53. My code: import pandas as pd import networkx as nx import matplotlib.pyplot as plt header_list = ["a","b","w"] E =pd.read_csv("aves-barn-swallow-contact-network.edges",sep=" ",header= None,names=header_list) # print(E.head()) # prints first few rows of the csv file G = nx.from_pandas_edgelist(E,"a","b",["w"]) nx.draw(G) plt.show() class GraphAnalyse: #finding nodes and edges def nodeEdge(self,Graph): nodes = Graph.nodes() edges = Graph.edges() ...