I am trying to read a large CSV file and then loading the data as JSON file. The following code is working fine but the data is writing as JSON object in each line on JSON file.
import dask.dataframe as dd
import pandas as pd
cols=['Name','State']
df=dd.read_csv('F:\csvs\stack.csv', low_memory=False, usecols=cols,
dtype={'Name':str,'State':str}
)
df.to_json(r'F:\csvs\stack', orient ='records', compression = 'infer')
Above code writing the data as JSON object in each line
{"Name":"John","State":"TS"}
{"Name":"Paha","State":"MK"}
How to write the data as JSON array like below?
[{"Name":"John","State":"TS"},{"Name":"Paha","State":"MK"}]
By default files are creating file type as .part
, How to create files with .json
extension.
source https://stackoverflow.com/questions/73534169/python-dask-dataframe-write-json-file-with-array-format
Comments
Post a Comment