Read .dat File in C++ Line by Line

A professional engineer and writer helping people discover creative ways to solve everyday problems.

how-to-read-data-from-a-file-in-python — Copyrights © Ali Khan (Author)

This article will provide y'all a detailed walk-through on how you can employ Python to read large text files. A fully functional, ready to execute lawmaking snippet is included in this walk-through to get you lot up to speed in x minutes after reading this article.

Permit us start familiarize you with the advanced information-structures available in Python that we will utilize to shop and procedure data from files, simply in case you are new to Python programming.

Advanced Information-Structures in Python

Python has ii advanced and powerful data structures that make it superior in functionality to C/C++ and make it an ideal language for numerical data intensive applications competing with manufacture dominant Matlab®.

ndarrays and dataframes

Adapted from Numpy and Pandas opensource documentation

NumPy Arrays

Numpy arrays or ndarrays are arrays that tin be scaled up to 'due north' dimensions. They are all-time used as 2-dimensional array structures to represent matrices. The Numpy module itself contains powerful office libraries for a variety of numerical and algebraic operations.

Pandas Data-Frames

Data-frames build upon the ii-dimensional ndarrays to add extra functionality. The two-dimensional ndarray now has a separate column for array index and all column headers are now individually addressable. More chiefly, each cavalcade tin now hold a different data type (int, float or string).

Text File to be Read by Python Lawmaking

Permit's move forward to the tutorial and acquaint you with the demonstration text file.

It is a 14 rows x xx column data table saved as a txt file. It contains data in all three information formats: int, bladder and string. File name is: BusData.

Fig 1: Data file that is to be parsed with our code.

IEEE xiv-Charabanc System CDF file

Next, view the code snippet given below, to read this file and nosotros will explicate this code line past line in the following department.

Python Code to Read Data From Text File

                                                            # Copyrights © Ali Khan (Author)                                                                              # Permitted to use with attribution                                                                                                                          import                  numpy                                                              import                  pandas                                                                                                                              def                    Read                    ():                                                                                                                                          global                  BusData, BusDataList, BusDataArray, BusDataReshaped                                                                                                                          X =                  open('C:/Users/user/OneDrive - Washington State University (electronic mail.wsu.edu)/EE - 521/BusData.txt',                  'r')                                                              BusData = X.read()                                                                                                                                          BusDataList = BusData.divide()                                                              BusDataArray = numpy.array(BusDataList)                                                              BusDataReshaped = BusDataArray.reshape(fourteen,20)                  #Brand a matrix out of 1D Array                                                                                                        Read()                                                                                        BusDataFrame = pandas.DataFrame(BusDataReshaped, columns =['BusNumber',                  'Coach',                  'Busx',                  'BusClass',                  'Unused1',                  'Unused2',                  'BusType',                  'Unused3',                  'Unused4',                  'BusLoadMW',                  'BusLoadMVAR',                  'BusGenMW',                  'BusGenMVAR',                  'InitialVoltAngle',                  'InitialVolt',                  'Qlimit+',                  'Qlimit-',                  'Unused5',                  'Shunt',                  'Unused'])                                                                                                          impress(BusDataFrame.BusClass[six])

Lawmaking Caption

Initialization: Import Numpy and Pandas

Line four: Import the numpy package in the project.

Line 5: Import the pandas packet in the project.

Line 7: Start a function definition Read(). Information technology is always a good practice to suspension your code in functions.

Line ix: Ascertain global variables.

Read More than From Owlcation

In Python only global variables will appear in the variable explorer and can be referenced outside functions. For demonstration, hither I have defined all four as global, otherwise only BusDataReshaped variable should have been declared global.

Fig 2: Spyder variable explorer showing details of every variable.

Copyrights © Ali Khan (Writer)

Open and Read the Target File

Line 11: The open up() part points to the directory location of file BusData.txt. Definition is assigned to random variable 10.

Line 12: read() function reads the entire file as a string and assigns information technology to variable BusData. Fig 2 shows that BusData is at present a string with 1792 characters.

Split up the File Graphic symbol-wise

Line fourteen: separate() role in Python, splits the string into a listing at the points where their is space. The data is now converted into a listing of 280 elements and assigned to variable BusDataList. Reference Fig 2.

Convert to Numpy Assortment

Line fifteen: The listing is converted into a numpy assortment past the numpy.array() function. Fig two shows that BusDataArray is now an array of datatype string and has 280 elements.

The trouble is that it withal does not look like our original information table in the file. Information technology needs to be reshaped.

Line 16: The numpy.reshape() function from the numpy packet reshapes the array into our desired dimensions of xiv x 20. Fig 2 shows that BusDataReshaped variable is at present an ndarray and has dimensions 14 x 20.

As we can run across, the data type of all the values is still string, but recall, the original file had integers and floats in the data also. To make sure that all the data is treated according to its correct data type we need to convert information technology into a Pandas Data-frame.

Fig 3: BusDataFrame, showing its first 13 columns in variable explorer — Fig iii: BusDataFrame, showing its beginning xiii columns in variable explorer

Copyrights © Ali Khan (Author)

Converting a Numpy Array to Data-frame

Line 20: line twenty finally does the task of converting an array of strings into a pandas dataframe.

Pandas.dataframe() part takes the reshaped numpy array, and the names of all 20 column headers every bit inputs. Fig 3 shows the formed dataframe and Fig 2 verifies this in the variable explorer.

Fig 4: Print result — Fig four: Print consequence

Referencing Values of a Pandas Dataframe

Line 22: Values of this dataframe can be very conveniently accessed past the dataframe.columnheader.[index] syntax.

A check of variable types volition show that all the three data types of cord, integer and floats of each column are automatically preserved by the dataframe.

This content is accurate and true to the all-time of the author's knowledge and is not meant to substitute for formal and individualized advice from a qualified professional person.

Read .dat File in C++ Line by Line

Source: https://owlcation.com/stem/How-to-Read-Data-From-a-File-in-Python