Data Analysis with Python

Python Data Analysis

A common task for scientists and engineers is to analyze data from an external source that may be in a text or comma separated value (CSV) format. By importing the data into Python, data analysis such as statistics, trending, or calculations can be made to synthesize the information into relevant and actionable information. This tutorial demonstrates how to import data, perform a basic analysis, trend the results, and export the results to another text file. Two examples are provided with Numpy and Pandas. Script files of the Python source code with sample data are below.

Import Data and Analyze with Numpy

Import Data and Analyze with Pandas

Source Code

 # import Numpy, Pandas, and Matplotlib
 import numpy as np
 import pandas as pd
 import matplotlib.pyplot as plt

 # load the data file
 data_file = pd.read_csv('data_with_headers.csv')

 # create time vector from imported data
 time = data_file['time']
 # parse good sensor data from imported data
 sensors = data_file.ix[:,'s1':'s4']

 # display the first 6 sensor rows
 print sensors[0:6]

 # adjust time to start at zero by subtracting the
 #  first element in the time vector (index = 0)
 time = time - time[0]

 # calculate the average of the sensor readings
 avg = np.mean(sensors,1) # over the 2nd dimension

 # export data
 my_data = [time, sensors, avg]
 result = pd.concat(my_data,axis=1)


 # generate a figure
 # add text labels to the plot
 plt.legend(['Sensor 2','Average'])
 plt.xlabel('Time (sec)')
 plt.ylabel('Sensor Values')
 # save the figure as a PNG file
 # show the figure on the screen

This tutorial can also be completed with Excel and Matlab. Click on the appropriate link for additional information.

comments powered by Disqus