CSV

CSV - Data Input

Measurements may be brought into the optimization problem via a Comma Separate Value (CSV) file. The file must have the same name as the model file, but with the .csv extension.

The CSV file consists of columns of data for a unique measurement. The column headings are the global variable names. The rows of the CSV file are the measurement at various times that correspond to steady-state conditions. If the data is not at steady-state, dynamic parameter estimation should be used instead.

Time/SetMeasurement #1Measurement #2Measurement #3
Time 1Meas 1 @ Time 1Meas 2 @ Time 1Meas 3 @ Time 1
Time 2Meas 1 @ Time 2Meas 2 @ Time 2Meas 3 @ Time 2
Time 3Meas 1 @ Time 3Meas 2 @ Time 3Meas 3 @ Time 3

There is no limit to the number of data rows that may be included. APMonitor automatically configures the model size based on the size of the file. The only limitation on problem size is the amount of computational resources that are available to run the case.

CSV File for MPU

For MPU, the number of steady-state sets is given by the number of data sets found in the file.

CSV File for Dynamic Cases

For dynamic problems, the time steps are given by the number of data rows. The spacing of the nodes in the time horizon is given by the column named time. If no time intervals are given, the NLC.ctrl_time is used by default.

CSV File for Replay

A replay data file (Replay.csv) is automatically generated when NLC.HIST_HOR >= 1. Replay data is stored for a minimum of 500 rows. If NLC.HIST_HOR is larger than 500, the replay file is truncated to that number of entries instead.

To replay the historical data, the user needs only to copy the appropriate Replay.csv file into the run directory and set NLC.Replay to the starting row of interest. Thereafter, the value of NLC.Replay is incremented after each complete cycle of the program, whether a good solution is obtained or not.

Missing Data

Missing data is indicated by non-numbers in the field. The missing data option is useful for infrequently measured data, intermittently available measurements, or to cancel periods of uncertainty. Fields such as X or Missing Data are acceptable indicators in the column locations.

Example File - tank.csv

 time,tank.m.v,tank.m.pct_open
 1.00E+00,2.68E+02,8.00E+01
 2.00E+01,6.13E+02,8.00E+01
 4.00E+01,9.55E+02,8.00E+01
 6.00E+01,1.26E+03,8.00E+01
 6.90E+01,???,     8.00E+01
 7.00E+01,1.42E+03,0.00E+00
 9.00E+01,1.34E+03,0.00E+00
 1.10E+02,1.24E+03,0.00E+00
 1.30E+02,X,       0.00E+00
 1.50E+02,1.04E+03,0.00E+00
 1.70E+02,9.55E+02,0.00E+00
 1.90E+02,Bad Data,0.00E+00
 2.10E+02,7.88E+02,0.00E+00
 2.30E+02,7.10E+02,0.00E+00
 2.40E+02,6.74E+02,0.00E+00
💬