Data Management

The DManager project propose classes and functions for managing the data. More...

Classes
class	STK::DataFrame
	DataFrame is a List of Variable with the same number of rows. More...
class	STK::ExportToCsv
	Export data to a Csv stream. More...
class	STK::ImportFromCsv
	import data from a Csv File. More...
class	STK::IPage
	A IPage is an interface base class for reading and/or writing a page of option in a file. More...
class	STK::IVariable
	IVariable is an Interface class for all Variables classes. More...
class	STK::Option
	the Option class allow to store the value of an option from a IPage. More...
class	STK::ReadWriteCsv
	the ReadWriteCsv class : allow to write and/or to read a csv file. More...
class	STK::ReadWritePages
	The ReadWritePages class allow to handle a file of option containing pages. More...
class	STK::Variable< TYPE >
	Variable is an implementation of the Base class IVariable using The Array1D class for storing the data. More...
Enumerations
enum	STK::Option::TypeOption {   STK::Option::unknown_ = 0, STK::Option::string_, STK::Option::real_, STK::Option::integer_,   STK::Option::range_, STK::Option::lstring_, STK::Option::lreal_, STK::Option::linteger_,   STK::Option::lrange_, STK::Option::page_ }
	A TypeOption is the kind of options that can read or write a program in a file. More...
enum	STK::Csv::readflags { STK::Csv::RF_APPEND_DATA = 0x00000004, STK::Csv::RF_REPLACE_DATA = 0x00000008 }
	enumeration of read flags These flags determine the behavior of the reading methods. More...
Functions
bool	STK::DManager::checkStringToBoolean (String const &str)
	check if a string represent a boolean.
bool	STK::DManager::StringToBoolean (String const &str)
	convert a string to a boolean.
void	STK::DManager::removeCharBeforeAndAfter (String &str, Char c)
	remove all occurrences of the char c at the beginning and the end of the string str.
String	STK::DManager::getField (istream &is, Char delimiter)
	Get the current field from the input stream.
template<class TYPE >
void	STK::DManager::readList (String const &strBuffer, std::list< TYPE > &lst, Char sep=CHAR_SEP)
	Read a list of value of type TYPE stored in a line.
template<class TYPE >
void	STK::DManager::writeList (ostream &os, std::list< TYPE > const &lst, Char sep=CHAR_SEP)
	Write a list of value of type TYPE stored in a line.
template<class TYPE , class TContainer1D >
void	STK::heapSort (ITContainer1D< TYPE, TContainer1D > &T)
	Sort the container T in ascending order.
template<class TYPE , class TContainer1D >
void	STK::heapSort (const ITContainer1D< TYPE, TContainer1D > &T, ITContainer1D< TYPE, TContainer1D > &Tsort)
	Sort the container T in ascending order and return the result in the container Tsort.
template<class TYPE , class TContainer1D >
void	STK::heapSort (Array1D< Integer > &I, const ITContainer1D< TYPE, TContainer1D > &T)
	Sort the container T in ascending order using index array.
template<class TYPE , class TContainer1D >
void	STK::applySort (ITContainer1D< TYPE, TContainer1D > &T, const Array1D< Integer > &I)
	Apply a sorting index array to the 1D container T.
template<class TYPE , class TContainerHo , class TContainerVe , class TContainer2D >
void	STK::applySort (ITContainer2D< TYPE, TContainerHo, TContainerVe, TContainer2D > &T, const Array1D< Integer > &I)
	Apply a sorting index array to the 2D container T row by row.
TypeImport	STK::Import::StringToTypeImport (String const &type)
	Convert a String to a TypeImport.
istream &	STK::operator>> (istream &is, ReadWriteCsv &df)
	Read the data from the stream and returns the stream when done.
ostream &	STK::operator<< (ostream &os, ReadWriteCsv const &df)
	write the data into the stream and returns the stream when done.
static Integer	STK::maxLength (const Variable< String > &V)
	Compute the maximal Length of the strings in V.
static Integer	STK::CountCols (String const &line, String const &delimiters, Variable< String > &typeDelimiter)
	Counts the number of columns in a line stored in a String and return the position of the delimiters and its types.
Variables
static const String	STK::STRING_EMPTY = String()
	Representation of Not Available String value.
static const String	STK::STRING_NL = _T("\n")
	Representation of a New Line String.
static const String	STK::STRING_BLANK = _T(" ")
	Representation of a blank value.
static const String	STK::STRING_VAR = _T("Var")
	default prefix of a variable name.
static const Char	STK::CHAR_COMMENT = _T('#')
	The char indicating the beginning of a comment in an option file.
static const Char	STK::CHAR_EQUAL = _T('=')
	The char indicating an equality in an option file.
static const Char	STK::CHAR_BLANK = _T(' ')
	The blank space char.
static const Char	STK::CHAR_TAB = _T('\t')
	The tab char.
static const Char	STK::CHAR_SEP = _T(',')
	The default separator char in list of option.
static const Char	STK::CHAR_OPENBRACKET = _T('[')
	The open bracket char.
static const Char	STK::CHAR_CLOSEBRACKET = _T(']')
	The close bracket char.
static const Char *	STK::Csv::DEFAULT_COLUMN_PREFIX = _T("Var")
	Defines the default prefix to used when naming the columns of the ReadWriteCsv.
static const Integer	STK::Csv::DEFAULT_RESERVE = Integer(0x0FFFF)
	Set Array1D<string>::reserve() with this value.
static const Char *	STK::Csv::DEFAULT_DELIMITER = _T(";")
	Defines the default field delimiter in a file.
static const std::string	STK::ERRORCODES []
	some errors messages.

---

Detailed Description

The DManager project propose classes and functions for managing the data.

The DManager project contains all the class related to data management. It provides

• an abstract base class for statistical variables,
• a templated implementation for arbitrary data,
• a Dataframe (Table) class,
• classes for read and write csv and (TODO) dbf files,
• various classes for importing/exporting data from different containers,
• methods for sorting one dimensional containers,
• abstract classes for reading pages of options.

---

Enumeration Type Documentation

enum STK::Option::TypeOption [inherited]

A TypeOption is the kind of options that can read or write a program in a file.

It can be:

• a String (e.g. toto)
• a Real (e.g. 3.14159)
• an Integer (e.g. 101)
• a Range of number (e.g. 2:10)
• a list of String
• a list of Real
• a list of Integer
• a list of Range
• a sub-page

Enumerator:

unknown_
string_	characters
real_	floating
integer_	discrete number
range_	range of number
lstring_	list of strings separated by commas
lreal_	list of floating numbers separated by commas
linteger_	list of discrete numbers separated by commas
lrange_	list of range of number separated by commas
page_	a page option

Definition at line 79 of file STK_Option.h.

    {
      unknown_ =0,
      string_,      
      real_,        
      integer_,     
      range_,       
      lstring_,     
      lreal_,       
      linteger_,    
      lrange_,       
      page_         
    };

enum STK::Csv::readflags

enumeration of read flags These flags determine the behavior of the reading methods.

Enumerator:

RF_APPEND_DATA
RF_REPLACE_DATA

Definition at line 64 of file STK_ReadWriteCsv.h.

  {
    RF_APPEND_DATA    = 0x00000004,
    RF_REPLACE_DATA   = 0x00000008
  };

---

Function Documentation

bool STK::DManager::checkStringToBoolean

(

String const &

str

)

check if a string represent a boolean.

A String is a boolean if it is written "TRUE" or "FALSE". There is no need to use upper case.

Parameters:

str	the string to check

Returns:

true if the String i a boolean, false otherwise.

Definition at line 53 of file STK_DManager_Util.cpp.

References _T, and STK::toUpperString().

{
  // is it TRUE ?
  if (str.compare(toUpperString(_T("true"))) == 0)
  { return true;}
  // is it FALSE ?
  if (str.compare(toUpperString(_T("false"))) == 0)
  { return true;}
  // not a bolean string
  return false;
}

bool STK::DManager::StringToBoolean

(

String const &

str

)

convert a string to a boolean.

A String is a boolean if it is written "TRUE" or "FALSE". There is no need to use upper case.

Parameters:

str	the string to convert

Returns:

true if the String is "TRUE, false otherwise.

Definition at line 73 of file STK_DManager_Util.cpp.

References _T, and STK::toUpperString().

{
  // is it TRUE ?
  if (str.compare(toUpperString(_T("true"))) == 0)
  { return true;}
  // if it's not true it's false
  return false;
}

void STK::DManager::removeCharBeforeAndAfter	(	String &	str,
		Char	c
	)

remove all occurrences of the char c at the beginning and the end of the string str.

Parameters:

str	the string to treat
c	the character to remove before and after

Definition at line 86 of file STK_DManager_Util.cpp.

Referenced by STK::IPage::findKeyword(), STK::DManager::getField(), STK::operator>>(), STK::IPage::processLine(), and STK::IPage::read().

{
  // erase first whitespaces
  str.erase(0, str.find_first_not_of(c));
  // erase remaining whitespaces
  size_t found =str.find_last_not_of(c);
  if (found != str.npos)
    str.erase(found+1);
  else
    str.clear(); // str is all whitespace
}

String STK::DManager::getField	(	istream &	is,
		Char	delimiter
	)

Get the current field from the input stream.

A field is between the current position and a delimiter or an end of line in the stream. All blank spaces and tabulations before and after the field are removed.

Parameters:

is	the stream to treat
delimiter	the delimiter of the current field

Returns:

the field extracted from the input stream

Definition at line 102 of file STK_DManager_Util.cpp.

References STK::CHAR_BLANK, STK::CHAR_TAB, and STK::DManager::removeCharBeforeAndAfter().

Referenced by STK::operator>>(), and STK::DManager::readList().

{
  String strbuff;
  std::getline( is, strbuff, delimiter);
  removeCharBeforeAndAfter(strbuff, CHAR_BLANK);
  removeCharBeforeAndAfter(strbuff, CHAR_TAB);
  return strbuff;
}

template<class TYPE >

void STK::DManager::readList	(	String const &	strBuffer,
		std::list< TYPE > &	lst,
		Char	sep = CHAR_SEP
	)

Read a list of value of type TYPE stored in a line.

Parameters:

strBuffer	the string with the list of value
lst	the resulting list
sep	the separator character

Definition at line 147 of file STK_DManager_Util.h.

References STK::DManager::getField(), and STK::stringToType().

Referenced by STK::Option::setValue().

{
  // Declare an input string stream
  istringstream instream;
  // Use strBuffer as source of input.
  instream.str(strBuffer);
  // read the line
  do
  {
    // get field
    String strbuff = getField(instream, sep);
    // check if it is a blank field
    if (strbuff.empty()) {  break;}
    // append Data to the list
    TYPE value;
    if (stringToType(value, strbuff))
      lst.push_back(value);
    // TODO: else emit warning or exception
  }
  while(1);
}

template<class TYPE >

void STK::DManager::writeList	(	ostream &	os,
		std::list< TYPE > const &	lst,
		Char	sep = CHAR_SEP
	)

Write a list of value of type TYPE stored in a line.

Parameters:

os	the output stream
lst	the list to write
sep	the separator character

Definition at line 176 of file STK_DManager_Util.h.

References STK::STRING_BLANK.

Referenced by STK::Option::write().

{
  if (lst.empty()) return;
  typename std::list<TYPE>::const_iterator it = lst.begin();
  if (it == lst.end()) return;
  os << *it;
  it++;
  for ( ; it != lst.end(); it++)
  {
    os << sep << STRING_BLANK << *it;
  }
}

template<class TYPE , class TContainer1D >

void STK::heapSort

(

ITContainer1D< TYPE, TContainer1D > &

T

)

Sort the container T in ascending order.

Parameters:

T	the container to sort

Definition at line 55 of file STK_HeapSort.h.

References STK::IRecursiveTemplate< TContainer1D >::asLeaf(), STK::ITContainer1D< TYPE, TContainer1D >::first(), and STK::ITContainer1D< TYPE, TContainer1D >::size().

Referenced by STK::Stat::Univariate< Real, TContainer1D >::compOrderStatistics(), and STK::BSplineCoefficients::computeDensityKnots().

{
  // number of elements
  const Integer nb_elt = T.size();
  if (nb_elt < 2) return;

  // if the container is base one, shift0 = 0 and shift1 = 1
  Integer shift1 = T.first(), shift0 = T.first() - 1;

  // create heap
  for (Integer first = nb_elt/2; first > 0; -- first)
  {
    // the value value to insert in the heap
    TYPE value = T[shift0 + first];
    // organize the heap
    Integer i=first, j=2*first;
    while (j <= nb_elt)
    {
      // j+1 is greatest child
      if ( j < nb_elt && T[shift0 + j] < T[shift1 + j] ) j++;
      // we have find a child gt value
      if (value >= T[shift0 + j]) break;
      // else shift the inner value
      T[shift0 + i] = T[shift0 + j];
      // go down in the tree
      i = j;
      j*= 2;
    }
    // plug value in its final location
    T[shift0 + i] = value;
  }
#ifdef STK_HEAPSORT_DEBUG
  std::cout << "T=\n" << T.asLeaf() << "\n";
#endif
  // sort T
  for (Integer last = nb_elt;;)
  { // the value to sort
    TYPE value = T[shift0 + last];
    // Put the top of the heap at the end
    T[shift0 + last] = T[shift1];
    // decrease last.  last==1 : we end the job
    if (--last == 1)
    { T[shift1] = value;
      break;
    }
    // organize the heap
    Integer i=1, j=2;
    while (j <= last)
    { // j+1 is greatest child
      if ( j < last && T[shift0 + j] < T[shift1 + j] ) j++;
      // we have find a child gt value
      if (value >= T[shift0 + j]) break;
      // else shift the inner value
      T[shift0 + i] = T[shift0 + j];
      // go down in the tree
      i = j;
      j*= 2;
    }
    // plug value in its final location
    T[shift0 + i] = value;
  }
}

template<class TYPE , class TContainer1D >

void STK::heapSort	(	const ITContainer1D< TYPE, TContainer1D > &	T,
		ITContainer1D< TYPE, TContainer1D > &	Tsort
	)

Sort the container T in ascending order and return the result in the container Tsort.

Parameters:

T	the container to sort
Tsort	the container with the result

Definition at line 125 of file STK_HeapSort.h.

References STK::IRecursiveTemplate< TContainer1D >::asLeaf(), STK::ITContainer1D< TYPE, TContainer1D >::first(), STK::ITContainer1D< TYPE, TContainer1D >::size(), and STK::Funct::T.

{
  // copy T in Tsort
  Tsort = T;
  // number of elements
  const Integer nb_elt = Tsort.size();
  if (nb_elt < 2) return;

  // if the container is base one, shift0 = 0 and shift1 = 1
  Integer shift1 = Tsort.first(), shift0 = Tsort.first() - 1;

  // create heap
  for (Integer first = nb_elt/2; first > 0; -- first)
  {
    // the value value to insert in the heap
    TYPE value = Tsort[shift0 + first];
    // organize the heap
    Integer i=first, j=2*first;
    while (j <= nb_elt)
    {
      // j+1 is greatest child
      if ( j < nb_elt && Tsort[shift0 + j] < Tsort[shift1 + j] ) j++;
      // we have find a child gt value
      if (value >= Tsort[shift0 + j]) break;
      // else shift the inner value
      Tsort[shift0 + i] = Tsort[shift0 + j];
      // go down in the tree
      i = j;
      j*= 2;
    }
    // plug value in its final location
    Tsort[shift0 + i] = value;
  }
#ifdef STK_HEAPSORT_DEBUG
  std::cout << "T=\n" << Tsort.asLeaf() << "\n";
#endif
  // sort T
  for (Integer last = nb_elt;;)
  { // the value to sort
    TYPE value = Tsort[shift0 + last];
    // Put the top of the heap at the end
    Tsort[shift0 + last] = Tsort[shift1];
    // decrease last.  last==1 : we end the job
    if (--last == 1)
    { Tsort[shift1] = value;
      break;
    }
    // organize the heap
    Integer i=1, j=2;
    while (j <= last)
    { // j+1 is greatest child
      if ( j < last && Tsort[shift0 + j] < Tsort[shift1 + j] ) j++;
      // we have find a child gt value
      if (value >= Tsort[shift0 + j]) break;
      // else shift the inner value
      Tsort[shift0 + i] = Tsort[shift0 + j];
      // go down in the tree
      i = j;
      j*= 2;
    }
    // plug value in its final location
    Tsort[shift0 + i] = value;
  }
}

template<class TYPE , class TContainer1D >

void STK::heapSort	(	Array1D< Integer > &	I,
		const ITContainer1D< TYPE, TContainer1D > &	T
	)

Sort the container T in ascending order using index array.

T is not modified, I contain the indices of the elements of T in ascending order.

Parameters:

I	the index array sorting T
T	the container to sort

Definition at line 200 of file STK_HeapSort.h.

References STK::ITContainer1D< TYPE, TContainer1D >::first(), STK::ITContainer1D< TYPE, TContainer1D >::last(), STK::ITContainer1D< TYPE, TContainer1D >::range(), STK::IContainer1D::resize(), and STK::ITContainer1D< TYPE, TContainer1D >::size().

{
  // number of elements
  Integer nb_elt = T.size();

  // create index array
  I.resize(T.range());
  Integer first = I.first(), last = I.last();
  for (Integer i=first; i<=last; i++)
  { I[i] = i;}

  if (nb_elt < 2) return;

  // if the container is base one, shift0 = 0 and shift1 = 1
  Integer shift1 = T.first(), shift0 = T.first() - 1;

  // create heap
  for (first = nb_elt/2; first > 0; --first)
  {
    // the value value to insert in the heap
    TYPE value = T[I[shift0 + first]];
    // organize the heap
    Integer i=first, j=2*first;
    while (j <= nb_elt)
    {
      // j+1 is greatest child
      if ( j < nb_elt && T[I[shift0 + j]] < T[I[shift1 + j]] ) j++;
      // we have find a child lt value
      if (value >= T[I[shift0 + j]]) break;
      // else shift the inner values
      I[shift0 + i] = I[shift0 + j];
      // go down in the tree
      i = j;
      j*= 2;
    }
    // plug value in its final location
    I[shift0 + i] = shift0 + first;
  }
#ifdef STK_HEAPSORT_DEBUG
  std::cout << "I=\n" << I <<"\n";
#endif
  // sort T
  for (Integer last = nb_elt;;)
  {
    // the value to sort
    Integer ivalue = I[shift0 + last];
    TYPE value = T[ivalue];
    // Put the top of the heap at the end
    //T[shift0 + last] = T[shift1];
    I[shift0 + last] = I[shift1];
    // decrease last.  last==1 : we end the job
    if (--last == 1)
    { //T[shift1] = value;
      I[shift1] = ivalue;
      break;
    }
    // organize the heap
    Integer i=1, j=2;
    while (j <= last)
    { // j+1 is greatest child
      if ( j < last && T[I[shift0 + j]] < T[I[shift1 + j]] ) j++;
      // we have find a child gt value
      if (value >= T[I[shift0 + j]]) break;
      // else shift the inner value
      // T[shift0 + i] = T[shift0 + j];
      I[shift0 + i] = I[shift0 + j];
      // go down in the tree
      i = j;
      j*= 2;
    }
    // plug value in its final location
    // T[shift0 + i] = value;
    I[shift0 + i] = ivalue;
  }
#ifdef STK_HEAPSORT_DEBUG
  std::cout << "I=\n" << I <<"\n";
#endif
}

template<class TYPE , class TContainer1D >

void STK::applySort	(	ITContainer1D< TYPE, TContainer1D > &	T,
		const Array1D< Integer > &	I
	)

Apply a sorting index array to the 1D container T.

Parameters:

I	the index array sorting T
T	the container to sort

Definition at line 288 of file STK_HeapSort.h.

References STK::IRecursiveTemplate< TContainer1D >::asLeaf(), STK::ITContainer1D< TYPE, TContainer1D >::first(), STK::ITContainer1D< TYPE, TContainer1D >::last(), and STK::ITContainer1D< TYPE, TContainer1D >::range().

Referenced by STK::Stat::Univariate< Real, TContainer1D >::compOrderStatistics().

{
#ifdef STK_DEBUG
  if (I.range() != T.range())
  { throw std::runtime_error("In applySort(T, I) "
                        "incompatible lengths\n");
  }
#endif
  TContainer1D A(T.range());
  const Integer first = I.first(), last = I.last();
  for (Integer i=first; i<= last; i++)
  {
    A[i] = T[I[i]];
  }
  T.asLeaf() = A.asLeaf();
}

template<class TYPE , class TContainerHo , class TContainerVe , class TContainer2D >

void STK::applySort	(	ITContainer2D< TYPE, TContainerHo, TContainerVe, TContainer2D > &	T,
		const Array1D< Integer > &	I
	)

Apply a sorting index array to the 2D container T row by row.

Parameters:

I	the index array sorting T
T	the container to sort

Definition at line 313 of file STK_HeapSort.h.

References STK::IRecursiveTemplate< TContainer2D >::asLeaf(), STK::ITContainer1D< TYPE, TContainer1D >::first(), STK::ITContainer1D< TYPE, TContainer1D >::last(), STK::ITContainer1D< TYPE, TContainer1D >::range(), STK::IContainer2D::rangeHo(), STK::IContainer2D::rangeVe(), and STK::Funct::T.

{
#ifdef STK_DEBUG
  if (I.range() != T.rangeVe())
  { throw std::runtime_error("In applySort(T, I) "
                        "incompatible lengths\n");
  }
#endif
  TContainer2D A(T.rangeVe(), T.rangeHo());
  const Integer first = I.first(), last = I.last();
  for (Integer i=first; i<= last; i++)
  {
    A(i) = T(I[i]);
  }
  T.asLeaf() = A.asLeaf();
}

TypeImport STK::Import::StringToTypeImport

(

String const &

type

)

Convert a String to a TypeImport.

Parameters:

type	the String we want to convert

Returns:

the TypeImport represented by the String type. if the string does not match any known name, the unknown_ type is returned.

Definition at line 51 of file STK_Import_Util.cpp.

References _T, STK::Import::directed_, STK::Import::intelligent_, STK::Import::numeric_, STK::Import::only_numeric_, STK::Import::string_, STK::toUpperString(), and STK::Import::unknown_.

{
  if (toUpperString(type) == toUpperString(_T("unknown"))) return unknown_;
  if (toUpperString(type) == toUpperString(_T("numeric")))  return numeric_;
  if (toUpperString(type) == toUpperString(_T("only_numeric"))) return only_numeric_;
  if (toUpperString(type) == toUpperString(_T("string"))) return string_;
  if (toUpperString(type) == toUpperString(_T("directed"))) return directed_;
  if (toUpperString(type) == toUpperString(_T("intelligent"))) return intelligent_;
  return unknown_;
}

istream & STK::operator>>	(	istream &	is,
		ReadWriteCsv &	df
	)

Read the data from the stream and returns the stream when done.

Parameters:

is	input stream
df	the ReadWriteCsv to read

Definition at line 610 of file STK_ReadWriteCsv.cpp.

References STK::ReadWriteCsv::appendData(), STK::ITContainer1D< TYPE, TContainer1D >::at(), STK::CHAR_BLANK, STK::ReadWriteCsv::clear(), STK::CountCols(), STK::ReadWriteCsv::delimiter_, STK::Variable< TYPE >::elt(), STK::ERRORCODES, STK::ReadWriteCsv::file_name_, STK::DManager::getField(), STK::IVariable::giveName(), STK::max(), STK::min(), STK::ReadWriteCsv::msg_error_, STK::ReadWriteCsv::push_back(), STK::ITContainer1D< TYPE, TContainer1D >::push_back(), STK::DManager::removeCharBeforeAndAfter(), STK::Csv::RF_APPEND_DATA, STK::IVariable::setName(), STK::ReadWriteCsv::size(), STK::ReadWriteCsv::source_file_names_, STK::ReadWriteCsv::str_data_, and STK::ReadWriteCsv::with_names_.

{
  try
  {
    // clear previous ReadCvs if we don't want to append
    if (!(Csv::RF_APPEND_DATA)) df.clear();
    // compute number of existing variables
    Integer colOffset = df.size();
    // initialize the initial number of variables to 0
    Integer nbVars = 0;

    // aux variable for handling delimiters
    Variable<String>    typeDelimiter;
    // set filname    
    df.source_file_names_.push_back(df.file_name_);
    // load file in memory
    stringstream inBuffer;
    inBuffer << is.rdbuf();

    // If the names are at the top line
    if (df.with_names_)
    {
      // get current line
      String lineBuffer;
      // Count the number of names of the first line
      Integer numField;
      do
      {
        // get current line in strBuffer
        std::getline(inBuffer, lineBuffer);
        DManager::removeCharBeforeAndAfter(lineBuffer, CHAR_BLANK);
        (lineBuffer.size() == 0) ?
          numField = 0
        : numField = CountCols( lineBuffer, df.delimiter_, typeDelimiter);
      }
      while ((numField == 0)&&(!inBuffer.eof()));
      // break if we get the end of file
      if (inBuffer.eof()) return is;

      // Declare an input string stream
      istringstream instream;
      // Reset from possible previous errors.
      instream.clear();
      // Use strBuffer as source of input.
      instream.str(lineBuffer);
      // Loop over the columns
      for(Integer icol=1; icol<=numField; icol++)
      {
        // Append a Col
        df.push_back(Variable<String>());
        df.str_data_.at(icol+colOffset).setName( DManager::getField( instream
                                                        , typeDelimiter.elt(icol).at(0))
                                                        );
      }
      // Update the number of var
      nbVars = STK::max(nbVars, numField);
    }

    // Read data
    for (Integer irow=1; true; irow++) // loop for all rows
    {
      // current line
      String lineBuffer;
      // number of fields in the line
      Integer numField;
      do
      {
        // get current line in strBuffer
        std::getline(inBuffer, lineBuffer);
        DManager::removeCharBeforeAndAfter(lineBuffer, CHAR_BLANK);
        (lineBuffer.size() == 0) ?
          numField = 0
        : numField = CountCols( lineBuffer, df.delimiter_, typeDelimiter);
      }
      while ((numField == 0)&&(!inBuffer.eof()));
      // check end of file and empty line
      if (inBuffer.eof()) break;
      if (numField == 0) continue;
      // Declare an input string stream
      istringstream instream;
      // Reset from possible previous errors.
      instream.clear();
      // Use lineBuffer as source of input.
      instream.str(lineBuffer);
      String fieldValue;
      // first loop on the exisiting cols with data
      Integer attCols = min(numField, nbVars);
      for (Integer icol=1; icol<=attCols; icol++)
      {
        // Read field
        fieldValue = DManager::getField( instream, typeDelimiter.elt(icol).at(0));
        // append Data to the columnn
        df.appendData(icol+colOffset, fieldValue);
      }

      // second loop on the existing cols without data
      // will be passed if (numField < nbVars)
      for (Integer icol=attCols+1; icol<=nbVars; icol++)
        df.str_data_.at(icol+colOffset).push_back(Arithmetic<String>::NA());

      // loop on the non-exisiting cols
      // will be passed if (numField > nbVars)
      for (Integer icol=nbVars+1; icol<=numField; icol++)
      {
        // we create each column with NA values
        df.push_back(Variable<String>( irow-1
                                     , Arithmetic<String>::NA()
                                     , IVariable::giveName(icol+colOffset)
                                     )
                    );
        // Read field
        fieldValue = DManager::getField( instream, typeDelimiter.elt(icol).at(0));
        df.appendData(icol+colOffset, fieldValue);
      }
      // Update the number of variables
      nbVars = STK::max(nbVars, numField);
      // break if we get the end of the file
      if (inBuffer.eof()) break; 
    } // irow loop
  }
  catch(const std::exception& e) 
  { 
    df.msg_error_ = e.what();
    throw e; 
  }
  catch(...) 
  { 
    df.msg_error_ = ERRORCODES[0];
    throw std::exception();
  }

  return is;
}

ostream & STK::operator<<	(	ostream &	os,
		ReadWriteCsv const &	df
	)

write the data into the stream and returns the stream when done.

Parameters:

os	output stream
df	the ReadWriteCsv to write

Definition at line 744 of file STK_ReadWriteCsv.cpp.

References STK::ITContainer1D< TYPE, TContainer1D >::first(), STK::ReadWriteCsv::firstVe(), STK::ITContainer1D< TYPE, TContainer1D >::last(), STK::ReadWriteCsv::lastVe(), STK::ReadWriteCsv::str_data_, and STK::ReadWriteCsv::writeSelection().

{
  try
  {
    df.writeSelection( os
                     , df.firstVe()
                     , df.lastVe()
                     , df.str_data_.first()
                     , df.str_data_.last()
                     );
  }
  // catch and re-throw any exceptions
  catch(const std::exception& e) { throw e; }
  catch(...) { throw std::exception(); }
  return os;
}

static Integer STK::maxLength

(

const Variable< String > &

V

)

[static]

Compute the maximal Length of the strings in V.

Parameters:

V	the String to treat.

Definition at line 69 of file STK_ReadWriteCsv.cpp.

References STK::IContainer1D::empty(), STK::IContainer1D::first(), STK::IContainer1D::last(), and STK::max().

Referenced by STK::ReadWriteCsv::writeSelection().

{
  if (V.empty()) return 0;

  // initialize
  Integer maxlength = -Arithmetic<Integer>::max();
  // loop over the values
  for (Integer i=V.first(); i<=V.last(); i++)
  {
    if (!Arithmetic<String>::isNA(V[i])) // update
      maxlength = max(maxlength, (Integer(V[i].size())));
  }
  return maxlength;
}

static Integer STK::CountCols	(	String const &	line,
		String const &	delimiters,
		Variable< String > &	typeDelimiter
	)		[static]

Counts the number of columns in a line stored in a String and return the position of the delimiters and its types.

Parameters:

line	The String to parse
delimiters	The String with the delimiters
typeDelimiter	The String with all the delimiters encountered

Returns:

the number of field in the line

Definition at line 93 of file STK_ReadWriteCsv.cpp.

References _T, STK::ITContainer1D< TYPE, TContainer1D >::push_back(), and STK::IContainer1D::resize().

Referenced by STK::operator>>().

{
  // number of fields in the file
  Integer numField = 0;
  // no delimiters
  typeDelimiter.resize();
  // Find first delimiter
  String::size_type idx = line.find_first_of(delimiters);
  // if the position of the delimiter char is found (there is a position)
  while (idx != line.npos)
  {
    // add a column
    ++numField;
    // save delimiter
    typeDelimiter.push_back(line.substr(idx,1));
    // find next delimiter
    idx = line.find_first_of(delimiters, idx+1);
  }
  // last delimiter is the end of line
  typeDelimiter.push_back(_T("\n"));
  // return the number of fields
  return ++numField;
}

---

Variable Documentation

const String STK::STRING_EMPTY = String() [static]

Representation of Not Available String value.

Empty String : the empty string is also the NA value of the class String.

Definition at line 54 of file STK_DManager_Util.h.

const String STK::STRING_NL = _T("\n") [static]

Representation of a New Line String.

Definition at line 58 of file STK_DManager_Util.h.

Referenced by STK::IPage::write().

const String STK::STRING_BLANK = _T(" ") [static]

Representation of a blank value.

Definition at line 62 of file STK_DManager_Util.h.

Referenced by STK::Option::write(), STK::DataFrame::writeDataFrame(), and STK::DManager::writeList().

const String STK::STRING_VAR = _T("Var") [static]

default prefix of a variable name.

Definition at line 66 of file STK_DManager_Util.h.

const Char STK::CHAR_COMMENT = _T('#') [static]

The char indicating the beginning of a comment in an option file.

Definition at line 72 of file STK_DManager_Util.h.

Referenced by STK::IPage::findKeyword(), and STK::IPage::read().

const Char STK::CHAR_EQUAL = _T('=') [static]

The char indicating an equality in an option file.

Definition at line 76 of file STK_DManager_Util.h.

Referenced by STK::IPage::processLine(), STK::IPage::read(), and STK::Option::write().

const Char STK::CHAR_BLANK = _T(' ') [static]

The blank space char.

Definition at line 80 of file STK_DManager_Util.h.

Referenced by STK::IPage::findKeyword(), STK::DManager::getField(), STK::operator>>(), STK::IPage::processLine(), STK::IPage::read(), and STK::IPage::write().

const Char STK::CHAR_TAB = _T('\t') [static]

The tab char.

Definition at line 84 of file STK_DManager_Util.h.

Referenced by STK::IPage::findKeyword(), STK::DManager::getField(), STK::IPage::processLine(), and STK::IPage::read().

const Char STK::CHAR_SEP = _T(',') [static]

The default separator char in list of option.

Definition at line 88 of file STK_DManager_Util.h.

const Char STK::CHAR_OPENBRACKET = _T('[') [static]

The open bracket char.

Definition at line 92 of file STK_DManager_Util.h.

Referenced by STK::createKeyWord(), and STK::IPage::read().

const Char STK::CHAR_CLOSEBRACKET = _T(']') [static]

The close bracket char.

Definition at line 96 of file STK_DManager_Util.h.

Referenced by STK::createKeyWord().

const Char* STK::Csv::DEFAULT_COLUMN_PREFIX = _T("Var") [static]

Defines the default prefix to used when naming the columns of the ReadWriteCsv.

Definition at line 56 of file STK_ExportToCsv.h.

const Integer STK::Csv::DEFAULT_RESERVE = Integer(0x0FFFF) [static]

Set Array1D<string>::reserve() with this value.

Definition at line 54 of file STK_ReadWriteCsv.h.

const Char* STK::Csv::DEFAULT_DELIMITER = _T(";") [static]

Defines the default field delimiter in a file.

Definition at line 58 of file STK_ReadWriteCsv.h.

const std::string STK::ERRORCODES[] [static]

Initial value:

{
  "In ReadWriteCsv UNKNOWN ERROR",
  "In ReadWriteCsv::variable() An unknown error occurred!",
  "In ReadWriteCsv Variable name not found!",
  "In ReadWriteCsv Filename name not found!",
  "In ReadWriteCsv File not found!",
  "In ReadWriteCsv The Number of Headers is different"
                   " from the Number of Data Columns!"
 }

some errors messages.

Definition at line 51 of file STK_ReadWriteCsv.cpp.

Referenced by STK::ReadWriteCsv::appendData(), STK::ReadWriteCsv::colIndex(), STK::ReadWriteCsv::data(), STK::ReadWriteCsv::eraseColumn(), STK::ReadWriteCsv::name(), STK::operator>>(), STK::ReadWriteCsv::push_back(), STK::ReadWriteCsv::push_front(), STK::ReadWriteCsv::read(), STK::ReadWriteCsv::setData(), STK::ReadWriteCsv::setName(), and STK::ReadWriteCsv::write().