Beware of integer data type in computing

Matlab and Python are probably the most used programming languages for research. These languages are commonly used for algorithm prototyping, solving optimisation problem, data analysis and other purposes.

However, in many circumstances, we will need to use C/C++ , either from the first time of developments or from translating Matlab or Python codes, due to the need for deployment into instruments (control software), for faster execution, and for other reasons. C/C++ is a compiled language so that it has faster execution time than non-compiled languages, such as Matlab and Python. This faster execution time is just one of many advantages of C/C++ programming languages.

In C/C++, and other statically-typed programming languages such as C# and JAVA, we need to define the variable types in advance before using them. Following variable definitions, we need to strictly consider the types when we perform calculations in codes to build algorithms.

Integer data type is very often used to define variables storing non-floating point numbers. With integer data type, the memory space used to store the data is less than floating point data type. However, when integer-defined variables store floating numbers, the variable will cut any value after the decimal place. For example, when an integer variable is given a value of 2.345, the variable will store 2. Similarly, if the variable is given a value of 0.345 (<1), then the variable will store 0.

A serious problem occurs when an integer variable stores 0 value for actual values < 1. The problem is that, when we have a complex numerical calculation, whatever numbers involved in the calculation, the results will be 0. This problem causes many computational algorithms do not give results as expected by models or theories. This small value conversion from < 1 to 0 is, very often, difficult to debug.

Here, we show with a very simple example. In this example, we want to calculate the value of a sine wave with amplitude A = 1 and frequency f = 2Hz. The code to calculate this sine wave is given below:

const float pi=3.14159265358979;
const float f=2; 
for(int i=0;i<n;i++){		
	sine[i]=sin(2*pi*f*(i/n));
}

From the given code above, when we plot the sine wave, we get an unexpected result that is all sine values are 0 (figure 1). These all zero values are caused by the section in the code: (i/n)

This section of code involves variable i and n that are integer. Hence, since their calculation is done first, when the value of i/n < 1, the result of this calculation will be stored as 0. This 0 result causes all the values of the sine wave become 0 (figure 1).

To solve this problem, we need to convert at least one of the variables involved in (i/n) to be float or convert the results if (i/n) to be float. The first solution for the problem is shown in the code below:

const float pi=3.14159265358979;
const float f=2; 
for(int i=0;i<n;i++){		
	sine[i]=sin(2*pi*f*(float(i)/n));
}

From the code above, we first convert the data type of variable i into float. By this conversion, the compiler will expect that the result of the calculation is float and can store the actual value. Another solution is to convert the results of (i/n) to be float as shown in the code below:

const float pi=3.14159265358979;
const float f=2; 
for(int i=0;i<n;i++){		
sine[i]=sin(2*pi*f*((float)i/n));
}

From the code above, the result of the calculation (i/n) is converted to float so that the actual value is used to calculate the rest of the calculation.

By applying the correction, we can calculate the sine wave correctly. The result of the correct calculation is shown in figure 2. In figure 2, we can see that the sine wave has amplitude of one and has two cycles within one second, that is 2 cycle/period.

To close, we need to be discipline in assigning and using data type in C/C++ other statically-typed programming languages, especially in numerical computation.

We sell all the source files, EXE file, include and LIB files as well as documentation of ellipse fitting by using C/C++, Qt framework, Eigen and OpenCV libraries in this link