www.digitalmars.com         C & C++   DMDScript  

c++ - The slowest free C compiler?

reply "Ronald Barrett" <ronaldb sebcorinc.com> writes:
Hello,

I did a short floating-point benchmark with a simple C source. I tested gcc
(MinGW version), Lcc-win32 and DMC with default compiler options on P4
platform . It's surprisal to see that compilers which use 12 bytes long
double generate faster code. The DMC -ff option does not affect
significantly the resultant cpu utilization in this test. Here are the
results (in relative units):

gcc v3.2.3 (mingw special 20030504-1)                1
Lcc-win32 v3.8                                                       1.36
DMC v8.38n                                                            1.89

gcc v3.2.3 (mingw special 20030504-1                  0.85
                   with Dinkum library v4.02(commercial) -
                   only for comparison)

Ronald
Jan 25 2004
next sibling parent reply Ilya Minkov <minkov cs.tum.edu> writes:
You cannot seriously use default options, since it is with all compilers 
non-optimised, fast compile! With optimised compiles, i would frankly 
expect small difference, with slowest code being generated by LCC-Win32. 
I believe LCC is the only one which cannot effectively use up the 
floating-point registers.

Use -o -ff on DMC and something like -O2 -ffastmath on GCC, plus the 
architecture switches. Can't recall the options on LCC-Win32.

-eye

Ronald Barrett wrote:

 Hello,
 
 I did a short floating-point benchmark with a simple C source. I tested gcc
 (MinGW version), Lcc-win32 and DMC with default compiler options on P4
 platform . It's surprisal to see that compilers which use 12 bytes long
 double generate faster code. The DMC -ff option does not affect
 significantly the resultant cpu utilization in this test. Here are the
 results (in relative units):
 
 gcc v3.2.3 (mingw special 20030504-1)                1
 Lcc-win32 v3.8                                                       1.36
 DMC v8.38n                                                            1.89
 
 gcc v3.2.3 (mingw special 20030504-1                  0.85
                    with Dinkum library v4.02(commercial) -
                    only for comparison)
 
 Ronald
Jan 25 2004
next sibling parent "Ronald Barrett" <ronaldb sebcorinc.com> writes:
Thanks for the fast replies,

I coducted more comprehensive tests with the same source file. Here are the
results:
gcc v3.2.3 (mingw special 20030504-1)                   1
result_a
gcc v3.2.3 (mingw special 20030504-1) with -O2    0.115     result_a1
gcc v3.2.3 (mingw special 20030504-1) with -O1    0.28       result_a1
DMC v8.38n with -o                                                1.19
result_b
Lcc-win32 v3.8 with/without -O                               1.36
result_a1
DMC v8.38n without -o                                           1.89
result_b


With optimised compiles, i would frankly expect small difference, with


slowest code being generated by LCC-Win32.
You are right. DMC -o generate faster code in this test than Lcc-Win32.

The result_a, result_a1 and result_b are the results of the computations.
Every of them contains 3000000 long double values.
result_a and result_b are incomparable bit by bit because of the different
long double length. I found that result_a and result_a1 have exactly 1000000
identical differences:
fc result_a result_a1:
0000000A: 00 22
0000002E: 00 22
00000052: 00 22
...
0225509E: 00 22
022550C2: 00 22
022550E6: 00 22

I constructed a simple code to visualize the raw results:
#include <stdio.h>

int main(){
 FILE *input = fopen("result_a1", "rb");
 long double value[1];
 unsigned long counter;

 for (counter = 0; counter < 10000; counter++)
 {
  fread(value, sizeof(long double), 1, input);
  printf("%.20Lf\n", *value);
 }

 fclose(input);
 return 0;
}

The problem is that

Or with printf("%.20Lg...
-1311.0351567603643 from a line of result_a

-1311.0351562537248 from the same line of result_a1



Which file contains the correct computations?

If this is result_a consequently gcc with -O1 or -O2 options and Lcc-win32
(with/without -O) does not generate accurate long double computations in all
cases.



Also, DMC has slower math functions because DMC does extra work in them to


ensure accuracy
and correct handling of NaN's and overflows, work that is frequently skipped
by other compilers.
The test code indeed contain some inf computations. If the result_a1 is the
correct (the all difference are identical) then

"Ilya Minkov" <minkov cs.tum.edu> wrote in message
news:bv12k6$7ep$1 digitaldaemon.com...

 You cannot seriously use default options, since it is with all compilers
 non-optimised, fast compile! With optimised compiles, i would frankly
 expect small difference, with slowest code being generated by LCC-Win32.
 I believe LCC is the only one which cannot effectively use up the
 floating-point registers.

 Use -o -ff on DMC and something like -O2 -ffastmath on GCC, plus the
 architecture switches. Can't recall the options on LCC-Win32.

 -eye

 Ronald Barrett wrote:

 Hello,

 I did a short floating-point benchmark with a simple C source. I tested




gcc

 (MinGW version), Lcc-win32 and DMC with default compiler options on P4
 platform . It's surprisal to see that compilers which use 12 bytes long
 double generate faster code. The DMC -ff option does not affect
 significantly the resultant cpu utilization in this test. Here are the
 results (in relative units):

 gcc v3.2.3 (mingw special 20030504-1)                1
 Lcc-win32 v3.8




1.36

 DMC v8.38n




1.89

 gcc v3.2.3 (mingw special 20030504-1                  0.85
                    with Dinkum library v4.02(commercial) -
                    only for comparison)

 Ronald
Jan 25 2004
prev sibling next sibling parent reply "Ronald Barrett" <ronaldb sebcorinc.com> writes:
Thanks for the fast replies,

I conducted more comprehensive tests with the same source file. Here are the
results:
gcc v3.2.3 (mingw special 20030504-1)                   1
result_a
gcc v3.2.3 (mingw special 20030504-1) with -O2    0.115     result_a1
gcc v3.2.3 (mingw special 20030504-1) with -O1    0.28       result_a1
DMC v8.38n with -o                                                1.19
result_b
Lcc-win32 v3.8 with/without -O                               1.36
result_a1
DMC v8.38n                                                            1.89
result_b

From Ilya Minkov <minkov cs.tum.edu>:

With optimised compiles, i would frankly expect small difference, with


slowest code being generated by LCC-Win32.
You are right - DMC with -o option generate faster code in this test than
Lcc-Win32.

It's interesting to note the performance with gcc -O2 compilation.

The result_a, result_a1 and result_b are the results from the test file.
Every of them contains exactly 3000000 long double values.
result_a and result_b are incomparable bit per bit because of the different
long double length. Also the results within result_a and result_b differ
significantly (in same zones) probably due to the different long double
length (the test code perform really many computations - I don't know how
big is the accumulated computational error).

I found that result_a and result_a1 have exactly 1000000 identical
differences:
fc result_a result_a1:
0000000A: 00 22
0000002E: 00 22
00000052: 00 22
...
0225509E: 00 22
022550C2: 00 22
022550E6: 00 22

To compare visually the raw results I constructed vdata.c:
#include <stdio.h>

int main(int argc, char *argv[]){
 FILE *input = fopen(argv[1], "rb");
 long double value[1];
 unsigned long counter;

 for (counter = 0; counter < 3000000; counter++)
 {
  fread(value, sizeof(long double), 1, input);
  printf("%.20Lf\n", *value);
 }

 fclose(input);
 return 0;
}

lcc vdata.c & lcclnk vdata.obj:
vdata result_a1
-0.43290277584473510800
0.75025220636165043600
-0.45220046017929632000
...

vdata result_a > a.txt
vdata result_a1 > a1.txt

fc a.txt a1.txt /b
FC: no differences encountered

I repeated this with vdata.c compiled with MinGW gcc (with Dinkum
libraries - printf from the the default MinGW gcc libraries have a problem
with long doubles values) and found no difference in the fc comparison.

It seem that there are no difference between result_a and result_a1. If this
is true, consequently the test code compiled with gcc -O2 executes more than
10 times faster than the DMC equivalent with all optimizations.

From Walter <walter digitalmars.com>:

Also, DMC has slower math functions because DMC does extra work in them to


ensure accuracy
and correct handling of NaN's and overflows, work that is frequently skipped
by other compilers.
The test code indeed contain some overflows computations, but only with
addition, subtraction and multiplication. If you show interest in this case
I'll send you the source file - it's short and simple algorithm.

From Scott Michel <scottm cs.ucla.edu>:

What's the benchmark's code, what does it do, what does it test?


Indeed this is not a test source for benchmark. It's part of scientific
software.

Thank you,
Ronald

"Ilya Minkov" <minkov cs.tum.edu> wrote in message
news:bv12k6$7ep$1 digitaldaemon.com...

 You cannot seriously use default options, since it is with all compilers
 non-optimised, fast compile! With optimised compiles, i would frankly
 expect small difference, with slowest code being generated by LCC-Win32.
 I believe LCC is the only one which cannot effectively use up the
 floating-point registers.

 Use -o -ff on DMC and something like -O2 -ffastmath on GCC, plus the
 architecture switches. Can't recall the options on LCC-Win32.

 -eye

 Ronald Barrett wrote:

 Hello,

 I did a short floating-point benchmark with a simple C source. I tested




gcc

 (MinGW version), Lcc-win32 and DMC with default compiler options on P4
 platform . It's surprisal to see that compilers which use 12 bytes long
 double generate faster code. The DMC -ff option does not affect
 significantly the resultant cpu utilization in this test. Here are the
 results (in relative units):

 gcc v3.2.3 (mingw special 20030504-1)                1
 Lcc-win32 v3.8




1.36

 DMC v8.38n




1.89

 gcc v3.2.3 (mingw special 20030504-1                  0.85
                    with Dinkum library v4.02(commercial) -
                    only for comparison)

 Ronald
Jan 25 2004
parent reply "Walter" <walter digitalmars.com> writes:
If you're testing long doubles, you should be aware that few compilers on
Win32 actually support true 80 bit long doubles. Most fake it with 64 bits,
which of course will compute faster, but much less accurately.

DMC++ does real 80 bit long doubles.

Benchmarking floating point isn't easy <g>.
Jan 25 2004
parent reply "Ronald Barrett" <ronaldb sebcorinc.com> writes:
I'm aware of this. I conducted again the MinGW gcc tests with
option -m96bit-long-double. There wasn't difference between results and the
cpu utilization with the previous tests.

How is possible gcc -O2 to produce10 times faster code (96 bit long double)
in some cases than DMC which uses 80 bit long double?
If the above statement is fully correct this mean that my project which
requires 50 days computations (yes I have such projects) with DMC will
execute for 5 days roughly with gcc -O2. In addition it will be more
precisely computed with gcc due to the longer long double.

Do you want to see the source?

Ronald

"Walter" <walter digitalmars.com> wrote in message
news:bv20n4$1m55$1 digitaldaemon.com...

 If you're testing long doubles, you should be aware that few compilers on
 Win32 actually support true 80 bit long doubles. Most fake it with 64


bits,

 which of course will compute faster, but much less accurately.

 DMC++ does real 80 bit long doubles.

 Benchmarking floating point isn't easy <g>.
Jan 26 2004
parent reply "Walter" <walter digitalmars.com> writes:
"Ronald Barrett" <ronaldb sebcorinc.com> wrote in message
news:bv2vvl$ah2$1 digitaldaemon.com...

 I'm aware of this. I conducted again the MinGW gcc tests with
 option -m96bit-long-double. There wasn't difference between results and


the

 cpu utilization with the previous tests.

 How is possible gcc -O2 to produce10 times faster code (96 bit long


double)

 in some cases than DMC which uses 80 bit long double?


There's something else going on in the benchmark code, then. Something else,
like perhaps file I/O, that is taking so much time it is swamping the
result. Or perhaps MinGW happens to determine that your benchmark
computation is all dead code and deletes it entirely.


 If the above statement is fully correct this mean that my project which
 requires 50 days computations (yes I have such projects) with DMC will
 execute for 5 days roughly with gcc -O2. In addition it will be more
 precisely computed with gcc due to the longer long double.

 Do you want to see the source?

 Ronald
Jan 26 2004
parent "Ronald Barrett" <ronaldb sebcorinc.com> writes:
There's something else going on in the benchmark code, then. Something


else,
like perhaps file I/O, that is taking so much time it is swamping the
result.
Yes, there are exactly 30 fwrite calls with buffer of 100000 long double
values (note that the buffer for gcc is 1.2 times larger than the DMC one
and therefore the resultant file).


 Or perhaps MinGW happens to determine that your benchmark


computation is all dead code and deletes it entirely.
No, I compared visually the data from the source code compiled with DMC and
with MinGW gcc. There are some zones which differ significantly, but there
are also zones which doesn't differ. According to my empiric observations
this is due to the different accumulated errors (which in some zones
probably deplete the long double precision) in the computational space.
In addition compilation with Lcc-win32 produces exactly the same resultant
file as compilation with gcc -O2, but about 11 times slower.

I'll send you the source code within an hour.
Is there precision difference between the 80 bit long double and 96 bit one
(LDBL_EPSILON, LDBL_MIN, LDBL_MAX & LDBL_DIG are the same in DMC, Lcc-Win32
& MinGW gcc with -m96bit-long-double option) on x86 platform?

Thank you,
Ronald
Jan 26 2004
prev sibling parent "Ronald Barrett" <ronaldb sebcorinc.com> writes:


 Jan 25 2004



prev sibling next sibling parent  "Walter" <walter digitalmars.com>  writes:


"Ronald Barrett" <ronaldb sebcorinc.com> wrote in message
news:bv0t6c$30rc$1 digitaldaemon.com...

 Hello,

 I did a short floating-point benchmark with a simple C source. I tested


gcc

 (MinGW version), Lcc-win32 and DMC with default compiler options on P4
 platform . It's surprisal to see that compilers which use 12 bytes long
 double generate faster code. The DMC -ff option does not affect
 significantly the resultant cpu utilization in this test. Here are the
 results (in relative units):

 gcc v3.2.3 (mingw special 20030504-1)                1
 Lcc-win32 v3.8                                                       1.36
 DMC v8.38n                                                            1.89

 gcc v3.2.3 (mingw special 20030504-1                  0.85
                    with Dinkum library v4.02(commercial) -
                    only for comparison)


Default means unoptimized for DMC. Use -o for optimization. Also, DMC has
slower math functions because DMC does extra work in them to ensure accuracy
and correct handling of NaN's and overflows, work that is frequently skipped
by other compilers.



 Jan 25 2004



prev sibling  parent  Scott Michel <scottm cs.ucla.edu>  writes:


These results are absolutely meaningless. What's the benchmark's code, what
does it do, what does it test? Are these normalized raw numbers, normalized
relative to what? What's the distribution's 95% percentile, std. dev.? How
did you come up with the timings?

You've successfully benchmarked something, but what, I can't tell.


-scooter

Ronald Barrett wrote:


 Hello,
 
 I did a short floating-point benchmark with a simple C source. I tested
 gcc (MinGW version), Lcc-win32 and DMC with default compiler options on P4
 platform . It's surprisal to see that compilers which use 12 bytes long
 double generate faster code. The DMC -ff option does not affect
 significantly the resultant cpu utilization in this test. Here are the
 results (in relative units):
 
 gcc v3.2.3 (mingw special 20030504-1)                1
 Lcc-win32 v3.8                                                       1.36
 DMC v8.38n                                                            1.89
 
 gcc v3.2.3 (mingw special 20030504-1                  0.85
                    with Dinkum library v4.02(commercial) -
                    only for comparison)
 
 Ronald





 Jan 25 2004