, 7 min read
Performance Comparison C vs. Java vs. Javascript vs. PHP vs. Python vs. Cobol vs. Dart
Original post is here eklausmeier.goip.de/blog/2023/03-25-performance-comparison-c-vs-java-vs-javascript-vs-php-vs-python-vs-cobol-dart.
1. Introduction. This post is a continuation of a previous benchmark in Performance Comparison C vs. Java vs. Javascript vs. LuaJIT vs. PyPy vs. PHP vs. Python vs. Perl. Here we compare
- C
- Java
- JavaScript
- Python
- Cobol
- GnuCOBOL
- gcobol (gcc based COBOL)
- Dart
I tried to run IBM COBOL as well, but installation as described in Installing IBM COBOL for Linux on Arch Linux failed.
Testing machine is a Ryzen 7 5700G (Cezanne) with 16 CPUs and 4672.0698 MHz max clock. Operating system is Arch Linux 6.2.6-arch1-1. Source code for the programs is in Performance Comparison C vs. Java vs. Javascript vs. LuaJIT vs. PyPy vs. PHP vs. Python vs. Perl.
2. Results. I again use the n-queens problem, i.e., how many times can n queens be put on an n x n chess board without attacking any other queen. Task at hand is to compute from n=1 to n=13 all possible combinations. For example, for C I call time xdamcnt2 13
. I ran the programs multiple times and took the lowest result. Of course, all programs produced exactly the same results -- all computations are integer computations. Expected results are:
n | 2 | 4 | 6 | 8 | 10 | 12 | 14 | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
combinations | 1 | 0 | 0 | 2 | 10 | 4 | 40 | 92 | 352 | 724 | 2,680 | 14,200 | 73,712 | 365,596 |
Runtimes in real and user seconds are given in below table. Times taken with time
command.
Language | real | user |
---|---|---|
C | 0.42 | 0.42 |
Java 19.0.2+7 | 0.59 | 0.62 |
GnuCOBOL Sobisch | 0.68 | 0.68 |
Dart 2.19.2 | 0.72 | 0.75 |
node.js 19.8.0 | 0.74 | 0.74 |
PHP 8.3.12 JIT abs() inlined | 3.53 | 3.50 |
PHP 8.3.3 JIT | 6.58 | 6.54 |
GnuCOBOL 12.2.1 | 17.45 | 17.42 |
PHP 8.2.4 | 18.16 | 18.13 |
Python 3.9.6 | 32.53 | 32.35 |
gcobol Sobisch | 191.13 | 190.39 |
gcobol 13.0.0 | 309.79 | 305.29 |
3. Special modifications in COBOL program. Simon Sobisch, in private e-mail from 13-Mar-2023 to me, made below changes to the COBOL program:
- No variable with sign any more, i.e., variable is always greater or equal zero
- Compiler optimization flag
-fno-trunc
Modifications by Simon Sobisch:
21c21
< 77 l pic s9(8) comp-5.
---
> 77 l pic 9(8) comp-5.
139,149c134,146
< compute l = z - A(j)
< if l = 0 then
< * exit section
< go to configOK-exit
< end-if
< if l < 0 then
< compute l = 0 - l
< end-if
< if l = k - j then
< * exit section
< go to configOK-exit
---
> evaluate true
> when z < A(j)
> move A(j) to l
> subtract z from l
> when z = A(j)
> exit section
> when other
> move z to l
> subtract A(j) from l
> end-evaluate
> add j to l
> if l = k then
> exit section
4. Conclusions.
- C is by far the fastest. Faster than Java by a factor of 1.5. Almost two times faster than Dart as its other closest competitor.
- Java is roughly 20% faster than Javascript.
- JavaScript and Dart have almost the same performance.
- Cobol can be made faster than JavaScript and Dart but slightly slower than Java.
- PHP 8 is two times faster than Python, but almost 25-times slower than Javascript.
- PHP with JIT is almost three times faster than normal PHP.
- PHP with JIT enabled is five times faster than Python, but nine times slower than Javascript.
5. Dart program. Dart is a programming language by Google and used for Flutter.
int abs(int x) { return ((x >= 0) ? x : -x); }
/* Check if k-th queen is attacked by any other prior queen.
Return nonzero if configuration is OK, zero otherwise.
*/
bool configOkay (int k, List<int> a) {
int z = a[k];
for (int j=1; j<k; ++j) {
int l = z - a[j];
if (l == 0 || abs(l) == k - j) return false;
}
return true;
}
int solve (int N, List<int> a) { // return number of positions
int cnt = 0;
int k = a[1] = 1;
int N2 = N; //(N + 1) / 2;
bool flag = false;
for (;;) {
if (configOkay(k,a)) {
if (k < N) { a[++k] = 1; continue; }
else ++cnt;
}
flag = false;
do
if (a[k] < N) { a[k] += 1; flag = true; break; }
while (--k > 1);
if (flag) continue;
a[1] += 1;
if (a[1] > N2) return cnt;
a[k=2] = 1;
}
}
void main (List<String> argv) {
int NMAX = 100;
List<int> a = [ for(var i=0;i<100;++i) 0 ];
int argc = argv.length;
print(" n-queens problem.\n"
" 2 4 6 8 10 12 14\n"
" 1 0 0 2 10 4 40 92 352 724 2680 14200 73712 365596\n"
);
int start = 1;
int end = 0;
if (argc >= 1) end = int.parse(argv[0]);
if (end <= 0 || end >= NMAX) end = 10;
if (argc >= 2) { start = end; end = int.parse(argv[1]); }
if (end <= 0 || end >= NMAX) end = 10;
for (int n=start; n<=end; ++n)
print(" D(${n}) = ${solve(n,a)}");
}
Added 28-Sep-2024: Francesco Cristiano on 24-Sep-2024 wrote to me and stated that inlining the abs()
function in PHP would reduce the runtime by one second.
That inlining is fully justified as the C program in question also uses a macro like:
#define abs(x) ((x >= 0) ? x : -x)
So doing the same in PHP:
if ($l == 0 || ($l >= 0 ? $l : -$l) == $k - $j) return 0;
But this change in the PHP program not only reduces the runtime by one second, it almost halves the runtime!
The original benchmark was done more than a year ago. The environment now has changed a bit: Linux is now 6.10.10, and PHP is 8.3.12. Old program under this changed environment:
$ time php xdamcnt2.php 13
array(2) {
[0]=>
string(12) "xdamcnt2.php"
[1]=>
string(2) "13"
}
n-queens problem.
2 4 6 8 10 12 14
1 0 0 2 10 4 40 92 352 724 2680 14200 73712 365596
D( 1) = 1
D( 2) = 0
D( 3) = 0
D( 4) = 2
D( 5) = 10
D( 6) = 4
D( 7) = 40
D( 8) = 92
D( 9) = 352
D(10) = 724
D(11) = 2680
D(12) = 14200
D(13) = 73712
real 6.71s
user 6.69s
sys 0
swapped 0
total space 0
Inlined program:
time php xdamcnt3.php 13
array(2) {
[0]=>
string(12) "xdamcnt3.php"
[1]=>
string(2) "13"
}
n-queens problem.
2 4 6 8 10 12 14
1 0 0 2 10 4 40 92 352 724 2680 14200 73712 365596
D( 1) = 1
D( 2) = 0
D( 3) = 0
D( 4) = 2
D( 5) = 10
D( 6) = 4
D( 7) = 40
D( 8) = 92
D( 9) = 352
D(10) = 724
D(11) = 2680
D(12) = 14200
D(13) = 73712
real 3.53s
user 3.50s
sys 0
swapped 0
total space 0
Thanks Francesco.
Added 28 Sept 2024 at 15:14, Francesco Cristiano wrote:
Hi Elmar,
Glad to have contributed to your benchmark. I have been using PHP 8-jit to implement a power-to-ammonia simulation plant (with 3 millions time-steps) all through CLI and the execution time is indeed impressive. I don't see why people keep saying that PHP is a scripting language for webserver only.
Cheers Fran
Added 02-Oct-2024, Francesco Cristiano wrote:
Hi Elmar,
I've been testing how close PHP can get to Java using your n-queen code as a benchmark, and after using some tricks to improve the PHP code I'm getting some interesting results: This is the tweaked PHP code:
function solve ( $N, &$a) { // return number of positions
$cnt = 0;
$k = $a[1] = 1;
$N2 = $N; //(N + 1) / 2;
for (;;) {
$flag = false;
//code equivalent to configOkay function
$configOkayFlag=true;
{
$z = $a[$k];
for ($j=1; $j<$k; ++$j) {
$l = $z - $a[$j];
if ($l == 0 || (($l >= 0) ? $l : -$l) == $k - $j) {
$configOkayFlag= false;
break;
}
}
}
// end of code equivalent to configOkay function
//if (configOkay ($k, $a)) {
if ( $configOkayFlag) {
if ($k < $N) {
$a[++$k] = 1;
continue;
}
else
++$cnt;
}
do
if ($a[$k] < $N) {
$a[$k] += 1;
$flag = true;
break;
}
while (--$k > 1);
if ($flag)
continue;
$a[1] += 1;
if ($a[1] > $N2)
return $cnt;
$k = 2 ;
$a[2] = 1;
}
}
$get_as_float = true;
$start = microtime(true);
$a= [];
for($i=1; $i <= 13; $i++){
$result=solve($i,$a);
echo "i=$i - number of combinations: $result\n";
}
$time_elapsed_secs = microtime(true) - $start;
echo "time_elapsed_secs:$time_elapsed_secs \n";
And the equivalent Java code is the following:
class bench {
public static long solve (int N, int[] a) { // return number of positions
long cnt = 0;
int k = a[1] = 1;
int N2 = N; //(N + 1) / 2;
boolean flag;
for (;;) {
flag = false;
//code equivalent to configOkay function
boolean configOkayFlag=true;
{
int z = a[k];
for (int j=1; j<k; ++j) {
int l = z - a[j];
if (l == 0 || ((l >= 0) ? l : -l) == k - j) {
configOkayFlag= false;
break;
}
}
}
// end of code equivalent to configOkay function
//if (configOkay(k,a)) {
if ( configOkayFlag) {
if (k < N) { a[++k] = 1; continue; }
else ++cnt;
}
do
if (a[k] < N) { a[k] += 1; flag = true; break; }
while (--k > 1);
if (flag) continue;
a[1] += 1;
if (a[1] > N2) return cnt;
k = 2; a[2] = 1;
}
}
public static void main(String[] args){
long startTime = System.currentTimeMillis();
int[] a = new int[73713];
for(int i=1; i<=13; i++){
System.out.println("i="+i+ " - number of combinations: "+ bench.solve(i, a));
}
long stopTime = System.currentTimeMillis();
long elapsedTime = stopTime - startTime;
System.out.println("Total time: " + elapsedTime +"ms");
}
}
These are the results I'm getting:
c:\php8.4\php.exe -f bench.php
i=1 - number of combinations: 1
i=2 - number of combinations: 0
i=3 - number of combinations: 0
i=4 - number of combinations: 2
i=5 - number of combinations: 10
i=6 - number of combinations: 4
i=7 - number of combinations: 40
i=8 - number of combinations: 92
i=9 - number of combinations: 352
i=10 - number of combinations: 724
i=11 - number of combinations: 2680
i=12 - number of combinations: 14200
i=13 - number of combinations: 73712
time_elapsed_secs:1.577357
java bench
i=1 - number of combinations: 1
i=2 - number of combinations: 0
i=3 - number of combinations: 0
i=4 - number of combinations: 2
i=5 - number of combinations: 10
i=6 - number of combinations: 4
i=7 - number of combinations: 40
i=8 - number of combinations: 92
i=9 - number of combinations: 352
i=10 - number of combinations: 724
i=11 - number of combinations: 2680
i=12 - number of combinations: 14200
i=13 - number of combinations: 73712
Total time: 875ms
Not bad, PHP is only taking 1.8 times the execution time taken by Java.
Cheers Fran