2010-02-18, 05:20 PM
I know that ZX Basic is amazing, but I was wondering how it stood up to other basic compilers that were around for use on the ZX Spectrum. We know that Hisoft basic was pretty fast, for example, and LCD mentioned another compiler the other day that was pretty amazing too.
Let me borrow from an article in Crash Magazine: http://www.crashonline.org.uk/19/compilers.htm
In this article, Simon Goodwin talks about several compilers. Hisoft Basic isn't one of them - it wasn't out yet. He doesn't list the benchmarks, either; but they can be interpolated from this:
Simon didn't use Benchmark 9, and I can see why - it's not clearly specified. BM1 to BM8 are pretty clear, however.
My own personal testing with Sinclair Basic gave very slightly differing results. In all cases, my programs were very slightly faster than the timings Goodwin gave in the magazine article. Perhaps he specified things a little differently, perhaps he was using a stopwatch in hand, and human error was the result. Perhaps it was a different version of the ZX Spectrum used. I got the computer to time the programs using the 50 frames per second interrupt timer. For very fast running programs I increased the number of loops by a factor 10 or 100 and estimated back down.
The compilers goodwin tested were:
A Mehmood's "Compiler".
MCODER
Softek's FP and IS
And a little cheekily, Zip 1.5. He wrote that himself, I believe.
The first two rows are for Sinclair Basic. The first being Simon Goodwin's numbers, the second being my own. All times are in seconds, smaller is better.
The actual code used is listed below. It's possible to Extrapolate what BM1-6 are, because they simply add code to end up with BM7. Bm 8's main loop is listed separately.
BM 8 replaces most of the code with:
This is changed from using constants to prevent constant folding optimizations.
RESULTS and DISCUSSION
First up, passing all the benchmarks and more, clearly Boriel's work is by far the most flexible and comprehensive compiler available. It blows the spots off everything else in terms of WHAT it can compile, and all credit to him for creating it. It is excellent!
In terms of performance, it's pretty amazing, too. It's the second fastest of all the compilers listed here. Only ZIP goes faster, generally. BM7 is a little disappointing, in that the produced code seems to be slower than both MCODER 2 and Zip by a quite significant margin. Perhaps some examination of array handling code could improve this. With version 1.25 beta, sadly, I couldn't use -O3 as an option - the programs all failed to compiler with this option enabled, so I couldn't see if peephole optimization would make a difference. It's worth noting that most On Spectrum compilers refused to deal with floating point numbers. In this roundup, only Softek FP could do it, and that barely faster than Basic. Boriel's compiler blew me away with the FP result, frankly. I had to check to see if it was doing it correctly, it was so amazing! There might be some sneaky optimization happening, but printing the numbers as it created them did seem to work fine. (Note: It WAS cheating. It was putting in constants at compile time. A clever option, but not what we were aiming to test. This number has been changed)
Fixed Hisoft Basic Numbers. These corrected numbers do in fact show it produces some of the fastest code available, sometimes beaten by ZIP 1.5. It far outmatches what ZIP can do, however, in that it deals with FP as well as integer - and it seems to do both faster than the competition. Of course ZX BASIC basic excels at being FP and Integer aware as well.
Added in Tobos. It's fully FP, so tends to be slow where integer math could improve things. But look at BM8!
ZX BASIC In short: Solid and well optimized. Seems to be slow in BM7 (array handling). Very clever use of constant insertion to produce good BM8 speed value of 0.1 but now times are corrected because that was cheating a little!
[Edit] - Array handling speed has been dramatically increased with later versions. Boriel has stated that he will be looking into further array optimizations similar to Hisoft Basic methods - so we can hope for another doubling of speed, perhaps!
hock:
Let me borrow from an article in Crash Magazine: http://www.crashonline.org.uk/19/compilers.htm
In this article, Simon Goodwin talks about several compilers. Hisoft Basic isn't one of them - it wasn't out yet. He doesn't list the benchmarks, either; but they can be interpolated from this:
Code:
Benchmark BM1 : A null-action FOR, REPEAT or DO loop, executed
1000 times.
Benchmark BM2 : A null-action explicitly-coded loop executed
1000 times.
Benchmark BM3 : BM2 plus A=K/K*K+K-K in the loop.
Benchmark BM4 : BM2 plus A=K/2*3+4-5 in the loop.
Benchmark BM5 : BM4 plus a branch to null-action subroutine
from inside the loop.
Benchmark BM6 : BM5 plus an array declaration M(5), and a
null-action FOR loop (of 1-5) also in the
loop.
Benchmark BM7 : BM6 plus M(L)=A in this 1-5 loop.
Benchmark BM8 : A square function, log function and sin
function in an explicitly-coded FOR loop,
repeated 100 times.
Benchmark BM9 : Prime numbers in the range 1-1000 are printed
to the screen, calculated in an outer loop of
1000 and an inner loop of 500, with no tricks
at all. This is a very bad prime number
routine indeed, but a very useful basis for
inter-machine, interpreter and compiler
comparisons.Simon didn't use Benchmark 9, and I can see why - it's not clearly specified. BM1 to BM8 are pretty clear, however.
My own personal testing with Sinclair Basic gave very slightly differing results. In all cases, my programs were very slightly faster than the timings Goodwin gave in the magazine article. Perhaps he specified things a little differently, perhaps he was using a stopwatch in hand, and human error was the result. Perhaps it was a different version of the ZX Spectrum used. I got the computer to time the programs using the 50 frames per second interrupt timer. For very fast running programs I increased the number of loops by a factor 10 or 100 and estimated back down.
The compilers goodwin tested were:
A Mehmood's "Compiler".
MCODER
Softek's FP and IS
And a little cheekily, Zip 1.5. He wrote that himself, I believe.
The first two rows are for Sinclair Basic. The first being Simon Goodwin's numbers, the second being my own. All times are in seconds, smaller is better.
Code:
BM1 BM2 BM3 BM4 BM5 BM6 BM7 BM8 BMDRAW
Sinclair 4.46 8.46 21.56 19.82 25.34 60.82 87.44 23.30 80.18
Boriel's ZX BASIC 0.038 0.032 0.30 0.15 0.16 0.328 2.20 24.0
ZX Basic 1.26-r1603 -O3 0.94 20.78 (17.14 with fSin)
ZX Basic 1.2.8-s682 -O3 0.88 20.56 (16.94 with fSin) 21.14
ZX Basic 1.2.8-s758 -O3 0.90 20.76 (17.10 with fSin) 21.32
HiSoft FP 0.82 1.34 7.26 7.30 7.32 12.52 14.40 21.9
HS Integer 0.042 0.67 0.08 0.088 0.334 0.50 10.76
Mehmood * 0.065 9.0 4.2 4.2 * * *
ZIP 1 .5 0.031 0.064 0.194 0.108 0.115 0.29 0.46 *
TOBOS 0.58 0.82 2.02 1.76 2.34 6.68 8.72 0.746
SOFTEK FP 1.75 2.1 8.7 9.4 9.4 19.7 24.0 22.5
SOFTEK IS 0.058 0.076 0.57 0.98 0.99 1.32 * *
MCODER2 0.043 0.097 0.62 0.90 0.92 1.17 1.47 *Code:
REM BM7
FUNCTION t() as uLong
asm
LD DE,(23674)
LD D,0
LD HL,(23672)
end asm
end function
goto start
subroutine:
return
start:
DIM time,i as uInteger
DIM k,var,j as uByte
let time =t()
LET k=5
LET i=0
label:
LET i=i+1
LET var=k/2*3+4-5
gosub subroutine
DIM M(5) as uInteger
FOR j=0 to 4
LET M(j)=i
NEXT j
IF i<1000 then GOTO label: END IF
print (CAST (FLOAT,t())-time)/50BM 8 replaces most of the code with:
Code:
REM BM8
DIM i,j as ubyte
j=2
FOR i=1 to 100
result=j^2
result=ln(j)
result=sin(j)
next iRESULTS and DISCUSSION
First up, passing all the benchmarks and more, clearly Boriel's work is by far the most flexible and comprehensive compiler available. It blows the spots off everything else in terms of WHAT it can compile, and all credit to him for creating it. It is excellent!
In terms of performance, it's pretty amazing, too. It's the second fastest of all the compilers listed here. Only ZIP goes faster, generally. BM7 is a little disappointing, in that the produced code seems to be slower than both MCODER 2 and Zip by a quite significant margin. Perhaps some examination of array handling code could improve this. With version 1.25 beta, sadly, I couldn't use -O3 as an option - the programs all failed to compiler with this option enabled, so I couldn't see if peephole optimization would make a difference. It's worth noting that most On Spectrum compilers refused to deal with floating point numbers. In this roundup, only Softek FP could do it, and that barely faster than Basic. Boriel's compiler blew me away with the FP result, frankly. I had to check to see if it was doing it correctly, it was so amazing! There might be some sneaky optimization happening, but printing the numbers as it created them did seem to work fine. (Note: It WAS cheating. It was putting in constants at compile time. A clever option, but not what we were aiming to test. This number has been changed)
Fixed Hisoft Basic Numbers. These corrected numbers do in fact show it produces some of the fastest code available, sometimes beaten by ZIP 1.5. It far outmatches what ZIP can do, however, in that it deals with FP as well as integer - and it seems to do both faster than the competition. Of course ZX BASIC basic excels at being FP and Integer aware as well.
Added in Tobos. It's fully FP, so tends to be slow where integer math could improve things. But look at BM8!
ZX BASIC In short: Solid and well optimized. Seems to be slow in BM7 (array handling). Very clever use of constant insertion to produce good BM8 speed value of 0.1 but now times are corrected because that was cheating a little!
[Edit] - Array handling speed has been dramatically increased with later versions. Boriel has stated that he will be looking into further array optimizations similar to Hisoft Basic methods - so we can hope for another doubling of speed, perhaps!
hock: