Cawen and the web

A comparison with php

In this page,

    • First, you will be able to compare in real time two web pages. One is written in PHP and is run by an apache server, the other is coded in Cawen and is executed by a mongoose server.
    • Then, you can have a look at the corresponding PHP and Cawen codes.
    • We also present the results that we have obtained with siege, a load testing utility. We simulated a constant flow of 10 simultaneous requests sent to the Cawen + mongoose and PHP + apache pages. 10000 requests in all are thus processed. For this last test, we added another solution to the list of competitors : Cawen + nxweb.

If you want to run a copy of this benchmark on your own server, you will find everything you need (php and cawen source code, generated C code, test scripts and our raw results files) in this archive : cwn_web_benchmark.tgz.

Real time comparison

Principles of the test

The cawen/mongoose and the php/apache pages perform the same processing : displaying their own execution times and the execution times of their concurrent in the same line graph. A moving average is also calculated over the two sets of numbers.

All the htttp servers are running on the same server :
ubuntu 3.5.0-17-generic / 5,4 Go / AMD E1-12000 APU / 64 bits / Cawen is compiled with gcc 4.7.2

Execution times are stored in two files that are unique per connection to this current page and that will be reset if you refresh it. The php’s execution times file is written by the php script and read by both servers, the cawen’s execution times file is written by the cawen page and read by both servers. Executions times are in microsecond, they record the elapsed time between the first line of code in the page and the last one.

Data from apache and mongoose are both compressed with zlib. The elapsed time does not include the compression process (because apache compresses the html to be sent once this text is ready, that is to say, after the execution of the last line of the php script).

With the various buttons, you can choose wether the data displayed comes from apache or mongoose (you must get the same values), you can decide wether requests are sent one by one or automatically (“step by step” and “automatic” button) and wether all results are displayed or only the last 256.

The light  blue line display the cawen page exec times, the dark blue line shows the moving average  calculated from cawen measures.  The orange and dark orange lines show php results.

Test conclusions

Maybe you could not have a complete view of our live benchmark (apache, mongoose or our server may have gone for a nap). So , you may or may not have obtained this kind of display :

It can be noted that :

    • The Cawen execution times are a lot smaller than php’s. However, at the beginning of the test, apache processes more requests per second than mongoose.
    • Unlike Cawen binary, the php script is extremely sensitive to the increasing of data size. When this size reaches only a few hundreds records, mongoose + Cawen become 20,30,..80 etc times quicker than apache + php.
    • This gap ultimately has a serious impact on the service level : mongoose + cawen can process more requests than apache + php and the difference grows indefinitely.

These conclusions are precised below in a less playfull but more relevant way.

Source code

Click here to read the php code.

And here to get the Cawen source file.

A few remarks about the Cawen code presented above:

  • Line count is similar to PHP’s.
  • The Cawen file only contains the functional part of the server. The technical part (mongoose_interface.hwn) that defines the translation of this file to mongoose/C files is 389 lines long.
  • To implement this benchmark with xxxx, another light http server whose mother tongue is C or to create CGI, fast CGI, Zend modules etc.. one just has to write a specific xxx_interface.hwn and a main source xxx.cwn : the functional part can be reused as is. And has been reused this way when we coded a nxweb version of our benchmark (see load tests below).
  • Please note the cawen tree structure in line 102 and 158. It is used here to produce html but it can be very helpulf in the coding of whatever tree-like data or open/close functionnality.
  • Also note the #raw directive in line 141 that allows you to implement user-defined pre-compile time transformation and apply them to raw blocks of the source file.
  • The cawen functional (cwn_vs_php.hwn) source is 260 lines long, the corresponding generated C is 843 lines long.

Load test

The scenario of the load test is the following : process 10000 requests (10 simultaneous) for producing html with 4 different sets of data containing respectively 10, 100, 200 and 250 execution time records. This time the files are only read, not written : four constant files have been prepared on the server and are processed over and over.We measured memory consumption, percentage of CPU used  and energy consumption.  Number of requests per second is given by siege at the end of each test.

Test sequences

The servers run 30 sec without being accessed / 10000 requests (10 records) are processed / 30 sec break /10000 requests (100 records) are processed / 30 sec break /10000 requests (200 records) are processed / 30 sec break /10000 requests (250 records) are processed / 30 sec break / the server is stopped.

We runned and tested the 3 HTTP servers one by one.

Memory consumption

We measured memory consumption in a very basic way : we recorded the “used” column from the “free” command results throughout test execution.

CPU usage

We recorded the results of a ps command. For apache, we added up the figures of the various processes.

You can see below the cpu usage progressions of the three solutions. Please note the four peaks corresponding to the four stages of the test.

To extract a single figure from each step (corresponding to a particular phase of the test) of these 3 curves, we looked for a standard way of taking into account that using 10% of CPU during 10 sec is not exactly the same that using 10% during half an hour. How can we estimate the total amount of work that each HTPP server needed to process each phase of the test ? In the following summaries, we used the average value of CPU usage, for lack of anything better

Power curve

We used the Green Code Lab software and followed its methodology.

For example, here is the power curve observed with Cawen + nxweb. The four steps of the test are clearly visible.

The results

The following spiderweb graphs summarize the observation for each criteria (in percentage of PHP+apache results) :

They are based on these raw figures :

TEST 10
APACHE
MONGOOSE
NXWEB
ENERGY (mWh)
50,47
14,74 (div 3,4)
9 (div 5,6)
CPU usage (%)
61,97
5,16 ( div 12)
16,47 (div 3,7)
MEMORY used (kb)
22245
2192 (div 10,1)
2424 (div 9,1)
EXEC TIME (sec)
70,34
210,23 (mul 2,9 )
15,51 ( div 4,5 )
CODE SIZE (line)
267
242 (-9%)
242 (-9%)
TEST 100
APACHE
MONGOOSE
NXWEB
ENERGY (mWh)
651,6
29,34 (div 22,2)
19,1(div 34,1)
CPU usage (%)
109,8
11,5 (div 9,5)
27,6 (div 4)
MEMORY used (kb)
28688
3363 (div 8,53)
2544 (div 11,3)
EXEC TIME (sec)
330,25
50,29 (div 6,6)
33,7( div 9,8)
TEST 200
APACHE
MONGOOSE
NXWEB
ENERGY (mWh)
2640,75
59,19 (div 44,61)
30,28(div 87,2)
CPU usage (%)
142,3
20,6 (div 6,9)
33,2 (div 4,2)
MEMORY used (kb)
61515
4144 (div 14,8)
2620 (div 23,5)
EXEC TIME (sec)
1167,75
59,86 (div 19,5)
60,26( div 19,4)
TEST 250
APACHE
MONGOOSE
NXWEB
ENERGY (mWh)
3862,9
68,77 (div 56,17)
35,44 (div 109)
CPU usage (%)
151,6
28,59 (div 5,3)
36,08(div 4,2)
MEMORY used (kb)
63376
4794 (div 13,2)
2843 (div 22,3)
EXEC TIME (sec)
1710,83
68,77 (div 24,9)
65.22( div 26,2)

Cawen + light HTTP servers do much better than apache + PHP except for the execution time of mongoose in test 10. We will investigate on this case, for it is very strange that mongoose needs more time to process 10000 “10 records” requests than to process 10000 “100 records” requests.

For all the other cases, the use of Cawen + light HTTP server, preferably to PHP + apache, made it possible to :

  • divide energy consumption by 3,4 to 109
  • divide average CPU usage by 3,7 to 12
  • divide memory usage by 8,5 to 23,5
  • divide execution times by 4,5 to 26,2

And, these results were obtained with a code that is 9% shorter than the PHP page.

Future work

We have compared 3 couples web server + languages, not 2 languages (Cawen, and php) which complicates the interpretation of our results. Who should be applauded, Cawen or nxweb ?
We did not associate Cawen with http servers that are more efficient than apache to ensure that their association wins the race, we did it because all efficient http servers offer the opportunity to code web pages in straight C code, without any verbose wrapping…

3 avenues still need to be explored to improve the php+apache solution that we present here :

    • We used the default apache configuration file, with two exceptions : we set KeepAliveTimeOut to 2 secs and MaxKeepAliveRequests to 0 (unlimited). This configuration may be optimized : we welcome proposals.
    • We did not use any php accelerator. Maybe someday we’ll have time to test some of them ?
    • The php source itself may be optimized. We remain open to any and all ideas.

We will be very happy to add some data to this quick overview : please do not hesitate to send us some new competitors : (mongoose+lua), (nxweb+python),(apache+python) etc…

We are going to gather all our future web experiments in this page.

Conclusion

Memory, CPU, energy : the nxweb+Cawen and mongoose+Cawen solutions are incredibly resource-saving compared to apache+php.
They also offer a far better scalability.

From the developer’s point of view, Cawen can be as concise as php.
A few shortcuts (dynamic typing for instance) will never be implemented by default in Cawen, whose pride is to be a precompilation tool for C.
So that, for a beginner, it will always require a better level of programing knowledge and a little more effort to produce a Cawen application than to write its php, lua, java or python equivalent.

But once a few mechanisms are understood, is it easier to write “< ? php  echo “Hello world!”; ? >” or “@output(“Hello world!”);” ?

Don’t worry, we won’t run a poll on this issue !

Leave a Reply

*


one × 9 =

-Your email address will not be published.-