In
my last post I discussed the Apache Benchmarking tool and how I used it to test my server against Apache.
Using the tool, I saw Apache leave my poor server in the dust. I could serve
550 rps (
requests per second) to Apache's
1800 rps!
To be expected, I suppose. After all I had been writing the tool in my spare time for two weeks on and off while Apache was an old and well tested beast.
Still, it bugged me. Not because Apache beat me, but because I couldn't figure out why! After all:
a) Apache is doing a TON of things that I am not. My server is simple and so should be able to do simple things well. I can't imagine doing the amazing things Apache is able to, but serving simple GET requests? Cmon I can do that at least!
b) My server is purely asynchronous and I don't have the overhead of thread switching that Apache is presumably dealing with.
Yet despite (a) and (b) Apache beat me hollow! How?
CACHING
Of course! Apache must be caching the pages it loads while I load from disk every time. That must be the reason it's so much faster!
Given that as a working hypothesis, I built a simple cache and served my pages from there. This time when I ran the benchmark, I expected significant improvements!
I got none!
I'm still chugging along at 550 rps while Apache whizzes by at 1800!
The OS caches the files I read so my simple cache didn't give me any improvement at all! I removed it.
CUSTOM ALLOCATORS
The next option I considered was memory management. I hadn't build a custom allocator for my simple server so maybe that was the problem.
No biggie - I could build a buddy allocator and see if that gave any benefits.
But once bitten, twice shy - I had already spent some time building and discarding a cache. What were the chances that the allocator was the problem? Was there something else?
Turns out there was - and it was all my own fault.
SPINNING
The problem was actually caused by the fact that my server was purely asynchronous. The main server loop would not wait for ANYTHING and so it spun at a terrific speed and thus became CPU bound. To deal with this, I actually added a little line forcing it to sleep:
/* Enter main loop to handle requests */
for (;;) {
...
/* We introduce a miniscule delay to stop the CPU spinning */
usleep (1);
...
}
A minor change to this would make the server sleep only if it was not serving requests:
/* Enter main loop to handle requests */
for (;;) {
...
/* In the middle of doing something - no sleeping on the job */
if (cctx[cc].uState != BD_READHDR) dosleep = 0;
...
/* We introduce a miniscule delay to stop the CPU spinning */
if (dosleep) usleep (1);
...
}
VOILA! Now serving
1800 rps!
My little server is now as fast as Apache!
Well, for simple GET requests of a static page at least. ;-)
I can probably speed it up a bit more by implementing a custom memory allocator, but I don't think I'll bother. This is good enough for me.
Moral of the story
Don't give up easily - catching up to the big guys may be easier than you think!
For those of you who are interested, here is the code: (bugs and comments welcome)
bd-server-1.c
/**
* @brief This is the server component of the application I'm building.
* I am publishing it only because of the performance benchmarks
* in my blog.
*
* This is an embedded purely asynchronous server that works
* as fast as Apache in serving simple GET requests.
*
* This test server will only serve "index.htm" from the
* current folder. The full application will come later as I
* develop it.
*
* @author
* Comments/Bugs/Feedback: the.brown.dragon.blog@gmail.com
* http://the-brown-dragon.com/
*
* @note
* Further improvements in performance may be possible if I build
* a custom memory allocator.
*
* @note
* Known bugs/issues are listed at the end of this file.
*/
/*****************************************************************************
* BASIC
****************************************************************************/
/**
* @brief Standard Includes
*/
#include <stdio.h>
#include <assert.h>
#include <errno.h>
#include <memory.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <fcntl.h>
#include <dirent.h>
/**
* @brief C99-like typedefs.
*/
typedef void Void;
typedef void* vptr_t;
typedef int8_t byte_t;
typedef int8_t* byteptr_t;
typedef char char_t;
typedef char_t* strptr_t;
typedef int bool_t;
/**
* @brief Basic allocator - crashes when unable to allocate.
* (good enough strategy for this application).
* @param[in] pSize The amount of memory to allocate in bytes.
* @return void* a pointer to the allocated memory.
* @note
* This function will exit() the application if it cannot
* succeed (i.e. we are not expecting to work in a
* limited-memory environment).
*/
static inline void *
xmalloc (size_t pSize)
{
vptr_t v;
/* Allocate or die! */
if (!(v = malloc (pSize))) {
perror ("Out of Memory!");
exit (EXIT_FAILURE);
}
/* Return triumphantly bearing memory */
return v;
}
/**
* @brief This macro performs the general function
* of expanding a given area in a block-wise
* fashion.
* This is useful in several cases. Some examples:
* - A string needs to resize as it is filled
* - An array needs to resize as it is filled
* - A keymap needs to resize as it is filled.
* @param w 'w'hat should be expanded.
* @param cs 'c'urrent 's'ize of whatever it is.
* @param us 'u'tilized 's'ize of whatever it is.
* @param exp 'exp'ansion block size.
* @note
* Like all non-hygienic macros, this suffers from variable capture.
* The variables that will be captured are tmp_232 and tmpsz_232,
* which in practice should not conflict with actual names.
*/
#define XPAND_(w,cs,us,exp) do {\
if ((us)==(cs)) {\
vptr_t tmp_232 = (w);\
uint32_t tmpsz_232 = (cs);\
(cs) += exp;\
(w) = xmalloc ((cs));\
memcpy ((w), tmp_232, tmpsz_232);\
free (tmp_232);\
}\
} while (0)
/*****************************************************************************
* HTTP SERVER
****************************************************************************/
/**
* @brief The various states that the user connection
* function must handle.
*/
typedef enum {
BD_CS_GOTCONNECTION,
BD_CS_CONNECTIONCLOSED,
BD_CS_GOTHEADER,
BD_CS_LOADRESPONSE,
BD_CS_RESPONSEDONE,
} bd_connstate_t;
/**
* @brief This structure contains the data used
* by the users's connection function.
*/
typedef struct {
/* This variable is set by the user if
* he/she requires a per-connection context */
vptr_t uCTxT;
/* The item requested */
strptr_t uRequest;
/* When state is BD_LOADRESPONSE
* the callback sets these values
* and finally setting uResponseDataSize to -1
* when finished writing.
* It is recommended the callback does as
* much as possible asynchronously to keep
* the server responsive */
vptr_t uResponseData;
int32_t uResponseDataSize;
} bd_conn_t;
/**
* @brief The size of the read buffer.
*/
#define BD_SVR_BUFSZ 256
/**
* @brief The maximum number of connections
* handled.
*/
#define BD_SVR_POOL_SZ 8
/**
* @brief The various states the server/a connection can be in.
*/
typedef enum {
BD_START = 1,
BD_MKSOCKET, BD_CONVERTIPADD, BD_BIND, BD_LISTEN, BD_FCTL,
BD_SERVERSTATESENDS = 100,
BD_ACCEPT_NONBLOCK, BD_GOTCONNECTION,
BD_CONNECTIONCLOSED,
BD_READHDR, BD_READLINE, BD_HANDLEREAD, BD_GOTLINE, BD_GOTHEADER,
BD_PARSEHDR_ERR,
BD_LOADRESPONSE, BD_WRITINGDATA, BD_RESPONSEDONE,
/* User uses these states to control the server. */
BD_CLOSECONNECTION, BD_SHUTDOWNSERVER, BD_CONTINUE,
} bd_svrstate_t;
/**
* @brief A connection context.
* @note
* This structure is also used
* as the server context when
* the states are < BD_SERVERSTATESENDS
*/
typedef struct {
/* User-data */
bd_conn_t uUD;
/* The current state of the connection */
bd_svrstate_t uState;
/* Read buffer */
byte_t uReadData[BD_SVR_BUFSZ];
int32_t uRdSz;
int32_t uRP;
strptr_t uCurLine;
uint32_t uCLSz;
uint32_t uCP;
/* Write Buffer (buffer in user data) */
int32_t uRsP;
} bd_conctx_t;
/**
* @brief Parses the first line of a HTTP request.
* Basically skips the request type and
* URLDecodes the request.
* @param[in] pHTTPReq The request line.
* @return strptr_t A newly allocated string containing
* the actual request.
*/
static strptr_t
sParseRequest (strptr_t pHTTPReq)
{
int32_t i , j, k;
char_t v;
strptr_t r;
/* We service only GET requests right now
* as that will let us proceed with other components.
* POST request support can come later if needed. */
if (strncmp (pHTTPReq, "GET ", 4)) return NULL;
/* Skip over the GET part. We are now at the start
* of the requested resource. */
pHTTPReq += 4;
/* Look for the end of the requested resource */
for (i = 0; pHTTPReq [i] && pHTTPReq [i] != ' ';++i);
/* Didn't find it! */
if (!pHTTPReq [i]) return NULL;
/* Copy the request into a newly created buffer */
r = xmalloc (i+1);
for (j = k = 0;j < i;++j,++k) {
/* Case: URL-encoded value */
if (pHTTPReq [j] == '%') {
/* Error: URL-encoded must have a 2-char hex */
if ((j + 3) > i) {
free (r);
return NULL;
}
#define PR_CHAR2NUM do {\
/* Maybe between 0-9 */\
v = pHTTPReq [j] - '0';\
/* No? */\
if (v > 9) {\
/* Maybe between a-f */\
v = (pHTTPReq [j] | 0x20) - 'a';\
/* No? - Quit with error */\
if ((v < 0) || (v > 5)) {\
free (r);\
return NULL;\
}\
/* Ok - adjust to actual value */\
v += 10;\
}\
/* Not in range - Quit with error */\
if (v < 0) {\
free (r);\
return NULL;\
}\
} while (0)
/* Convert the first character to a number */
++j; PR_CHAR2NUM;
r [k] = v;
/* And add the second character as a number */
++j; PR_CHAR2NUM;
r [k] = (r[k] * 0x10) + v;
} else {
/* Copy source to destination */
r [k] = pHTTPReq [j];
}
}
/* NULL-terminate the string */
r [i] = 0;
/* Return it. */
return r;
#undef PR_CHAR2NUM
}
/**
* @brief This macro is used to define a state handling
* function for the server (sConnectionDo).
*/
#define BD_SVRSTATE_HANDLER(h) \
static bd_svrstate_t\
s##h (\
int pConn, bd_conctx_t *pCC,\
int32_t (*pErrFunc)(bd_svrstate_t pState, vptr_t pCTxT),\
int32_t (*pConnectionFunc) (bd_connstate_t pState,\
bd_conn_t *pConn, vptr_t pCTxT),\
vptr_t pCTxT)
/**
* @brief BD_READHDR state handler.
*/
BD_SVRSTATE_HANDLER (BD_READHDR)
{
/* If needed, allocate space for current line */
if (!pCC->uCLSz) {
assert (!pCC->uCurLine);
pCC->uCLSz = BD_SVR_BUFSZ;
pCC->uCurLine = xmalloc (pCC->uCLSz);
pCC->uCP = 0;
}
/* Clean up old line and read header information */
pCC->uCP = 0;
pCC->uRdSz = 0;
pCC->uRP = 0;
free (pCC->uUD.uRequest);
pCC->uUD.uRequest = NULL;
/* Enter BD_READLINE state */
return BD_READLINE;
}
/**
* @brief BD_READLINE state handler.
*/
BD_SVRSTATE_HANDLER (BD_READLINE)
{
ssize_t rd;
bd_svrstate_t ua;
/* Non-blocking read */
if ((rd=read (pConn, pCC->uReadData, BD_SVR_BUFSZ)) != -1) {
/* Read something - enter BD_HANDLEREAD */
pCC->uRdSz = rd;
pCC->uRP = 0;
return BD_HANDLEREAD;
}
/* Nothing read? Is it an i/o block? Continue with other things */
if (errno == EAGAIN) return BD_CONTINUE;
/* Is it a connection problem? Close the connection */
if (errno == ECONNRESET) return BD_CLOSECONNECTION;
if (errno == ENOTCONN) return BD_CLOSECONNECTION;
if (errno == ETIMEDOUT) return BD_CLOSECONNECTION;
/* No? Inform user about error and do what he asks */
ua = pErrFunc (pCC->uState, pCTxT);
if ((ua == BD_CLOSECONNECTION) || (ua == BD_SHUTDOWNSERVER)) return ua;
/* Ok just continue in current state */
return pCC->uState;
}
/**
* @brief BD_HANDLEREAD state handler.
*/
BD_SVRSTATE_HANDLER (BD_HANDLEREAD)
{
/* Walk read and unprocessed data */
while (pCC->uRP < pCC->uRdSz) {
/* Ignore CR as specified in RFC 1945 Appendix B */
if (pCC->uReadData [pCC->uRP] != '\r') {
/* Expand buffer if needed */
XPAND_(pCC->uCurLine, pCC->uCLSz, pCC->uCP, BD_SVR_BUFSZ);
/* EOL? */
if (pCC->uReadData [pCC->uRP] == '\n') {
/* End string */
pCC->uCurLine [pCC->uCP] = 0;
++pCC->uRP;
/* Enter BD_GOTLINE state */
return BD_GOTLINE;
/* Not EOL? */
} else {
/* Copy into buffer */
pCC->uCurLine [pCC->uCP] = pCC->uReadData [pCC->uRP];
}
/* Point to next location */
++pCC->uCP;
}
/* Continue walk */
++pCC->uRP;
}
/* Walk done? Read more data */
return BD_READLINE;
}
/**
* @brief BD_GOTLINE state handler.
*/
BD_SVRSTATE_HANDLER (BD_GOTLINE)
{
/* Read buffer empty? Signals end of header */
if (!*pCC->uCurLine) return BD_GOTHEADER;
/* Is this first line? */
if (!pCC->uUD.uRequest) {
/* Parse first line */
pCC->uUD.uRequest = sParseRequest (pCC->uCurLine);
/* If we fail to parse request - close the connection */
if (!pCC->uUD.uRequest) {
pErrFunc (BD_PARSEHDR_ERR, pCTxT);
return BD_CLOSECONNECTION;
}
/* Otherwise */
} else {
/* TODO: Parse name/value pairs */
}
/* Finished processing content */
pCC->uCP = 0;
/* Enter BD_HANDLEREAD state to process remaining read data */
return BD_HANDLEREAD;
}
/**
* @brief BD_GOTHEADER state handler.
*/
BD_SVRSTATE_HANDLER (BD_GOTHEADER)
{
bd_svrstate_t ua;
/* Call handler function and adjust flow if needed */
ua = pConnectionFunc (BD_CS_GOTHEADER, (bd_conn_t*)pCC, pCTxT);
if ((ua == BD_CLOSECONNECTION) || (ua == BD_SHUTDOWNSERVER)) return ua;
/* Switch state to BD_LOADRESPONSE */
return BD_LOADRESPONSE;
}
/**
* @brief BD_LOADRESPONSE state handler.
*/
BD_SVRSTATE_HANDLER (BD_LOADRESPONSE)
{
bd_svrstate_t ua;
/* Call handler function and adjust flow if needed
* NB: User must change state here to BD_RESPONSEDONE
* otherwise he will stay here forever (we can't
* know when user has finished writing). */
ua = pConnectionFunc (BD_CS_LOADRESPONSE, (bd_conn_t*)pCC, pCTxT);
if ((ua == BD_CLOSECONNECTION) || (ua == BD_SHUTDOWNSERVER)) return ua;
/* Reset response pointer */
pCC->uRsP = 0;
/* If user has finished writing,
* we change state to BD_RESPONSEDONE */
if (pCC->uUD.uResponseDataSize == -1) {
pCC->uUD.uResponseDataSize = 0;
return BD_RESPONSEDONE;
}
/* Otherwise we just write the current response (BD_WRITINGDATA) */
return BD_WRITINGDATA;
}
/**
* @brief BD_WRITINGDATA state handler.
*/
BD_SVRSTATE_HANDLER (BD_WRITINGDATA)
{
ssize_t wr;
bd_svrstate_t ua;
/* Nothing left to write? Go back to BD_LOADRESPONSE to get more data */
if (pCC->uRsP == pCC->uUD.uResponseDataSize) return BD_LOADRESPONSE;
assert (pCC->uRsP < pCC->uUD.uResponseDataSize);
/* Non blocking write */
if ((wr = write (pConn,
((byteptr_t)pCC->uUD.uResponseData) + pCC->uRsP,
pCC->uUD.uResponseDataSize - pCC->uRsP)) != -1) {
/* Update written amount */
pCC->uRsP += wr;
/* Just continue in same state */
return pCC->uState;
}
/* Write failed. Is it an i/o block? Continue with other things */
if (errno == EAGAIN) return BD_CONTINUE;
/* Is it a connection error? Close the connection */
if (errno == ECONNRESET) return BD_CLOSECONNECTION;
if (errno == EPIPE) return BD_CLOSECONNECTION;
/* Call error function and adjust flow if needed */
ua = pErrFunc (pCC->uState, pCTxT);
if ((ua == BD_CLOSECONNECTION) || (ua == BD_SHUTDOWNSERVER)) return ua;
/* Just keep writing */
return pCC->uState;
}
/**
* @brief BD_RESPONSEDONE state handler.
*/
BD_SVRSTATE_HANDLER (BD_RESPONSEDONE)
{
bd_svrstate_t ua;
/* BD_RESPONSEDONE is simply the ending state of BD_WRITINGDATA */
ua = sBD_WRITINGDATA (pConn, pCC, pErrFunc, pConnectionFunc, pCTxT);
/* BD_WRITINGDATA is not yet done - continue */
if (ua != BD_LOADRESPONSE) return ua;
/* Inform user to clean up his data */
ua = pConnectionFunc (BD_CS_RESPONSEDONE, (bd_conn_t*)pCC, pCTxT);
if ((ua == BD_CLOSECONNECTION) || (ua == BD_SHUTDOWNSERVER)) return ua;
/* Go back to reading next request */
return BD_READHDR;
}
/**
* @brief BD_RunServer helper function than handles each connection.
* The state of each connection is managed in it's context
* and this function is basically a state machine.
* @param[in,out] pConn The Connection.
* @param[in,out] pCC The state of the connection.
* @param[in] pErrFunc Callback for handling errors.
* @param[in] pConnectionFunc Callback to handle connections.
* @param[in] pCTxT User context (passed to callbacks).
* @return int Passes through the return values of the error function
* (pErrFunc) and the connection function (pConnectionFunc).
* @note
* The return values control the flow of the server:
* BD_CONTINUE - the server will continue normal flow
* BD_CLOSECONNECTION - the server will discard the current connection
* BD_SHUTDOWNSERVER - the server will shutdown
* any other value will simply be ignored and the server will continue.
*/
static int
sConnectionDo (
int pConn, bd_conctx_t *pCC,
int32_t (*pErrFunc)(bd_svrstate_t pState, vptr_t pCTxT),
int32_t (*pConnectionFunc) (bd_connstate_t pState,
bd_conn_t *pConn, vptr_t pCTxT),
vptr_t pCTxT)
{
bd_svrstate_t ns;
#define CD_SM_HANDLE(st) case st: \
ns = s##st (pConn, pCC,\
pErrFunc, pConnectionFunc,\
pCTxT);\
break
/* This is the SM that processes the current connection
* Everything happens here and we only exit if explicitly
* asked by the user or if any operation would result
* in a blocking call.
* NB: Users should also not block here depending on their
* volume requirements because it would slow down
* processing of other requests (if any). */
for (;;) {
/* State Machine */
switch (pCC->uState) {
CD_SM_HANDLE (BD_READHDR);
CD_SM_HANDLE (BD_READLINE);
CD_SM_HANDLE (BD_HANDLEREAD);
CD_SM_HANDLE (BD_GOTLINE);
CD_SM_HANDLE (BD_GOTHEADER);
CD_SM_HANDLE (BD_LOADRESPONSE);
CD_SM_HANDLE (BD_WRITINGDATA);
/* CD_SM_HANDLE (BD_RESPONSEDONE); */
case BD_RESPONSEDONE:
ns = sBD_RESPONSEDONE (pConn, pCC,
pErrFunc, pConnectionFunc,
pCTxT);
if (ns == BD_READHDR) {
/* This line makes our server a simple HTTP 1.0 server
* - it closes the the connection after each response.
* Note that it is simple to extend this framework to
* become HTTP 1.1 compliant if needed. */
return BD_CLOSECONNECTION;
}
break;
default:
/* Should never reach here! */
assert (0);
}
switch (ns) {
case BD_CLOSECONNECTION:
case BD_SHUTDOWNSERVER:
case BD_CONTINUE:
/* These states break out of the state machine */
return ns;
default:
/* Continue with new state */
pCC->uState = ns;
}
}
#undef CD_SM_HANDLE
}
/**
* @brief Start and run a HTTP server.
* @param[in] pIP IP to bind to (NULL implies INADDR_ANY)
* @param[in] pPort Port to bind to.
* @param[in] pErrFunc Callback for handling errors.
* @param[in] pBackgroundTask Callback to perform background tasks.
* @param[in] pConnectionFunc Callback to handle connections.
* @param[in] pCTxT User context (passed to callbacks).
* @return int EXIT_SUCCESS on completion, else EXIT_FAILURE
* @note
* The error function (pErrFunc) and connection function (pConnectionFunc)
* can return values to control the flow of the server.
* If they return:
* BD_CONTINUE - the server will continue normal flow
* BD_CLOSECONNECTION - the server will discard the current connection
* BD_SHUTDOWNSERVER - the server will shutdown
* any other value will simply be ignored and the server will continue.
*/
int BD_RunServer (
strptr_t pIP,
int32_t pPort,
int32_t (*pErrFunc)(bd_svrstate_t pState, vptr_t pCTxT),
void (*pBackgroundTask) (vptr_t pCTxT),
int32_t (*pConnectionFunc) (bd_connstate_t pState,
bd_conn_t *pConn, vptr_t pCTxT),
vptr_t pCTxT
)
{
int sd;
struct sockaddr_in sa;
int cc;
int cd[BD_SVR_POOL_SZ + 1]; /* +1 = guard value */
bd_conctx_t *cctx;
bool_t dosleep;
/* This macro is helpful in setting the current state */
#define BD_SETCURSTATE(s) (cctx[cc].uState = (s))
/* This macro is useful when handling user control flow */
#define BD_DOUSERFLOW(f) \
switch ((f)) {\
case BD_CLOSECONNECTION:\
/* Inform user about abrupt closure */\
(Void)pConnectionFunc (BD_CS_CONNECTIONCLOSED,\
(bd_conn_t*)&cctx[cc], pCTxT);\
/* Flush any pending data */\
(Void)shutdown (cd[cc], SHUT_RDWR);\
/* Release associated resources */\
(Void)close (cd[cc]);\
/* Clean up */\
cd[cc] = 0;\
memset (&cctx[cc], 0, sizeof (bd_conctx_t));\
break;\
case BD_SHUTDOWNSERVER:\
/* Refer [religious] comment on goto usage */\
goto done;\
}
/* This macro helps in socket cleanup */
#define BD_SHUTDOWNCONNECTIONS do {\
for (cc = 0;cc < BD_SVR_POOL_SZ;++cc) {\
if (cd[cc]) {\
(Void)shutdown (cd[cc], SHUT_RDWR);\
(Void)close (cd[cc]);\
cd[cc] = 0;\
}\
}\
} while (0)
/* Set up the connection context pool */
cctx = xmalloc (sizeof (bd_conctx_t) * BD_SVR_POOL_SZ);
memset (cctx, 0, sizeof (bd_conctx_t) * BD_SVR_POOL_SZ);
/* Initialize the connection descriptors */
memset (cd, 0, sizeof (cd));
/* Start off with the first connection context
* The error callbacks will recognize this as a server context
* because the states will be < BD_SERVERSTATESENDS */
cc = 0;
/* Create a socket */
BD_SETCURSTATE (BD_MKSOCKET);
if ((sd = socket (PF_INET, SOCK_STREAM, IPPROTO_TCP)) == -1) goto err;
/* Bind to the port and Ip given */
memset (&sa, 0, sizeof (sa));
sa.sin_family = AF_INET;
sa.sin_port = htons (pPort);
if (pIP) {
BD_SETCURSTATE (BD_CONVERTIPADD);
if (!inet_pton (AF_INET, pIP, &sa.sin_addr)) goto err;
} else {
sa.sin_addr.s_addr = INADDR_ANY;
}
BD_SETCURSTATE (BD_BIND);
if (bind (sd, (const void*)&sa, sizeof (sa)) == -1) goto err;
/* Prepare for stream-based connections */
BD_SETCURSTATE (BD_LISTEN);
if (listen (sd, SOMAXCONN) == -1) goto err;
/* Set mode to non-blocking so we don't need multiple threads */
BD_SETCURSTATE (BD_FCTL);
if (fcntl (sd, F_SETFL, O_NONBLOCK) == -1) goto err;
/* Enter main loop to handle requests */
for (;;) {
/* Find a currently free connection slot */
for (cc = 0;cd [cc];++cc);
/* If we have not hit the guard value
* we have slots left for accepting connections */
if (cc < BD_SVR_POOL_SZ) {
/* Check if there is an incoming connection */
BD_SETCURSTATE (BD_ACCEPT_NONBLOCK);
if ((cd [cc] = accept (sd, NULL, NULL)) == -1) {
/* No? Clear connection */
cd [cc] = 0;
/* Is it an error */
if (errno != EAGAIN) {
/* Inform caller and do what he asks */
if (pErrFunc(cctx[cc].uState, pCTxT)==BD_SHUTDOWNSERVER) {
/* Springing out of the loop ( with apologies to
* Wirth/Dijkstra! :-D )
* NB: If you have religious problems with the use
* of GOTO, please re-read the original paper.
* This usage of goto does not invalidate any
* independent coordinates (i.e. it is structured).*/
goto done;
}
}
} else {
/* Got a connection! Inform caller and do what he asks */
BD_SETCURSTATE (BD_GOTCONNECTION);
BD_DOUSERFLOW (pConnectionFunc (BD_CS_GOTCONNECTION,
(bd_conn_t*)&cctx[cc],
pCTxT));
/* If the connection is still valid we enter reading state */
if (cd[cc]) BD_SETCURSTATE (BD_READHDR);
}
}
/* Let's give our caller a chance to do background stuff */
if (pBackgroundTask) pBackgroundTask (pCTxT);
/* Prepare to sleep if we don't do anything */
dosleep = 1;
/* Ok - let's perform read/writes on valid contexts */
for (cc = 0;cc < BD_SVR_POOL_SZ;++cc) {
if (cd [cc]) {
BD_DOUSERFLOW (sConnectionDo (cd [cc], &cctx [cc],
pErrFunc, pConnectionFunc, pCTxT));
/* In the middle of doing something - no sleeping on the job */
if (cctx[cc].uState != BD_READHDR) dosleep = 0;
}
}
/* We introduce a miniscule delay to stop the CPU spinning */
if (dosleep) usleep (1);
/* And so we go again.... */
}
done:
/* Done path */
BD_SHUTDOWNCONNECTIONS;
(Void)close (sd);
free (cctx);
return EXIT_SUCCESS;
err:
/* Error path */
if (pErrFunc) (Void)pErrFunc (cctx[cc].uState, pCTxT);
BD_SHUTDOWNCONNECTIONS;
if (sd != -1) (Void)close (sd);
free (cctx);
return EXIT_FAILURE;
#undef BD_SHUTDOWNCONNECTIONS
#undef BD_DOUSERFLOW
#undef BD_SETCURSTATE
}
/*****************************************************************************
* APPLICATION
****************************************************************************/
/**
* @brief Context state.
*/
typedef enum {
AS_START = 0,
AS_GOTFILE,
AS_DONE,
AS_ERRORS = 1024,
AS_ERR_GOTFILE,
AS_ERR_GOTFILEASYNC,
AS_ERR_GOTFILEINFO,
AS_ERR_GOTEMPTYFILE,
} bd_appconstate_t;
/**
* @brief Application context information per connection.
*/
typedef struct {
/* Connection state */
bd_appconstate_t uState;
/* Currently serving this file */
int uFile;
struct stat uFInfo;
} bd_appcon_t;
/**
* @brief The server error function.
* @param[in] pState The state of the server.
* @param[in/out] pCTxT The application context.
* @return int32_t Server control value (see note).
* @note
* BD_CONTINUE - the server will continue normal flow
* BD_CLOSECONNECTION - the server will discard the current connection
* BD_SHUTDOWNSERVER - the server will shutdown
* any other value will simply be ignored and the server will continue.
*/
static int32_t
sErrFunc (bd_svrstate_t pState, vptr_t pCTxT)
{
char_t msg[128];
strptr_t s;
switch (pState) {
case BD_START : s = "BD_START"; break;
case BD_MKSOCKET : s = "BD_MKSOCKET"; break;
case BD_CONVERTIPADD : s = "BD_CONVERTIPADD"; break;
case BD_BIND : s = "BD_BIND"; break;
case BD_LISTEN : s = "BD_LISTEN"; break;
case BD_FCTL : s = "BD_FCTL"; break;
case BD_SERVERSTATESENDS : s = "BD_SERVERSTATESENDS"; break;
case BD_ACCEPT_NONBLOCK : s = "BD_ACCEPT_NONBLOCK"; break;
case BD_GOTCONNECTION : s = "BD_GOTCONNECTION"; break;
case BD_READHDR : s = "BD_READHDR"; break;
case BD_READLINE : s = "BD_READLINE"; break;
case BD_HANDLEREAD : s = "BD_HANDLEREAD"; break;
case BD_GOTLINE : s = "BD_GOTLINE"; break;
case BD_PARSEHDR_ERR : s = "BD_PARSEHDR_ERR"; break;
case BD_GOTHEADER : s = "BD_GOTHEADER"; break;
case BD_LOADRESPONSE : s = "BD_LOADRESPONSE"; break;
case BD_WRITINGDATA : s = "BD_WRITINGDATA"; break;
case BD_RESPONSEDONE : s = "BD_RESPONSEDONE"; break;
default: assert (0); s = NULL;
}
if (s) {
sprintf (msg, "ERROR: [State %s]", s);
} else {
sprintf (msg, "ERROR: [State %d]: Please contact developer!", pState);
}
/* Display error */
perror (msg);
/* Just continue */
return BD_CONTINUE;
}
#define BD_APPCONSTATE_HANDLER(h) \
static bd_appconstate_t s##h (bd_conn_t *pConn,\
bd_appcon_t *pCCTxT,\
vptr_t ign)
BD_APPCONSTATE_HANDLER (BD_CS_GOTCONNECTION)
{
/* Since we have a new connection, we'll populate it with a context */
pConn->uCTxT = xmalloc (sizeof (bd_appcon_t));
memset (pConn->uCTxT, 0, sizeof (bd_appcon_t));
/* Ready to rock & roll! */
return AS_START;
}
BD_APPCONSTATE_HANDLER (BD_CS_RESPONSEDONE)
{
/* Clean up response */
if (pConn->uResponseData) free (pConn->uResponseData);
pConn->uResponseData = NULL;
/* Clean up currently serving file */
if (pCCTxT->uFile && pCCTxT->uFile != -1) close (pCCTxT->uFile);
pCCTxT->uFile = 0;
/* Continue from beginning */
return AS_START;
}
BD_APPCONSTATE_HANDLER (BD_CS_CONNECTIONCLOSED)
{
/* Clean up response */
(Void)sBD_CS_RESPONSEDONE (pConn, pCCTxT, ign);
/* Clean up context itself */
free (pConn->uCTxT);
pConn->uCTxT = NULL;
/* All done! */
return AS_DONE;
}
BD_APPCONSTATE_HANDLER (BD_CS_GOTHEADER)
{
/* Check the request */
if (!strcmp (pConn->uRequest, "/")) {
/* Handle the default request */
pCCTxT->uFile = open ("index.htm", O_RDONLY);
} else {
pCCTxT->uFile = open (pConn->uRequest + 1, O_RDONLY);
}
/* Failed to get file to serve */
if (pCCTxT->uFile == -1) return AS_ERR_GOTFILE;
/* Set in async mode */
if (fcntl (pCCTxT->uFile,
F_SETFL, O_NONBLOCK) == -1) return AS_ERR_GOTFILEASYNC;
/* Get file information */
if (fstat (pCCTxT->uFile,
&pCCTxT->uFInfo) == -1) return AS_ERR_GOTFILEINFO;
/* Empty file! */
if (!pCCTxT->uFInfo.st_size) return AS_ERR_GOTEMPTYFILE;
/* Set up response data */
pConn->uResponseData = xmalloc (pCCTxT->uFInfo.st_size);
/* Success! */
return AS_GOTFILE;
}
BD_APPCONSTATE_HANDLER (BD_CS_LOADRESPONSE)
{
strptr_t e = NULL;
int rd;
switch (pCCTxT->uState) {
case AS_DONE:
/* Inform server */
pConn->uResponseDataSize = -1;
return pCCTxT->uState;
case AS_GOTFILE:
/* Reset response data */
pConn->uResponseDataSize = 0;
/* All done? */
if (!pCCTxT->uFInfo.st_size) return AS_DONE;
/* Read as much as possible without blocking */
if ((rd = read (pCCTxT->uFile, pConn->uResponseData,
pCCTxT->uFInfo.st_size)) != -1) {
/* Ok - decrease amount left to read */
pCCTxT->uFInfo.st_size -= rd;
/* Inform server how much we have read */
pConn->uResponseDataSize = rd;
/* Keep going */
return pCCTxT->uState;
}
/* Didn't read anything because i/o not ready */
if (errno == EAGAIN) return pCCTxT->uState;
/* Error! */
e = "Failed to read file!";
/* Fall-through */
case AS_ERR_GOTFILE:
if (!e) e = "Failed to open file!";
/* Fall-through */
case AS_ERR_GOTFILEASYNC:
if (!e) e = "Failed to set Async on file!";
/* Fall-through */
case AS_ERR_GOTFILEINFO:
if (!e) e = "Failed to get file size!";
/* Fall-through */
case AS_ERR_GOTEMPTYFILE:
if (!e) e = "File is empty!";
pConn->uResponseData = xmalloc (strlen (e) + 1);
pConn->uResponseDataSize = sprintf (pConn->uResponseData,
"%s", e);
return AS_DONE;
default:
/* @warning This block has potential for buffer overrun
* but it should never reach here anyway! */
pConn->uResponseData = xmalloc (128);
pConn->uResponseDataSize = sprintf (pConn->uResponseData,
"Invalid state! Please contact developer! (%d)",
pCCTxT->uState);
return AS_DONE;
}
}
static int32_t
sConnectionFunc (bd_connstate_t pState,
bd_conn_t *pConn, vptr_t pCTxT)
{
bd_appconstate_t ns;
#define CF_SM_HANDLE(st) case st: \
ns = s##st (pConn, pConn->uCTxT, NULL);\
break
switch (pState) {
CF_SM_HANDLE (BD_CS_GOTCONNECTION);
CF_SM_HANDLE (BD_CS_CONNECTIONCLOSED);
CF_SM_HANDLE (BD_CS_GOTHEADER);
CF_SM_HANDLE (BD_CS_LOADRESPONSE);
CF_SM_HANDLE (BD_CS_RESPONSEDONE);
default:
/* Should never reach here */
assert (0);
}
/* Set new state (if not cleaned up) */
if (pConn->uCTxT) ((bd_appcon_t *)pConn->uCTxT)->uState = ns;
assert ((pConn->uCTxT) || (pState == BD_CS_CONNECTIONCLOSED));
/* And continue */
return BD_CONTINUE;
#undef CF_SM_HANDLE
}
/*****************************************************************************
* MAIN ENTRY POINT
****************************************************************************/
int main (int argc, char* argv[])
{
/* Run the HTTP server */
return BD_RunServer (NULL, 8899,
sErrFunc,
NULL,
sConnectionFunc,
NULL);
}
/**
* @brief Known bugs/issues.
*
* TODO: Support POST requests
* TODO: Support HTTP 1.1
*/
