samples: sockets: Add a dumb HTTP server example #1378

pfalcon · 2017-09-06T14:20:28Z

It's dumb, because it doesn't really parse HTTP request, just always
sends the same page in response. Even such, it's useful for socket
load testing with tools like Apache Bench (ab) and for regression
checking in the net subsystem.

Signed-off-by: Paul Sokolovsky [email protected]

pfalcon · 2017-09-06T14:29:08Z

This is a sockets test application I wrote while developing BSD Sockets API. I didn't submit it at that time, because it seemed to dumb, plus I wanted to it to be an example of serving a real page, not a handful of characters, and that hit issues deep in the stack which I wanted to investigate first.

Anyway, it's definitely useful to do load testing on BSD Sockets API and to uncover regressions like https://jira.zephyrproject.org/browse/ZEP-2593 .

pfalcon · 2017-09-06T14:41:01Z

https://jira.zephyrproject.org/browse/ZEP-2593?focusedCommentId=19957&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-19957 was updated with instructions how to use this sample to reproduce queue operations performance regressions introduced by 84db641

jukkar · 2017-09-08T13:50:05Z

samples/net/sockets/socket_dumb_http/Makefile

+src/socket_dumb_http.o: src/response.inc
+
+response.inc: response.html
+	bin2h 8 <$^ >$@


This requires that you have bin2h somewhere. At least in my host (Fedora 25) I do not have it.
One option could be that I send the patch "samples: net: common: Add rules to generate a header file" (currently found in websocket patchset) separately as there could be some use for it here.

Those lines are actually artifact of experimentation, which didn't work still. Instead, I included pre-generated *.inc files into this PR (and standalone Makefile.gen file to re-generate them). But indeed, this:

that I send the patch "samples: net: common: Add rules to generate a header file" (currently found in websocket patchset) separately as there could be some use for it here.

would be better, I'd appreciate it.

I created a separate PR #1447 that contains one way to create an include file.

pfalcon · 2017-09-13T10:24:17Z

Updated to use the latest version of #1447.

pfalcon · 2017-09-18T17:34:36Z

Updated to the latest version of #1447

pfalcon · 2017-09-19T10:25:20Z

CI fails due to false positive:

b'-:90: ERROR:DOS_LINE_ENDINGS: DOS line endings\n#90: FILE: samples/net/sockets/dumb_http_server/src/response_big.html.bin:1:\n+HTTP/1.0 200 OK^M$\n\n-:91: ERROR:DOS_LINE_ENDINGS: DOS line endings\n#91: FILE: samples/net/sockets/dumb_http_server/src/response_big.html.bin:2:\n+Content-Type: text/html; charset=utf-8^M$\n\n-:92: ERROR:DOS_LINE_ENDINGS: DOS line endings\n#92: FILE: samples/net/sockets/dumb_http_server/src/response_big.html.bin

That file should have \r\n line endings in the HTTP response part, as can be seen I event tried to rename it to .bin to deter checkpatch from poking into it, but that didn't help.

@jukkar : Please re-review/approve.

jukkar

LGTM

jukkar · 2017-09-19T14:36:52Z

That file should have \r\n line endings in the HTTP response part, as can be seen I event tried to rename it to .bin to deter checkpatch from poking into it, but that didn't help.

What about just having \n and remove \r, usually the clients are not very strict about that.

pfalcon · 2017-09-19T14:42:32Z

What about just having \n and remove \r, usually the clients are not very strict about that.

Usually, yes, but that's violation of the protocol!

jukkar · 2017-09-20T07:15:29Z

Usually, yes, but that's violation of the protocol!

Hardly a big issue for a sample application with test content.

pfalcon · 2017-09-20T07:39:39Z

Hardly a big issue for a sample application with test content.

I think it is. This app tests (per 2nd in row intention behind it) ability to handle multiple client connections. It's not intended to test how much clients are tolerable to malformed protocol replies, and testing both at once is a bit too much (the best practice is to test one thing at time).

@jukkar, so, I'd appreciate if this can be merged as is. If you can't do that because of a false positive CI failure, and it requires override from e.g. @nashif, please let me know that, so I can pursue it. Thanks!

nashif

This needs a readme explain what exactly are you doing here and why this is useful and we also need a sample.yaml to build this in CI.

pfalcon · 2017-09-20T17:48:50Z

@nashif : Both are now added.

nashif · 2017-09-20T19:41:14Z

@nashif : Both are now added.

thanks, waiting for @dbkinder to review doc

dbkinder · 2017-09-20T21:27:14Z

samples/net/sockets/dumb_http_server/README.rst

+Finally, you can run an HTTP profiling/load tool like Apache Bench
+(``ab``) against the server:
+
+    $ ab -n10 http://192.0.2.1:8080/


I think there's a space required here, -n 10

Not "required", but optionally can be used. Unless there's strong affinity for Zephyr docs to use space form when quoting params for any external apps which supports both, let's leave it as is (at least it's shorter)?

dbkinder · 2017-09-20T21:29:35Z

samples/net/sockets/dumb_http_server/README.rst

+
+    $ ./socket_dumb_http
+
+To test, connect to http://127.0.0.1:8080/ .


add, "as described in the previous section."

Taken literally, that sounds confusing, because previous section didn't describe connecting to http://127.0.0.1:8080/ . Instead, rewrote as:

"To test, connect to http://127.0.0.1:8080/ , and follow the steps in the previous section."

dbkinder · 2017-09-20T21:32:31Z

samples/net/sockets/dumb_http_server/src/response_big.html.bin

+<p>
+The Zephyr kernel is derived from Wind River's commercial VxWorks Microkernel Profile for VxWorks. Microkernel Profile has evolved over 20 years from DSP RTOS technology known as Virtuoso. The RTOS has been used in several commercial applications including satellites, military command and control communications, radar, telecommunications and image processing. The most recent example of the technology’s success is the successful Philae Landing on Comet Churyumov–Gerasimenko and the accompanying Rosetta Orbiter.
+</p>
+


and you're missing the closing </td></tr>

Browsers can deal with that, no worries ;-). And yep, that was more or less crude copy-paste from the Zephyr website homepage at that time. Fixed though.

dbkinder · 2017-09-20T21:42:05Z

samples/net/sockets/dumb_http_server/README.rst

+
+``-n`` parameter specified the number of HTTP requests to issue against
+a server.
+


Maybe add here a section that shows how you can update the sample to reply with an bigger HTML response (change an #if 0 to #if 1)?

Well, I don't know if it's worth to describe such features. That feature is developer-oriented, the docs is user-facing. There can be various different developers' features, they may change over time.

dbkinder · 2017-09-20T21:42:15Z

samples/net/sockets/dumb_http_server/src/response_big.html.bin

+Content-Type: text/html; charset=utf-8
+
+<html>
+<style>


Technically this is malformed HTML. Need to add a section (where the <style> would go).
Also, you don't talk about this "big" response in the documentation. Looks like you could tweak the code to get this big response if you wanted to...

Technically this is malformed HTML.

Depends on which HTML ;-). For pre-5, based on XML with omitted and implied tags, it may be even valid, and browsers definitely can deal with it. Happy to add <head> there of course. The point - if you keep looking, you may find more artifacts, but it was tested to work with the browsers...

dbkinder

suggested edits

It's dumb, because it doesn't really parse HTTP request, just always sends the same page in response. Even such, it's useful for socket load testing with tools like Apache Bench (ab) and for regression checking in the net subsystem. Signed-off-by: Paul Sokolovsky <[email protected]>

pfalcon · 2017-09-21T10:26:23Z

@dbkinder : Addressed what made sense IMHO, would prefer to keep "response_big" feature an internal one - those who'll poke in the source will find it. Will be happy to make it the default later - when the IP stack will be able to run reliably with it.

…phyrproject-rtos#1378) Signed-off-by: Geoff Gustafson <[email protected]>

pfalcon requested review from jukkar and Vudentz September 6, 2017 14:29

pfalcon mentioned this pull request Sep 7, 2017

poll: Minimize overhead when polling single event #1396

Closed

pfalcon added the net label Sep 8, 2017

jukkar requested changes Sep 8, 2017

View reviewed changes

pfalcon force-pushed the sockets-dumb-http-server branch from 35dce9a to 98d1803 Compare September 12, 2017 10:23

ghost assigned pfalcon Sep 12, 2017

ghost added the In progress For PRs: is work in progress and should not be merged yet. For issues: Is being worked on label Sep 12, 2017

pfalcon mentioned this pull request Sep 12, 2017

build: Add rules in Makefile.gen to generate a hex file #1447

Merged

pfalcon force-pushed the sockets-dumb-http-server branch 2 times, most recently from 839866f to 237e5a9 Compare September 13, 2017 10:23

nashif mentioned this pull request Sep 18, 2017

net: "queue: Use k_poll if enabled" commit regressed BSD Sockets performance zephyriot/zep-jira14#2422

Open

pfalcon force-pushed the sockets-dumb-http-server branch from 237e5a9 to 7c2ce51 Compare September 18, 2017 17:34

pfalcon force-pushed the sockets-dumb-http-server branch 2 times, most recently from 6532b53 to 30e0d9c Compare September 19, 2017 06:02

jukkar previously approved these changes Sep 19, 2017

View reviewed changes

nashif requested changes Sep 20, 2017

View reviewed changes

pfalcon dismissed jukkar’s stale review via da7548a September 20, 2017 15:30

pfalcon force-pushed the sockets-dumb-http-server branch 2 times, most recently from da7548a to ebb21f6 Compare September 20, 2017 16:05

pfalcon requested a review from dbkinder as a code owner September 20, 2017 16:05

pfalcon force-pushed the sockets-dumb-http-server branch 2 times, most recently from 88e7fee to 2860a83 Compare September 20, 2017 17:21

dbkinder reviewed Sep 20, 2017

View reviewed changes

dbkinder suggested changes Sep 20, 2017

View reviewed changes

pfalcon force-pushed the sockets-dumb-http-server branch from 2860a83 to 758f9e2 Compare September 21, 2017 10:23

nashif approved these changes Sep 21, 2017

View reviewed changes

nashif merged commit ef39533 into zephyrproject-rtos:master Sep 22, 2017

ghost removed the In progress For PRs: is work in progress and should not be merged yet. For issues: Is being worked on label Sep 22, 2017

nashif mentioned this pull request Sep 23, 2017

net: "queue: Use k_poll if enabled" commit regressed BSD Sockets performance zephyriot/zephyr-issues#2422

Open

zephyrbot mentioned this pull request Sep 23, 2017

net: "queue: Use k_poll if enabled" commit regressed BSD Sockets performance #4022

Closed

nagineni pushed a commit to nagineni/zephyr that referenced this pull request Nov 20, 2017

[callbacks] Adjust cb_size in new_id() instead of every call site (ze…

fceee73

…phyrproject-rtos#1378) Signed-off-by: Geoff Gustafson <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

samples: sockets: Add a dumb HTTP server example #1378

samples: sockets: Add a dumb HTTP server example #1378

pfalcon commented Sep 6, 2017

pfalcon commented Sep 6, 2017

pfalcon commented Sep 6, 2017

jukkar Sep 8, 2017

pfalcon Sep 8, 2017

jukkar Sep 12, 2017

pfalcon commented Sep 13, 2017

pfalcon commented Sep 18, 2017

pfalcon commented Sep 19, 2017

jukkar left a comment

jukkar commented Sep 19, 2017

pfalcon commented Sep 19, 2017

jukkar commented Sep 20, 2017

pfalcon commented Sep 20, 2017

nashif left a comment

pfalcon commented Sep 20, 2017

nashif commented Sep 20, 2017

dbkinder Sep 20, 2017

pfalcon Sep 21, 2017

dbkinder Sep 20, 2017

pfalcon Sep 21, 2017

dbkinder Sep 20, 2017 •

edited

Loading

pfalcon Sep 21, 2017

dbkinder Sep 20, 2017

pfalcon Sep 21, 2017

dbkinder Sep 20, 2017

pfalcon Sep 21, 2017

dbkinder left a comment

pfalcon commented Sep 21, 2017


		$ ./socket_dumb_http

		To test, connect to http://127.0.0.1:8080/ .


		``-n`` parameter specified the number of HTTP requests to issue against
		a server.

samples: sockets: Add a dumb HTTP server example #1378

samples: sockets: Add a dumb HTTP server example #1378

Conversation

pfalcon commented Sep 6, 2017

pfalcon commented Sep 6, 2017

pfalcon commented Sep 6, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pfalcon commented Sep 13, 2017

pfalcon commented Sep 18, 2017

pfalcon commented Sep 19, 2017

jukkar left a comment

Choose a reason for hiding this comment

jukkar commented Sep 19, 2017

pfalcon commented Sep 19, 2017

jukkar commented Sep 20, 2017

pfalcon commented Sep 20, 2017

nashif left a comment

Choose a reason for hiding this comment

pfalcon commented Sep 20, 2017

nashif commented Sep 20, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dbkinder Sep 20, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dbkinder left a comment

Choose a reason for hiding this comment

pfalcon commented Sep 21, 2017

dbkinder Sep 20, 2017 •

edited

Loading