Back

--------------------------

More about Rumprun Unikernelz

--------------------------

...and nginx modules

Hey there, kidz! Not many moons ago we had our first zine about Rumprun Unikernels. I know, they were so much phun, right? So, let's party harder! Let's see how you can run Nginx modules (written in C! Just how Gawd Igor intended them to be!). [---------Prepare thy environment We will be using same kek1 container, built up from unikernix Docker image. If you do not have it/removed it/first time around here, no worries! Check the first link, down below, on how to deploy unikernels on Docker containers and more! Just in case you are bored to check... here are the steps, folks! root@tr0n:/home/buildz# more Dockerfile FROM ubuntu:16.04 RUN apt-get update -y RUN apt-get install gcc -y RUN apt-get update -y && apt-get install git -y # install a bunch of necessary packages RUN apt-get install libc6-armel-cross libc6-dev-armel-cross \ binutils-arm-linux-gnueabi \ libncurses5-dev \ qemu-user-static \ inotify-tools qemu -y RUN apt-get install genisoimage -y # ...well, more packages RUN apt-get install -y pkg-config \ && apt-get install -y openjdk-8-jdk \ && apt-get install -y cpio \ && apt-get install -y mercurial \ && apt-get install -y unzip \ && apt-get install -y zip \ && apt-get install zlib1g-dev -y RUN cd /opt/ && git clone http://repo.rumpkernel.org/rumprun &&\ cd /opt/rumprun &&\ git submodule update --init WORKDIR /opt/rumprun #build it RUN ./build-rr.sh hw WORKDIR /opt/rumprun And build it as usual... root@tr0n:/home/buildz# docker build -t unikernix . [ .... Wild break to sip more coffee .... ] ...and once it's done, do a flip from excitement, and a check: root@tr0n:/home/buildz# docker images | grep '.rnix*' unikernix latest 36280uc0832 2 days ago 2.23GB Provide some access to devices filesystem root@tr0n:~# docker run --name kek1 --privileged=true \ --hostname=kek1 -v /dev:/dev -ti unikernix /bin/bash root@kek1:/opt/rumprun# My container's environment: root@kek1:/opt/rumprun# more /etc/environment PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/rumprun" export PATH=$PATH:rumprun/bin root@kek1:/opt/rumprun# Ahkay enough with the recap! ... so I fired up my kek1 container: root@tr0n:/ch00k33ty# #start and execute kek1 commands, just in case root@tr0n:/ch00k33ty# docker start kek1 root@tr0n:/ch00k33ty# docker exec -ti kek1 /bin/bash root@kek1:/opt/rumprun# Last time we created folder rumpish under /opt/rumprun My actual path for this example starts here: root@kek1:/opt/rumprun/rumprish/nginx_test# #hi! root@kek1:/opt/rumprun/rumprish/nginx_test# Let's create a new folder under nginx_test: root@kek1:/opt/rumprun/rumprish/nginx_test# mkdir abc123 root@kek1:/opt/rumprun/rumprish/nginx_test# This will be the folder where we will be installing all the necessary packages for our nginx service. Luckily for us, the Rumprun Unikernels have amazingly good repositories, so no big headaches here. Keep in mind the dependencies of nginx. Below a small example for what it takes to run nginx (output taken from a different machine) root@ch00k33ty:/home/test# ldd nginx linux-vdso.so.1 => (0x00007ffc9dbe2000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2d4c5e4000) libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f2d4c3ac000) libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f2d4c13c000) libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f2d4bcf7000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f2d4badd000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2d4b713000) /lib64/ld-linux-x86-64.so.2 (0x00007f2d4c801000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2d4b50f000) So, just by analyzing that, you could tell you'd be needing pcre, libssl, some libcrypto. Now, let's go back to our kek1 container, and download the nginx repository under abc123 folder, but first, install svn ( apt-get install subversion ) root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# svn export https://github.com/rumpkernel/rumprun-packages.git/trunk/nginx A nginx [...] A nginx/patches/0004-auto-lib-libxslt-conf-allow-to-override-ngx_feature_.patch Exported revision 420. Time to grab pcre and libssl: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# svn export https://github.com/rumpkernel/rumprun-packages.git/trunk/libressl A libressl A libressl/Makefile A libressl/README.md Exported revision 420. root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# svn export https://github.com/rumpkernel/rumprun-packages.git/trunk/pcre A pcre A pcre/.gitignore A pcre/Makefile A pcre/README.md Exported revision 420. Hey, almost there! If you take a look at nginx' Makefile (under path /opt/rumprun/rumprish/nginx_test/abc123/nginx), you'll notice you need following files: 1) line Makefile.inc in ./nginx/Makefile include ../Makefile.inc Now, let's grab it: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# svn export https://github.com/rumpkernel/rumprun-packages.git/trunk/Makefile.inc A Makefile.inc Export complete. 2) line scripts in ./nginx/Makefile build/stamp_patch: build/configure patches/* ../scripts/apply-patches.sh build/ patches/* touch $@ Grab'em: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# svn export https://github.com/rumpkernel/rumprun-packages.git/trunk/scripts A scripts A scripts/apply-patches.sh A scripts/fetch.sh Exported revision 420. 3) ... and line Makefile.deps in ./nginx/Makefile: include ../Makefile.deps Grab it fast: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# svn export https://github.com/rumpkernel/rumprun-packages.git/trunk/Makefile.deps A Makefile.deps Export complete. ..and grab some pkgs while you're still here: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# svn export https://github.com/rumpkernel/rumprun-packages.git/trunk/pkgs A pkgs A pkgs/.gitkeep Exported revision 420. ... Bu77 weight... there's moar! ... Ah, yes... we need one more file, so that we can start compiling and building our unikernel: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# svn export https://github.com/rumpkernel/rumprun-packages.git/trunk/config.mk.dist config.mk.dist Export complete. This file helps you to configure the package builder. Copy the config.mk.dist to config.mk: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123# cp config.mk.dist config.mk and update RUMPRUN_TOOLCHAIN_TUPLE to RUMPRUN_TOOLCHAIN_TUPLE=x86_64-rumprun-netbsd. The copied file, after the above change, should look as below(change implemented in red) root@kek1:/opt/rumprun/rumprish/nginx/abc123# more config.mk # # rumprun-packages "configuration" # # Set to the name of the rumprun compiler toolchain you want to use for # building packages. (eg. x86_64-rumprun-netbsd, i486-rumprun-netbsdelf). RUMPRUN_TOOLCHAIN_TUPLE=x86_64-rumprun-netbsd # Select ssl package (another option is openssl) #RUMPRUN_SSL= libressl ... and a small recap of what my abc123 folder contains: root@kek1:/opt/rumprun/rumprish/nginx/abc123# ls -ltr total 52 -rw-r--r-- 1 root root 284 Aug 15 2016 config.mk.dist -rw-r--r-- 1 root root 1096 Aug 15 2016 Makefile.inc -rw-r--r-- 1 root root 168 Aug 15 2016 Makefile.deps drwxr-xr-x 2 root root 4096 Feb 23 20:32 scripts -rw-r--r-- 1 root root 305 Feb 23 20:37 config.mk -rwxr-xr-x 1 root root 15094 Feb 23 21:02 build-rr.sh drwxr-xr-x 4 root root 4096 Feb 23 21:03 libressl drwxr-xr-x 7 root root 4096 Feb 23 21:10 pkgs drwxr-xr-x 4 root root 4096 Feb 23 21:10 pcre drwxr-xr-x 7 root root 4096 Feb 24 00:35 nginx It looks nice! ------------] Exciting times for writing a Gumped nginx module Well, kidz, before building our nginx *sniffs* and testing our module *sniffs*snifss*, one has to modify the nginx.conf. root@kek1:/opt/rumprun/rumprish/nginx/abc123/nginx# more images/data/conf/nginx.conf worker_processes 1; [..........] events { worker_connections 128; } http { [..........] server { listen 80; listen [::]:80; server_name localhost; location / { root /data/www; index index.html; }

location /greetz { oh_hi; }

} } Marked with red what has been added in the existent file nginx.conf. So, everytime I access http://[some IP]:80/greetz, the module oh_hi is called, and will pop on the page whatever I programmed it to...
Eh... module example... And now, kidz, create a new folder, where you will put the C code for module and its config file. My stuff looks like this - files are under /home/Hiya_Gump_Module: root@kek1:/home/Hiya_Gump_Module# ls -ltr total 8 -rw-r--r-- 1 root root 172 Feb 23 23:18 config -rw-r--r-- 1 root root 4009 Feb 23 23:42 ngx_http_hiya_gump_module.c root@kek1:/home/Hiya_Gump_Module# Let's check the config file: root@kek1:/home/Hiya_Gump_Module# more config ngx_addon_name=ngx_http_hiya_gump_module HTTP_MODULES="$HTTP_MODULES ngx_http_hiya_gump_module" NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_hiya_gump_module.c" root@kek1:/home/Hiya_Gump_Module# , where HTTP MODULES - the list for HTTP modules NGX_ADDON_SRCS - list of all addons, that are to be compiled And our wild C code -- comments included; documentation to read is included as usual (and u'll weep while reading it). Nginx is an entire new world by itself. root@kek1:/home/Hiya_Gump_Module# more ngx_http_hiya_gump_module.c #include <nginx.h> //this is always needed #include <ngx_config.h> //this is always needed #include <ngx_core.h> #include <ngx_http.h> //for http static char *ngx_http_hiya_gump_conf_handler( ngx_conf_t *cf, ngx_command_t *cmd, void *conf ); //declare directive hiya_gump // when providing our location in nginx.conf ("greetz" in our case), // this module will run when finding "oh_hi;" static ngx_command_t ngx_http_hiya_gump_commands[] = { { ngx_string("oh_hi"), NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS , ngx_http_hiya_gump_conf_handler, 0, 0, NULL }, ngx_null_command }; //declare callback function of nginx events static ngx_http_module_t ngx_http_hiya_gump_module_ctx = { NULL, /* preconfiguration */ NULL, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; //module that binds context and commands ngx_module_t ngx_http_hiya_gump_module = { NGX_MODULE_V1, &ngx_http_hiya_gump_module_ctx, /* module context */ ngx_http_hiya_gump_commands, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; //main handler function of the module ngx_int_t ngx_http_hiya_gump_handler(ngx_http_request_t *r) { ngx_buf_t *b; ngx_chain_t out; ngx_int_t rc; static u_char webish[] = "<html><head><title>FeelinGumpy</title> <style>body{background-color:#F444FF}\ </style></head>\ <body><center><h2>Hiya, Gump!</h2><br> \ <img src=\"https://66.media.tumblr.com/66237e7e905eb9cd2555ab422ccf817c/tumblr_oo3gndn3ly1uaiqeco2_500.gif\">\ </center></body></html>"; //set the 'Content-Type' header r->headers_out.content_type.len = ngx_strlen("text/html"); r->headers_out.content_type.data = (u_char *)"text/html"; //set the status r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = sizeof( webish ) - 1 ; // allocate buffer from nginx pool for the response body b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); if(b == NULL) { ngx_log_error( NGX_LOG_ERR, r->connection->log, 0, "Failed to allocate buffer." ); return NGX_HTTP_INTERNAL_SERVER_ERROR; } //attach buffer to buffer chain out.buf = b; out.next = NULL; // adjust pointers of the buffer b->pos = webish ; b->last = webish + sizeof( webish ) - 1 ; b->memory = 1; // this buffer is now in memory b->last_buf = 1; //the last buffer in the buffer chain rc = ngx_http_send_header(r); if(rc != NGX_OK) { return rc; } //send your response's buffer chain return ngx_http_output_filter(r, &out); } //directive handler static char *ngx_http_hiya_gump_conf_handler( ngx_conf_t *cf, ngx_command_t *cmd, void *conf ) { ngx_http_core_loc_conf_t *clcf; clcf = ngx_http_conf_get_module_loc_conf( cf, ngx_http_core_module ) ; //specify request handler clcf->handler = ngx_http_hiya_gump_handler ; return NGX_CONF_OK ; } root@kek1:/home/Hiya_Gump_Module# Let's go back to our nginx folder: root@kek1:/home/Hiya_Gump_Module# cd /opt/rumprun/rumprish/nginx_test/abc123/nginx root@kek1:/opt/rumprun/rumprish/nginx/abc123/nginx# ls Makefile Makefile root@kek1:/opt/rumprun/rumprish/nginx/abc123/nginx# And let's edit the Makefile, so that we can run our nginx module, by using --add-module parameter. Provide the entire path of folder, where the config and code are located: NGINX_CONF_OPTS += \ --crossbuild=NetBSD \ [...............] --with-ipv6 \ --add-module=/home/Hiya_Gump_Module \ --with-pcre --------------------] "Are we there yet?!" Just in case: under /opt/rumprun, run once again the export for tools: root@kek1:/opt/rumprun# export PATH=${PATH}:$(pwd)/rumprun/bin We are now ready to build the Nginx unikernel (along with the module) - simply run make: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# make [......if no errors you should see the module....] objs/src/http/modules/ngx_http_upstream_keepalive_module.o \ objs/addon/Hiya_Gump_Module/ngx_http_hiya_gump_module.o \ objs/ngx_modules.o \ [................................................] cp build/objs/nginx bin/nginx genisoimage -l -r -o images/data.iso images/data Result Size: 200 x 446 Total translation table size: 0 Total rockridge attributes bytes: 1254 Total directory bytes: 4560 Path table size(bytes): 34 Max brk space used 0 189 extents written (0 MB) root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# You can also check the new addon on container's side: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# ls build/objs/addon/ Hiya_Gump_Module root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# ls build/objs/addon/Hiya_Gump_Module/ ngx_http_hiya_gump_module.o root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# ... bake the image: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# rumprun-bake hw_virtio ./nginx.bin bin/nginx root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# ls nginx.bin nginx.bin root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx Create a tap device and give it a network: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# ip tuntap add tap0 mode tap root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# ip addr add 10.0.120.100/24 dev tap0 root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# ip link set dev tap0 up Check, just in case... root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# ip a | grep tap0 7: tap0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 inet 10.0.120.100/24 scope global tap0 ... and run it: root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# rumprun qemu -i -g '-curses' -M 128 \ -I if,vioif,'-net tap,script=no,ifname=tap0' \ -W if,inet,static,10.0.120.101/24 \ -b images/data.iso,/data -- ./nginx.bin -c /data/conf/nginx.conf Same purple on black output should appear (just like in the 1st zine) Now, open a terminal, log in to the kek1, and check with a curl what you get at http://10.0.120.101:80 root@tr0n:/ch00k33ty# docker exec -ti kek1 /bin/bash root@kek1:/opt/rumprun/# root@kek1:/opt/rumprun/# curl -v http://10.0.120.101 * Rebuilt URL to: http://10.0.120.101/ * Trying 10.0.120.101... * Connected to 10.0.120.101 (10.0.120.101) port 80 (#0) > GET / HTTP/1.1 > Host: 10.0.120.101 > User-Agent: curl/7.47.0 > Accept: */* > < HTTP/1.1 200 OK < Server: nginx/1.8.0 < Date: Sun, 24 Feb 2019 20:03:00 GMT < Content-Type: text/html < Content-Length: 212 < Last-Modified: Sun, 01 Jan 2017 19:02:22 GMT < Connection: keep-alive < ETag: "5869523e-d4" < Accept-Ranges: bytes < <html> <body style="font-size: 14pt;"> <img src="logo150.png"/> Served to you by <a href="http://nginx.org/">nginx</a>, running on a <a href="http://rumpkernel.org">rump kernel</a>... </body> </html> * Connection #0 to host 10.0.120.101 left intact Kewl! So, we can see the index.html works on port 80! Check if our module works on http://10.0.120.101:80/greetz : root@kek1:/opt/rumprun/# curl -v http://10.0.120.101/greetz * Trying 10.0.120.101... * Connected to 10.0.120.101 (10.0.120.101) port 80 (#0) > GET /greetz HTTP/1.1 > Host: 10.0.120.101 > User-Agent: curl/7.47.0 > Accept: */* > < HTTP/1.1 200 OK < Server: nginx/1.8.0 < Date: Sun, 24 Feb 2019 20:03:02 GMT < Content-Type: text/html < Content-Length: 261 < Connection: keep-alive < * Connection #0 to host 10.0.120.101 left intact <html><head><title>FeelinGumpy</title> <style>body{background-color:#F444FF} </style></head><body> <center><h2>Hiya, Gump!</h2><br> <img src="https://66.media.tumblr.com/66237e7e905eb9cd2555ab422ccf817c/tumblr_oo3gndn3ly1uaiqeco2_500.gif"> </center></body></html> Noice! For a fast check, let's use netstat-traditional once again, like we did in first unikernel zine: [From terminal one] forward trafic from port 80 to 172.17.0.2 (container's IP) on port 8080 root@kek1:/opt/rumprun/rumpish# nc -l -p 8080 -c 'nc 10.0.120.101 80' [From terminal two - from vm ] forward traffic from port 8080 container to localhost vm on port 8666 root@tr0n:/ch00k33ty# nc -l -p 8666 -c 'nc 172.17.0.2 8080' Open a browser, and check link at http://127.0.0.1:8666/greetz Some wild gif should appear!

Try same thing for http://127.0.0.1:8666 Extra stuff: feel free to check src code under build/src/ root@kek1:/opt/rumprun/rumprish/nginx_test/abc123/nginx# ls build/src/ core event http mail misc os For more horrors, do not forget to check the 1st zine about unikernels (in case you want to skip netstat-traditional...) Oh, this was a long zine... And it was quite superficially written. Heh! I hope you were able to pay attention all this time. If you did, congrats! ... And thank you! Buh-bye... for now! ^^ =========================== To be read: [First zine about Rumprun unikernelz] https://jnc0x24dd099bb870.tumblr.com/NetworkZ/unikZ [Nginx Guts - http-modules /really kewl site] http://www.nginxguts.com/2011/02/http-modules/ [Nginx - hello world module /oldies, but goldies!] https://nutrun.com/weblog/2009/08/15/hello-world-nginx-module.html [Rumprun-package] https://github.com/rumpkernel/rumprun-packages [Rumprun - Serve a static website as a Unikernel] https://github.com/rumpkernel/wiki/wiki/Tutorial%3A-Serve-a-static-website-as-a-Unikernel [Nginx - documentation] http://nginx.org/en/docs/ -------EOF---------