--------------------------Packet capture &Data analysis -------------------------- ...with a side of R &Jupyter env
Hey, kidz!
How about learning a bit about R programming and Jupyter, today?
For networking purposes, of `corpse`!
It`s gonna be fast, I promise!
[---------------Prepare the env
Today we'll be using jupyter/r-notebook - an R kernel for Jupyter on container (what else!) ,
to which we will apply a few more changes, for own interest.
1) Create the container:
docker run --user=root -it --rm -p 8888:8888 jupyter/r-notebook
2) Once in the container:
> Create a link to /bin/tar:
root@bba43b7777d:~# ln -s /bin/tar /bin/gtar
> Run this bash script for a bunch of updates:
root@bba43b7777d:~# more run_it.sh
#!/bin/bash
apt-get update
apt-get install software-properties-common -y
add-apt-repository -y "ppa:marutter/rrutter"
add-apt-repository -y "ppa:marutter/c2d4u"
apt-get update
apt-get install r-cran-igraph -y
> Run below inline R for installation of igraph:
root@bba43b7777d:~# R -e "require(devtools);
install_version('igraph', version='1.0.1', repos='https://cran.rstudio.org/ ')"
Good, now you can draw graphs.
3) Onto the next one...
We will be needing to install on this Jupyter container a very nice R package
to work with pcap!
You can get it from this repository link: https://github.com/hrbrmstr/crafter
(it offers info on installation, too!)
...but we'll do all that right now:
a) Install git
root@baa43b7777d:~# apt-get install git
b) Install the necessary packages in order to proceed with R package installation:
root@baa43b7777d:~# apt-get install autoconf libtool
root@baa43b7777d:~# apt-get install libsqlite3-0 libpcap0.8 libpcap0.8-dev
c) Clone R package repository:
root@baa43b7777d:~# git clone https://github.com/pellegre/libcrafter
d) root@baa43b7777d:~# cd libcrafter/libcrafter
From there, run autogen.sh script:
root@baa43b7777d:libcrafter/libcrafter# ./autogen.sh
..then,
root@baa43b7777d:libcrafter/libcrafter# make && make install
you probably need to install make...
The environment is ready now, you should be able to install the R package from Jupyter.
[-----------Power up the Jupyter
Since you are a root user inside this container, you'll have to start the jupyter notebook
as per below example (with --allow-rot option):
root@bba43b97756d:~# jupyter notebook --allow-root
[W 22:38:02.522 NotebookApp] WARNING: The notebook server is listening on all IP addresses
and not using encryption. This is not recommended.
[I 22:38:02.523 NotebookApp] The port 8888 is already in use, trying another port.
[..........................snip..........................]
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8889/?token=loooooongnumbeeeeroverrrheeeere
Copy that link into a browser. The localhost should be replaced with the
container's IP.
root@baa43b7777d:/home# ip -f inet a s eth0 | grep -Po 'inet \K[\d.]+'
172.17.0.2
You should reach Jupyter's home. And if you go in the right, New-> R, and a new tab
should open, allowing you to program in R.
[-----------Briefly, about R
From terminal, simply type R:
root@baa43b7777dd:/home# R
R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
>
> R.version
_
platform x86_64-pc-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 3
minor 3.2
year 2016
month 10
day 31
svn rev 71607
language R
version.string R version 3.3.2 (2016-10-31)
nickname Sincere Pumpkin Patch
>
Cramin` time!
Variable:
> x<-12
> x
[1] 12
Functions:
> rep("sup!", times = sum(1, 3, 6) )
[1] "sup!" "sup!" "sup!" "sup!" "sup!" "sup!" "sup!" "sup!" "sup!" "sup!"
Vectors:
> 6:8
[1] 6 7 8
> seq(1, 9)
[1] 1 2 3 4 5 6 7 8 9
>
> wordz <- c('walk', 'the', 'plank')
> wordz[1]
[1] "walk"
>
> length(wordz)
[1] 3
>
Matrices:
> matrix<- 1:4
> dim(matrix)<-c(2,2)
> matrix
[,1] [,2]
[1,] 1 3
[2,] 2 4
>
Factors (store categorical data):
> pets <- c('meow', 'hamham', 'meow', 'hamham', 'moo')
> pets.factor <-factor(c('meow', 'hamham', 'meow', 'hamham', 'moo'))
> pets
[1] "meow" "hamham" "meow" "hamham" "moo"
> pets.factor
[1] meow hamham meow hamham moo
Levels: hamham meow moo
>
Lists:
> wild_list <- list("potatoes",w2="Vodka",desert="Coffee!!")
> wild_list[[1]]
[1] "potatoes"
>
> wild_list$w2
[1] "Vodka"
>
> wild_list[["desert"]]
[1] "Coffee!!"
>
Data Frames:
> Wild_dfr <- data.frame( count=1:3,
tiny_pets=c("piglet","kitten","puppy"),
food_portions=c(5,3,4) )
> Wild_dfr
count tiny_pets food_portions
1 1 piglet 5
2 2 kitten 3
3 3 puppy 4
>
> Wild_dfr$tiny_pets
[1] piglet kitten puppy
Levels: kitten piglet puppy
> Wild_dfr$tiny_pets[1]
[1] piglet
Levels: kitten piglet puppy
> Wild_dfr$food_portions <-as.vector(Wild_dfr$food_portions)
> Wild_dfr$food_portions
[1] 5 3 4
>
Use q() to quit R.
Good! We have assimilated some info. We're a bit comfortable with R.
Now, go back to yer browser, where you have opened the R notebook, and let's practice a bit
drawing graphs (well, you`ll give it a try!)
Networks in igraph
Short example - You mostly must focus on plot()
How will your graph look like if you run below line in Jupyter?
full_g1 <- make_full_graph(20)
plot(empty_g1, vertez.size=10, vertex.label=NA)
Yuh, boy! We R done here!
[---------------PCAP or it didn't happen!
Good! Let's make a small packet capture on our container.
root@baa43b7777dd:~# tcpdump -w test.pcap
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C363 packets captured
363 packets received by filter
0 packets dropped by kernel
Move the capture under crafter/pcaps
root@bba43b97756d:~# mv test.pcap /opt/conda/lib/R/library/crafter/pcaps
Now, back to Jupyter-R in browser. Time to check our pcap capture:
1) Load/reload/install necessary packages (the crafter package, as well)
2) Check out that fabulous capture:
3) You can also apply tail/head on the capture:
4) Let's gather only TCP packets (no ARP, no IP, no Ethernet)
5) ...and just for fun, create a filter (using chksum and windowsize)
6) Install scatterplot3d
install.packages("scatterplot3d")
...and let's try to 3D the capture
7) ...and the result
...if u curious enough, try to make a 2D representation of the packets
You did well for a first time ever introduction with R!
Apologies for my bad cropping. I never have the patience... /le sigh
See ya!
===========================
To be read:
http://tryr.codeschool.com/
https://plot.ly/r/
https://github.com/IRkernel/IRkernel
https://github.com/hrbrmstr/crafter
http://www.lovemytool.com/blog/2010/07/practical-tcp-series-the-tcp-window-by-chris-greer.html
http://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/execute.html
-------EOF---------
|