Run your Selenium driven tests in parallel using TestNG.

Sometime ago I came across this post:
http://rationaleemotions.wordpress.com/2013/07/31/parallel-webdriver-executions-using-testng/
And I thought it’d be good to have it in a repo to save you time setting up your own project.

Basically this example project runs JUnit tests in parallel using TestNG.
Tests are grouped by the browser in which they’re going to be executed (have a look at the TestNG XML Suite files in src/test/resources).
As you might have guessed to drive the browsers we use Selenium WebDriver 🙂
And of course you can run your tests locally or remotely using Selenium GRID.

You can find the repo here: https://github.com/kowalcj0/parallel-selenium-with-testng

And here’s a video showing this project in action 🙂

GEE [Jmeter-ec2] – video tutorials part 2

Hi All,

I just added few more videos showing how to configure your test environment to run your JMeter tests:

  • locally on a Vagrant box
  • using remote machines
  • and on Amazon EC2

04 – create a first test plan, run it using Gee and a vagrant box
Shows how to create a new test plan from a template project. Then how to run it using GEE on our local vagrant box.

05 – configure passwordless ssh access to a linux box
Shows how to configure a passwordless SSH access to a remote Linux box. Such access is required by Gee to run your JMeter tests on a remote machine.

06 – automatically install JMeter with plugins on a remote machine
Shows how to use download-jmeter.sh script to download JMeter and JMeter-plugins automatically on a linux host that will be used by GEE to run your load tests.

07 – run your JMeter test on remote linux hosts
Shows how to configure GEE and remote Linux hosts to run your JMeter tests. This video might come handy when you’d like to run your tests using internal network resources.

08 – run your JMeter test on Amazon EC2
Shows how to configure GEE to run your JMeter tests using Amazon EC2 services.

Vim – tips and tricks

Record a macro that will delete everything after matching character and repeat it N-times.
In this example we’re going to use ‘=’ as the matching character.

qaf=D⏎q

Where:
‘qa’ – to start recording and save it under ‘a’ buffer
‘f=’ – find ‘=’ character
‘D’ – Delete everything to the EOL
‘⏎’ – press ‘enter’ to go to the beginning of next line
‘q’ – to stop recording the macro

then to repeat the macro N-times, i.e. repeat it 5 times:

5@a

‘5’ – the number of times you’d like to repeat the macro
‘@a’ – play macro recorded under ‘a’ buffer

VIM macro delete everything after matching character and repeat N-times

VIM macro delete everything after matching character and repeat N-times

GEE [Jmeter-ec2] – video tutorials part 1

I finally started recording videos explaining how to use Gee.
Gee is a tool based on a JMeter-EC2 project by Oliver Lloyd.
It allows you to run JMeter tests on:

  • Amazon EC2
  • Vagrant boxes
  • Linux hosts

Running your tests using this tool is better than using the standard distributed mode in Jmeter, because it doesn’t require constant communication between the master node and slave nodes.

Here are first few videos explaining how to clone the project, spawn new vagrant box and finally how to run an example project on that virtual box.
Hope you all will find in useful 🙂

GEE [Jmeter-ec2] – 01 – cloning the project

GEE [Jmeter-ec2] – 02 – initialize vagrant box

GEE [Jmeter-ec2] – 03 – running an example test on a vagrant box

How to compile and install latest version of Vim with support for: X11 clipboard, ruby, python 2 or python 3

This was tested with vim 7.4.(1-398)
(Update: 2013-12-03 apparently you can’t build vim with support for both python 2 and python 3, so I had to update this tutorial a bit 🙂 )
(Update: 2014-08-09 Added three commands: “hg pull”, “hg update” and “hg status” to pull the latest version of the repo)

First of all install all the dependencies required when compiling Vim with additional options:

@:~> sudo apt-get install mercurial python python-dev python3 python3-dev ruby ruby-dev libx11-dev libxt-dev libgtk2.0-dev  libncurses5  ncurses-dev

Then clone official vim repo, configure it with support for: X11 clipboard, ruby, python 2 or 3 and few other cool options 🙂

@:~> hg clone https://vim.googlecode.com/hg/ vim
## if you already had this repo cloned, then update it:
@:~> hg pull
@:~> hg update
## and then check if you definitely have the latest version:
@:~> hg summary
parent: 6121:913d16b4904c
 Added tag v7-4-398 for changeset f62b2e76dd80
branch: default
commit: (clean)
update: 52 new changesets (update)
@:~> cd vim/src
##
## this will configure it with python 2
##
@:~> ./configure \
    --enable-perlinterp \
    --enable-pythoninterp \
    --enable-rubyinterp \
    --enable-cscope \
    --enable-gui=auto \
    --enable-gtk2-check \
    --enable-gnome-check \
    --with-features=huge \
    --enable-multibyte \
    --with-x \
    --with-compiledby="Senor QA <senor@qa>" \
    --with-python-config-dir=/usr/lib/python2.7/config-x86_64-linux-gnu
##
## and with python 3
##
@:~> ./configure \
    --enable-perlinterp \
    --enable-python3interp \
    --enable-rubyinterp \
    --enable-cscope \
    --enable-gui=auto \
    --enable-gtk2-check \
    --enable-gnome-check \
    --with-features=huge \
    --enable-multibyte \
    --with-x \
    --with-compiledby="Senor QA <senor@qa>" \
    --with-python3-config-dir=/usr/lib/python3.3/config-3.3m-x86_64-linux-gnu

Next step is to compile our setup:

@:~> make

Then if you want you can check if all the features were compiled properly:

VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Aug  9 2014 15:37:52)
Included patches: 1-398
Compiled by Me <senor@qa>
Huge version with GTK2 GUI.  Features included (+) or not (-):
+acl             +farsi           +mouse_netterm   +syntax
+arabic          +file_in_path    +mouse_sgr       +tag_binary
+autocmd         +find_in_path    -mouse_sysmouse  +tag_old_static
+balloon_eval    +float           +mouse_urxvt     -tag_any_white
+browse          +folding         +mouse_xterm     -tcl
++builtin_terms  -footer          +multi_byte      +terminfo
+byte_offset     +fork()          +multi_lang      +termresponse
+cindent         +gettext         -mzscheme        +textobjects
+clientserver    -hangul_input    +netbeans_intg   +title
+clipboard       +iconv           +path_extra      +toolbar
+cmdline_compl   +insert_expand   -perl            +user_commands
+cmdline_hist    +jumplist        +persistent_undo +vertsplit
+cmdline_info    +keymap          +postscript      +virtualedit
+comments        +langmap         +printer         +visual
+conceal         +libcall         +profile         +visualextra
+cryptv          +linebreak       +python          +viminfo
+cscope          +lispindent      -python3         +vreplace
+cursorbind      +listcmds        +quickfix        +wildignore
+cursorshape     +localmap        +reltime         +wildmenu
+dialog_con_gui  -lua             +rightleft       +windows
+diff            +menu            +ruby            +writebackup
+digraphs        +mksession       +scrollbind      +X11
+dnd             +modify_fname    +signs           -xfontset
-ebcdic          +mouse           +smartindent     +xim
+emacs_tags      +mouseshape      -sniff           +xsmp_interact
+eval            +mouse_dec       +startuptime     +xterm_clipboard
+ex_extra        -mouse_gpm       +statusline      -xterm_save
+extra_search    -mouse_jsbterm   -sun_workshop    -xpm
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/usr/local/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -pthread -I/usr/include/gtk-2.0 -I/usr/lib/x86_64-linux-gnu/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/pango-1.0 -I/usr/include/gio-unix-2.0/ -I/usr/include/freetype2 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/libpng12 -I/usr/include/harfbuzz     -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1      
Linking: gcc   -L. -Wl,-Bsymbolic-functions -Wl,-z,relro -L/build/buildd/ruby1.9.1-1.9.3.484/debian/lib -rdynamic -Wl,-export-dynamic  -L/usr/local/lib -Wl,--as-needed -o vim   -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgio-2.0 -lpangoft2-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo -lpango-1.0 -lfontconfig -lgobject-2.0 -lglib-2.0 -lfreetype   -lSM -lICE -lXt -lX11 -lXdmcp -lSM -lICE  -lm -ltinfo -lnsl   -ldl    -L/usr/lib/python2.7/config-x86_64-linux-gnu -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions   -lruby-1.9.1 -lpthread -lrt -ldl -lcrypt -lm  -L/usr/lib

Just to make sure it works as it should, launch newly compiled ./vim, and type:

:echo has("python")

It should return 1 if it was compiled properly 🙂

If you’re happy then install it 🙂

@:~> sudo make install

Then locate the vim command

@:~> which vim
vim is /usr/local/bin/vim
vim is /usr/bin/vim

As you can see, there are two installed on my system.
To make newly installed version “/usr/local/bin/vim” the default one, we’ll use “update-alternatives”.

@:~> sudo update-alternatives --install "/usr/bin/vim" "vim" "/usr/local/bin/vim" 1
@:~> sudo update-alternatives --install "/usr/bin/vi" "vi" "/usr/local/bin/vim" 1

Then use “update-alternatives” to switch between installed versions 🙂

@:~> sudo update-alternatives --config vim
There are 2 choices for the alternative vim (providing /usr/bin/vim).

  Selection    Path                Priority   Status
------------------------------------------------------------
* 0            /usr/bin/vim.basic   30        auto mode
  1            /usr/bin/vim.basic   30        manual mode
  2            /usr/local/bin/vim   1         manual mode

Press enter to keep the current choice[*], or type selection number: 2
@:~> 
@:~> # now let's change symbolic link for vi
@:~> 
@:~> sudo update-alternatives --config vi
There are 3 choices for the alternative vi (providing /usr/bin/vi).

  Selection    Path                Priority   Status
------------------------------------------------------------
* 0            /usr/bin/vim.basic   30        auto mode
  1            /usr/bin/vim.basic   30        manual mode
  2            /usr/bin/vim.tiny    10        manual mode
  3            /usr/local/bin/vim   1         manual mode

Press enter to keep the current choice[*], or type selection number: 3
update-alternatives: using /usr/local/bin/vim to provide /usr/bin/vi (vi) in manual mode

Last thing to do is to check where the /usr/bin/vim points at:

@:~> ll /usr/bin/vim
lrwxrwxrwx 1 root root 21 Jun  7 11:29 /usr/bin/vim -> /etc/alternatives/vim*

Then check where the /etc/alternatives/vim points at.
If everything went fine, it should be pointing at: /usr/local/bin/vim

@:~> ll /etc/alternatives/vim
lrwxrwxrwx 1 root root 18 Nov 19 12:17 /etc/alternatives/vim -> /usr/local/bin/vim*

Happy Vimming 🙂
J

How to add official Chromium-browser PPA repository to Linux Mint 15 and install the latest stable build

Here’s the official Chromium-browser PPA repo: https://launchpad.net/~chromium-daily/+archive/stable

To install latest stable build follow 3 simple steps:

Step 1:
add “deb http://ppa.launchpad.net/chromium-daily/stable/ubuntu raring main” to /etc/apt/sources.list

Step 2:
add repo’s pub key so that we can avoid problems like the one below:

W: GPG error: http://ppa.launchpad.net raring Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 5A9BF3BB4E5E17B5
sudo gpg --keyserver hkp://keys.gnupg.net:80 --recv 5A9BF3BB4E5E17B5
sudo gpg --export FBEF0D696DE1C72BA5A835FE5A9BF3BB4E5E17B5 | sudo apt-key add -

btw. using hkp://keys.gnupg.net:80 will work even if you’re behind a firewall or proxy 🙂

Step 3:
Update package list and install Chromium

sudo apt-get update
sudo apt-get install chromium-browser

Step 4:
Happy browsing 🙂

Monitoring JVM metrics via JMX managemnet interface in JMeter

What do we need:

Once you have plugins installed them in the jmeter’s/lib/ext folder, then:
1 – On the box you want to monitor, copy templates: jmxremote.password & jmxremote.access from $JAVA_HOME/lib/management to for example: /srv/play/
2 – edit them according to your liking
3 – launch your JAVA application with additional parameters:

java -jar your_application.jar -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=10006 -Dcom.sun.management.jmxremote.password.file=/srv/play/jmxremote.password -Dcom.sun.management.jmxremote.access.file=/srv/play/jmxremote.access

4 – start a server-agent (http://jmeter-plugins.org/wiki/PerfMonAgent/) on the host which JVM metrics you want to monitor using command like:

cd ${jmeter_folder}/lib/ext/
java -jar ./CMDRunner.jar --tool PerfMonAgent --udp-port 0 --tcp-port 7777

You can also run it as background process and detach from your session, so it will continue to work even if you disconnect from the server.

cd ${jmeter_folder}/lib/ext/
nohup java -jar ./CMDRunner.jar --tool PerfMonAgent --udp-port 0 --tcp-port 7777 &

5 – add “PerfMon Metrics Collector” to your test plan
6 – provide the hostname/IP address of the box running server-agent in the “Host/IP” field (don’t forget about the port 🙂 )
7 – select JMX as the “Metric to collect”
8 – double click on “Metric parameter” field and then click on the “…” button to the right
9 – enter all the credentials that the server-agent will use to connect to the local JVM via JMX port. Here’s an example config:

url=localhost\:10006:user=role:password=password:gc-time

where role & password are of course defined in the jmxremote.password & jmxremote.access files

btw. Here’s an example test plan with preconfigured PerfMon listener.

If everything was configured properly the you should see something like that on the PerfMon graph:

example PerfMon graph with JVM Metrics

example PerfMon graph with JVM Metrics

Useful links to all owners of a kindle or any other ebook reader

sending web pages and RSS directly to Kindle
/
wysyłanie stron internetowych i RSS’ów na kindle’a
http://www.klip.me/
http://sendtoreader.com/
http://kindlefeeder.com/

ebook management, file converter etc
/
program do zarządzanie ebookami, ściąganie newsów, konwertowanie itp itd
http://calibre-ebook.com/

free en-pl dictionary
/
darmowy sł ang-pl
http://www.bumato.pl/

pl website with news about ebook readers, ebooks and everything related
/
newsy o czytnikach darmowych książkach i promocjach itp
http://swiatczytnikow.pl/

free and legit EN & PL books
/
darmowe i legalne EN i PL książki
http://wolnelektury.pl/
http://www.gutenberg.org/
http://archive.org/details/texts
http://www.amazon.com/s/?node=2245146011
http://ksiazki.pl/e-book-za-0-zl/
http://manybooks.net/
http://www.mobipocket.com/freebooks/Default.aspx
http://www.ebook.pl/category.php?id_category=1&mode=free
http://openlibrary.org/
http://www.openculture.com/free_ebooks

FR books
/
FR książki
http://www.ebooksgratuits.com/
http://beq.ebooksgratuits.com/
http://www.livrespourtous.com/
http://www.amazon.fr/b?ie=UTF8&node=927173031

EN & FR ebooks
http://www.ebouquin.fr/2010/02/27/ou-trouver-des-livres-electroniques/
http://pix-geeks.com/geeks/ebook-gratuit/

Magic catalogs
/
“Magiczne” katalogi:
Katalogi legalnych i darmowych książek na kidnle’a, które można ściągnąć bezpośrednio na urządzenie, bez potrzeby kopiowania na urządzenie czy wysyłania na maila itp.
Wystarczy je zapisać na urządzenie w katalogu documents, potem właczyć wifi, otworzyć wybrany katalog, wybrać sobie jakąś pozycję do czytania, potwierdzić chęć ściągnięcia i ta-dam, książka jest już dostępna na ekranie głównym 🙂
książki po PL – Magiczny katalog bookini.pl
http://wuub.net/mkb2.mobi
Książki głównie po Ang – The Magic Catalog of Project Gutenberg E-Books:
http://www.freekindlebooks.org/MagicCatalog/MagicCatalog.mobi

How to generate a cURL command out of SoapUI Pro REST request

After few attempts I finally made it working properly 🙂

To make use it, simply add a new script assertion to your REST request, then paste those two lines:

import com.yelllabs.soapy.helpers.CurlGenerator;
new CurlGenerator(context, messageExchange, log);

and check what will come up in the Script log tab 🙂
A nicely formatted cURL command with all the query parameters, headers and msg body.

ps. of course before that create a CurlGenerator.groovy file in {script_bibrary}/com/yelllabs/soapy/helpers/ directory 🙂

package com.yelllabs.soapy.helpers;

import com.eviware.soapui.impl.wsdl.teststeps.RestResponseMessageExchange;
import com.eviware.soapui.impl.wsdl.testcase.WsdlTestRunContext;
import com.eviware.soapui.support.types.StringToStringsMap;
import org.apache.log4j.Logger;


/**
* @brief can create cURL command out of REST request.
* @author Janusz Kowalczyk
* @created 2012-07-04
* @updated 2012-11-14 Janusz Kowalczyk
*/
class CurlGenerator{

    private def context;
    private def messageExchange;
    private def request;
    private Logger log;
    private String testCaseName;
    private String reqHttpMethod;
    private String reqContent;
    private String reqContext;
    private String reqEndpoint;
    private byte[] reqRawData;
    private StringToStringsMap reqHeaders;
    private def reqParams;

    /**
    * @brief Default constructor
    *
    * @param context - test run context
    * @param messageExchange - message exchange availabe in a assertion script
    * @param log - Logger
    * @param curlParams - cURL params that will be placed at the begining of cmd
    */
    CurlGenerator( WsdlTestRunContext context, RestResponseMessageExchange messageExchange, Logger log, String curlParams ) {
        this.context = context;
        this.messageExchange = messageExchange;
        this.log = log;

        if ( this.context.getCurrentStep().isDisabled() == false ) {
            init();
            log.info getCurlCmd(curlParams);
        }
    }


    /**
    * @brief Another constructor, this one will use "-ki" as default cURL params
    *
    */
    CurlGenerator( WsdlTestRunContext context, RestResponseMessageExchange messageExchange, Logger log) {
        this(context, messageExchange, log, "ki");
    }


    /**
    * @brief Init method that gets all the needed data from context and msgExchng
    *
    */
    private void init() {
        this.request = messageExchange.getRestRequest();
        this.testCaseName = this.context.getCurrentStep().getLabel();
        this.reqHttpMethod = this.request.getMethod();
        
        // gets the req body if there's one
        // if not the an empty List is returned
        this.reqContent = (this.request.hasRequestBody()) ? this.context.expand(this.request.getRequestContent()) : ""; 

        this.reqParams = this.request.getProperties();

        this.reqContext = this.context.expand(this.request.getResource().getFullPath());
        this.reqEndpoint = this.context.expand(this.request.getEndpoint());

        // get request headers
        // works only when request is made manually for the second time, in other case it will be always [:]
        this.reqHeaders = (request.getResponse() != null) ? request.getResponse().getRequestHeaders() : [:];
        this.reqRawData = (request.getResponse() != null) ? request.getResponse().getRawRequestData() : [];
    }


    /**
     * @brief Will use "-ki" as default curl parameters
     * 
     * @return a cURL command with "-ki" set as default parameters
     */
    public String getCurlCmd() {
        return getCurl("ki");
    }


    /**
     * @brief 
     * 
     * @param String curlParams - Pass the curl parameters. If "" is used then no params will be added to the command line
     *
     * @return A cURL command with custom parameters
     */
    public String getCurlCmd(String curlParams) {
        return getCurl(curlParams);
    }


    /**
    * @brief Will create a cURL command with provided params
    *
    * @param curlParams custom cURL params
    *
    * @return a cURL command with custom params
    */
    private String getCurl(String curlParams)
    {
        String params = ( curlParams.isEmpty() ) ? "" : "-" + curlParams;
        String cmd = "curl %s %s \"%s%s\" %s %s";
        return String.format(cmd, params, getMethod(), getUri(), getParams(), getHeaders(), getContent());
    }


    public byte[] getRawRequestData(){
        return this.reqRawData;
    }

    /**
    * @brief return a list of maps of sent non-empty query parameters
    *
    * @return return a list of maps of sent non-empty query parameters
    */
    private List getSentParams(){
        List params = [];
        this.reqParams.each{
            k,v ->
                if ( !v.getValue().isEmpty() )  {
                    def param = [ "name": v.getName(), "val" :  context.expand( v.getValue() ) ]
                    params.push( param )
            }
        }
        return params;
    }

    /**
    * @brief Converts list of query params into a nicely formatted string
    *
    * @return A string representing all the query params
    */
    private String getParams(){
        String qp = "";
        if ( false == getSentParams().isEmpty() ) {
            getSentParams().each{
                p ->
                // insert ? when processing first param, else insert & 
                    qp += ( qp  == "" ) ? "?" + p.name + "=" + p.val : "&" + p.name + "=" + p.val;
            }
        }
        return qp;
    }

    private String getMethod(){
        return "-X" + reqHttpMethod;
    }

    private String getUri() {
        if ( this.reqContext.toString().toLowerCase().contains( this.reqEndpoint.toString().toLowerCase() ) ){
            return this.reqContext;
        } else {
            return this.reqEndpoint + reqContext;
        }
    }

    private String getHeaders() {
        if ( this.reqHeaders.isEmpty()) {
            return "";
        } else {
            String H = "";
            reqHeaders.each{
                key, val ->
                    H += ' -H "' +key + ':' + val[0] +'"'
            }
            return H;
        }
    }

    private String getContent() {
        if ( this.reqContent.isEmpty() ) {
            return "";
        } else {
            return "-d '" + this.reqContent+"'";
        }
    } 

}//end