Some useful settings and plugins for VI Editor : Part 4 - CodeSniffer Integration

In this part 4 of this series, I am going to discuss CodeSniffer and its integration with VIM editor.

What is CodeSniffer

CodeSniffer is a code analysis tool. This tool allows you to check your php code against a coding standard. It allows you to apply a set of rules (standard) to your source code. These rules can be used to detect common programming errors. It can also be used to define a set of coding standards for your project.

A coding standard in CodeSniffer is a collection of sniff files. Each sniff file checks one part of the coding standard only. CodeSniffer comes with a set of coding standards already defined. These are:

• MySource
• PEAR
• PHPCS
• Squiz
• Zend

The default coding standard used by CodeSniffer is the PEAR coding standard. By integrating CodeSniffer into vim you can get the list of violations in a separate error window.

How to install CodeSniffer

You can install CodeSniffer using pear.

$ sudo pear install PHP_CodeSniffer

This will download and install the CodeSniffer from the PEAR repository.

How to run CodeSniffer

To run CodeSniffer execute

$ phpcs --standard=<standard> <path to file or directory>

The output will be a list of errors and warnings in your code as per your coding standard. 

CodeSniffer Integration with VIM editor

Add folowing lines in your ~/.vimrc file

function! RunPhpcs()
    let l:filename=@%
    let l:phpcs_output=system('phpcs --report=full --standard=PEAR '.l:filename)
    let l:phpcs_list=split(l:phpcs_output, "\n")
    unlet l:phpcs_list[0]
    cexpr l:phpcs_list
    cwindow
endfunction

set errorformat+=\"%f\"\\,%l\\,%c\\,%t%*[a-zA-Z]\\,\"%m\"
command! Phpcs execute RunPhpcs()

Now you can run CodeSniffer for the current file using command

:Phpcs

After that run

:cope

This will open a window with a list of all the errors and warnings in your code as per your coding standard. Alternatively you can use quickfix also to navigate through the error window. For that see help quickfix in the vim help.


For other posts related to VIM settings and plugins you can also visit

Some useful settings and plugins for VI Editor : Part 1 - General settings
Some useful settings and plugins for VI Editor : Part 2 - Autocompletion
Some useful settings and plugins for VI Editor : Part 3 - PHP documentor
Some useful settings and plugins for VI Editor : Part 5 - Comment a code block

Some useful settings and plugins for VI Editor : Part 3 - PHP documentor

In the part 1 and part 2 I talked about some useful vim settings and how to enable autocompletion in vim. In this part I am going to talk about a very useful plugin written by Tobias Schlitt, for generating comment blocks or docblocks for php scripts.

We do document our code with proper inline comments. To make this task easier, Tobias Schlitt wrote a VIM plugin which automatically lookup some characteristics of the item you want to document and creates a docblock skeleton for it. This plugin provides functions to generate documentation blocks for your PHP code. The script currently documents:

- Classes
- Methods/Functions
- Attributes

This plugin supports PHP 4 and 5 syntax elements. It also allows you to define default values for phpDocumentor tags like @version, @author, @license and so on. For function/method parameters and attributes, the script tries to guess the type as good as possible from PHP5 type hints or default values (array, bool, int, string etc).

Steps to install this plugin

• Download the plugin file (php-doc.vim) from here.
• After downloading just place the php-doc.vim file in ~/.vim/plugin/ folder
• Add folowing lines in your ~/.vimrc file

        source ~/.vim/plugin/php-doc.vim
        inoremap <c-p> <esc>:call PhpDocSingle()<cr>i
        nnoremap <c-p> :call PhpDocSingle()<cr>
        vnoremap <c-p> :call PhpDocRange()<cr>

This includes the script and maps the combination <ctrl> + p to the doc functions.

How to use

Just hit <ctrl>+p on the line where the element to document resides and the doc block will be created directly above that line.

For other posts related to VIM settings and plugins you can also visit

Some useful settings and plugins for VI Editor : Part 1 - General settings
Some useful settings and plugins for VI Editor : Part 2 - Autocompletion
Some useful settings and plugins for VI Editor : Part 4 - CodeSniffer Integration
Some useful settings and plugins for VI Editor : Part 5 - Comment a code block

Some useful settings and plugins for VI Editor : Part 2 - Autocompletion

In the first part I talked about some useful settings, In this part I am going to explain how to use Autocompletion with VI editor.

To set autocompletion on, add the following settings in .vimrc file in your home directory or alternatively you can add them to /etc/vim/vimrc file, that will enable these settings for all users on the system.

   
set ofu=syntaxcomplete

Vim has autocomplete functionality for all common web development contexts. Now, If you are editing a file in vim which ends with .php, .html, .css, .js, .sql, .rb, or .py. Vim's "omnifunc" feature combined with its built-in autocomplete feature will show autocomplete options specific to the corresponding language. You can type a few words/chars and can press (in insert mode) following commands to autocomplete words from different context, vim shows a box below the cursor containing the options, with the first entry highlighted:

CTRL-X_CTRL-O - To search matching words in coding language manual
CTRL-X_CTRL-L - To search matching words in whole lines
CTRL-X_CTRL-N - To search matching words in the current file
CTRL-X_CTRL-K - To search matching words in dictionary
CTRL-X_CTRL-I - To search matching words in the current and included files
CTRL-X_CTRL-F - To search matching words in file names
CTRL-X_CTRL-] - To search matching words in tags
CTRL-N        - To search matching words in all of above

Autocompletion using the TAB key

Using above commands are little difficult. For easy use we can remap them to TAB key. Add the following function to your vimrc file. This function determines, wether we are on the start of the line text (then tab indents) or if we want to try autocompletion

func! InsertTabWrapper()
    let col = col('.') - 1
    if !col || getline('.')[col - 1] !~ '\k'
        return "\<tab>"
    else
        return "\<c-p>"
    endif
endfunction

Remap the TAB key to select action with InsertTabWrapper, add the following line in vimrc file

inoremap <buffer> <tab> <c-r>=InsertTabWrapper()<cr>

Now, when you will press a TAB key, it will check wheather you are on the start of a line, if yes it will indent your code, otherwise it will try to show you autocompletion window.

For other posts related to VIM settings and plugins you can also visit

Some useful settings and plugins for VI Editor : Part 1 - General settings
Some useful settings and plugins for VI Editor : Part 3 - PHP documentor
Some useful settings and plugins for VI Editor : Part 4 - CodeSniffer Integration
Some useful settings and plugins for VI Editor : Part 5 - Comment a code block

Some useful settings and plugins for VI Editor : Part 1 - General settings

Here are some useful settings, commands and plugins to make VI editor more developer friendy (some settings are there only for PHP users).

You need to add the following settings in .vimrc file in your home directory or alternatively you can add them to /etc/vim/vimrc file, that will enable these settings for all users on the system.

"For highlighting the code add
 syntax on

"Show a ruler at the bottom of screen
  set ruler
  set laststatus=2

"Show matching brackets.
  set showmatch

"To do a case insesnitive search.
  set ignorecase

"To replace TAB with shift of 4 spaces.
  set tabstop=4
  set shiftwidth=4
  set expandtab

"Show line numbers.
 set number

"Jump 5 lines when running out of the screen
 set scrolljump=5

"Indicate jump out of the screen when 3 lines before end of the screen
 set scrolloff=3

"Set indentation rules
 setlocal autoindent
 setlocal smartindent

"Correct indentation after opening a docblock and automatic * on every line
  setlocal formatoptions=qroct

"Append ending brackets whenever open a bracket
 inoremap [ []
 inoremap ( ( )

"Spell Checking
 set spell spelllang=en_us
 Now use can move to next or previous misspelled word using ]s and [s commands, also you can use following
 zg   Add word under cursor as good word
 z=   Suggest corrections for the word under cursor

For other posts related to VIM settings and plugins you can also visit

Some useful settings and plugins for VI Editor : Part 2 - Autocompletion
Some useful settings and plugins for VI Editor : Part 3 - PHP documentor
Some useful settings and plugins for VI Editor : Part 4 - CodeSniffer Integration
Some useful settings and plugins for VI Editor : Part 5 - Comment a code block

MongoDB vs MySQL: speed test part 2, Select queries

In the first part of this post I compared the performance of Insert operations for mongoDB and mysql. In this part I tried to compare the performance for different select operations. Test setup is same as used in the part 1.

1. Selects on an indexed column with different limit clauses

To check the performance of selects on an indexed column with limit clause, a number of queries with different limit clauses were executed on both the databases.

sample mysql query:
SELECT ID, NAME, BIRTH_DT, CONTACT_ADDRESS, CITY, TOTAL_EXP, ENTRY_DT, PROFILE, SUMMARY from db.resume where ID > 1000 limit 100000, 1000

sample MongoDB query:
$collection->find(array('ID' => array(':gt'=>1000)))->skip(100000)->limit(1000);

Start Limit	Total Records Fetched	MySQL	MongoDB
0	1000	0.846 ms	0.0710ms
100000	1000	0.903 ms	0.0391ms
200000	1000	0.969 ms	0.0209ms
300000	1000	1.029 ms	0.0889ms
400000	1000	1.058 ms	0.0488ms
500000	1000	1.149 ms	0.0482ms
600000	1000	1.214 ms	0.0460ms
700000	1000	1.170 ms	0.0469ms
800000	1000	1.196 ms	0.0450ms
900000	1000	1.216 ms	0.0460ms

2. Selects on a non indexed column with different limit clauses

sample mysql query:
SELECT ID, NAME, BIRTH_DT, CONTACT_ADDRESS, CITY, TOTAL_EXP, ENTRY_DT, PROFILE, SUMMARY from db.resume where TOTAL_EXP > 5 limit 100000, 1000

sample MongoDB query:
$collection->find(array('TOTAL_EXP' => array(':gt'=>5)))->skip(100000)->limit(1000);

Start Limit	Total Records Fetched	MySQL	MongoDB
0	1000	1.133 ms	0.0679 ms
100000	1000	1.166 ms	0.0469 ms
200000	1000	1.334 ms	0.0469 ms
300000	1000	1.293 ms	0.0438 ms
400000	1000	2.047 ms	0.0450 ms

3. Selects on an indexed column with sorting

sample mysql query:

SELECT ID, NAME, BIRTH_DT, CONTACT_ADDRESS, CITY, TOTAL_EXP, ENTRY_DT, PROFILE, SUMMARY from db.resume where ID > 1000 order by USERNAME asc

sample MongoDB query:

$collection->find(array('ID' => array(':gt'=>1000)))->sort(array("USERNAME"=>1));

Avg time in Mysql : 1.973 sec
Avg time in MongoDB : 0.138 ms

4. Selects with IN clause on an Indexed Column

To check the performance of select queries on an indexed key with IN clause, a number of queries were executed on both the databases and an avearge is taken. Each query had 100 random ID values in the IN clause.

Avg time in Mysql : 4.865 ms
Avg time in MongoDB : 1.570 ms

Its clear from the above results that MongoDB outperformed mysql in each case by a large margin.

MongoDB vs MySQL: speed test part 1, Insert queries

Recently I started exploring NoSQL databases as an alternative for some of our high traffic mysql tables. After going through a number of articles on net, I decided to explore MongoDB. I tried to compare the performance of different database operations (inserts/ different types of selects) in mongoDB and in MySQL. Performance comparison of insert operations are given here.

Test setup:
For testing I used a 3.16 GHz, Intel Xeon CPU with 2 GB of memory and 350 GB of disk.

MySQL:

key_buffer = 128M
sort_buffer_size = 512K
read_buffer_size = 256K
max_allowed_packet = 1M

Table schema:

ID int(11)
NAME varchar(35)
BIRTH_DT date
CONTACT_ADDRESS varchar(150)
CITY int(11)
TOTAL_EXP varchar(5)
ENTRY_DT date
PROFILE varchar(250)
SUMMARY varchar(250)

MongoDB:

For mongo two shard servers, one config server and one mongos were satarted on the same machine with chunk size set to 10.

Sample document:

{"_id" : ObjectId("4ca6cca6a87305c90b000000"),
"ID" : "5839427",
"NAME" : "Gaurav Asthana",
"BIRTH_DT" : "1981-06-29",
"CONTACT_ADDRESS" : "Noida, India",
"CITY" : "19",
"TOTAL_EXP" : "06.10",
"ENTRY_DT" : "2010-11-26",
"PROFILE" : "zxzzzzz zzzzzzzzzz zzzzzzzzzzzzzzzzzz zzzzzzzz",
"SUMMARY" : "abcfsf fsdfs gdgdfg gdfgdh dfghdh dfhdh" }

An index is also created on ID field.

I have created a simple php script to perform the benchmark. This script inserted a total of 15 Lac records both in the mysql and mongodb. I have recorded time for each batch of 100 records that were inserted. So, in total I recorded 15000 readings. The average time taken by both the databases is given below.

Average time per batch of 100 records :

Mysql : 18.77 ms
MongoDB : 5.53 ms

Size on disk:

Mysql : 390 MB
MongoDB : 1.6 GB

In my benchmark, MongoDB came out three times faster that mysql in case of insert queries. But it occupied four times more disk than mysql.

Classification of NoSQL Databases

NoSQL databases can be broadly classified as:

1. Distributed vs. Not-distributed databases

Distributed databases take the responsibility of data partitioning (for scalability) and replication (for availability) and do not leave that to the client. Non-distributed databases leaves the responsibility of data partitioning and replication on the clients.

Table 1: Distributed and Non-distributed databases

Distributed	Not Distributed
Amazon Dynamo Amazon S3 Scalaris Voldemort CouchDb (thru Lounge) Riak MongoDb BigTable Cassandra HyperTable HBase	Redis Tokyo Tyrant MemcacheDb Amazon SimpleDb

2. Disk vs. Memory databases

An useful dimension is whether the database is memory-driven or disk-driven. This is important since in the latter case an explicit cache would be required, while in the former case data is not durable.

Table 2: Memory driven and disk driven databases

Memory	Configurable	Disk
Scalaris Redis	BigTable Cassandra Hbase HyperTable	CouchDb MongoDb Riak Voldemort

On one end of the spectrum is Scalaris which is entirely memory-driven, and Redis which is primarily memory oriented. Cassandra, BigTable, Hypertable, Hbase allow configuring how large the Memtable can get, so that provides a lot of control. CouchDb, MongoDb and Riak all use on-disk B+ trees, and Voldemort uses BDB and MySQL.

3. Data Model richness

On the basis of data model the various NoSQL databases can be grouped in following three groups.

3.1 Key-value Stores

These systems store values and an index to find them, based on a programmer-defined key. These data stores use a data model similar to the popular memcached distributed in-memory cache, with a single key-value index for all the data. Like memcached, none of these systems offer secondary indices or keys.

3.2 Document Stores

These systems store documents. The documents are indexed and a simple query mechanism may be provided. Document stores support more complex data than the key-value stores. The term “document store” is not ideal, because these systems store objects (generally objects without pointers, described in JSON notation), not necessarily documents. Unlike the key-value stores, they generally support multiple indexes and multiple types of documents (objects) per database, and they support complex values.

3.3 Column Stores

These systems store extensible records that can be partitioned across nodes. They are also refered as “Extensible Record Stores”. Their basic data model is rows and columns, and their basic scalability model is splitting both rows and columns over multiple nodes. Rows are split across nodes through conventional sharding, on the primary key. They typically split by range rather than a hash function (this means that queries on ranges of values do not have to go to every node). Columns of a table are distributed over multiple nodes by using “column groups”.

These may seem like a new complexity, but column groups are simply a way for the customer to indicate which columns are best grouped together. These two partitionings (horizontal and vertical) can be used simultaneously on the same table. The column groups must be pre-defined with the extensible record stores. However, that is not a big constraint, as new attributes can be defined at any time. Rows are not that dissimilar from documents: they can have a variable number of attributes (fields), the attribute names must be unique, rows are grouped into collections (tables), and an individual row’s attributes can be of any type.

Table 3: Classification of NoSQL databases based on data model

Key-Value store	Document store	Column-Store
Amazon Dynamo Amazon S3 Redis Scalaris Voldemort	SimpleDb Couchdb MongoDb Riak	Cassandra Google BigTable HBase Hyperbase

Multi Version Concurrency Control

Multi Version Concurrency Control or MVCC is an efficient method to let multiple processes access the same data in parallel without corrupting the data and the possibility of deadlocks. It is an alternative to the Lock based approaches, where every process first has to request an exclusive lock on a data item, before it can be read or updated. MVCC is used in some relational databases as well as in most distributed databases.

Instead of letting each process access the data exclusively for a certain amount of time, MVCC allows processes to read the data in parallel, even if a process is updating the data. To maintain consistency, each data item has some kind of time stamp or revision. If a process reads a data item, it does not only get the value of it, the process also retrieves the revision of the data item. So if this process attempts to update this data item, then it writes the new value with the previously read revision number to the database. If the actual revision in the store is the same, then the new value is written and the revision of the data item is incremented.

But if the revision in the store is not the same as the revision read by the writing process, then there must have been another process which has updated the data item in the meantime. In a relational database, the writing process would be a transaction, which would in this case be aborted or restarted.

In distributed databases, there are at least two cases for such a conflict: The first one is that two processes are attempting to write the same data item on the same node. In this case, the database could detect the conflict during the write operation and abort it, so the client would need to re-read the data item and retry its desired update, like the behavior of a RDBMS in this situation.

Another case is that multiple clients update the same data item on different nodes. If the distributed database uses asynchronous replication, then this conflict can not be detected and handled during the write operations. The nodes first have to be synchronized before they can handle the conflict. The conflict resolution can happen during the replication or during the first read operation on the conflicting data item.

Some databases, which implement this, store all conflicting revisions of the data items and let the client decide how the conflict should be handled. In such systems a read request returns all conflicting versions of the value and the client has to choose one or has to merge the versions and to write the corrected revision back to the database.

Using Thrift with Java and PHP

Thrift is a software framework for scalable cross-language services development. Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages.

This post provides a step by step guide to install thrift and write a server (in java) and a client (in php) using it.

1. Download Thrift

Basic requirements
Please go through this link for a list of prerequisite or basic requirements for thrift compiler.

Download the latest stable release of from here and extract it. OR do a svn checkout

$ svn co http://svn.apache.org/repos/asf/thrift/trunk thrift

2. Build and Install

Now go to the thrift directory and run

$ ./bootstrap.sh
$ ./configure
$ make
$ make install

this will install thrift on your system.

3. Writing a Thrift file

Next step is to write a thrift definition or .thrift file. This file describes the data structures, and functions available to your remote service. For this post I am going to write a simple service for getting a user profile.

profileservice.thrift

namespace php ProfileService #client
namespace java test.services.profile.thrift #server

enum JobType {
P, //Permanent
T //Temporary
}

enum EmploymentStatus {
F, //Full Time
P, //Part Time
}

exception ProfileServiceException {
1: i32 code,
2: string message
}

struct Profile {
1: i32 profileId,
2: string name,
3: string birthDate,
4: string contactAddress,
5: i32 cityId,
6: double totalExperience,
7: JobType jobType,
8: EmploymentStatus employmentStatus,
9: string summary,
}

service ProfileService {
Profile getProfileById(1:i32 profileId) throws (1: ProfileServiceException e),
Profile getProfileByName(1:string name) throws (1: ProfileServiceException e),
}

4. Using the Thrift Compiler

Now its time to generate the thrift code for server and client. For java server run the command

thrift --gen java profileservice.thrift

After you run the thrift generation for java, it’ll make a directory called gen-java/. Under this, you can find relevant files and classes to do work based on your Thrift definition. For my thrift its generated the following files under the directory gen-java/test/services/profile/thrift/ (its based on package name or namespace provided in the .thrift file)

$ ls gen-java/test/services/profile/thrift/
EmploymentStatus.java
JobType.java
Profile.java
ProfileServiceException.java
ProfileService.java

For php client run

thrift --gen php profileservice.thrift

for php, it’ll make a directory called gen-php/. For my thrift its generated the following files under the directory gen-php/profileservice/ (its based on package name or namespace provided in the .thrift file)

$ ls gen-php/profileservice/
ProfileService.php
profileservice_types.php

5. Creating a Thrift Server using Java

The next step is to create a java source file for implementing the interface (functions that we had defined in the profileservice.thrift file). The name of the interface is our case is ProfileService.Iface. We named the java class that implemented this interface in our case "ProfileServiceImpl". You will also need thrift java library for this. You can get lib/java/libthrift.jar file from your thrift source directory.

ProfileServiceImpl.java

package server;

import java.util.*;
import org.apache.thrift.*;
import test.services.profile.thrift.*;

class ProfileServiceImpl implements ProfileService.Iface
{
public Profile getProfileById(int profileId) throws ProfileServiceException, TException {
// your code goes here
return profile;
}

public Profile getProfileByName(String name) throws ProfileServiceException, TException {
// your code goes here
return profile;
}
}

Now write a java server for this service.

Server.java

package server;

import java.io.*;
import org.apache.thrift.protocol.*;
import org.apache.thrift.protocol.TBinaryProtocol.*;
import org.apache.thrift.server.*;
import org.apache.thrift.transport.*;
import test.services.profile.thrift.*;

public class Server
{
private void start()
{
try
{
TServerSocket serverTransport = new TServerSocket(7911);
ProfileService.Processor processor = new ProfileService.Processor(new ProfileServiceImpl());
Factory protFactory = new TBinaryProtocol.Factory(true, true);
TServer server = new TThreadPoolServer(processor, serverTransport, protFactory);
System.out.println("Starting server on port 7911 ...");
server.serve();
}catch(TTransportException e)
{
e.printStackTrace();
}
}

public static void main(String[] args)
{
Server srv = new Server();
srv.start();
}
}

This program simply has a main function which binds the service to a particular port and makes the server ready to accept connections and provide response. This code will generally remain constant unless you want to provide additional functionality at server level.

Compile all the files and start the server.

6. Creating a Thrift Client using PHP

Now its time to write a thrift client in php to use this service. You'll need to include the language specific libraries to facilitate access to thrift. Look for the folder ./lib/php/src/ in your thrift source directory which contains the library files you will need.

For this tutorial I have created a folder testclient in my home directory. Now create a subfoder named src-php, and copy all the library files in this folder. You will also need to mv or cp the autogenerated thrift files (from gen-php folder) for this project into the packages folder of these library files. Here’s a screenshot of my directorys structure for this project.

testclient
..src-php
....autoload.php
....ext
....packages
......profileservice
........ProfileService.php
........profileservice_types.php
....protocol
....server
....Thrift.php
....transport

Write a php client script to connect to the thrfit ProfileService server

ProfileServiceClient.php

// Setup the path to the thrift library folder
$GLOBALS['THRIFT_ROOT'] = 'thrift';
// Load up all the thrift stuff
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift.php';
require_once $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TSocket.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TBufferedTransport.php';

// Load the package that we autogenerated for this tutorial
require_once $GLOBALS['THRIFT_ROOT'].'/packages/profileservice/ProfileService.php';

try {
// Create a thrift connection
$socket = new TSocket('localhost', '9090');
$transport = new TBufferedTransport($socket);
$protocol = new TBinaryProtocol($transport);

// Create a profile service client
$client = new ProfileServiceClient($protocol);

// Open up the connection
$transport->open();
$data = $this->client->getProfileById(123);
$this->transport->close();
$this->socket->close();
print_r($data);
}
catch (TException $tx) {
// a general thrift exception
echo "ThriftException: ".$tx->getMessage()."\r\n";
}
?>

to run the client execute
php ProfileServiceClient.php

An Intro to NoSQL

What is NoSQL

For a quarter of a century, the relational database (RDBMS) has been the dominant model for database management. In the past, relation databases were used for nearly everything. Because of their rich set of features, query capabilities and transaction management they seemed to be fit for almost every possible task one could imagine to do with a database. But their feature richness is also their flaw, because it makes building distributed RDBMSs very complex. In particular it is difficult and not very efficient to make transactions and join operations in a distributed system.

This is why, there are now some non relational databases with limited feature sets and no full ACID support, which are more suitable for the usage in a distributed environment. These databases are currently called NoSQL databases. The need to look at Non SQL systems arises out of scalability issues with relational databases, which are a function of the fact that relational databases were not designed to be distributed (which is key to write scalability), and could thus afford to provide abstractions like ACID transactions and a rich high-level query model. All NoSQL databases try and address the scalability issue in many ways – by being distributed, by providing a simpler data / query model, by relaxing consistency requirements, etc.

The name first suggests that these databases do not support the SQL query language and are not relational. But it also means "Not Only SQL", which is not so aggressive against relational databases. This stands for a new paradigm: One database technology alone is not fit for everything. Instead it is necessary to have different kinds of databases for different demands. Most NoSQL databases are developed to run on clusters consisting of commodity computers and therefore have to be distributed and failure tolerant. To achieve this, they have to make different trade-offs regarding the ACID properties, transaction management, query capabilities and performance. They are usually designed to fit the requirements of most web services and most of them are schema free and bring their own query languages.

Why NoSQL

Even though RDBMS have provided database users with the best mix of simplicity, robustness, flexibility, performance, scalability, and compatibility, their performance in each of these areas is not necessarily better than that of an alternate solution pursuing one of these benefits in isolation. Today, the situation is slightly different. For an increasing number of applications, one of these benefits is becoming more and more critical; and while still considered a niche, it is rapidly becoming mainstream, so much so that for an increasing number of database users this requirement is beginning to eclipse others in importance. That benefit is scalability.

Relational databases scale well, but usually only when that scaling happens on a single server node. When the capacity of that single node is reached, you need to scale out and distribute that load across multiple server nodes. This is when the complexity of relational databases starts to rub against their potential to scale. Try scaling to hundreds or thousands of nodes, rather than a few, and the complexities become overwhelming, and the characteristics that make RDBMS so appealing drastically reduce their viability as platforms for large distributed systems.

Cloud computing also has placed new challenges on the database. The economic vision for cloud computing is to provide computing resources on demand with a "pay-as-you-go" model. A pool of computing resources can exploit economies of scale and a levelling of variable demand by adding or subtracting computing resources as workload demand changes. The traditional RDBMS has been unable to provide these types of elastic services. For cloud services to be viable, vendors have had to address this limitation, because a cloud platform without a scalable data store is not much of a platform at all. So, to provide customers with a scalable place to store application data, vendors had only one real option. They had to implement a new type of database system that focuses on scalability, at the expense of the other benefits that
come with relational databases.