Logging Messages using Scribe and PHP

What's Scribe

Scribe is developed and open sourced by facebook. Its their internal logging system, capable of logging 10s of billions of messages per day. These messages include access logs, performance statistics, actions and many others. Scribe is a server for aggregating log data that's streamed in real time from clients. It is designed to be scalable and reliable. It is designed to scale to a very large number of nodes and be robust to network and node failures. There is a scribe server running on every node in the system, configured to aggregate messages and send them to a central scribe server (or servers) in larger groups. If the central scribe server isn't available the local scribe server writes the messages to a file on local disk and sends them when the central server recovers. The central scribe server(s) can write the messages to the files that are their final destination, typically on an nfs filer or a distributed file system, or send them to another layer of scribe servers.

To know more about scribe visit scribe on github

Install Scribe:

Prerequisite:

--libevent, Event Notification library
--boost, Boost C++ library (version 1.36 or later)
Note: For latest scribe version 2.2 you need to install boost library version 1.45 or lower.
scribe 2.2 is not compatible with boost 1.46 or higher.
you can download boost 1.45 from here.
--thrift, version 0.5.0 or later
--fb303, Facebook Bassline (included in thrift/contrib/fb303/) r697294 or later is required.

You can download latest version of scribe from here.

Steps to install:

1. Install libevent if not already installed
2. Install boost version between 1.36 to 1.45
3. If boost is installed in a non-default location or there are multiple boost versions installed, you will need to set the Boost path and library names
export BOOST_ROOT=/opt/boost_1_45_1
export LD_LIBRARY_PATH=/opt/boost_1_45_1/stage/lib
4. Install thrift
5. Install fb303
6. Now install scribe
untar the source and run
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
./bootstrap.sh
make
make install

A sample scribe config file:

#This file configures Scribe to listen for messages on port 1463 and write them to /var/log/scribelogs

port=1463
max_msg_per_second=2000000
check_interval=3

# DEFAULT
<store>
category=default
type=buffer

target_write_size=20480
max_write_interval=1
buffer_send_rate=2
retry_interval=30
retry_interval_range=10

<primary>
type=file
fs_type=std
file_path=/var/log/scribelogs
base_filename=thisisoverwritten
max_size=1000000
add_newlines=1
</primary>

<secondary>
type=file
fs_type=std
file_path=/tmp
base_filename=thisisoverwritten
max_size=3000000
</secondary>
</store>
Please go through this README file to learn more advance scribe config options.

Running a scribe server:

run the follwoing command to run scribe server

path_to_scribe_dir/src/scribed path_to_config_file/config.conf


Write a Scribe Client in PHP:

In order to write a client in PHP, you will need the following:

• Thrift PHP library files
• Client libraries for scribe
• Client libraries for fb303

Step 0: Create a folder for your client files, for example /home/gaurav/scribephplibs

Step 1: Get Thrift PHP library files
Thrift php library files are bundled with thrift source, you can just copy them to your folder
cp -R path_to_thrift_source/lib/php/src /home/gaurav/scribephplibs

Step 2: Generate client library for scribe
thrift -o /home/gaurav/scribephplibs --gen php path_to_scribe_source/if/scribe.thrift

Step 3: Generate client library for fb303
thrift -o /home/gaurav/scribephplibs --gen php path_to_thrift_source/contrib/fb303/if/fb303.thrift

Step 4: By default Thrift will place the generated files in a folder named gen-php. But Scribe client will expect them in folder named packages, so we need to rename that folder.

mv /home/gaurav/scribephplibs/gen-php /home/gaurav/scribephplibs/packages

A sample PHP client script (ScribeLogger.php):

<?php

class ScribeLogger {

public static function Log($msg, $category){

// Set this to where you have the Scribe and Thrift libary files.
$GLOBALS['THRIFT_ROOT'] = '/home/gaurav/scribephplibs';

// Include all of the lib files we need.
require_once $GLOBALS['THRIFT_ROOT'].'/packages/scribe/scribe.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TSocket.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TFramedTransport.php';
require_once $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php';

// A message in Scribe is made up of two parts: the category (a key) and the actual message.
$message = array();
$message['category'] = $category;
$message['message'] = $msg;

// Create a new LogEntry instance to hold our message, then add it to a new $messages array (we can submit multiple messages at the same time if we want to).

$entry = new LogEntry($message);
$messages = array($entry);

$socket = new TSocket('localhost', 1463, true);
$transport = new TFramedTransport($socket);
$protocol = new TBinaryProtocol($transport, false, false);

$scribeClient = new scribeClient($protocol, $protocol);

try {
$transport->open();
$scribeClient->Log($messages);
$transport->close();
}catch (TException $e) {
echo $e->getMessage();
}
}
}

ScribeLogger::Log("This is a test message.", 'TEST');

?>

Running the script:

php ScribeLogger.php

Now go to the /var/log/scribelogs folder, there you will see a new folder created named TEST. Scribe creates a new folder for each category. Each of these folder may contain multiple files versioned like Category_0000, Category_0001 etc and one symlink Category_current pointing to the current file scribe is writing to for that particular category.

An Intro to Phing

What is Phing

Phing is a build system for php projects, like Ant is for java. Moreover phing is based on Apache Ant. Like any traditional build systems you can do a lot of things with Phing. Phing uses XML build files similar to ant build files. Phing is useful in cases where you have to write custom scripts for testing, packaging and deploying your application code. Phing provides a number of built in tasks or operational modules, and it also allows you to add custom tasks as per your need.

In short Phing provides the following features:

• Simple XML buildfiles
• Rich set of provided tasks
• Easily extendable via PHP classes
• Platform-independent: works on UNIX, Windows, MacOSX
• No required external dependencies

How Phing Works

Phing uses XML buildfiles, these buildfiles contain a description of the things to do. A buildfile contains one or more targets. These targets contain the actual commands to execute or perform (e.g. create a directory, delete a directory, copy a file from one directory to another, create a archive file etc.). In order to use Phing, you need to write a buildfile first and then you need to run phing, specifying the target in your buildfile that you want to execute.

$ phing [-f buildfile.xml] [target]

If name of a buildfile is not given, then Phing will search for a file name build.xml in the directory from where the Phing command is run. You can also define default target in your buildfile. In that case you can skip the target also while executing the phing command.

Installation

Phing requires PHP version 5.2 or above compiled with --with-libxml2 at minimum. For advanced usage of Phing you may need other libararies and softwares. For a detailed list of dependencies you can visit this page.

The easiest way to install phing is by using the PEAR installer. Just run the following commands from the command line:
$ pear channel-discover pear.phing.info
$ pear install phing/phing

A sample Phing build file:

Given below is a small simple build file that I used for one of my projects. I have added comments before each line or section to explain what is that doing.

<?xml version="1.0"  encoding="UTF-8" ?>

<!-- project is the root element, a buildfile shoud start with project tag after the document prolog  -->
<!-- name : name of the project -->
<!-- basedir : the base project directory -->
<!-- default : the default target to build if phing is executed without specifying any target -->

<project name="applyservice" basedir="." default="main">

    <!-- define some project properties like package name, build directory and source directory -->
    <property name="package"  value="${phing.project.name}" override="true" />
    <property name="builddir" value="./build/" override="true" />
    <property name="srcdir"   value="${project.basedir}" override="true" />

    <!-- You can define set of all files to include/exclude in the build, give fileset an id for later reference -->
    <fileset dir="${srcdir}" id="allfiles">
        <include name="**/*.php" />
        <include name="**/*.yml" />
        <exclude name="**/*.log" >
    </fileset>

    <!-- Target: prepare, this will create a build directory if not already there -->
    <target name="prepare" description="initial setup">
        <echo msg="Creating build directory....." />
        <mkdir dir="./build" />
    </target>

    <!-- Target: main, Its our default target as defined in project tag -->
    <!-- A target can depend on other targets. You can define dependencies using depends attribute -->
    <!-- Our main target is dependent on prepare target to complete first -->
    <!-- You can define multiple targets in depends attribute using a comma separated list like A, B, C -->

    <target name="main" description="main target" depends="prepare">
        <echo msg="Copying files to build directory..." />
        <!-- copy all source files from the fileset defined above to the build directory -->
        <copy todir="${builddir}">
            <fileset refid="allfiles" />
        </copy>
       
        <!-- create a archive file of all the files in build directory using gzip compression -->
        <echo msg="Creating archive from build..." />
        <tar destfile="./build/build.tar.gz" compression="gzip">
            <fileset dir="./build">
                <include name="**" />
            </fileset>
        </tar>

        <echo msg="Build completed successfully........" />
    </target>

    <!-- Traget: rebuild, this target is for rebuilding, its first deletes previously build files and creates a new one by calling main target. -->
    <target name="rebuild" description="rebuilds this package">
    <echo msg="Deleting old build directory........" />
        <delete dir="${builddir}" />
        <phingcall target="main" />
    </target>
</project>

For a detailed list of all available options. Please go through the Phing User Guide.

Understanding MySQL Persistent Connections with mysql_pconnect()

What is mysql_pconnect():

The main purpose of using mysql_pconnect() function is to maintain a persistent connection to the mysql server. A persistent connection is a connection that do not get closed even after the excecution of the script is over which opened the connection. Even function mysql_close() can't close a persisitent connection. In contrast a normal connection opened using mysql_connect() gets closed either by mysql_close() or at the end of the scripts execution.

Why mysql_pconnect():

In one word the answer is "Efficiency". Persistent connections are good if the overhead to create a link to your db server is high. This overhead may be high due to various reasons. Persistent connections can help you considerably if the connection overhead is high.

How mysql_pconnect() works:

The most popular method to run PHP is to run it as a module in a multiprocess web server like Apache. A multiprocess server typically has one parent process which coordinates with a set of child processes. These child processes actually do the work of serving up web pages. When a request comes in from a client, it is handed over to one of the children that is free. This means that when the same client makes a second request to the server, it may be served by a different child process than the first time.

When opening a persistent connection, the function would first try to find a persistent link that is already open with the same host, username and password combination. If one is found, instead of opening a new connection an identifier for it will be returned. It causes the child process to simply connect only once for its entire lifespan, instead of every time it processes a page that requires connecting to the same db server. Every child that opened a persistent connection will have its own open persistent connection to the db server. For example, if you had 20 different child processes that ran a script that made a persistent connection to your db server, you would have 20 different connections to the db server, one from each child.

Issues with mysql_pconnect():

• Persistent database connections don't necessarily reflect subsequent privilege changes.
• Be very careful when using persistent connections and temporary tables on MySQL. With normal connections temporary tables are visible only to the current connection, but if you have a persistent connection the temporary tables will be visible to everybody sharing the same persistent connection. This can lead to major trouble.
• Don't use mysql_pconnect() in situations where multiple MySQL servers are running on multiple ports of the same host. The connection pooling algo in php apparently only checks for the host, username and password combination but not the port. Therefore, if you use the same host, username and password but a different port, you might get a connection that is connected to a different port than the one you asked for.
• Do not use transactions with persistent connections.  If your script stops or exits for any reason, your transaction will be left open and your locks will be left on.  You have to reset MySQL to release them. They won't rollback automatically on error, like they ought to. When you restart the script, you'll get a new connection, so you can't rollback or commit for the previous script.
• You should be very careful when using LOCK TABLES with persistent connections. If the script terminates before the UNLOCK TABLES is executed, the the table(s) will stay locked, and very likely hang future scripts.

Things to remember:

• Persistent connections were designed to have one-to-one mapping to regular connections. That means that you should always be able to replace persistent connections with non-persistent connections, and it won't change the way your script behaves. It may change the efficiency of the script, but not its behavior.
• Any script with a start transaction, rollback, or commit SQL statement should use regular, not persistent connections.
• Use totally random temporary table names when using persistent connections to avoid major problems.
• Make damn sure that max connections limit in your my.cnf has a limit with a few more connections than your httpd.conf has for number of apache children. Less MySQL connections than apache children means some apache children will be starved for db.
• Leave a few extra mysql connections, so that in case of a problem it can leave you with the ability to log in from shell to diagnose/fix it. Otherwise, you will have to bring down all of apache to get into your database.
• You can also use register_shutdown_function() to register a simple cleanup function to unlock your tables or roll back your transactions. But it's better to avoid the problem entirely by not using persistent connections in scripts which use table locks or transactions.
• Instead of use wait_timeout, you can set interactive_timeout to short period of time (for ex. 20 sec.) this is a lot better solution in apache + mysql environment than wait_timeout.

Some useful settings and plugins for VI Editor : Part 5 - Comment a code block

In the part 5 of this series, I am going to explain how to Comment/Uncomment a code block in VIM editor.

Method 1

Add folowing lines in your ~/.vimrc file

func! PhpUnComment() range
    let l:paste = &g:paste
    let &g:paste = 0

    let l:line        = a:firstline
    let l:endline     = a:lastline

    while l:line <= l:endline
        if getline (l:line) =~ '^\s*\/\/.*$'
            let l:newline = substitute (getline (l:line), '^\(\s*\)\/\/ \(.*\).*$', '\1\2', '')
        else
            let l:newline = substitute (getline (l:line), '^\(\s*\)\(.*\)$', '\1// \2', '')
        endif
        call setline (l:line, l:newline)
        let l:line = l:line + 1
    endwhile

    let &g:paste = l:paste
endfunc

vnoremap <buffer> <C-c> :call PhpUnComment()<CR>

Now In visual mode select the lines to comment/uncomment and press <CTRL-c>. This will comment all the uncommented lines, and uncomment all the commented lines.


Method 2

Using blockwise visual mode (CTRL-V) select the block to be commented.
Press I (capital i) and write the text you want to prepend to each line of the selected block (e.g. // or #). Then press ESC and the text will be inserted to the left of each line of the selected block.


For other posts related to VIM settings and plugins you can also visit

Some useful settings and plugins for VI Editor : Part 1 - General settings
Some useful settings and plugins for VI Editor : Part 2 - Autocompletion
Some useful settings and plugins for VI Editor : Part 3 - PHP documentor
Some useful settings and plugins for VI Editor : Part 4 - CodeSniffer Integration

Some useful settings and plugins for VI Editor : Part 4 - CodeSniffer Integration

In this part 4 of this series, I am going to discuss CodeSniffer and its integration with VIM editor.

What is CodeSniffer

CodeSniffer is a code analysis tool. This tool allows you to check your php code against a coding standard. It allows you to apply a set of rules (standard) to your source code. These rules can be used to detect common programming errors. It can also be used to define a set of coding standards for your project.

A coding standard in CodeSniffer is a collection of sniff files. Each sniff file checks one part of the coding standard only. CodeSniffer comes with a set of coding standards already defined. These are:

• MySource
• PEAR
• PHPCS
• Squiz
• Zend

The default coding standard used by CodeSniffer is the PEAR coding standard. By integrating CodeSniffer into vim you can get the list of violations in a separate error window.

How to install CodeSniffer

You can install CodeSniffer using pear.

$ sudo pear install PHP_CodeSniffer

This will download and install the CodeSniffer from the PEAR repository.

How to run CodeSniffer

To run CodeSniffer execute

$ phpcs --standard=<standard> <path to file or directory>

The output will be a list of errors and warnings in your code as per your coding standard. 

CodeSniffer Integration with VIM editor

Add folowing lines in your ~/.vimrc file

function! RunPhpcs()
    let l:filename=@%
    let l:phpcs_output=system('phpcs --report=full --standard=PEAR '.l:filename)
    let l:phpcs_list=split(l:phpcs_output, "\n")
    unlet l:phpcs_list[0]
    cexpr l:phpcs_list
    cwindow
endfunction

set errorformat+=\"%f\"\\,%l\\,%c\\,%t%*[a-zA-Z]\\,\"%m\"
command! Phpcs execute RunPhpcs()

Now you can run CodeSniffer for the current file using command

:Phpcs

After that run

:cope

This will open a window with a list of all the errors and warnings in your code as per your coding standard. Alternatively you can use quickfix also to navigate through the error window. For that see help quickfix in the vim help.


For other posts related to VIM settings and plugins you can also visit

Some useful settings and plugins for VI Editor : Part 1 - General settings
Some useful settings and plugins for VI Editor : Part 2 - Autocompletion
Some useful settings and plugins for VI Editor : Part 3 - PHP documentor
Some useful settings and plugins for VI Editor : Part 5 - Comment a code block

Some useful settings and plugins for VI Editor : Part 3 - PHP documentor

In the part 1 and part 2 I talked about some useful vim settings and how to enable autocompletion in vim. In this part I am going to talk about a very useful plugin written by Tobias Schlitt, for generating comment blocks or docblocks for php scripts.

We do document our code with proper inline comments. To make this task easier, Tobias Schlitt wrote a VIM plugin which automatically lookup some characteristics of the item you want to document and creates a docblock skeleton for it. This plugin provides functions to generate documentation blocks for your PHP code. The script currently documents:

- Classes
- Methods/Functions
- Attributes

This plugin supports PHP 4 and 5 syntax elements. It also allows you to define default values for phpDocumentor tags like @version, @author, @license and so on. For function/method parameters and attributes, the script tries to guess the type as good as possible from PHP5 type hints or default values (array, bool, int, string etc).

Steps to install this plugin

• Download the plugin file (php-doc.vim) from here.
• After downloading just place the php-doc.vim file in ~/.vim/plugin/ folder
• Add folowing lines in your ~/.vimrc file

        source ~/.vim/plugin/php-doc.vim
        inoremap <c-p> <esc>:call PhpDocSingle()<cr>i
        nnoremap <c-p> :call PhpDocSingle()<cr>
        vnoremap <c-p> :call PhpDocRange()<cr>

This includes the script and maps the combination <ctrl> + p to the doc functions.

How to use

Just hit <ctrl>+p on the line where the element to document resides and the doc block will be created directly above that line.

For other posts related to VIM settings and plugins you can also visit

Some useful settings and plugins for VI Editor : Part 1 - General settings
Some useful settings and plugins for VI Editor : Part 2 - Autocompletion
Some useful settings and plugins for VI Editor : Part 4 - CodeSniffer Integration
Some useful settings and plugins for VI Editor : Part 5 - Comment a code block

Using Thrift with Java and PHP

Thrift is a software framework for scalable cross-language services development. Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages.

This post provides a step by step guide to install thrift and write a server (in java) and a client (in php) using it.

1. Download Thrift

Basic requirements
Please go through this link for a list of prerequisite or basic requirements for thrift compiler.

Download the latest stable release of from here and extract it. OR do a svn checkout

$ svn co http://svn.apache.org/repos/asf/thrift/trunk thrift

2. Build and Install

Now go to the thrift directory and run

$ ./bootstrap.sh
$ ./configure
$ make
$ make install

this will install thrift on your system.

3. Writing a Thrift file

Next step is to write a thrift definition or .thrift file. This file describes the data structures, and functions available to your remote service. For this post I am going to write a simple service for getting a user profile.

profileservice.thrift

namespace php ProfileService #client
namespace java test.services.profile.thrift #server

enum JobType {
P, //Permanent
T //Temporary
}

enum EmploymentStatus {
F, //Full Time
P, //Part Time
}

exception ProfileServiceException {
1: i32 code,
2: string message
}

struct Profile {
1: i32 profileId,
2: string name,
3: string birthDate,
4: string contactAddress,
5: i32 cityId,
6: double totalExperience,
7: JobType jobType,
8: EmploymentStatus employmentStatus,
9: string summary,
}

service ProfileService {
Profile getProfileById(1:i32 profileId) throws (1: ProfileServiceException e),
Profile getProfileByName(1:string name) throws (1: ProfileServiceException e),
}

4. Using the Thrift Compiler

Now its time to generate the thrift code for server and client. For java server run the command

thrift --gen java profileservice.thrift

After you run the thrift generation for java, it’ll make a directory called gen-java/. Under this, you can find relevant files and classes to do work based on your Thrift definition. For my thrift its generated the following files under the directory gen-java/test/services/profile/thrift/ (its based on package name or namespace provided in the .thrift file)

$ ls gen-java/test/services/profile/thrift/
EmploymentStatus.java
JobType.java
Profile.java
ProfileServiceException.java
ProfileService.java

For php client run

thrift --gen php profileservice.thrift

for php, it’ll make a directory called gen-php/. For my thrift its generated the following files under the directory gen-php/profileservice/ (its based on package name or namespace provided in the .thrift file)

$ ls gen-php/profileservice/
ProfileService.php
profileservice_types.php

5. Creating a Thrift Server using Java

The next step is to create a java source file for implementing the interface (functions that we had defined in the profileservice.thrift file). The name of the interface is our case is ProfileService.Iface. We named the java class that implemented this interface in our case "ProfileServiceImpl". You will also need thrift java library for this. You can get lib/java/libthrift.jar file from your thrift source directory.

ProfileServiceImpl.java

package server;

import java.util.*;
import org.apache.thrift.*;
import test.services.profile.thrift.*;

class ProfileServiceImpl implements ProfileService.Iface
{
public Profile getProfileById(int profileId) throws ProfileServiceException, TException {
// your code goes here
return profile;
}

public Profile getProfileByName(String name) throws ProfileServiceException, TException {
// your code goes here
return profile;
}
}

Now write a java server for this service.

Server.java

package server;

import java.io.*;
import org.apache.thrift.protocol.*;
import org.apache.thrift.protocol.TBinaryProtocol.*;
import org.apache.thrift.server.*;
import org.apache.thrift.transport.*;
import test.services.profile.thrift.*;

public class Server
{
private void start()
{
try
{
TServerSocket serverTransport = new TServerSocket(7911);
ProfileService.Processor processor = new ProfileService.Processor(new ProfileServiceImpl());
Factory protFactory = new TBinaryProtocol.Factory(true, true);
TServer server = new TThreadPoolServer(processor, serverTransport, protFactory);
System.out.println("Starting server on port 7911 ...");
server.serve();
}catch(TTransportException e)
{
e.printStackTrace();
}
}

public static void main(String[] args)
{
Server srv = new Server();
srv.start();
}
}

This program simply has a main function which binds the service to a particular port and makes the server ready to accept connections and provide response. This code will generally remain constant unless you want to provide additional functionality at server level.

Compile all the files and start the server.

6. Creating a Thrift Client using PHP

Now its time to write a thrift client in php to use this service. You'll need to include the language specific libraries to facilitate access to thrift. Look for the folder ./lib/php/src/ in your thrift source directory which contains the library files you will need.

For this tutorial I have created a folder testclient in my home directory. Now create a subfoder named src-php, and copy all the library files in this folder. You will also need to mv or cp the autogenerated thrift files (from gen-php folder) for this project into the packages folder of these library files. Here’s a screenshot of my directorys structure for this project.

testclient
..src-php
....autoload.php
....ext
....packages
......profileservice
........ProfileService.php
........profileservice_types.php
....protocol
....server
....Thrift.php
....transport

Write a php client script to connect to the thrfit ProfileService server

ProfileServiceClient.php

// Setup the path to the thrift library folder
$GLOBALS['THRIFT_ROOT'] = 'thrift';
// Load up all the thrift stuff
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift.php';
require_once $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TSocket.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TBufferedTransport.php';

// Load the package that we autogenerated for this tutorial
require_once $GLOBALS['THRIFT_ROOT'].'/packages/profileservice/ProfileService.php';

try {
// Create a thrift connection
$socket = new TSocket('localhost', '9090');
$transport = new TBufferedTransport($socket);
$protocol = new TBinaryProtocol($transport);

// Create a profile service client
$client = new ProfileServiceClient($protocol);

// Open up the connection
$transport->open();
$data = $this->client->getProfileById(123);
$this->transport->close();
$this->socket->close();
print_r($data);
}
catch (TException $tx) {
// a general thrift exception
echo "ThriftException: ".$tx->getMessage()."\r\n";
}
?>

to run the client execute
php ProfileServiceClient.php

Php scripts : problem with whitespace

Recently while working on some project I faced a strange problem. In one of our app we were trying to implement and call a service using hessian. But somehow we were getting malformed reply for all the requests. We checked all the scripts and there were no problems in any of them. Finally, after a lot of efforts we got the problem. The problem was very simple, in one of the scripts there were some blank lines before the php start tag (<?php)

Leaving whitespaces (spaces/line breaks etc) before or after php scripts can be problematic and can result in unexpected or undesirable behaviour. Since these whitespaces will be echoed to the browser along with the normal output, It can break any script attempting to send headers (e.g. initializing session, sending content type, etc.) and can distort the web page layout.

Here is a simple php script that will scan php files in a given directory (and its subdirectories) and will remove all the whitespaces.

<?php
$maindir = "/path_to_the_project_dir";

define("PRE", "/^[\n\r|\r\n|\n|\r|\s]+<\?php/");
define("POST", "/\?>[\n\r|\r\n|\n|\r|\s]+$/");

clearstatcache();

if(scan_dir( $maindir, "removeWSpace", true ) === false)
{
echo "'{$maindir}' is not a valid directory\n";
}

function scan_dir( $maindir, $callback, $recursive = true )
{
$dh = @opendir( $maindir );
if( $dh === false)
return false;

while( $file = readdir( $dh ))
{
if( "." == $file || ".." == $file )
{
continue;
}
call_user_func( $callback, "{$maindir}/{$file}" );
if( $recursive !== false && is_dir( "{$maindir}/{$file}" ))
{
scan_dir( "{$maindir}/{$file}", $callback, $recursive );
}
}
closedir( $dh );
return true;
}

function removeWSpace( $path )
{
if( !is_dir( $path ) && substr($path, -4) == ".php")
{
$fh = file_get_contents($path);
$fh = preg_replace(PRE, ' 0 || $c2 > 0)
{
if(file_put_contents($path, $fh))
echo $path . " -- modified \n";
}
}
}

?>