Svilen Spasov's Dev Corner

XHProf 0.9.2 is out and ready for Mac OS X

2009-06-04T01:47:00.009+03:00

Version 0.9.2 of Facebook's XHProf PHP module is out!

Thanks to Kannan from Facebook's dev team the patch for OS X is now a part of the official release.

Go get it Mac users! It compiles on OS X perfectly.

Two weeks ago I wrote that I've made a patch for version 0.9.1 to build under OS X.

Then I decided to post a request on XHProf's bug track and supply the patch there.

Kannan (leader of the project) answered and we collaborated in the next week for applying the patch in the next official release of XHProf.

You can read more about the whole story on PECL.

Thank you, Kannan, for your time and efforts!

I am glad that version 0.9.2 has a port for Mac OS X and I was able to help for that.

Facebook's XHProf PHP module patch for Mac OS X Leopard Build

2009-05-17T23:33:00.018+03:00

*) note (05 Jun 2009): Thanks to Kannan from Facebook dev team the patch is now available as part of the official 0.9.2 release. You can download it from PECL.

Download full patched XHProf 0.9.1 for MAC OS X Leopard

or get the patch file only.

Bug submitted on PECL.

Full story

-----------

Yesterday (15 May 2009) I ran into XHProf module made by Facebook and published open source in March 2009.

Right after I read about its great features I was sure I had to try it and eventually put it on production servers. I was looking for such kind of PHP profiler with stats since 2005 when I found XDebug, which is really cool while developing but I wouldn't put it on a live server with high load like Tupalka.com's.

Eventually later that day our server hosting Tupalka.com was hit by a huge traffic and it was very loaded from time to time when the eAccelerator caches expires and most of the heavy mysql queries have to be ran once per 15 mins. That force me to try the XHProf.

Yup, I am sure that the problem was in the queries then why I was going for the XHProf? Good question. Curiosity.

These very moments are the once that push me to find and try different and new tools for optimization of web application/web sites.

So I was desparate to put XHProf on the production server at that moment, but I wanted just to take a look at it on my laptop for 30minutes before going live.

And here starts my real story with XHProf.

I downloaded latest PHP 5.2.9 and copied XHProf/extension folder to php_source/ext, ran:

#rm ./configure; ./buildconf --force; ./configure --prefix=/usr/local/php --with-xhprof; make

and ... it did not compile. The make exited with:

php_source/ext/xhprof/xhprof.c:202: error: syntax error before 'cpu_set_t'

php_source/ext/xhprof/xhprof.c:202: warning: no semicolon at end of struct or union

php_source/ext/xhprof/xhprof.c:213: error: syntax error before '}' token

php_source/ext/xhprof/xhprof.c:213: warning: data definition has no type or storage class

php_source/ext/xhprof/xhprof.c:222: error: syntax error before 'hp_globals'

php_source/ext/xhprof/xhprof.c:222: warning: data definition has no type or storage class

"Ooohh, gush" I thought and I opened xhprof.c

It's kinda miss-reading the docs on my side, cause I opened then and saw:

"Note: A windows port hasn't been implemented yet. We have tested xhprof on Linux/FreeBSD so far.

Note: XHProf uses the RDTSC instruction (time stamp counter) to implement a really low overhead timer for elapsed time. So at the moment xhprof only works on x86 architecture. Also, since RDTSC values may not be synchronized across CPUs, xhprof binds the program to a single CPU during the profiling period."

Ahh, they haven't tested the module on MAC OS X. And why should they? All of their servers run on Linux or FreeBSD. It's clear.

Ok, now I have to look and see if I can patch it.

(Oh, I forgot to tell that meanwhile I decided to change eAccelerator's cache expire on the loaded server to 2 hours, so load problem was resolved for a day or two and I had time to look at the XHProf)

Clearly XHProf documentation says that the module has to be bind to a single CPU and I knew that the "Thread Affinity" is implemented in a little bit different way in MAC OS X than in Linux or FreeBSD. Maybe that was the problem?

Yup, it is. Facebook team have made a patch for the affinity function in FreeBSD, but that's it. Now I have to add a similar solution for OS X.

I started to read more about thread affinity in Leopard and I found this document, which explains Apple's affinity API. Then after reading again and again, looking for some code snippets over the net I got the solution:

on line 30 to 49 replace with this:

#ifdef __FreeBSD__

# if __FreeBSD_version >= 700110

# include

# define cpu_set_t cpuset_t

# define SET_AFFINITY(pid, size, mask) \

cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_TID, -1, size, mask)

# define GET_AFFINITY(pid, size, mask) \

cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_TID, -1, size, mask)

# else

# error "This version of FreeBSD does not support cpusets"

# endif /* __FreeBSD_version */

#elif __APPLE__

# include

# define cpu_set_t thread_affinity_policy_data_t

# define CPU_SET(cpu_id, new_mask) \

*new_mask.affinity_tag = (cpu_id + 1)

# define CPU_ZERO(new_mask) \

*new_mask.affinity_tag == THREAD_AFFINITY_TAG_NULL

# define SET_AFFINITY(pid, size, mask) \

thread_policy_set(mach_thread_self(), THREAD_AFFINITY_POLICY, mask, THREAD_AFFINITY_POLICY_COUNT)

# define GET_AFFINITY(pid, size, mask) \

thread_policy_get(mach_thread_self(), THREAD_AFFINITY_POLICY, mask, THREAD_AFFINITY_POLICY_COUNT)

#else

/* To enable CPU_ZERO and CPU_SET, etc. */

# define __USE_GNU

/* For sched_getaffinity, sched_setaffinity */

# include

# define SET_AFFINITY(pid, size, mask) sched_setaffinity(0, size, mask)

# define GET_AFFINITY(pid, size, mask) sched_getaffinity(0, size, mask)

#endif /* __FreeBSD__ */

and on line 375 to 378 replace with:

#ifndef __APPLE__

if (GET_AFFINITY(0, sizeof(cpu_set_t), &hp_globals.prev_mask) <>

perror("getaffinity");

return FAILURE;

}

#endif

You can download whole the patched version of XHProf 0.9.1 from here.

Or get only the diff patch.

After successful compile and run I think it works - sharing the same L2 cache for a given thread when running on multiple cores, but NOT on different CPUs!

At least you can compile and see what's this module and implement it into your project. Then on the live server running Linux/FreeBSD you will definitely have correct data comming from XHProf.

If you find some errors or mistakes please let me know.

I also submitted a bug on the topic on PECL.

Thanks in advance for your oppinion and suggestions.

Tupalka.com (aka databg.svilen.com)

2009-05-13T15:44:00.004+03:00

Few weeks ago we launched new design of Tupalka.com along with new features and the whole core rewritten and moved to SFrameWork.

SFrameWork is my own PHP framework born in 2006 (some internal classes date from 2005). It is fully MVC and I am thinking of publishing a open source project at SorceFourge. Some of you may find it useful.

Some of the new key features of Tupalka.com

- Implemented Sphinx search engine (it's very very fast and efficient compared to MySQL build-in fulltext index)
- New design
- New statistics: most popular searches, most recent searches, random seaches
- Added new data sources dir.bg, youtube.com along with the old rock solid one - data.bg
- Truly enabled cyrillic support for search queries
- Sort by list columns (just click on column name, e.g. filename, filesize, date)
- Categories trasferred to tabs

I hope these will help you for better experience while using Tupalka.com

And I would like to thank to all the people involved in some way in this new release of Tupalka.com - Georgi, Evgeni, Nikolay, Marian, Dimitar (Co-founder and owner of the domain name)

Lighttpd the Book

2009-05-06T20:19:00.007+03:00

Following my post "Using mod_negotiation with Lighttpd" from April 23, 2009

I received a nice email from Packt Publishing who has published a new book about Lighttpd ("Lighttpd" written by Andre Bogus) asking me to review it.

I said - "ok, I will be glad to read your book."

And the story goes on...

Fortunately, I was nicely surprised that it deeply describes not only the Lighty 1.4.x, but the author has focused on 1.5.0, too. As some of you may know there are a lot of improvements in Lighttpd v.1.5.0, a lot of re-coded stuff, my beloved mod_uploadprogress was added again, etc., which is very essential in production servers.

Good. Now let's read on.

What's the first question come into your mind?

Maybe - Why Lighttpd? Huh?

Simple - cause it's fast, light, flexible and FREE. :)

Just look at the sample chapter of the book and you will see what I am talking about.

First few chapters are standard:

Introduction, Configuring & Compiling (I must say that it's very very well explained even for a beginner in UNIX systems), Virtual Hosts, CGI ...

oh wait - CGI? Isn't it slow?

Heh, yup CGI is a bit slow, but FastCGI is really fast! That's the key thing here - a whole chapter about FastCGI & SCGI support and how you should use it and configure in production environment.

Next is - "Optimizing Lighttpd".

Believe or not this is one of the most important things Andre (the author) has written about in his book.

How we can test our Lighttpd setup under stress with http_load (which I use, too) and how we can make some optimizations.

For example a simple mod_magnet caching script can deal easily with lots of load. And of course - mesuring system load is very handy here.

Are you Migrating from Apache?

Ok, then look at chapter 10 when you have a chance or here.

And now my favourite - "Using Lua with Lighttpd" (of course it means "using mod_magnet in Lighttpd").

This is one of the powers of Lighty where it leaves in the dust the good old Apache.

Lua is lightweight, embeddable language nevertheless it is very powerful. Combined with a lightweight and poweful webserver makes the unstoppable "killing" machine for huge amounts of web traffic.

Learn how to manipulate URLs, requested file names, caches, how to simulate Apache'с modules, etc.

This is a MUST READ topic!

And finally "Writing custom modules/plugins for the Lighttpd API" is for you C lovers.

Yes, that's right. A simple explanation how you can write a strong, stable module for Lighty in C.

It's pretty simple, although keep in mind this quote taken from the book:

"Do not write a module where a mod_magnetscript will do

A mod_magnetscript is usually smaller and much easier to write. The

higher level of abstraction leaves less chances of making mistakes. C,

unlike Lua, leaves the memory handling to the programmer. Also the

standard string-handling functions open security holes when used

improperly. Even if we are hell-bent on writing a module, it makes sense

to write a prototype mod_magnetscript first, presuming, of course, that

the function is possible to implement in Lua."

I cannot say anything else but Congratulations to Andre Bogus and Packt Publishing for this great book!

I recommend you to read it if you're going to deal with heavy load and big websites with lots of users. Also newbies can learn how to test their websites/web application and configure/compile a webserver by themselves.

Go get it!

Using mod_negotiation with Lighttpd

2009-04-23T12:20:00.002+03:00

Yesterday we finally decided to switch our web servers to Lighttpd 1.5.0

Everything went fine on the test environments until I ran into a problem with Lighty and Apache's mod_negotiation that we are using on some websites.

It turned out that there is no easy solution for a technique like mod_negotiation in Lighttpd, but fortunately Lighty has one very powerful module called Mod_Magnet.

With Mod_Magnet you actually can write your our LUA scripts for use with Lighttpd.

I was eager to try it. And here it comes my first LUA script emulating mod_negotiation and rewriting to a predefined file if nothing was found while "negotiating".

Use it like this:

# set document root for domains

$HTTP["host"] =~ "^(.*)(domain\.com)$" {

magnet.attract-physical-path-to = ( "/usr/local/lighttpd/etc/lighttpd-multiviews-to-php+rewrite.lua" ) # multiviews module

}

To compile Lighty with LUA support use:

./configure --prefix=/usr/local/lighttpd --with-lua

but before that you should install LUA 5.1

Bulding Lighttpd 1.5.0 r2914 on MAC OS X Leopard

2009-04-18T10:38:00.008+03:00

I haven't blogger for almost a year and it's time to start sharing my experience from the past months again.

Last week I went into trouble trying to build/compile Lighttpd 1.5.0 (latest prerelease r2914) on Mac OS X for test purposes.

One of the major features of Lighty 1.5.0 is the mod_uploadprogress, but unfortunately it is avaialable only in 1.5.0 version for now. So I was desparate to build it. :) (see mod_uploadprogress in action here)

What happened. After downloading the r2914 I ran the configure script with:

./configure --with-zlib --with-bzip2 --prefix=/usr/local/lighttpd --with-lua --sysconfdir=/usr/local/lighttpd/etc

(oh, yeah I wanted mod_magnet, too, to emulate Apache's mod_negotiation, so I added --with-lua)

All went ok with the configure, but then the make exits with:

Undefined symbols:

"_network_get_backends", referenced from:

_mod_status_handle_server_config in mod_status.o

"_network_get_backend_info_by_type", referenced from:

_mod_status_handle_server_config in mod_status.o

ld: symbol(s) not found

collect2: ld returned 1 exit status

make[4]: *** [mod_status.la] Error 1

make[3]: *** [all-recursive] Error 1

make[2]: *** [all] Error 2

make[1]: *** [all-recursive] Error 1

make: *** [all] Error 2

Ok, so now what? :) Of course, googling, googling and finally I got it! Hurray! :)

A patch was released on 07/08/2008 (download it directly from here), but why it is not applied in the official prerelease? Hm.. I hope the guys from Lighttpd's team will fix this in future.

Building Lighttpd 1.5.0 r2419 and later with applied patch for Mac OS X platform:

(directions for users that are not so much into command line tools)

1) Download and install XCode (if you already have it skip this step)

2) Download Lighttpd 1.5.0 latest prerelease

3) Extract Lighty

4) Download patch and copy it to Lighty's folder

5) Run following commands inside Lighty's folder to apply the patch:

patch -p1 < 0001-fix-build-on-mac-os-x.patch

aclocal

autoheader

glibtoolize --automake --force

automake --add-missing --force

autoconf

6) Configure Lighty with:

./configure --with-zlib --with-bzip2 --prefix=/usr/local/lighttpd

if you need mod_magnet install LUA 5.1 with fink (command: sudo fink install lua51) and configure with:

./configure --with-zlib --with-bzip2 --prefix=/usr/local/lighttpd --with-lua --sysconfdir=/usr/local/lighttpd/etc

7) Build Lighty:

make

8) Install Lighty:

sudo make install

9) Now you have Lighttpd 1.5.0 installed into /usr/local/lighttpd

10) Enjoy!

In an absolute world

2008-06-13T14:45:00.002+03:00

It's the first and last time I am playing blogger games ever and it's because Petya challenged me.

Hmm... let me see what should an absolute world be.

- only MAC computers available

- no managers/team leaders with unworkable requests all the time

- women with no timeframe restrictions for having sex

- free Mazda RX-8 with a full tank on every corder

- unrestricted joint and coffee shops (Deen Haag's affect ... )

And now it's my turn - Evgeni, Genadi show me your worlds. :)

SWFUpload & Sessions

2008-05-15T11:40:00.004+03:00

Have you tried SWFUpload?

It's cool flash uploader which is ideal for uploading big files (over 5-10MB) and having visual status about how much it's left.

Everything seems fine from the demos, etc. but! Yeah there is one clause that I figured out by myself while trying to use the tool in Drupal.

If you want to use the SWFUpload in restricted areas with scripts that require sessions (e.g. cookie sessions) you will fall into the hole.

For solving the issue add session read from $_GET also cause under Safari/Firefox/MAC OS X; Firefox/Linux the SWFUploader won't pass any cookies the the "upload.php" script.

Example solution for drupal:

if (isset($_GET['fmauthsessid']) {

if ($_GET['fmauthsessid']) {

$_COOKIE[session_name()] = $_GET['fmauthsessid'];

$key = $_GET['fmauthsessid'];

}

else {

if (!isset($_COOKIE[session_name()])) {

$user = drupal_anonymous_user();

return '';

}

Put this in the beginning of sess_read() in includes/session.inc

More experience lessons will be comming these days.

Wanna send email to Hotmail.com from PHP mail()? SUX...

2008-03-12T13:42:00.004+02:00

Hello everyone,

I know you've noticed that I am not quite often blogging, but I am and will try to post interesting tech stuff here.

Today's topic: Sending email to Hotmail account from PHP.

Sounds easy. Some of will say "just use PHP's mail() function with correct headers". Yeah, but not. :)

I read almost all posts on this topic around the net with no success. So finally I started trying and trying ... after the 5th or 6th hour (God bless I started in the morning of the day else I would probably throw my MacBook out of the window) I managed to send emails to Hotmail successfully!

Hints:

1) Do NOT put any "a href" tags in the email

2) Put ONLY email address in the TO: and FROM: headers of the email

If you do one of the above your email won't even be in the Junk!

3) Do NOT send email from address like: no_reply@domain.com or noreply@domain.com

or it will go to Junk! folders in Hotmail accounts

That's it! Enjoy! :)

Ouh, I forgot to mention that I wanted to send HTML emails. ;-)

Looking for Dedicated Hosting Provider?

2008-01-11T12:44:00.000+02:00

As many of you may know I do use dedicated server for 3 years by now. I started with MegaNetServe and I was satisfied till the previous year (2007) when they started to have frequently network downtimes with no reason or at least they didn't gave one.

Now I moved to Aplus and I can strongly recommend you them. Becides a value dedicated server you can get Plesk Control Panel with licence for 30 domains for no additional costs, which is great!

Plesk can save you a lot of time configuring the whole machine's services.

No, I haven't experienced a long term partnership with them by guys from MySQL Performance Blog are very pleased with Aplus for years.

Try them - www.aplus.net! ;-)

You will thank me later. :)

Why not Microsoft?

2007-12-20T10:28:00.000+02:00

It's simple. Because of this. :))

Very nice M$ car indeed.

PHP advice

2007-11-20T20:02:00.000+02:00

As a PHP developer some of the important things in my work are tools. Yeah, tools read more about them here.

Then you might want to take a look at these PHP optimization tips. Believe me you won't regret. :)

How to play .flac files in iTunes

2007-11-14T02:47:00.000+02:00

Everybody that knows me personally is aware that I am an Apple fan, expecially Mac OS X one.

I bought my first Mac laptop 2 years ago and an year ago I had a need to play FLAC music files on my laptop.

Yes it was possible, but as much as I searched for a good OS X music player I couldn't find one better than iTunes. That's why I needed iTunes to play FLAC files.

Finally I found it! :) Although Apple are not going to make it cause they want us to use their ALAC format.

So here's how you can make it work for you:

1. Exit all iTunes and Quicktime applications

2. Download xiph-qt-0.1.8.dmg, flac_import_0.5b1_p0.1.dmg, set-OggS-0.1.dmg

3. Drag XiphQT.component (xiph-qt-0.1.8.dmg) to /Library/Components/

4. Drag FLACImport.component (flac_import_0.5b1_p0.1.dmg) to /Library/QuickTime/

5. Drag Clear OggS.app and Set OggS.app (set-OggS-0.1.dmg) to your Applications folder

6. Drag you FLAC files and drop them on Set OggS.app. A window will popup to show that all FLAC files have been "converted".

7. Add your "converted" FLAC files to you iTunes

8. Enjoy! :)

Hope it will work for you, too. :)

The End of PHP4

2007-10-29T17:27:00.000+02:00

While thinking of installing a new PHP5 on a one of my new Windows servers (don't ask why Windows :) I went through the php.net's announcements "PHP 4 end of life announcement".

Zend Team will shut down PHP4's development - Finally! :)

Now I hope all hosting provider out there will switch their f*****g server to PHP5 and we all will bless them for this. :)

Go ahead!

За Б. Шопов :)

2007-10-05T22:04:00.000+03:00

Да започваме да, яицазимитпо лъгуг, говорим на,яицазимитпо лъгуг, марсиански. яицазимитпо лъгуг, край.

MySQL on road

2007-08-03T15:30:00.000+03:00

As I am on the MySQL topic these days I found (why I wasn't already...?!?) this mysql performance blog.

Once again I convinced myself that using MySQL in all kind of projects is good choice ... and read this post. It is clear that using different servers for different services, e.g. separate Web server from MySQL server, is a very very good decision and we will work on this for databg.svilen.com. Plus installing and using Sphinx full-text search engine.

Wish me luck. :)

Back To MySQL

2007-07-26T00:52:00.001+03:00

As I am working very hard this summer I decided to blog more from now on...

So while moving to the latest project I have in my task list I decided to code it back with MySQL as backend DB. I started converting all PostgreSQL tables/data and it turned out that it was a simple task. Time consuming but simple and I convinced myself one more time that simple is better e.g. MySQL is better. :)

What do you think?

Jobs vs. Gates

2007-02-21T12:58:00.000+02:00

Recently I bought a new MacBook. It's incredible. I admire Apple how they make their products. Those guys have thought almost of everything that man can want from their new notebook. :)

It has a remote control, too. So very much cool. :) I will post some photos tomorrow or so, but before that look at this funny video about Bill Gates & Steve Jobs :).

http://www.youtube.com/watch?v=qHO8l-Bd1O4

PHP & XML

2007-01-11T18:39:00.000+02:00

Have you ever tried using XML in your PHP applications?

It's not so easy task using PHP4. I definitely understood that yesterday when I tried to use the PHP's internal XML parser.
Why TF (The Fuck) it doesn't returns a multi-dimentional array build on the XML's data?
Insted of this PHP's XML parser returns an one-level array with all the tags. Gosh that's awful. :)

Finally I have to made my own class to handle XML files. I'm sick of sucking PHP software. :)

CC Fraud

2007-01-05T20:06:00.000+02:00

Finally, it catched me, too. :)

Yesterday while I was checking my online bank account I found out that there were some transactions from Microsoft Adcenter, which weren't mine!

Oh, god! Microsoft?

I've immediately called Microsoft Adcenter's Support and they admitted they fault.
"What the fuck?" will say one of my colleague. Is Microsoft credit card verification system so bad?
Or they don't have such at all. :) I guess it's the second reason.

At least how can someone use my credit card and they hadn't check him by phone before making a charge to the credit card.
Bad, bad for them.

Hope Google's Anti-Fraud System is better. I'm using Adwords. :)

Freelance Projects

2006-11-14T03:52:00.000+02:00

So working as a freelancer is not so ugly. :)
Finally I have enough free time for my friends and family.

These days I had to add new MSSQL db layer to my framework. All went fine and I did it in no time.
hm... wasn't thinking that using my frame will save me so much time.

Tomorrow I am going to code some project for google adwords api. Should be interesting. ;-)

My Framework

2006-10-25T12:02:00.000+03:00

As I decided to leave my ex-company I started to freelance.
Now I'm working on my own framework. I tried Zend's framework, but unfortunately it work only on PHP5 platforms and PDO enabled modules for DB access.

I need something universal. Some of the projects are hosted on PHP4 platforms that's why I can't use those different PHP5 frameworks that have come up these months.

Anyway, I always prefer to use my own code. :)

work on new payment system

2006-08-22T18:09:00.000+03:00

I am working on a new payment system for my current company.
It should work with multiple payment gateways (e.g. Worldpay, Paypal) and be easily extensible.

Till now I've coded a couple of PHP classes and all seems fine, although there are some hindrances.

Hope I will release it soon :)

http://databg.svilen.com fully automated

2006-08-22T17:59:00.000+03:00

After moving http://databg.svilen.com to a new server, collocated in Bulgaria, I started coding the automated crawl and update scripts.

Finally all is done and set. Now this search engine is working all alone, without any support from me. :)

Hope users will like it more and more since the database will be fully updated once a week.

Lucene Search

2006-08-14T13:33:00.000+03:00

Zend Framework has one great module called Zend Search Lucene, which I'm going to try for a new project :) :

"Zend_Search_Lucene is a general purpose text search engine written entirely in PHP 5. Since it stores its index on the filesystem and does not require a database server, it can add search capabilities to almost any PHP-driven website. Zend_Search_Lucene supports the following features:

Ranked searching - best results returned first

Many powerful query types: phrase queries, wildcard queries, proximity queries, range queries and more [5]

Search by specific field (e.g., title, author, contents)"