Paul Beach's Blog

Firebird /Java Plugin Milestone One

2009-07-06T00:13:00.000-07:00

Posted by Adriano in Firebird Development:

"I'm pleased to announce availability [1] of the first test version of
FB/Java plugin.

FB/Java is a plugin for FB external engines feature, to be presented in
v3.0, that allows execution of Java code at the server side. The plugin
allows classes/resources in the file system or stored in the database.
File system classes/resources are global and the code run with all
permissions. Database classes/resources are per-database and code runs
accordingly with per-engine configured security policy.

The plugin has been made with the assumption that anything could be done
using existing FB infrastructure. You don't need to setup FTP server or
shared directory in the server to update your code or edit security police.

Installation/update of database JARs works from client command line
utility or from SQLJ package procedures. Actually, the same Java class
that run in the client is the one that runs in the server. You can put
JARs in the database using a Java URL, so lookup of them is not
restricted to client or server.

Security handling is configured in a central database, with a single
PERMISSION table. Permissions are granted to users that executes the
Java code. This way to configure permissions is not adequate for every
(or most) cases, and roles are not supported at the moment. I suppose
and desire we can have a better design in the future, before a final
release.

It's welcome any attempt to bypass the security system. But please,
report it. :-)

This version also lacks some notification mechanism for when the code
are going to be unloaded. You may note the problem in the example, right
after disconnect all attachments from the database the log file remains
locked, as it wasn't been garbaged collected. And no kind of "hot
redeployment" support had been implemented or tested. However, I have
ideas of how these problems can be solved in very good ways, based on
existing models. For unloading (or any other type of notification),
JPA-like listeners. For maintain state after redeployments, serializable
sessions.

In the file plugins\java-engine\examples\fbjava\README, you found
instructions for a "quick start" with it. In
plugins\java-engine\examples\tests there are functional tests for many
of its feature (i.e., how type mappings works, etc).

The plugin is based on Jaybird 2.2 (HEAD). I've put it in a maven
repository at [2], with its version defined as 2.2-jrt-SNAPSHOT.

The server includes my v3.0 changes, supporting packages, DDL triggers,
over clause and some internal changes. It may have others instabilities.

If you have a JAVA_HOME environment variable defined, it should just
works without any other configuration. The quick start assumes you had
defined FIREBIRD, ISC_USER and ISC_PASSWORD. In [1], I've zipped plugin
and engine together, but plugin installation consists in just unpacking
it in FB directory.

Documentation has room for improvements, and is at [3]. I've done it
with FB docbook tools.

Source code is in [4].

Happy testing and thanks.
Adriano"

[1] http://www.firebirdsql.org/downloads/rabbits/asfernandes/fbjava/fbjava-m1.zip
[win32]
[2] http://www.firebirdsql.org/downloads/rabbits/asfernandes/maven2
[3] http://www.firebirdsql.org/downloads/rabbits/asfernandes/fbjava/fbjava.pdf
(included in [1])
[4] https://firebird.svn.sourceforge.net/svnroot/firebird/java-plugin/tags/2.2-jrt-M1

IBPhoenix Web Site

2009-06-30T00:35:00.001-07:00

I need to post a note on my blog about the IBPhoenix Web site, it seems we are having problems with the T1 line. It looks like there is a problem in the central phone office, where our T1 line originates. It's been a problem now for several days. The issue has been escalated, so hopefully things will improve shortly.

../src/include/types_pub.h:88: error: conflicting declaration 'typedef int ISC_LONG'

2009-06-25T04:16:00.000-07:00

Building Firebird 64bit on Solaris (sparc or intel) can sometimes produce the following error:
-c ../src/jrd/dsc.cpp -o ../temp/boot/jrd/dsc.o
../src/include/types_pub.h:88: error: conflicting declaration 'typedef int ISC_LONG'
../src/include/fb_types.h:69: error: 'ISC_LONG' has a previous declaration as 'typedef SLONG ISC_LONG'
../src/include/types_pub.h:89: error: conflicting declaration 'typedef unsigned int ISC_ULONG'
../src/include/fb_types.h:70: error: 'ISC_ULONG' has a previous declaration as 'typedef ULONG ISC_ULONG

If you see this - then you made a mistake in the build process. Configure.in when you invoked it, assumed that the size of long was 4, and you are attempting to build Firebird for a 32bit environment. As per the prefix.solaris files, you must set the following before running configure:

export CXXFLAGS='-m64'
export CFLAGS='-m64'
export LDFLAGS='-m64'

Configure will now automtically realise that you want to build in a 64 bit environment and set the size of long to 8.

Some Positive Feedback for Firebird

2009-06-24T23:33:00.001-07:00

This message appeared on the Firebird Development list, from Jens Zurawski

"I think from time to time, one should really send some positive feedback
to the people who are involved into the development of the firebird
DBMS. Thank you very much for your efforts to make this software this good!

Just before a week or so we eventually upgraded our production server
from linux kernel 2.4 to 2.6.x and the firebird engine from 2.0.x to
2.1.2. As some of you might remember, long time ago we had some big
troubles with kernel 2.6.x and firebird, because our system is under
very heavy contention and we stumbled over many freezes, so we were
stuck to the 2.4.x kernel. (In average there are 75-150 simultaneous
connections to one database, in peak times up to 300. The database is
more than 20GBs in size and is serving for a pool of approx. 7 different
applications, starting from web server (php/java) over some internal
logistics processings (java) up to the CRM front end (Win32 native), and
additionally there are frequent update tasks with long running
transactions (about 10-30 minutes) and massive changes to the database
records in them).

Now our old database server broke down (hardware problem) and we had to
buy a new one and this time we needed to go for a 2.6.x kernel because
the new hardware was not supported from the old kernel. To my pleasant
surprise, the actual FB 2.1.2 is working very well with a 2.6 kernel and
the heavy contention we still have on our database. Only thing one has
to do is to dramatically increase the semaphores. In the first place we
increased the LockSemCount to 250 (which was the limit of the default
kernel setup) but that wasn't enough. After half of the day firebird
crashed down with a "Fatal lock manager error: semop failed (acquire),
errno: 22", accepted no more new connections and also didn't serve the
existing connections any more. Now after we increased the kernel limit
and set the LockSemCount up to 1024, everything is working very well. No
more freezes, no more crashes. (BTW: ironically this hint I was getting
from a guy who just has collected some experiences with an Oracle
installation on Linux ;-)

So, what I want to say: Thanks a lot for all the good work! firebird is
really my most favourite open source project.

Ah, and the new monitoring tables in FB 2.1 are a gift from heaven! Now
I'm able to track down problems from applications which are not written
by me, because now I can simply monitor hanging transactions or long
running "sub-optimal" queries and send the developer a hint ;) Also
thanks a lot for this feature!"

64 bit Firebird 2.5, and Solaris Sparc

2009-05-07T00:18:00.000-07:00

Looks like the job is already done... I read the following post on Firebird Development from Bill Oliver (SAS)

"I confirm that the issues are fixed in HEAD. I confirmed this on 64-bit
Solaris Sparc, built with the Sun Studio native compiler - using pthreads!!!

"How cool is that! Thanks very much guys for the quick turn-around.

Also, big thanks to Alex for his assitance with this port!!"

Excellent work!

GCC 4.01 MacOSX & Weak Symbols

2009-04-22T03:01:00.000-07:00

During the QA process for Firebird V2.1.2 we found that there was a problem with creating a database remotely. In effect the inet server would crash.

On further analysis we found that we have a problem where the function free()(standard C dynamic memory deallocator) was being invoked with the incorrect parameter, an address which had not been allocated using malloc().

After setting up the code to make the inet server call abort we got the following stack trace

(fragment)

#0 0x90005efc in free ()
#1 0x010440b8 in Firebird::AbstractString::~AbstractString (this=0xbfffcb04)
at ../src/include/../common/classes/fb_string.h:375
#2 0x010440f8 Firebird::StringBase::~StringBase
(this=0xbfffcb04) at ../src/include/../common/classes/fb_string.h:391
#3 0x0101593c in LD_setup_attributes (textTypeName=0xbfffcd60 "ES_ES",
charSetName=0xbfffccdc "ISO8859_1",
configInfo=0x6a1204 "filename=$(this)/fbintl;icu_versions=default",
srcLen=39, src=0x69a898 "DISABLE-COMPRESSIONS=1;SPECIALS-FIRST=1",
dstLen=512, dst=0xbfffcdb8 "DISABLE-COMPRESSIONS=1;SPECIALS-FIRST=1,")
at ../src/intl/ld.cpp:457
#4 0x3011b1b4 in Jrd::IntlManager::setupCollationAttributes
(collationName=@0xbfffd0e4, charSetName=@0xbfffd0b8,
specificAttributes=@0xbfffd14c, newSpecificAttributes=@0xbfffd178)
at ../src/jrd/IntlManager.cpp:317

From this back trace you can see that free() is called from the operator delete[], where we would have expected MemoryPool::deallocate() to be used. For some
reason the default operator delete[] is used.

Strange, how did this code arrive at address 0x010440b8?

After an analysis of nm output for fbintl and the linker's map of symbols Alex found that at this address is in fact located the redefined operator delete[], which was compiled in via ld.cpp. However in the assembly version of the file (ld.cpp), it was correctly redefined (i.e. it was calling MemoryPool::deallocate()) operator delete[].

Houston it looks as if we have a buggy toolchain.

But we also need to take into account that AbstractString dtor() and operator delete[] are also WEAK symbols. Weak meaning that such symbols may be present in many object files, and the linker at link-time will take it from the relevant one. But in this case it looks as if it was taken from another file...

But after we compiled into assembler all of the files that are used in the fbintl
library we found that they all contained the correctly redefined operator
delete[]. As such there is no use of operator delete[], calling std free() function in our code, but anyway we get such use in the resulting binary.

In effect it looks like weak symbols are broken on MacOSX GCC 4.01 (Apple build 5747). And the only way to fix a problem is to avoid them.

We are currently compiling Firebird on PPC 10.4 (Tiger) using a new switch -fno-weak

fno-weak:
"Do not use weak symbol support, even if it is provided by the linker. By default, G++ will use weak symbols if they are available. This option exists only for testing, and should not be used by end-users; it will result in inferior code and has no benefits."

MallocLogFile, MacOSX & LaunchDaemons

2009-04-16T08:07:00.001-07:00

I have been debugging the fb_inet_server trying to find a problem in Firebird 2.1.2 Classic for PowerPC. That takes a little bit of setting up, but I will try and explain that in another blog post. Part of one of the things that I wanted to check was what malloc was doing. However by default any "error" messages get sent to standard out, however you can set the environment variable MallocLogFile to capture messages to a specified file. But setting that in my normal environment is going to work because the fb_inet_server is actually launched by the LaunchDaemon and is then also forked. So we need to set the environment variable in the LaunchDaemon script directly.

The solution:

You need to edit the xml script that "launches" the fb_inet_server. This file can be found in /Library/LaunchDaemons and is called org.firebird.gds.plist

First of all unload the current launch definition

launchctl unload /Library/LaunchDaemons/org.firebird.gds.plist

Then edit the file as follows
After

<string>org.firebird.gds</string>

Add

<Key>EnvironmentVariables</Key>
<dict>
<Key>MallocLogFile</Key>
<String>/Library/Frameworks/Firebird.framework/Malloc.log</String>
</dict>

Save and

launchctl load /Library/LaunchDaemons/org.firebird.gds.plist

Modern Music

2009-03-04T10:23:00.000-08:00

The Brits - Hmmm - Not my list of recommended music....
Try listening to:

1. The Killers (excellent)
2. Editors (good)
3. The View (acquired taste)

Enjoy

Bosmere Products Ltd

2009-02-23T03:20:00.000-08:00

I don't normally do this, but about a month ago I sent the following letter to a company in the UK. I am still waiting for a reply. I am not expecting to receive one, nor do I expect a refund.

So if anybody out there is googling for Patio Set Covers, and happens to bump into this post...

We had bought a patio set cover for our patio tables and chairs in November 2008, and covered them up using this cover. By February 2009, the cover was falling apart.
Letter follows.

Customer Services
Bosmere Products Ltd,
Unit 5 Mitchell Way,
Airport Service Road,
Portsmouth,
PO3 5PR,
England.

Dear Sir,

I am writing to express my disappointment with the quality of your 6 seater circular patio set cover.

I purchased this cover from Chinnor Garden Centre, Chinnor, Oxfordshire for £39.99 on 4th November 2008 while on a short trip to the UK from my home in France.

Within a week the first of the supposedly heavy duty ties had broken – followed by the other 3 within the next month. Attempts to re-tie the broken cords were useless (they simply snapped again) and now the metal eyelets have ripped away from the plastic cover, now making it impossible to secure the cover even with heavy duty polypropylene rope which I purchased.

On removing the cover, it is also apparent that large areas of the plastic coating on the inside of the cover have completely worn away and it is no longer waterproof.
We do not live in the Outer Hebrides but south of the Loire valley in France, therefore it has most certainly not been exposed to unusually severe weather here!
It is clear that this product is not fit for purpose – it is marketed as being top quality PVC backed polyester, secure in windy conditions and giving full winter protection – it is obvious that after just two and a half months use that it is none of these.

As we live in France and very seldom travel to the UK we are unable to return the cover to the place where we purchased it. I would ask you therefore to refund the cost of the cover.

Yours faithfully
Mr Paul Beach.

Daily Telegraph Cryptic No 25851

2009-02-13T03:41:00.000-08:00

I am a great fan of the Daily Telegraph Cryptic crossword, and have been doing, or attempting to do this crossword, just about every day since I was a student at University.

I was introduced to it by my Grandmother many years ago, when stayed with her in Banbury.

"We" (and I use We, advisedly) used to do the crossword when we had breakfast, usually my Grandmother doing the crossword and explaining how things worked, whilst every now and then I would have a eureka moment. Anyway enough digression.

I found the following web site that specialises in helping people through the Telegraph Cryptic crossword in a way that is much more friendly than normal "specialist" cryptic sites.

Big Dave's Telegraph Crossword Blog

For those of you who don't know Libellule, is french for Dragonfly.

Memory Utilisation by the Firebird Classic Server

2009-01-30T04:04:00.000-08:00

The following was posted by Dimitry Yemanov on the Firebird Development list in response to some questions about Memory Usage within Firebird (Classic)

"Need some help understanding how memory allocation is done per process on Linux 1.5.5 CS. It's understandable that there is some amount of working memory needed to support fb process, plus per connection amount, plus cache depending on database page size and buffer count."

Per process (independent) memory usage:
- page buffers (40MB in your case)
- metadata cache (cannot be predicted/controlled, depends on the schema complexity, especially number of interdependent PSQL objects)
- sort buffers (turned off by default on Classic, see SortMemUpperLimit in firebird.conf)
- undo log (cannot be predicted/controlled, wastes memory mostly for multiple batch updates in the same transaction)

Shared memory usage (depends on other connections):
- lock table (starts with LockMemSize and auto-growing)
- event table (starts with EventMemSize and auto-growing)

"What makes me wander is how concurrent user connection count affects memory utilization per process. Somehow the more users we have on the server - the bigger overhead is added per connection. When comparing mem utilization on same database having single connection and 55 for example - utilized memory delta increases by ~33MB. On server having 160 connections it's almost 2-3 times more. Is this lock manager related data? 30MB just for that?"

The lock manager data is shared, but it's mapped into the address space of the every Classic process. I dunno whether the Linux tools include the mmap'ed memory into the counters you check or not. If so, it gets easy to explain: yes, this increase is caused by the lock table. You can draftly calculate the lock manager memory requirements by this formula:

number of connections * number of page buffers * 100 bytes.

The exact size could be found in the output of fb_lock_print (header section) at runtime. With your quite large (for Classic) page cache settings, the numbers should be nearly the same as you experience.

Firebird Trace and Audit Services

2009-01-30T03:58:00.000-08:00

Posted by Vlad Horsun to the Firebird Development List.

I am glad to introduce new Firebird facility, based on TraceAPI initially developed by Nickolay Samofatov and maintained and given to us by RedSoft (thanks!).

Below is brief overview of what was done. It will be included into HEAD soon, I hope.

Trace session - set of engine-generated events to trace and placement of trace output. Trace session also have its unique ID, optional name, state, flags, creator user and creation timestamp.

List of available trace events is fixed and determined by the Firebird engine. List of events to trace, which data items to trace and placement of trace output is specified by trace configuration when trace session created.

There are two kinds of trace sessions: system audit and user trace. System audit session is started by the engine itself and obtains configuration from the text file. This file name is set via new setting in firebird.conf and by default is "none", i.e. no system audit is configured. There may be only one system audit trace session, obviously. Configuration file contains list of traced events and placement of trace log(s). It is very flexible and allows to log different set of events
for different databases into different log files. Example configuration file is attached.

User trace session is managed by user via Services API. There are five new services for this purposes - to start, stop, suspend, resume trace session and to enlist all known trace sessions.

When user application starts trace session it passed optional name and (mandatory) session configuration. Session configuration is a text. Rules and syntax the same as for configuration file mentioned above except of log placement. Engine stores output of user session in set of temporary files and deleted it automatically. After application starts user trace session it must read session's output from service (using isc_service_query). When application decide to stop its trace session it just do detach from service. Also user may manage (suspend\resume\stop) any own trace session. Administrator allowed to manage any trace session.

If user trace session was created by ordinary user it will trace only connections established by this user.

User trace sessions is not preserved when all Firebird processes is stopped. I.e. if user started trace session and Firebird SS or SC process is shutted down, this session is stopped and will not be restarted with new Firebird process. For CS this is not the case as user trace session can't live without connection with service manager and dedicated CS process.

There are three main use cases:

1. Constant audit of engine.
This is served by system audit trace.

2. On demand interactive trace of some (or all) activity in some (or all) databases.
Application start user trace session and show traced events to user in real time on the screen. User may pause\continue trace and at last stop it.

3. Aggregate of some activity for a relatively long period of time (few hours or even whole day) to analyse it later.
Application start user trace session and save trace log in file (or set of files). Session must be stopped manually by the same application or by another one.

Firebird's service manager utility (fbsvcmgr.exe) may be used to manage trace sessions. Later specialized console utility will be introduced (at least i plan to develop it soon after beta). Of course third party vendors are welcome to implement own GUI based monitoring and trace applications.

InterBase 2009 comes with multicore support

2008-09-09T00:56:00.000-07:00

I have just read the following news article on SDTimes:
InterBase 2009 comes with multicore support
A couple of aspects to this story seeme a little peculiar, while some comments require some clarification.

"Thus the update for InterBase 2009 is relatively subdued"

correct, I do not see any major new features in this new version, however a couple of new features highlighted are

"such as over-the-wire encryption and AES as an encryption scheme".

If you want over the wire encryption for Firebird you could use Zebedee. There have been a number of papers re. this in the past. However this is something that could be implemented in Firebird fairly easily if we so wished.

In terms of database encryption there is code to allow it now, however the original implementation was not reliable. We did put together a more detailed proposal, but after some lengthy discussions it was decided that such functionality was going to be too complicated for the majority of users, and most of them didn't want it any way.

I do have trouble working out what

"The biggest change, perhaps, is the removal of any non-symmetric multiprocessing (SMP) compatible versions of the database. All versions of InterBase will now be able to use multiple processors and cores."

means.. CodeGear have been stating that InterBase was SMP enabled since InterBase 7.0. So what does this mean? Was InterBase non SMP capable/SMP friendly on some platforms? Was there code that wasn't SMP enabled? Has the code been cleaned up, so that all the old platforms have been removed. Or is it just a licensing issue where the new SMP license now supports up to 8 cores/CPUs, while previously you had to buy extra CPU licenses?

One particular quote was very confusing

"For the existing open-source Firebird database, which is based on InterBase, these changes won't trickle down; that free database does not include SMP, disaster recovery or high availability features."

Firebird was originally based on the open sourced version of InterBase 6.0, but the code bases are now quite different after 8 years of independent development work. So to state that Firebird is based on InterBase, although this is technically correct, it is a little disingenuous.

Stating that changes in InterBase will not trickle down to Firebird is correct. InterBase as per Inprise/Borland/CodeGear/Embarcadero is a closed source database. After the code was originally open sourced on SourceForge by Borland, it was forked and Firebird was created. InterBase was then "closed sourced" by Borland, and development work has continued on InterBase in a closed source manner. No code or feature at any time has come from InterBase to Firebird. Although it is possible that Firebird code may have made it into InterBase. However we have no way of verifying that.

In terms of SMP, by re-introducing the process based Classic Arhitecture, Firebird does support SMP. However a process based architecture isn't multi-threaded. That's an issue that's being addressed in Firebird V2.5 (SuperClassic) and Firebird 3.0.

I assume by "Disaster Recovery" this refers to "Journaling with Write Ahead Logging (WAL)" something that has not been implemented in Firebird. I am unsure what the other "High Availability" features are.

Finally

"In October, he said that InterBase ToGo will be made available. Normally, InterBase has a memory footprint of around 100MB. ToGo will be slimmed to around 3MB"

I am lost on this one. I am sure by this he means a disk footprint, not memory. And most of the size of the normal disk footprint is taken up by a stupid installer (imho) and docs. My assumption is that InterBase ToGo is effectively a variant on Firebird Embedded, but since ToGo will not be available until October it is difficult to comment.

Solaris 10, Firebird SuperServer and the Service Management Facility

2008-09-04T02:03:00.001-07:00

I recently released Firebird V2.04 SuperServer for Solaris 10. This new build uses the Solaris 10 Service Management Facility to stop/start Firebird. Here are some notes on how it works.
I created the following manifest file (it can be found in /opt/firebird/ as gds_db-tcp.xml):

<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>


<service_bundle type='manifest' name='firebird'>
<service
name='application/firebird'
type='service'
version='1'>
<create_default_instance enabled='true' />

<exec_method
type='method'
name='start'
exec='/opt/firebird/bin/fbmgr.bin -start'
timeout_seconds='10'>
<method_context>
<method_credential user='firebird' group='firebird' />
</method_context>
</exec_method>

<exec_method
type='method'
name='stop'
exec='/opt/firebird/bin/fbmgr.bin -stop'
timeout_seconds='10'>
<method_context>
<method_credential user='firebird' group='firebird' />
</method_context>
</exec_method>

<template>
<common_name>
<loctext xml:lang='C'>
Firebird
</loctext>
</common_name>
</template>
</service>
</service_bundle>

This manifest file is automatically installed by the pkg installer.
You can manually stop and start firebird using the following commands, instead of using fbmgr.bin

svcadm enable svc:/application/firebird:default
svcadm disable svc:/application/firebird:default

You can verify whether the Firebird "service" is running by using the following:

svcs -a | grep firebird

Supposedly the new Service Management Facility provides automatic recovery from software and hardware errors, so theoretically SuperServer on Solaris 10 no longer has need of the Firebird Guardian, since SMF should do the same job that the Guardian was designed to do. This is going to need some testing ....

Solaris 10 and iscsi

2008-07-21T03:46:00.000-07:00

Seems that applying all of the patches discussed previously has broken my version of Solaris, as it now seems to have problems booting.

It seems to hang during the boot cycle, so I had to do some hunting to try and find out the cause.
Boooting with a -v (verbose) shows that the system is hanging in iscsi.
After some googling it seems the only way to fix this, so you can get in is to do the following..

1. Boot using -m milestone=none
2. Login as root
3. svcadm enable -r filesystem/minimal
4. svcadm enable -r filesystem/local
5. cd /etc/iscsi
6. mv iscsi_v1.dbc iscsi_v1.dbc.bad
7. reboot

Now you can get in, I suggest you go and install the latest iscsi path from sunsolve:
Patch number 119091 for x86 Solaris.

reboot and hopefully all will be well.

Solaris and the 256 File Descriptor Problem

2008-07-02T05:25:00.000-07:00

Some of you reading this may remember that back when InterBase 6.0 was released on Solaris we ran into a fundamental problem on Solaris. It seems that since 1992 there has been an open bug documenting Solaris' file descriptor problem: "32-bit stdio routines should support file descriptors >255."

Bug id:
1085341

When this issue was discovered, we chose to use SFIO, a stdio variant from AT&T
Comments on this issue can be found in jrd/common.h which forces a superserver build to use our own internal stdio.h found in extern/SfIO/include.

Well it looks like in Solaris 10 Sun have finally fixed the issue.
More details can be found in the article entitled:
Solaris OS Solutions to 32-Bit stdio's 256 File-Descriptors Limitation by Giri Mandalika dated May 2007

Suffice it say - the bug has been fixed, but in a fairly typical Sun way.... the fix is more complicated than it should be... why they didn't just change the char to a short and be done with it I really don't know.

Anyway it should be possible on Solaris 10 to use this fix instead of our "own" implementation of stdio.h, just by adding a couple of lines to gds.cpp
e.g.

#ifdef SOLARIS
#include <stdio_ext.h>
#endif
after including

and then adding the following at around about line 3535 (Firebird 2.0.4)
#ifdef SOLARIS
enable_extended_FILE_stdio (-1, -1);
#endif

Hmm that seems to be a nice simple solution, except that its always more complicated than that.
It seems that this fix is only available from later releases of Solaris 10, so if you have a version of Solaris from Solaris 10 3/05 through Solaris 10 11/06 you will need to install just 3 patches...
Well no - thats not the case. I had to do the following to get this to work...

Download and install the following patches:

1. 118855-36
2. 118919-21
3. 119255-53
4. 120273-21
5. 123840-04
6. 127756-01
7. 125504-02
8. 122661-08
9. 125548-02
10. 126420-01
11. 126424-03

And then finally in single user mode, followed by a reconfigure reboot:
To boot Solaris x86 in single user mode, requires some fun and games too...

At the GRUB menu, type 'e' before it times out and takes the default boot option.
Scroll to the default multiuser boot line, then type 'e' egain.
This will bring you in to edit mode. Add a ' -s' at the end of the line.
Then type 'b' to boot.

12. 120012-14

Install this patch - then reconfigure

At the GRUB menu, type 'e' before it times out and takes the default boot option.
Scroll to the default multiuser boot line, then type 'e' egain.
This will bring you in to edit mode. Add a ' -r' at the end of the line.
Then type 'b' to boot.

I am not sure that we can ask Solaris users of Firebird if they do not have the fix already installed to go through this, so we will have a look at keeping our own SfIO. However currently that will not compile using gcc 4.2.1 and thats something else we need to try and fix.

Addendum:
sftp is now broken - so you need to add two more patches to the list above

13. 119253-26
14. 124629-07

MacOSX and stdin,stdout,stderr

2008-06-05T03:00:00.000-07:00

I am in the process of porting a friends software to MacOSX. Don't ask what it does yet, all will be revealed soon. The code compiles quite happily on Solaris and Linux, but fails with the following error on MacOSX...

Why?

dbFile.c: In function ‘_setLogfile’:
dbFile.c:243: error: invalid lvalue in assignment
dbFile.c:244: error: invalid lvalue in assignment
dbFile.c: In function ‘fieldFillGen’:

I know what the problem is -
Seems on MacOSX 10.4 and below:
stdin, stdout, and stderr are addresses of a FILE rather than a FILE*
On MacOSX 10.5 they are FILE*

So depending on the OS version you are targeting will determine whether the code compiles or not. The issue goes back to the way FreeBSD originally implemented this in stdio.h which is the base for MacOSX, and it looks like it finally became "correct" i.e. portable and adhering to other implementations in MacOSX 10.5
A quick look at /usr/include/stdio.h confirms this...
the FILE* appears within the ifdef for _DARWIN_UNIX_03

__BEGIN_DECLS
#if __DARWIN_UNIX03
extern FILE *__stdinp;
extern FILE *__stdoutp;
extern FILE *__stderrp;
#else /* !__DARWIN_UNIX03 */
extern FILE __sF[];
#endif /* __DARWIN_UNIX03 */
__END_DECLS

So I need to come up with a simple workaround for what was originally valid code on other operating systems:

stderr = wk;
stdout = wk;

the following seems to work:

dup2(wk->_file ,STDERR_FILENO);
dup2(wk->_file ,STDOUT_FILENO);

Standard delete v's Firebird delete

2008-06-04T05:04:00.000-07:00

Alex and I have just committed a slew of changes to the code in Firebird 2.1.
This all started when we did the 32 bit builds for Firebird on MacOSX.

We found that in remote we would get asserts trying to create a database. On deeper and further investigation over a period of a couple of weeks we realised that in some cases the standard delete was being called instead of the redefined version in Firebird i.e. it invoked free() instead of
MemoryPool::globalFree(). After some extensive googling we found other people have also run into similar problems.

As a solution it was suggested to use:
1. Use switches -fvisibility=hidden and -fvisibility-inlines-hidden together.
2. Use switch -fvisibility=hidden and __attribute__((always_inline)).

Trying with second combination we found that we sometimes got (depending on the version of gcc) an error:
sorry, unimplemented: inlining failed in call to 'void operator delete(void*) throw ()'.

It seems that without -fvisibility=hidden and __attribute__((always_inline)) gcc is
using non-inline body for operator delete. And is randomly using the wrong body, the one from the standard operator delete.

The first combination helps to avoid this bug - but any entrypoint, which is to be exported, now has to be marked with __attribute__((visibility="default")).

64bit PPC build for MacOSX

2008-04-21T03:00:00.000-07:00

Someone asked me why there wasn't a 64bit version of Firebird for MacOS powerpc... so I tried a build on the powerpc MacOSX box we have...
sw_vers -productVersion = 10.4.11
Build fails when we try to build config_root.cpp.

/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/MachineExceptions.h:272: error: typedef 'ExceptionHandlerProcPtr' is initialized (use __typeof__ instead)
/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/MachineExceptions.h:272: error: 'ExceptionInformation' was not declared in this scope
/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/MachineExceptions.h:272: error: 'theException' was not declared in this scope
/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/MachineExceptions.h:306: error: 'ExceptionInformation' was not declared in this scope
/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/MachineExceptions.h:306: error: 'theException' was not declared in this scope
/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/MachineExceptions.h:307: error: expected primary-expression before 'userUPP'
/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/MachineExceptions.h:307: error: initializer expression list treated as compound expression
/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/fp.h:1338: error: 'SIGDIGLEN' was not declared in this scope

Some googling explains what the problem is. Seems MacOSX is 10.4 is not 100% 64bit friendly and the carbon framework will not support a full 64bit build until MacOSX 10.5.

We use the carbon framework in a few, but very specific places, mainly to load UDF libraries as bundles and also the fbintl library.

So....

I either need to upgrade our powerpc Mac to MacOSX 10.5 to do the build...
or
Anybody out there got a powerpc Mac running MacOSX 10.5 that I could use to check the build?

Editline, Curses and Solaris AMD64

2008-04-08T07:23:00.000-07:00

We finally solved the last major problem in the Firebird 2.0x build for Solaris 10 AMD 64bit. Whenever we tried to include editline in the build isql_static would coredump. We tracked the problem down to the following:

Seems there is a problem in the Solaris' native implementation of libcurses, the function tgetstr() returns a bad pointer (non-NULL, but out of the process space). This means that until Sun fix it we can't use libcurses to link with editline on Solaris 64bit AMD.

So we came up with a solution...
We downloaded and built ncurses... (5.5) then copied libncurses.a to gen/firebird/lib along with the normal editline.a
Then adjusted make.defaults to remove references to -lcurses for LINK_LIBS and STATICLINK_LIBS and added -lncurses.

We now have a succesful build that seems to work properly. I hope to put the 64bit Solaris AMD builds up in the firebird prerelease area for testing purposes, then commit the changes made to the code base. Then wen 2.04 is ready to be released will create proper versions for full release.

Hope that somebody out there finds this useful, or at least the builds anyway.

Wrongly Named libfbclient on Solaris

2008-04-07T05:51:00.000-07:00

There is a small problem with the currently shipping versions of Firebird 2.0.3 for Solaris.
The problem doesn't affect the general workings of Firebird, but if you try to link with the libfbclient library using your own program, you will run into a problem where you will consistently get the following error:

ld: fatal: library -lfbclient: not found

The problem is due to a misnamed libfbclient.so in /opt/firebird/lib

To correct:
cd /opt/firebird/lib
rm libfbclient.so
ln -s libfbclient.so.2 libfbclient.so

This has already been corrected in CVS

64bit Firebird for Solaris 10 AMD64

2008-04-01T02:33:00.000-07:00

Looks like having a 64bit build of Firebird on Solaris 10 AMD64 is going to be a non-starter.
Alex and I have done all the hard work, and we have a build (currently without editline, since thats a problem in itself), but when we try and run isql created by the build we get the following:

ld.so.1: ./isql: fatal: relocation error: R_AMD^$_32S:
file /usr/lib/64/libfbembed.so.2: symbol (unknown): vakue 0xfffffd7ffee00000
does not fit

or in debug mode trying to run isql from the build directly:
ld.so.1: isql: fatal: relocation error: R_AMD64_PC32
file /home/pbeach/firebird64/gen/firebird/lib/libfbembed.so.2:
symbol main: value 0x2800155d494 does not fit.

A bit of googling finds this:

Bug ID 5102797
Synopsis AMD64: static relocation in dynamic library fails

If you read the details, seems the bug has been around since 15th Sept 2004
and because of its nature, there are no plans to fix it until Solaris 11.

Addendum: 2nd April 2007
Looks like we have found the problem.
LIB_LINK_OPTIONS in the prefix build file was -G
-G on Solaris should tell the linker to create a shared library
However because we are using gcc, something somewhere was getting confused.
If we pass -fPIC and -shared instead the problem is resolved.

From the gcc manaual:

-shared

Produce a shared object which can then be linked with other objects to form an executable. Not all systems support this option. For predictable results, you must also specify the same set of options that were used to generate code (-fpic, -fPIC, or model suboptions) when you specify this option

On some systems, `gcc -shared' needs to build supplementary stub code for constructors to work. On multi-libbed systems, `gcc -shared' must select the correct support libraries to link against. Failing to supply the correct flags may lead to subtle defects. Supplying them in cases where they are not necessary is innocuous.

FreeAdhocUDF for MacOSX

2008-03-28T04:37:00.000-07:00

I have just finished porting FreeAdhocUDF to MacOSX and sent the Makefiles, code, plus 32bit UDF libraries for Firebird 1.5.5 and Firebird 2.x to the FreeAdhocUDF types to be included in the distribution.

Issues that you might need to be aware of.
1. In Firebird 1.5.5 we don't ship libib_util.dylib, and even if we did you wouldn't be able to link to it. It seems that Firebird 1.5.5's installer doesn't copy the library to the Framework. This has now been fixed for the next release. Secondly the library was being built as a bundle. All Firebird UDF's need to be created as bundles to work on MacOSX. You can't link a bundle to a bundle. This has also been fixed.
2. In Firebird 1.5.5 you must remove any extension from the UDF library (e.g dylib) to get it to work. So for example FreeAdhocUDF.dylib needs to be renamed to FreeAdhocUDF. This bug has been fixed in flu.cpp for the next version.

Firebird 2.x does not have any of these problems :-)

In compiling FreeAdhocUDF for MacOSX, it has to be noted that MacOSX has no idea of sysinfo.h, so I had to add a chunk of code to handle the getting of a MAC address, and to get the random_info/randomness for GUIDs. This does seem to work nicely now.

The FreeAdhocUDF libraries I have provided are 32bit intel for Firebird 1.5.5 and Firebird 2.x, if you want to build ppc libraries, or 64bit libraries enjoy. If you have problems, let me know.

On ppc you "might" have to add the following to the Makefile
i.e. change
LDFLAGS=$(CF) $(LIB) -bundle -flat_namespace -undefined suppress to
LDFLAGS=$(CF) $(LIB) -bundle -flat_namespace -undefined suppress /usr/sys/bundle.o

Exceptions, gcc and Solaris 10 AMD 64bit

2008-03-21T08:06:00.000-07:00

For a little while now Alex and I have been battling a problem on Solaris 10 64bit AMD. We have been trying to port Firebird 2.0x to 64bit Solaris. However after fixing a few issues, e.g. defining the platform etc, we found that isql_static would core dump on trying to create a database. This is where the fun begins...

The default debugger for gcc shipped with Solaris 10 can't handle 64bit applications. Solution - build your own from a later version. At least this way we can now step through the Firebird code and try to see whats happening..

After some careful debugging sessions we find that Firebird seems to be OK, when you try to create a database, Firebird first trys to open it as an existing database, if it can't find an existing database we throw an internal exception and go on to create it.

The problem is isql_static is occurring when the exception is thrown...

Program received signal SIGSEGV, Segmentation fault.
0x0000000000488385 in __EH_FRAME_BEGIN__ ()

(gdb) bt
#0 0x0000000000488385 in __EH_FRAME_BEGIN__ ()
#1 0xfffffd7ffedacf3c in _Unwind_RaiseException_Body () from /lib/64/libc.so.1
#2 0xfffffd7ffedad129 in _Unwind_RaiseException () from /lib/64/libc.so.1
#3 0xfffffd7ffef3c71e in __cxa_throw (obj=0x1, tinfo=0x1, dest=0x474e5543432b2b00)
at /builds/sfw10-gate/usr/src/cmd/gcc/gcc-3.4.3/libstdc++-v3/libsupc++/eh_throw.cc:75
#4 0x000000000056510b in Firebird::status_exception::raise (status_vector=0x8d19e0)
at ../src/common/fb_exception.cpp:197
#5 0x0000000000694137 in ERR_punt () at ../src/jrd/err.cpp:562
#6 0x0000000000693ced in ERR_post (status=335544344) at ../src/jrd/err.cpp:441
#7 0x00000000005c66d4 in PIO_open (dbb=0xfffffd7ffec29050, string=@0xfffffd7fffdec9e0,
trace_flag=false, connection=0x0, file_name=@0xfffffd7fffdeca20, share_delete=false)
at ../src/jrd/os/posix/unix.cpp:646

So Firebird looks OK, i.e. its doing what it should do, however Solaris is causing a crash dump on the exception.

Some more detective work uncovered the following:

The problem is caused by the use of the -lc flag which was explicitly added to linker command line by autoconf.

Because we always use g++/gcc as linker, there is no need for us to need add in libc explicitly - it's done by compiler itself when it invokes ld. But autoconf seems to be doing it anyway.

When it is added to the command line, libc (Solaris native library) then happens to be added to ld's command line before libgcc_s where we would find gcc's own exception support library. If this happens any call to functions that have the same name, that happen to be present in both libraries, will go to libc, not libgcc_s.

There is at least one such function - Unwind_RaiseException(), which is called by g++ generated
code when exception is thrown. But instead of libgcc_s::Unwind_RaiseException() libc::Unwind_RaiseException() is called, but not with the parameters the Solaris native function expects - cue core dump.

Solution:

Don't use -lc

Where is libfbclient.dylib on MacOSX?

2008-03-13T02:13:00.000-07:00

There was a post to the Firebird Support list, asking how you link Qt to the libfbclient.dylib on MacOSX Firebird SuperServer. The problem being, that libfbclient.dylib is not "shipped" as part of the MacOSX framework... heres the detailed answer on how it works.

Firebird on MacOSX is delivered as a framework,

For more information on a frameworks look on : http://developer.apple.com

In a nutshell - "A framework is a hierarchical directory that encapsulates shared resources, such as a dynamic shared library, nib files, image files, localized strings, header files, and reference documentation in a single package. Multiple applications can use all of these resources simultaneously. The system loads them into memory as needed and shares the one copy of the resource among all applications whenever possible."

This is the "best" way to deliver an application on MacOSX.

However to do this the main dylibs (shared libraries) that run Firebird become part of the framework itself.

The main dylib that drives Classic is libfbembed, for SuperServer its libfbclient.

With Classic we ship both libfbembed and libfbclient, libfbclient as a separate dylib can be found in the libraries directory.

for SuperServer we ship libfbclient

However libfbembed (Classic) and libfbclient (SuperServer) are not directly obvious as shared libraries, i.e. if you look for them by that name with a dylib extension you are not going to find them. However if you took a close look at postfix.darwin in the Firebird source builds/posix (this is the code that sets up the framework) you will find that they become Firebird.framework/Firebird which is a link to Firebird.framework/Versions/Current/Firebird.

Where Firebird.framework/Firebird is libfbembed.dylib for Classic and
Firebird.framework/Firebird is libfbclient.dylib for SuperServer