Upgrading Drupal on Oracle (again)

drush vset maintenance_mode 0

Doesn’t bloody work with an Oracle install. Typical.

Well – today I don’t have the time nor the inclination to get to investigate, so, went in via the GUI, and set to maintenance mode.

Then I got on with the command line options.

drush dl
chown -R user:web-user drupal-7.xy
cd drupal-7.xy
rm -rf ./sites
cd ../
cp -pR html/sites/ ./drupal-7.xy/
chmod 000 drupal-7.xy/*.txt
chmod 644 drupal-7.xy/robots.txt

 

 

Yip-de-deee.

So – step one done, downloaded and copied most of the things as required.
Now need to patch the update.in file as I detailed here.

cp html/includes/update.inc.patch drupal-7.xy/includes/
cd drupal-7.xy/includes/
cp update.inc update.inc.bak
patch --dry-run -i update.inc.patch
patch -i update.inc.patch
diff update.inc.bak update.inc
chown user update.inc

Phew – now done the patch.

One last thing – copy the Oracle Database driver to the new area.

cp -pR html/includes/database/oracle ./drupal-7.xy/includes/database/

Now move the new directory into place, and run the update.php

mv html/ html-old
mv drupal-7.xy/ html/

Browser to to http://www.site.name/update.php

All done..

Use Gui to turn off maintenance mode.

Clear cache from CLI

cd html/
drush cc all

Yippeeee….

Reckon I should (could) script that now…..

Flush, flush – and flush again.

I am not talking toilets here – although for a short while, I wish I was – it would have been less frustrating.

I work and play in a LAMP environment, for example, this blog and site lives on a LAMP stack, and have recently needed to look at tweaking some performance handles.

A number of things have made me properly swear this week.

memchached, APC, and suexec primarily, along with the caches that exist in the software “running” the sites – like WordPress and Drupal.

Caches are so vitally important for anything in computing, as the direct path for repetitive data from storage to consumption, either via a computer or ultimately a human is simply, even in this era of super fast computing, just too damn slow.

We have files from disk cached in physical RAM done by the Linux kernel to increase the performance by dropping the latency at which frequently accessed files are actually read off the disk.

We have caching on the storage controllers for much the same reason – disk storage is slow ( SSD isn’t that common – and is still relatively slow when comparing access times of CPU cache )

We have caching of snippets of “compiled” code with APC (or similar) because it is computationally more expensive, which means time expensive, to recompile the code for an interpreter (PHP) than it is to store that compiled code in memory.

We use caches of commonly requested SQL queries because requesting the data from the actual database every time is expensive time wise – this is memcached.

We have the application itself (WordPress/Drupal) caching pages, or most if the pages and some calls it makes to save time when that is required.

All of these caches improve the user experience, even if that user is a computer doing automated queries.
We’ve come a long way.

The downside comes when data in one of these caches can trample over the data in another one of these caches, not from a memory perspective, because the areas of memory that each if these caches uses is very definitely separate (if it wasn’t – then there’d be huge opportunity for data trampling and associated corruption) – but from a a re-use perspective.

A page has been requested by a user, the application has done it’s caching, and thus could serve that page again without having to re-execute some parts (or all) of the request.
Great …..

Until we, the systems engineer wants to tweak more performance by adding another level of caching.

And due to the nature if the beast – the application’s cached results try to get executed at the next level in the chain – and end up having problems with, for example “Cannot redeclare class insert MySQL”

Enter the world of hair-pulling and head-table banging.

Enable APC, from the php/apache configuration perspective, first page off the site works, second, the infamous PHP white screen of death, and a nasty message in the apache log.

All worked perfectly on a test environment, but the only difference in the test environments is that there was one application installed on the server, not two or more.

Different site, same server, works properly.
Eh ? go figure!

Now, this other site, was the one that I had configured with APC/memcached first, so were were talking cached prefix keys trampling over each other ?

This thought ends in this from a google search.

Bottom line, yes, the APC module configuration from the first site could easily have been stamping all over the requests from the second (and third) sites.

So – configure the

$conf['cache_prefix']

as appropriate on both sites, and restart apache to clear the APC cache.

The problem was still apparent !!!!!!

Stop/start apache to clear APC cache, first page, fine, second page, no matter what that page, CRASH.

Bugger.
Now what.

Damn – disable it all, and put to the side for the day.
Next day, start from a clean slate.

And that term, clean slate was what got it for me.

Clean, totally, clear everything.
There is a reason there are a lot of instructions in Drupal for “drush cc all” when deploying new code or functionality, or modules etc, it clears all of Drupal’s internal cache, and starts everthing afresh.

So, clear all caches, Drupal, wordpress, restart memcached, restart apache, – then try accessing the website(s).

All perfect!!

Multiple pages, one ofter another work perfectly now.

We also get some interesting statistics with the use of APC, out of the 512MB we assigned, we are using about 380MB of it at the moment, but, by God the sites are faster.

Therefore, the words of wisdom for today – like you do after visiting the toilet – pull that chain, and flush all caches in a system every time you flush one. Flush ’em all.

Drupal and Oracle….. Pain

You may or may not realise it, but my “other” site, a pure testing site, www.macgyver.yi.org runs Drupal on an Oracle database.

I’ve done it to actually to learn Oracle, and give me a site I don’t care (so much) about breaking when I am learning.

Well for a good couple of weeks now, I’ve been getting the “There are new releases available for MacGyver.yi.org” email as there has been a new point version released.

OK – good, I like it that I am getting the emails, fantastic, I know that I need to do the work to do the upgrade.

So, today, as part of the bank holiday relaxation, I decided to do that upgrade (update as Drupal defines it, upgrade is from one major version to the next, update is a point release within the same major version)

Great – using this link, there appear to be some nice instructions.

As far as I was concerned, all worked great until step 6.

To quote:-

Run update.php by visiting http://www.example.com/update.php (replace www.example.com with your domain name). This will update the core database tables.

If you are unable to access update.php do the following:

Open settings.php with a text editor.

Find the line that says:

$update_free_access = FALSE;

Change it into:

$update_free_access = TRUE;

Once the upgrade is done, $update_free_access must be reverted to FALSE.

And running the <http://site/update.php> where I came a cropper:-

A PDO database driver is required!

You need to enable the PDO_ORACLE database driver for PHP 5.2.4 or higher so that Drupal 7 can access the database.

See the system requirements page for more information.

Two things here:

1) Version of PHP stated in the error message is wrong (very wrong – not even major revision close)

2) PDO driver for Oracle ? Eh ? Already have Drupal talking to Oracle, from the installation, so what goes ????

Damn, it’s not a simple as I was hoping, and now I was looking at a little bit of a fight to get this working.

Double damn – I had some more plans for today, other than relaxing 😀

Google has some answers, in the form of this link.

I’m going to copy the instructions here – at least then I have a copy of them.

Please note: the patch you will do uses “11.1” as the version number. It will work with version 11.2 and later (but unless you update the patch, you should continue using “11.1” in the ./configure command).

## Download the following instantclient files from Oracle’s website

http://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html

NOTE – You will need to have an Oracle account, and accept the terms and conditions.

Once downloaded, and copied to your webserver, unzip them.

unzip instantclient-basic-linux-x86-64-11.2.0.2.0.zip
unzip instantclient-sdk-linux-x86-64-11.2.0.2.0.zip

## Move the files to our install location, /usr/lib/oracle/instantclient

mkdir /usr/lib/oracle
mv instantclient_11_2/ /usr/lib/oracle/instantclient

## Fix some poorly named files and add them to our system’s library index:

cd /usr/lib/oracle/instantclient
ln -s libclntsh.so.* libclntsh.so
ln -s libocci.so.* libocci.so
echo /usr/lib/oracle/instantclient &gt;&gt; /etc/ld.so.conf
ldconfig

## Fix more stupid paths:

mkdir -p include/oracle/11.1/
cd include/oracle/11.1/
ln -s ../../../sdk/include client
cd /usr/lib/oracle/instantclient
mkdir -p lib/oracle/11.1/client
cd lib/oracle/11.1/client
ln -s ../../../../ lib

## Download PDO_OCI

mkdir -p /tmp/pear/download/
cd /tmp/pear/download/
pecl download pdo_oci
tar -xvzf PDO_OCI*.tgz
cd PDO_OCI*

## Patch PDO_OCI since it hasn’t been updated since 2005

# copy the lines below into the file “config.m4.patch”

*** config.m4 2005-09-24 17:23:24.000000000 -0600
--- config.m4 2009-07-07 17:32:14.000000000 -0600
***************
*** 7,12 ****
--- 7,14 ----
if test -s "$PDO_OCI_DIR/orainst/unix.rgs"; then
PDO_OCI_VERSION=`grep '"ocommon"' $PDO_OCI_DIR/orainst/unix.rgs | sed 's/[ ][ ]*/:/g' | cut -d: -f 6 | cut -c 2-4`
test -z "$PDO_OCI_VERSION" &amp;&amp; PDO_OCI_VERSION=7.3
+ elif test -f $PDO_OCI_DIR/lib/libclntsh.$SHLIB_SUFFIX_NAME.11.1; then
+ PDO_OCI_VERSION=11.1
elif test -f $PDO_OCI_DIR/lib/libclntsh.$SHLIB_SUFFIX_NAME.10.1; then
PDO_OCI_VERSION=10.1
elif test -f $PDO_OCI_DIR/lib/libclntsh.$SHLIB_SUFFIX_NAME.9.0; then
***************
*** 119,124 ****
--- 121,129 ----
10.2)
PHP_ADD_LIBRARY(clntsh, 1, PDO_OCI_SHARED_LIBADD)
;;
+ 11.1)
+ PHP_ADD_LIBRARY(clntsh, 1, PDO_OCI_SHARED_LIBADD)
+ ;;
*)
AC_MSG_ERROR(Unsupported Oracle version! $PDO_OCI_VERSION)
;;

## Attempt to compile (this is where you’re probably stuck, make sure you’re in your PDO_OCI folder!)

export ORACLE_HOME=/usr/lib/oracle/instantclient
patch --dry-run -i config.m4.patch
patch -i config.m4.patch
phpize
./configure --with-pdo-oci=instantclient,/usr/lib/oracle/instantclient,11.1

##

If you get an error as follows…

checking for PDO includes… checking for PDO includes…
configure: error: Cannot find php_pdo_driver.h.

Then you may get this fixed by doing…

ln -s /usr/include/php5 /usr/include/php

And you can continue by retrying the configure.

make
make test
make install

## Add extensions to PHP

# Create /etc/php5/apache2/conf.d/pdo_oci.ini

echo "extension=pdo_oci.so" &gt;&gt; /etc/php5/apache2/conf.d/pdo_oci.ini

## restart Apache

/etc/init.d/apache2 restart

Congratulations you made it!

## install Drupal!
Read the INSTALL file in the Drupal oracle module. It must be put in a special place in Drupal’s filesystem!

Now, although this all worked for me, in the sense that a phpinfo page returned an “Enabled” for PDO_OCI, but, crucially still failed on the database “upgrade.php” step from Drupal.

Arggggggggggggggggggghhhhhhhhhh 😡

Thankfully, Google to the rescue again.

http://drupal.org/node/1029080

cd includes/
ls -al
cp update.inc update.inc.bak

So, pulling down the patch (copy/pasting from http://drupal.org/files/1029080-update-database-pdo-rev3_2.patch )

Put this into a file, update.inc.patch

vi update.inc.patch

Or pulling down the file directly to the patch file.

wget -c http://drupal.org/files/1029080-update-database-pdo-rev3_2.patch -O ./update.inc.patch

Now – here I do something different, I edited the file because the file locations were different in my instance- i.e. my file wasn’t in a/includes, and I was running the patch directly from <drupal_install/includes/>

From:

diff --git a/includes/update.inc b/includes/update.inc
index f7c7b66..83fa6e4 100644
--- a/includes/update.inc
+++ b/includes/update.inc

To:

--- update.inc
+++ update.inc

Now to do the patch

patch --dry-run -i update.inc.patch
patch -i update.inc.patch
ls -altr
diff update.inc.bak update.inc

The output of the “diff update.inc.bak update.inc” should match the “update.inc.patch” file.

If it does, everything has gone to plan.

Now re-run <http://site/update.php>; et voila!!! All working.

Wow, what a pain on a Bank Holiday.