Wednesday 29 August 2012

Drush (aka drupal from bash)



Drush is cool...

# put a link to drush in your bin
ln /usr/local/bin/drush/drush bin/drush

mkdir ~/.drush
touch ~/.drush/aliases.drushrc.php 
# pinch another users alias file to get you started:
cp /home/michaeldance/.drush/aliases.drushrc.php .drush/

# edit alias file:
vim ~/.drush/aliases.drushrc.php

# see your aliases
drush sa

# chose an alias
drush use @prod

# check your good to go:
drush st

#drop into an sql shell
drush @ubox sqlc

# check the watchdog
drush watchdog-show

# clear the menu cache
drush @prod cc

# list the packages (eg modules/themes)
drush @prod pml
# filtering them with grep
drush @prod pml | grep agc

# review the watchdog-list
drush @ubox watchdog-list

#fire up the cron
drush @ubox cron

# list your packages 
drush @ubox pml


# disable a package
drush @ubox dis activitystream_feed

Saturday 25 August 2012

Coping with funny file names

I thought that spaces and other odd things in file names was just an irritating Windows thing, but no, it appears the contagion is now part of linux too.

Both the Swiss-army-knife commands find and xargs will cope tho, with just a little tweaking:
find /myfolder -type f -size +1M -print0 | xargs -0 ls -l
Find, unsurprisingly finds a list of files matching a criteria. Xargs then does something with them. In this case we're just listing files bigger than 1 meg.

Slightly related to this is the challenge of removing files with odd file names. A good tip is the -i flag on rm:
rm -i c*

This will check with you for each file starting with c.

Tuesday 21 August 2012

MySQL from bash

It turns out I've been accessing my favourite database from bash the hard way.

I just needed to make a MySQL configuration file in my home directory and set the permissions:

echo "
[client]
user=purple_drupal6
password=metoknow
host=ord-mysql-001-sn.bananas.net.au
[mysql]
database=nz_drupal" > .my.cnf
chmod 0600 ~/.my.cnf
 So that's easy.

Now I can just call mysql as a single word.... or better still do stuff like back up a table:

t=agc_geo_set
mysqldump au_drupal $t | gzip > $t.sql.gz

then to reload

cat $t.sql.gz  | gunzip | mysql

Running sql into the database from the shell is also a joy.

Having a quick look at a query is simple, and I can page through the results with less.
echo "
SELECT *
FROM agc_geo_set
" | mysql | less
Because I'm doing this from bash I can do variable substitution and reformat the output as I like it:

start=5
finish=10
echo "
SELECT *
FROM $t
WHERE id BETWEEN $start AND $finish
" | mysql | tocsv > $t.csv

Or copy permissions from one user to another:
u1=billblogs
u2=janeayre
echo "show grants for $u1;" | mysql | sed "s/$u1/$u2/" | grep "GRANT SELECT" | mysql


Thursday 9 August 2012

vim oh how I love thee

every day I get to love it more and more...

my favourite .vimrc line is, by a million miles:

imap jj <esc>

i also like this one:
" make f6 do an inner select on here docs:
nmap <F6> ?<<<<CR>w*kV?<<<<CR>j

This finds all lines containing word and adds them to a file:

:g/word/.w!>>filename

sort (unique) a range (as with all things works in visual mode too!):

:1,16sort u


but note to self must add these to my vimrc:

" Treat long lines as break lines (useful when moving around in them)
map j gj
map k gk

" Smart way to move between windows
map <C-j> <C-W>j
map <C-k> <C-W>k
map <C-h> <C-W>h
map <C-l> <C-W>l

Deleting or backing up a directory in linux

You'd think I'd be able to remember this:

    rm -rf directory

Nuh

or this
mkdir ../bin2
cp -r * ../bin2
ls ../bin2

its not really rocket science

This will pick up all the files in a directory (but skip the svn files):

tar -cvz $d/* > dust.tar.gz

Tuesday 7 August 2012

Postgres from bash

So I do most of my database stuff in MySQL these days... but every now and then I'm forced to use PostgreSQL for the GIS functionality.

First thing to do when using pg from bash is to make your ".pgpass" file which is a one liner in this format:

hostname:port:database:username:password

and set the permissions:

echo "pgserver.example.com:*:db:user:YourPassord" > ~/.pgpass
chmod 0600 ~/.pgpass

Then these commands are at your finger tips:

psql -h pgserver.example.com -d db -U user
pgsql2shp -h pgserver.example.com -u user db $t1
pg_dump -h pgserver.example.com -U user -t $t1

Saturday 4 August 2012

Getting started with automated web-interface testing

If you have a complex system with lots of interdependent components it is easy to break things whenever you change anything. This means whenever anything changes you have to go through your entire system to check for breakages or 'regressions'.

Thankfully, for web-applications there is selenium.

The selenium IDE (SIDE) is a firefox plug-in taking about 2 minutes to install and is very easy to use. It is great for recording the individual steps you go through to do a test: open this page, click on that link, type these values into a form, verify the results are as they are supposed to be. Just push the red record button and SIDE will start adding rows to your test script. You will also get extra options in your context menus that let you do extra things like "verify text present". Using SIDE it is very simple to record the steps you normally go through to test a component of a website.

Once you have recorded a simple test sequence you can save the test and replay it as you like (setting the start and end/break points). The SIDE lets you cluster a bunch of individual tests into a group or test suite.

The tests recorded by the SIDE are stored as a simple 3 column table. The SIDE itself stores them in a html table, allowing you to easily cut and paste the individual rows to "edit the script" as you like.The SIDE displays your test scripts either a neatly formatted table, or as raw html - I've found it works well to just open the file directly in the browser to be presented with a simple table that can be cut and paste.

So what are the challenges?
  1. Avoiding brittle tests - ie you want your tests to fail because something is wrong, not because your website has changed. The trick here is use well thought through locators to describe to selenium what part of the page you are referring to, and also to use smart pattern matching when "verifying text is present". Even though SIDE won't record great locators and text patterns for you, it is easy to over-write the initial values SIDE records. All the power of regular expressions and CSS/DOM/XML locators is at your finger tips.
  2. Managing the high level organisation of tests. This is really beyond the capacity of the SIDE. Think of making a comprehensive test battery for a large site as analogous to making a TV series. The role of the SIDE is to act as video camera enabling you to rapidly gather footage. Once the hours of footage are shot it needs to go through the special effects department and editing room. It turns out this editing process is pretty painless. You're in an environment that produces html pages for breakfast right? Managing you test suite is just as simple. That said, there are lots of naff TV series out there.
Next step? First of all download the SIDE and record a bit of footage.