A Developer's Arsenal of Productivity Hacks

Ben Buckman

New Leaf Digital

DrupalCampNYC8 July 2010

View the blog post & comments about this session

Welcome

Drupal runs on *nix servers, it helps to know how to use them
Non-developer with dev skills = Productivity Boost

some macro some micro
some basic some advanced
some sophisticated some ghetto

I hope everyone will learn something & then we can share our own tricks

We're mostly self-taught or learn through human osmosis ∴ we can all learn from each other.

Going thru everything fast, but will all be
@ http://benbuckman.net/hacks

What We'll Cover

• Terminal fu
  • SVN
  • Drush
  • Scripts
• Mac
  • Textmate
• Misc
  • Apache
  • LessCSS
  • etc

Terminal fu

• "cmd fu: The art of doing battle with the command line" (UrbanDictionary.com)

• Works natively on all non-Windows OS's (Unix, Linux, Mac OSX)

  • on Windows: Cygwin
• "Primitive" but also simple, fast, versatile
• Several shells available, I mostly use bash
• Programming mindset: think of tasks algorithmically

Basics

• cd, mkdir, ls
• . current dir (pwd)
• | pipe
• >, >> output
• VARIABLE="VALUE", $VARIABLE
• line breaks or ; to separate lines/commands
• alias
• .bash_profile startup script
• man help docs

Basics (cont.)

• history, !999 past commands, re-run specific command
• !! re-run last command, e.g. sudo !!
• & run as background process
• &&, || chain commands
  • great for error handling
• ⇥ autocomplete, ⇥⇥ list
• $? last exit code (0=success)

Output

• cat full output
• head top
• tail bottom
  • tail -f follow: e.g. for log
  • -n# number of lines
• more paginated

Handling Output

• Output stdout to file: script.sh > script.log

  • doesn't catch ERRORS

  script.sh 2&> script.log

  • or ONLY log errors: script.sh 2> error.log

  • (and good output goes to screen)

  • output to oblivion: dont-wanna-know.sh > /dev/null

• stdout is zero-sum, tee is stdout+file

Editing:

• nano
  • export EDITOR=nano
  • Ctrl+W writes, Ctrl+X exits

find

• find all modules: find . -name "*.module"
• by size, e.g. find . -size +1024k
• -maxdepth, -mindepth
• execute command on results, e.g. remove .svn folders:
  find . -type d -name ".svn" -exec rm -rf {} \;
• by date, e.g. to remove old backups:
  find /var/backup/ -name "*.sql.gz" -type f -mtime +21 -exec rm {} \;
• man find

Loops

• for
  • for VAR in $LIST; do SOMETHING; done;

• while

  • while CONDITION; do SOMETHING; done; (generic)
    • avoid ∞
  • LIST | while read VAR; do SOMETHING; done;
    (actually useful)

• while vs for -- line break treatment:

Loops cont.

• find+while:

  • turn on every core module:

  find . -name "*.module" | while read MODULE; do drush pm-enable `basename $MODULE .module` -y
done

grep

• Pattern Matching

• Basic text search: grep -r "some text" .

  • (grep is slow)
  • awk alternative

  

• A few flags:

  • -i case-insensitive
  • -v exclude

• Look for all ubercart modules:
  drush sm | grep -i uc

Shell Stew

• Find + Grep: Search for hook implementation in all modules:
  find . -name "*.module" -exec grep -i "_form_alter" {} \;
  OR
  grep "_form_alter" $(find . -name '*.module')
  • multiple file types:
    grep "_form_alter" $(find . -name '*.php' -or -name '*.inc' -or -name '*.tpl')

Shell Stew

• count words: wc, count lines: wc -l
  • e.g. Count templates:

find . -name "*.tpl.php" | wc -l
if [[ `find . -name "*.tpl.php" | wc -l` -gt 100 ]];
then echo "Jeez you have a lot of templates!"; fi

• File/dir size: du, du -h -s *

  • Find the largest file in a dir:
    du -k -s * | sort -n | tail -n1

• Free space: df

Symlinks

• (Essentially a pointer to a file)
• (in OSX aliases/shortcuts are symlinks)
  ln -s FILE LINK
  • any time the same file needs to be accessible at multiple paths
  • e.g. same Views tpl in 2 themes

More shell goodness

• Group commands with {}:
  { echo "hello"; echo "goodbye"; } | cat

• Backup (zipped):
  tar -czf /var/backup/site.zip /var/sites/site

  • exclusion pattern (e.g. for keeping "files" out of code backups):
    tar --exclude=PATTERN

• zip SQL dumps:
  mysqldump | gzip -c > dump.sql.gz
gunzip -c | mysql
  (same -c, different meaning)

• rsync for file transfers (local & remote)

  • rsync -rC to avoid SVN
  • -pl

More shell goodness

• Add to /etc/hosts:
  echo "echo \"\n127.0.0.1 local.site\" >> /etc/hosts" | sudo sh

• Identify variant of *nix:

  • uname -a (Linux kernel version)
  • lsb_release -a (distro information)
  • cat /etc/issue (more basic info)

• ssh: use keys instead of passwords (ssh-keygen)

• Automate/schedule anything with cron

  • critical for Drupal health in general

SVN

• svnurl: function svnurl() { svn info $1 | egrep '^URL: (.*)' | sed s/URL\:\ //; }
  • svn diff $(svnurl prod)@HEAD $(svnurl dev)@HEAD
• svnaddall: alias svnaddall='svn status | sed -ne "/^?/ {s/^? *//;s/ /\\\ /g;p;}" | xargs svn add'
• svnrmmissing: alias svnrmmissing='svn status | sed -ne "/^\!/ {s/^\! *//;s/ /\\\ /g;p;}" | xargs svn rm'
• svnnew:
  alias svnnew='svn st | grep -e "^[!M?~ACDR ]"'
• svnmerge (haven't tried yet)

Drush

• dl, en

• drush cc all

• Make sure you correctly added a hook in your module:
  drush hook theme_status_messages

• Calculate the size of your database:

  DB_NAME=`drush sql-conf | grep "database" | awk '{print $3}'`
  drush sql-query "SELECT table_name, 
  (data_length+index_length)/1024/1024 'Size_MB', 
  data_free/1024/1024 'Free_MB' FROM information_schema.TABLES 
  WHERE table_schema='${DB_NAME}' ORDER BY table_name;"
  <!-- `drush sql-query "SELECT table_schema 'DB', sum(data_length+index_length)/1024/1024 'Size_MB', sum(data_free)/1024/1024 'Free_MB' FROM information_schema.TABLES WHERE table_schema='${DB_NAME}' GROUP BY table_schema;" ` -->

• drush features-update-all -y, drush features-revert-all -y --force
• Read the list from drush ...

Scripting

• .sh, chmod +x

• declare language: #! /bin/bash, #! /usr/bin/env bash

• make sure file exists: if [ ! -f $SOMEFILE ]; then
echo "Missing file $SOMEFILE"; exit
fi

• make sure drush exists:

  if [ ! `which drush` ]; then echo "Missing Drush!"; exit; fi

• Catch errors by chaining: command && command && command || { echo "Error."; exit; }

Some of my scripts

• DB Sync Creator script

  • gets around MySql remote-connection blocks

• Simple/clunky way to compare changes between 2 DBs

  • (see comments for more sophisticated methods)

Apache: Environment detection with mod_rewrite

• mod_rewrite:

  RewriteEngine On
  <!--## set env vars for domains
    ## dash indicates no subst
    ## '.*' is critical!! thanks http://www.askapache.com/htaccess/crazy-advanced-mod_rewrite-tutorial.html
    ## no commas between flags!-->
  #  siteenv = local, live, dev
  RewriteCond %{SERVER_NAME} localhost
  RewriteRule .* - [E=siteenv:local]
 
  RewriteCond %{SERVER_NAME} ^site.com$ [OR]
  RewriteCond %{SERVER_NAME} ^www.site.com$
  RewriteRule .* - [E=siteenv:live]
 
  RewriteCond %{SERVER_NAME} ^dev.site.com$
  RewriteRule .* - [E=siteenv:dev]

• in php: getenv('siteenv')

Mac

• Spaces (built-in, not to be confused with Drupal API)

• Spotlight for math (built-in)
  

• AppleScript

  • e.g. script to turn on SSH tunnel + SOCKS proxy (e.g. in a cafe)
    • based on script @ http://bit.ly/bNF7k2
  • "Do Not Disturb" toggle @ http://tech.benbuck.net/6174

• Skitch

• ExpanDrive (& Cyberduck)

• Clyppan

• open

Textmate

• PHP bundle: validation, docs, snippets, etc
• Markdown, MultiMarkdown, Textile
  • (this presentation is in Markdown)
  • Markdown -> HTML conversion (e.g. for blogging)
• Drupal bundle: docs, instant hooks, instant module
  • GetBundles bundle

Capistrano/Webistrano + Drush

    namespace :drush do
 
    set :webroot, "#{deploy_to}/current"
    set :drush_cmd, "drush --root=#{webroot} --uri=#{drush_uri} "
 
    def drush_intro
      logger.info "Running drush with root=#{webroot} and uri=#{drush_uri}"
    end
 
    # status/info
    task :drush_status do
      drush_intro
      # hide the DB password
      logger.info capture "#{drush_cmd} status  | grep -v \"Password\" "
    end

Misc

• LessCSS

  • for CSS sprites
  • Less.app for OSX

• Chrome Search Drupal extension

• Firebug for Drupal (Firefox & Chrome)

• Google Command Line Tool

  • http://code.google.com/p/googlecl/

• ConnectBot on Android

Resources

• commandlinefu.com
• delicious.com/thebuckst0p/unix
• Mac specific:
  • Mac Power Users podcast: macpowerusers.com
  • especially ep.23 "Workflows" with Merlin Mann (43Folders.com)
• The Google

Thanks to

• Chris G
• Yuri B
• Brian Silverstein
• Moshe Weitzman
• chx
• Scott McCabe
• (& many others)

Contact Me

• [work] newleafdigital.com
• [email] ben@newleafdigital.com
• [tech] tech.benbuck.net
• [twitter] @thebuckst0p

http://benbuck.net/hacks

Thank You