Tech Blog :: git

Jan 30 '12 11:39am

Git trick: Cleanup end-of-line changes

I was working on a site recently that was moved from one VPS hoster to another, and the ops team that moved it somehow introduced an end-of-line change to every file. I didn't want to commit these junk changes, so I wrote this little snippet to clean them. For each modified file (not new ones), it counts the number of changed lines, excluding EOL changes, and if there are none, it checks out a clean copy of the file:

git st --porcelain | grep -r "^ M" | awk '{ print $2 }' | while read FILE; do
  LINES=`git diff --ignore-space-at-eol "$FILE" | wc -l`;
  if [[ "$LINES" -eq "0" ]]; then
    git checkout "$FILE"; 
    echo "$FILE"; 
Apr 6 '11 9:48pm

Good Git GUIs for Mac

Today on Twitter, I saw two Git apps for OSX worth spreading:

Via @jayroh: Brotherbard has an "experimental fork" of GitX which has a nice GUI for branch trees, local and remote branches, staging, cherry-picking, and rebasing. (Some time I have to try the last two with the app in particular.) It's free.

Via @mortendk, if you're willing to spend $59 (and I generally don't mind paying for great Mac software -developers have to eat!) - check out Tower, which claims to be and likely is "the most powerful Git client for Mac."

Of course there's always the terminal, which I'll still use for most operations - git log --graph shows a rudimentary graph, for instance. Visualizing complex branch trees in a GUI is really nice, and GitK (the Java-based app which comes with the Mac Git package) doesn't have a lot going for it.

(If you're new to Git, check out the excellent (and free) Pro Git book, or (via @danigrrl), a Git tutorial for designers.

Feb 25 '11 3:13pm
Tags on Git: how to preserve GitHub repositories for existing modules

I've been keeping all my code on GitHub, waiting for to move from CVS to Git. Well, it just did! No more bitching about the lousy CVS process, time to start maintaining my module releases again.

I want to keep my GitHub repositories' histories, of course. (Keeping the actual code on GitHub isn't critical, it's the commit/branches/tags that are important.) The new repositories on d.o were copied from the old CVS repositories, which contained (because I was lazy / holding out for Git) only final releases at best, or nothing at all, so I want the GitHub history to take precedence.

Git is "distributed," meaning the full repository is cloned everywhere it's used, including multiple "remotes." (This is unlike CVS or SVN, where there is a single remote server, and local "checkouts.") Anyway, this isn't the place for a Git tutorial (see the great [free] Pro-Git book to learn Git and some of my past Git tricks). I had assumed when d.o moved to Git, I could simply add a new remote and it would work automagically. It almost does, but needs a little extra work.

There is some official d.o documentation for Copying your repository onto from an existing repository. It clones the GitHub repo as a "mirror" and pushes the merged repo to d.o. This didn't work for me, for some reason. Maybe it'll work for you - try it first and see if it does. This worked for me instead.

This is all done through the terminal, from the directory where I've cloned my existing GitHub repository. My remote name for Github is origin and the branch is master (the standard convention). I'm going to leave GitHub at origin and add as drupal. To get the exact path to your repository, go to the Git Instructions tab in your project page. (I assume you've already agreed to the new TOS and set up SSH keys as it explains there.)

Add a new remote to the existing (but different) d.o repository:
git remote add drupal

(A digression: if you try at this point, git push drupal master, it'll throw an error -

 ! [rejected] master -> master (non-fast-forward)

- because the repository histories are different and can't be merged normally.)

Instead, pull the git.d.o branch alongside your existing one. (Note that the new 'drupal' remote has its own 'master' branch separate from the one we want. Hence we're fetching and not pulling.)
git fetch drupal

Then merge, but keeping the existing history ("ours") as the correct version:
git merge remotes/drupal/master --strategy=ours

One problem at this point: the git.d.o migration made a good change: "Stripping CVS keywords" (like $Id$). That's now gone, because we've dismissed the d.o history. So we get it back with a cherry-pick: git log shows the commit hash from the migration, so copy the start of the hash, and re-apply it:
git cherry-pick ####.

Check your code to make sure it's good... then
git push drupal master
And it's all up!

To create a tag for a new release (example):
git tag 6.x-1.0-alpha1
git push drupal 6.x-1.0-alpha1

(Interested in any feedback saying why this is stupid / why the other approaches should have worked / why this is causing the d.o infrastructure horrible damage... all I know is, it seemed to work for me.)

Dec 28 '10 12:50am

Alias to validate PHP and stage to Git in one step

One of the biggest faux pas that can happen working with PHP apps like Drupal (or any interpreted language for that matter) is deploying invalid syntax that brings down a whole site. So I like to run my PHP files through the PHP validator before committing. I'm working with Git, so this alias (put in .bash_profile for example) validates PHP files and then stages them to Git for committing (or doesn't stage them if there are errors). I'm including the extensions .php, .module, .install, and .inc as PHP files (please let me know if I'm missing any common ones):

[Update 1/9/11: I replaced the original alias syntax with a new function syntax that works more reliably.]

function git-v-add {
 git st --short | while read FILE; do echo $FILE | awk '{ print $2 }' | grep -E "(\.module|\.php|\.install|\.inc)$" | xargs php -l || return 1;
 done && \
 git add -A && git st

Then just run git-v-add && git commit.

Nov 16 '10 12:27am

Incremental database backups with Git

I'm trying a new method for database (MySql) backups: incremental tracking every 10 minutes with Git. Basically I have a Git repository cloned to a directory on the server, every 10 minutes on cron the database is dumped to the directory, and all changes are committed. I added a changed-line counter to make the logs more interesting.

I'm not sure how this will work in production, or what strain mysqldump puts on the database (I should run this off a slave when it goes live, I'm thinking), but the ability to revert the database to any 10-minute increment in the past and view changes to each table line by line seems pretty cool. This is the script:

#! /bin/bash
cd /var/backups/db-git
$DRUSH --root=/path/to/webroot sql-dump > /var/backups/db-git/site-db.sql
syncDT=`date "+%Y-%m-%d_%H-%M"`
## count # of lines changed
NUMLINES=$(git diff | grep -e "^[+-]" | wc -l)
## don't count 2 header lines
git ci -a -m"DB backup DEV: $syncDT ($NUMLINES lines changed)"
git push origin

Any thoughts?

Nov 14 '10 4:02am

Using MongoDB for Watchdog (logging) in Drupal

Much has been made of the MongoDB support in Drupal, mostly with Fields in Drupal 7, but with some limited backporting to D6 as well. MongoDB is a member of the new "NoSQL" family of databases, potentially offering better performance and scalability than relational databases like MySQL.

I've wanted to start using MongoDB for a while, so Watchdog (Drupal's error logging API) - being a particularly MySql-intensive process - seems like a good start. (One solution is to use syslog instead of MySql for watchdog logging, but then there's no backend UI to view the logs.)

The caveat is that the MongoDB module for Drupal 6 is very limited and officially no longer supported. The watchdog module in the latest release (not updated since July) seems to log events correctly, but had no page to view individual event details (so only truncated message summaries were available).

I contributed event pages as a patch here. I also submitted another patch to make the variable names in the module more flexible and consistent.

We'll see how this works. My MongoDB experience is very limited, so if this becomes a high-maintenance part of the site in the short term, I'll have to turn it off. I'm hoping it just works for now, and now that the MongoDB database is running, I'll start using it for mini-apps within the site that don't need MySQL or could benefit from Mongo's performance.

Oct 31 '10 1:55pm

Using git branches as folders to track config files

I keep coming back to this method, so I'm putting it here for my own reference:

If you want to use git branches as folders (e.g. to store config files in a single repository), you can initialize a repo (git init), then set the starting branch (instead of the default "master"):

git symbolic-ref HEAD refs/heads/NEWBRANCH (replace NEWBRANCH)

then git add your files, git commit, git remote add REMOTE (the Github repo URL), git push origin NEWBRANCH.

Do that for each config directory you want to track (I have "subdirectory" branch names like server1/apache) and you'll have version control on all your configuration.

Oct 2 '10 11:48pm

Git: syntax for generating a generic patch

In the base directory of the project:

git diff --no-prefix --relative --cc -u -- . > fixed-big-bug.patch

(It still has diff --git at the top, not sure how to get rid of that.)

Sep 29 '10 7:57pm

Drupal Dojo 9/29: Git workflows

Learned tonight at the unofficial Drupal Dojo in JP (copied from BHirsch):

Helpful links about forking (w github)

Undo a local change to a specific file do:
(note this checkout trick only works with modified files, not new files)
$ git checkout filename
$ git checkout path/to/file

Note: -- (double dash) separates all your options from the file path. E.g...
$ git command op1 op2 op3 -- path/to/file

Undo a committed change to a single file
$ git checkout 915108a7 -- filename

Undo all uncommitted local changes
$ git reset --hard

Undo everything up to this point
$ git reset --hard thispoint
$ git reset --hard HEAD
$ git reset --hard 915108a71552

Undo a specific commit (only that one commit), use revert
$ git revert 915108a71552

Undo a bunch of commits all at once...
$ git revert 915108a71..b05f9b935e29fd
Note: This (above) makes you log a message for each commit that is undone here, and each individual reversion is its own commit.

Deleting a local branch (where example is the name of an example branch)
$ git branch example
$ git checkout example
$ git push origin example
$ git checkout master
$ git branch -D example # now local branch is deleted
$ git push origin :example # this deletes the branch from remote

Undo a bunch of commits all at once with a single commit...
$ git revert -n 915108a71..b05f9b935e29fd
$ git commit -m "this is the only message i need because it all gets done in one commit."

Sep 23 '10 2:22pm

How to undo a file in a previous commit with Git

Suppose you've made a bunch of commits in a Git repository, and need to undo the changes made to a specific file. git revert, git checkout, and git reset can all be used to undo/revert changes in different stages (see here), but this syntax seems to be best when individual files need to be rolled back:

git checkout REV -- path/to/file
where REV is the SHA of the last good commit