SexpPath

SexpPath is a ruby DSL for pattern matching S-Expressions. Think of it as XPath or Regular Expressions for Ruby code, and you’re most of the way there.

Here is an example of an S-Expression in Ruby:

pets = s(:pets,
  s(:cat, :fluffy, s(:color, :white)),
  s(:cat, :snuggles, s(:color, :grey)),
  s(:fish, :bubbles, s(:drinks, :water)
)

This query extracts all the cats from our example:

pattern = Q?{ s(:cat, atom, _ ) }
cats = pets.search pattern

The SexpPath query above looks for expressions that start with the symbol :cat. The atom part says that we will match any symbol, so in the example above it would match both :fluffy and :snuggles. The underscore at the end will match anything at all, in this case, the S-Expression for color.

You can also match nested expressions with SexpPath:

pattern = Q?{ s(:cat, atom, s(:color, grey) ) }
grey_cats = pets.search pattern

SexpPath also has a notion of named matches. This query will place each cat’s name in the query result:

pattern = Q?{ s(:cat, atom % 'name', _ ) }
cat_matches = pets.search pattern
cat_names = cat_matches.map{|match| match['name']}

The % operator tells SexpPath where to stash a matching value.

Here is an example of using SexpPath with Ryan Davis’ excellent ParseTree library to extract all the methods in a given file:

# read in a ruby file
code = File.read('pony_factory.rb')
# parse the file with ParseTree
sexp = Sexp.from_array(ParseTree.new.parse_tree_for_string(code))
# create a SexpPath to find all the methods
pattern = Q?{ s(:defn, atom % 'name', _ ) }
# print all the methods in the ruby file
sexp.search(pattern).each do |match|
  puts match['name']
end

Now you’re doing static analysis on Ruby code! To learn more take a look at the Readme on GitHub, or skim over some of the examples.

If you have any suggestions or critiques of the code, API, etc. I would love to hear from you so comment, fork, or open an issue on GitHub.

Excel ruby-prof Report Printer

I got tired of re-sorting ruby-prof’s profiles in TextMate ( sort -n -r -k 2 is neat, but cumbersome ). So I whipped up a little gem for printing out the reports as Excel documents with excel_printer.

If nothing else, it’s a nice example of using the Spreadsheet gem in ruby.

Go fork it on Github and do something neat with it.-

OpenGem 1.3.1

Quick note, I just updated OpenGem so that you can now do:

gem read activerecord

If you the rdoc has not been generated yet, OpenGem will generate it for you and then open it up for your browsing pleasure (or horror).

So go ahead and update.

sudo gem update open_gem

Testing for existence with “blank?”

I know, I know, all ruby posts should be about slightly new takes on testing paradigms or other amazing best practices, but sometimes I think we skip the basiscs. Today it’s blank?.

ActiveSupport, the Rails extensions to ruby’s standard library introduce the method blank?. It’s pretty handy, and I think people tend to take it for granted without realizing exactly what its semantics are. blank? is handy if you want to treat an empty string, array, or hash as false.

Here’s a slightly haphazard table showing nil?, empty?, and blank? all work.

Input	nil?	empty?	blank?
” “	FALSE	FALSE	TRUE
nil	TRUE	ERROR	TRUE
1	FALSE	ERROR	FALSE
0	FALSE	ERROR	FALSE
“hi”	FALSE	FALSE	FALSE
[]	FALSE	TRUE	TRUE
[ [] ]	FALSE	FALSE	FALSE
[ nil ]	FALSE	FALSE	FALSE
[ 1 ]	FALSE	FALSE	FALSE
true	FALSE	ERROR	FALSE
false	FALSE	ERROR	TRUE
{}	FALSE	TRUE	TRUE
{1=>2}	FALSE	FALSE	FALSE

So if you want a flexible way of testing whether something exists, blank? will often give you what you want. I’m sure that was just thrilling!

OpenGem is on RubyForge

Hey quick note, OpenGem is now on RubyForge, so you can all just do:

gem install open_gem

Hoorah!

Creating Sample Data

Nothing earth shattering today, but I thought it was neat.  Every now and then I come across a document I would like to add to my test suites for a project, but there’s a problem.  The document might contain email addresses, phone numbers, who knows what, and I don’t really want that to be sitting around in my codebase, but the structure of the document matters.  The solution? Pipe it through this:

tr 'A-Za-z0-9' 'a-ff-aa-fa-ff-aa-fa-ff-aa-f001122334455'

Now all of your punctuation and such is preserved for your parsing pleasure, but the data is somewhat obfuscated. Of course use it at your discretion.

For bonus points, I actually made it into a TextMate command. Now I can have my test data, and feel reasonably safe about it.

OpenGem

Ok, trying to post a little more frequently, and less ambitiously.

This is just a tiny note to mention OpenGem a plugin for rubygems 1.3.2.

It’s quite handy, just do:

gem open rails

And rails will open up in your favorite text editor ($EDITOR).

The New Client Side

What happens when you separate your client from your business logic?

I was thinking about Martin Fowler’s recent post about the Database Thaw, and was reminded of an article published by by Dave Thomas. He pointed out that we could build our clients independently from our web frameworks if we use RESTful back ends.  Such an approach gives us a number of advantages in scalability, integration, and code separation.

Quite simply, when we work in terms of resources manipulated via HTTP, we introduce a new place where we can cache data easily.  HTTP GETs are (or should be) side effect free, and can be cached, or run through a web proxy.  Frameworks are also including increasing support for caching at the HTTP level, Rails for instance is introducing more support for ETags in its next release.

RESTFul back ends also create an excellent integration point for your applications.  As Martin Fowler pointed out, they are quickly becoming a viable alternative to Integration Databases.  They also make exposing functionality much simpler, and easier to pick up, for example the Twitter REST API follows similar conventions as Harvest’s REST API, and as a result, both are fairly easy to understand.

These are great reasons to look into a RESTful back end, and have been covered a lot.  What I think has been largely ignored is the possibility of increasing code separation between the layers of your application.  Right now, a common stack looks something like this:

	Database	Business Logic	Front End
Product	MySQL, Postgres, SQL Server, etc.	Rails, Django, CakePHP, ASP.NET MVC, etc	HTML
Language	SQL	Ruby, Python, C#, etc.	Framework’s Custom View Language + HTML + Javascript

Ignoring the fact that I probably left out your favorite language or framework, there’s something weird about this picture.  While we typically have a clean distinction between our database layer and our business logic, the front end typically gets injected with a lot of support code from our framework.  There is usually a template language, and often helpers which generate support JavaScript.

Over the next few posts I am going to explore building completely separated interfaces which are backed by a simple RESTful server.  My premise is that we should be able to build simpler, cleaner interfaces, that play nicely with the web.  I’m planning on creating a backend that is as bare bones as possible, and interfaces with examples using ExtJS, jQuery, SproutCore, and possibly Cappuccino.  If everything goes well, we might just be able to add a Flex and Java client as well, who knows. Let me know if you’d like to see any other frameworks used, or if you would like to contribute one once things get going.

RubyDiff 0.2, now with Subversion support

I saw a post over on Polishing Ruby about using the proper version requirements, and figured that it was about time for me to push out a new version of RubyDiff.  The major new feature is support for Subversion:

ruby_diff --svn http://project.svnserver.org/lib --svn BASE:lib

There are also some bugfixes dealing with classes referenced like: Module::SubModule::Class.

Playing with Ubiquity

Mozilla’s Ubiquity seems like a fairly neat idea.  It has its rough edges and glitches, but the idea seems promising.  I decided to give it a shot, and make a command to lookup bus routes here in Seattle.

CmdUtils.CreateCommand({
    name: "bus-schedule",
    takes: {"route": noun_arb_text},
    icon: "http://transit.metrokc.gov/favicon.ico",
    homepage: "http://endofline.wordpress.com",
    author: {name: "Adam Sanderson", email: "netghost@gmail.com"},
    license: "MPL",
    description: "Look up King County Metro bus route",
    help: "Select a bus route",

    _url: function(directObject){
      var url = "http://transit.metrokc.gov/cftemplates/show_schedule.cfm?BUS_ROUTE={QUERY}"
      var urlString = url.replace("{QUERY}", directObject.text);
      return urlString;
    },

    execute: function(directObject){
        var urlString = this._url(directObject);
        Utils.openUrlInBrowser(urlString);
    },

    preview: function(pblock, directObject){
        searchText = jQuery.trim(directObject.text);
        if(searchText.length <= 0) {
          pblock.innerHTML = "Look up King County Metro bus route";
        } else {
          jQuery.get( this._url(directObject), function(res) {
            var previewTemplate = "Look up route ${query} (${ok})";
            var ok = !res.match("No Schedule") ? "<em>Found</em>" : "<em> Not Found</em>";
            var previewData = {query: searchText, ok: ok};
            pblock.innerHTML = CmdUtils.renderTemplate(previewTemplate, previewData);
          });
        }
    }
});

One thing I really like about Ubiquity is that you can get a fairly rich preview.  This command for instance lets you know if the bus route you’re looking for is listed, not rocket science, but handy all the same.  For a nicely formatted version of this command, or to install it, check out the gist.

One thing left me a little puzzled about Ubiquity, why another text interface?  We already have the url bar, couldn’t Ubiquity launch from there?