The Processing program language

The Processing program language looks like an interesting tool for developing powerful data visualizations, or at least the skills and insight to do so with another language. Its author Ben Fry has several books about the language for various uses, including Visualizing Data. The book has few graphs of "visualized" data than I'd expect, and based on the introduction seems to emphasize Fry's philosophy and theory for all stages of data acquisition, analysis, and visualization.

From Wikipedia:
"Processing is an open source programming language and integrated development environment (IDE) built for the electronic arts and visual design communities with the purpose of teaching the basics of computer programming in a visual context, and to serve as the foundation for electronic sketchbooks. The project was initiated in 2001 by Casey Reas and Benjamin Fry, both formerly of the Aesthetics and Computation Group at the MIT Media Lab. One of the stated aims of Processing is to act as a tool to get non-programmers started with programming, through the instant gratification of visual feedback. The language builds on the Java programming language, but uses a simplified syntax and graphics programming model."

The grep() command for wild card searches

A cool way to search through your data is using the grep() command. grep is a "natural language" tool in R that lets you do wildcard searches for particular sequences of characters or words. I don't know the lingo or how or use it very well, but its not too hard to get some usefulness out of this command.

My understanding is that using natural language tools are a big part of intelligent computer programming. For data analysis, you could use it to pick out typos, ID #s that fit a particular pattern, multi-character factor variables, and probably many other creative applications. If you do mark-recapture analysis in program MARK this could be a useful way to identify capture histories that fit a pattern of interest.

Here's what I've used grep for. In my data set I have plants that have been observed each year from 2003 until 2007. I've made a column of data that compiles that "capture history" of each plant. 0 means that the plant wasn't there and 1 means it was there. These plants often spend a year underground in some kind of dormant state and we're trying to understand the biology of this phenomena.

I want to search my "capture history" column to identify all the plants that were dormant for one year and then grew at least two years in a row. This is so I can determine how quickly a plant grows the first few years after its been dormant. There are many possible combinations that fit my criteria since the capture histories have 5 entries
0 1 1 1 1
0 1 1 1 0
0 1 1 0 0
0 1 0 1 1
0 1 1 0 1
and so on.

To find all the plants that meet my criteria I used the grep() command like this

pattern_011<-grep("0 1 1",capture_history)

grep extracts all values that have the pattern "0 1 1" regardless of what comes before the 0 or after the last 1. The result is a list of row IDs for plants that fit the pattern.