Groovy / Grails Snippets

Running multithreaded example groovy

2011-02-24T13:57:00.001-06:00


#!/bin/env groovy

def MAX_THREADS = 24
class MyThread extends Thread { 

 private i ; 

 public  MyThread( int code ) { 
  this.i = code;
 }
 public void run()  { 
  def rand = new java.util.Random()
  sleep(1 + rand.nextInt(2000) ); 
  println "HELLO " + i 
 }
}

println " Started " 
def Thread[] results = new Thread[MAX_THREADS]
for ( int i = 0 ; i < MAX_THREADS ; i++ ) { 
 println "Started thread " + i 
 results[i] = new MyThread(i);
 results[i].start()
}
for (int i = 0; i < results.length; i++) {
    try {
       results[i].join();
    } catch (InterruptedException ignore) {}
}
println "Done ... "

New Android Site ...

2009-12-31T11:02:00.002-06:00

I have a new site where I'll be posting further Android code . .. See: GeorgeKowalski.com

Geocoding with Groovy and Google API

2008-12-09T08:33:00.000-06:00

Nice article on how to geocode data with Groovy and the Google API : Here..

Deploying Grails Application on Oracle Database

2008-12-05T14:41:00.000-06:00

The following article presents a couple of lessons learned on Deploying a Grails app on Oracle DB Here.

API for Twitpic

2008-12-04T10:14:00.002-06:00

Check it out here: http://twitpic.com/api.do

quick tutorial on installing grails application on glassfish prelude

2008-11-16T20:14:00.000-06:00

See suns demo Here

Download images of twitter users from the public timeline

2008-11-13T12:58:00.000-06:00

I wanted to take the previous program and mod it to download twitter user pictures from the public timeline. I found that a number of them were just the default picture , so I through of generating a Digest ( checksum ) on the images downloaded. You may note that it takes only the first 1285 bytes of the user's picture. That's because that is the size of this default picture as provided by Twitter. "3yLyyOGkvQJNArEKPzvRzA==" is the digest for this image.

It might be cool to next mod this to stitch these pictures together into a wallpaper, especially if it was on ones friends instead of the random public.



#!groovy
// Program to download images of twitter users from the public timeline
// Does not download the default image if the user has not changed it. 
// George Kowalski 2008
//

import sun.misc.BASE64Encoder;
import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;

// Create images dir if one does not exit
def inDir = new File("images").mkdir()

// program to parse RSS Feed
def rssurl = "http://twitter.com/statuses/public_timeline.xml"
def slurp = new XmlSlurper()
def rssObj = slurp.parse(rssurl)
rssObj.status.eachWithIndex { item , num ->
 println "-------------------------------------------"
 def name =  item.user.screen_name.toString()
 def url = item.user.profile_image_url.toString()
 if ( url ) { 
  download(num, url, name)
 }

}

// Download the picture to a file names after the user's screen name
def download(num , address, screenName)
{
 def filename =  address.tokenize("/")[-2]
 def tmp = address.tokenize("/")[-1] // get the extenstion off the last part
 def ext = tmp.tokenize(".")[-1]
 filename = screenName << "." << ext
 println "saving image file : " << filename
 def outfile = new FileOutputStream( "images/" + filename.toString())
 def out = new BufferedOutputStream(outfile)
 try { 
  out <<  new URL(address).openStream()
 } catch ( FileNotFoundException e ) { 
  println "Error : Could not download image"
  out.close()
  // delete existing file that was started
  new File( "images/" + filename.toString()).delete()
  return
 } 
 out.close()
 def infile = new FileInputStream( "images/" + filename.toString())
 def input = new BufferedInputStream(infile)
 byte[] testBytes= new byte[1285]
 input.read(testBytes)
 input.close()
 def digestResults = getMessageDigest(testBytes)
 // println "getMessageDigest " << digestResults
 // remove default twitter image
 if ( digestResults == "3yLyyOGkvQJNArEKPzvRzA==" ) { 
  println "removing default pic for  " << screenName
  new File( "images/" + filename.toString()).delete()
 }
 
}

// Calculate a digest ( checksum ) for the bytes passed in
static public String getMessageDigest(byte[] bytes) throws UnsupportedEncodingException {
        String result = null
        String algorithm = "MD5";

        if (bytes != null) {
            MessageDigest md = null;
            try {
                md = MessageDigest.getInstance(algorithm);
            } catch (NoSuchAlgorithmException e) {

            }
            md.update(bytes); //set digest source
            byte[] raw = md.digest(); //transformed digest
            result = (new BASE64Encoder()).encode(raw); //string representation of digest
        }

        return result;
    }

groovy script to parse RSS with namespaces and download images

2008-11-10T17:13:00.001-06:00



#!groovy
// program to parse RSS Feed and download images from digg
def rssurl = "http://feeds.digg.com/digg/news/popular.rss"
def slurp = new XmlSlurper()
def rssObj = slurp.parse(rssurl).declareNamespace(digg: "http://digg.com/docs/diggrss/", media: "http://search.yahoo.com/mrss/")
rssObj.channel.item.eachWithIndex { item , num ->
        println "-------------------------------------------"
        println item.title
        println item."digg:category"
        def url = item."media:thumbnail".@url.text()
        if ( url ) {
                println url
                download(num, url)
        }

}


def download(num , address)
{
        def filename =  address.tokenize("/")[-2]
        def tmp = address.tokenize("/")[-1] 
        def ext = tmp.tokenize(".")[-1]
        filename = filename << "." << ext
        println "saving image file : " << filename
        def file = new FileOutputStream(filename.toString())
        def out = new BufferedOutputStream(file)
        out << new URL(address).openStream()
        out.close()
}

Script to parse Rss feed from Digg

2008-11-10T16:26:00.001-06:00


#!/usr/bin/env groovy
// program to parse RSS Feed
def rssurl = "http://feeds.digg.com/digg/news/popular.rss"
def rssObj = new XmlSlurper().parse(rssurl)
rssObj.channel.item.eachWithIndex { item , num ->
        println "-------------------------------------------"
        println num.toString() << " " << item.description
}

groovy screen scraping example

2008-11-07T09:29:00.000-06:00


#!/usr/bin/env groovy
// Depends on tagsoup library:
//      http://ccil.org/~cowan/XML/tagsoup/

def slurper = new XmlSlurper(new org.ccil.cowan.tagsoup.Parser())

def url = new URL("http://fcd.mcw.edu/?module=faculty&func=view&id=1674")

url.withReader { reader ->

        html = slurper.parse(reader)

 

        //we should now have a parsed file

 

        def value = html.body.div.div.div[2].ul.li

        value.list().each { f ->

                println "\nPub : " << f.toString()[0..80] << "..."

        }

 

}

[gkowalski]$ ./screenScrape.groovy
Pub : Role of cannabinoids and endocannabinoids in cerebral ischemia. (Hillard CJ) Cur...
Pub : Regional alterations in the endocannabinoid system in an animal model of depressi...
Pub : Mediation of Cannabidiol Anti-inflammation in the Retina by Equilibrative Nucleos...

Groovy Magazine started

2008-11-03T11:29:00.000-06:00

Picked up the first issues of Groovy Magazine

http://www.groovymag.com

I'll do a full review later , but it does look promising.

Grails: How to query a database with SQL outside GORM and return an HTML SELECT

2008-10-29T12:00:00.000-05:00

Here is an example of how to perform a query outside GORM and pass it to a view for display in a drop down select ...



import groovy.sql.Sql

class DbTableListController {

  // The datasource for this grails app is injected
  // automatically by it's name 

  def dataSource

  def create = {
        groovy.sql.Sql sql = new groovy.sql.Sql(dataSource)

        def tableNames = []
        sql.eachRow("""
                SELECT
                    unique ( table_name ) 
                FROM
                    ALL_TAB_COLUMNS
                WHERE
                    OWNER = 'CRDEV'
          """, { row ->
            println "Found " << row.table_name
            tableNames << row.table_name
        }
                )

        def dbTableList = new DbTableList()
        dbTableList.properties = params
        return ['dbTableList':dbTableList, 'tableNames' : tableNames]
    }

    // Other Methods deleted ...
}

Then it's displayed in the create.gsp for this controller with this code:

< g:select id="tableName" name="tableName" from="${tableNames}" />

Grails Script DB Connection parameter storing

2008-10-15T09:26:00.000-05:00

Storing Userids / passwords for multiple connections securely in one central file in one's home directory.



// Load DB settings

def env = System.getenv()

String propfile = "${env['HOME']}/.dbconfig.properties"

def config = new ConfigSlurper().parse(new File(propfile).toURL())

println "Running with DB User ID : " << config.db.snp.dev.userid



def db

if ( Config.DBTOUPDATE == 'prod' ) {

        db = Sql.newInstance(config.db.snp.prod.jdbcURL, config.db.snp.prod.userid, config.db.snp.prod.password, config.db

        dbgcrd = Sql.newInstance(config.db.gcrd.dev.jdbcURL, config.db.gcrd.dev.userid, config.db.gcrd.dev.password, confi

} else {

        db = Sql.newInstance(config.db.snp.dev.jdbcURL, config.db.snp.dev.userid, config.db.snp.dev.password, config.db.sn

        dbgcrd = Sql.newInstance(config.db.gcrd.dev.jdbcURL, config.db.gcrd.dev.userid, config.db.gcrd.dev.password, confi

}

The format of the config params is :


[gkowalski@scott:~/projects/snpDatabaseImport]$ ls -l ~/.dbconfig.properties

-rwx------+ 1 gkowalski gkowalski 682 Sep 18 12:13 .dbconfig.properties

[gkowalski]$ cat ~/.dbconfig.properties

// Development Database settings

db.snp.dev.userid="devUserid"

db.snp.dev.password="xxxx"

db.snp.dev.jdbcURL="jdbc:oracle:thin:@server.mcw.edu:1521:SID"

db.snp.dev.jdbcDriver="oracle.jdbc.driver.OracleDriver"

// Production Database settings

db.snp.prod.userid="prodUserId"

db.snp.prod.password="xxxxxx"

db.snp.prod.jdbcURL="jdbc:oracle:thin:@server.com:1521:SID1"

db.snp.prod.jdbcDriver="oracle.jdbc.driver.OracleDriver"

This keeps one from checking in DB passwords into SVN… Also keeps them all in one central spot for updating.

Oracle DB ORA-01722: invalid number error

2008-08-22T10:21:00.000-05:00

I was getting a ORA-01722: invalid number error with the following code:


BigDecimal sequence = 1000595201
db.execute "alter sequence snp_calls_seq increment by ${sequence}"

Instead use :


BigDecimal sequence = 1000595201
String str = "alter sequence snp_calls_seq increment by ${sequence}"
db.execute str

Why ?

StringBuffer in groovy

2008-07-09T14:05:00.000-05:00

Remember that when one is using a StringBuffer , one needs to close() it or your program will exit with a half written file. I use this more often than not when writing large files for performance reasons.

Groovy Script to rename all TXT files in a directory

2008-07-09T13:57:00.000-05:00



#!/usr/bin/env groovy
// Script to rename files that end in .txt replacing the XX to YY in the name

dirName = "/tmp"
new File(dirName).eachFile() { file ->
    if ( file.getName() =~ /.txt/ ) {
            def newName = (file.getName()).replaceFirst("XX", "YY")
            File f = new File(dirName + "/" + newName)
            file.renameTo(f)
            println file.getName() + " -> " + f.getName()
    } else {
            print "Skipping file " << file.getName() << "\n"
    }
}

Sparklines

2008-06-10T14:59:00.000-05:00

From: http://www.testearly.com/2008/05/22/groovy-sparklines/ Original Article



package com.stelligent.gsparkyimport org.jfree.chart.JFreeChart
import org.jfree.chart.axis.NumberAxis
import org.jfree.chart.plot.XYPlot
import org.jfree.chart.renderer.xy.StandardXYItemRenderer
import org.jfree.data.general.Dataset
import org.jfree.data.xy.XYSeries
import org.jfree.data.xy.XYSeriesCollection
import org.jfree.chart.ChartUtilities

class GSparky {

    def DEFAULT_HEIGHT = 30
    def DEFAULT_WIDTH  = 150

    boolean build( def data, def imgPath, def height = DEFAULT_HEIGHT, def width = DEFAULT_WIDTH ) {

        def chart = buildChartFromData(data)
        return buildImageFromChart( chart: chart, height: height, width: width, path: imgPath)
    }

    private JFreeChart buildChartFromData( data ) {

        def dataset = generateDataset(data)

        def plot = new XYPlot()
        plot.dataset = dataset

        plot.domainAxis =  minimalAxis()
        plot.rangeAxis = minimalAxis()

        plot.domainGridlinesVisible = false
        plot.domainCrosshairVisible = false
        plot.rangeGridlinesVisible = false
        plot.rangeCrosshairVisible = false
        plot.outlinePaint = null
        plot.renderer = new StandardXYItemRenderer(StandardXYItemRenderer.LINES)
        plot.insets = new RectangleInsets(-1, -1, 0, 0)

        def chart = new JFreeChart(null, JFreeChart.DEFAULT_TITLE_FONT, plot, false)
        chart.borderVisible = false

        return chart
    }

    private boolean buildImageFromChart( args ) {

        ChartUtilities.saveChartAsPNG( new File(args.path), args.chart, args.width, args.height )

        return true
    }

    private Object minimalAxis() {

      def a = new NumberAxis()
      a.tickLabelsVisible = false
      a.tickMarksVisible = false
      a.axisLineVisible = false
      a.negativeArrowVisible = false
      a.positiveArrowVisible = false
      a.visible = false;

      return a
    }

    private Dataset generateDataset(def data) {

        def series = new XYSeries("Sparkline")

        def i = 0
        data.each { y -> series.add(i++, y)  }

        def dataset = new XYSeriesCollection()
        dataset.addSeries(series)

        return dataset

    }
}

Testing program:


package com.stelligent.gsparky

class GSparklineTest extends GroovyTestCase {

    public void testBuildSparklineImage() {

        def gs = new GSparky()

        def data = [20]

        def r = new Random( new Date().getTime() )

        (0..99).each { x ->
           def y = data.get(x) + (x/2 - r.nextInt(x + 1))
           data << y
        }

        assert gs.build( data, "out/GSparklineTest_test1.png")

    }

}

Requires: http://www.jfree.org/jfreechart/ and

Jasper Reports from Grails

2008-04-18T14:47:00.000-05:00

See this Tutorial http://grails.codehaus.org/Jasper+Plugin

Executing External Processes From Groovy - Improved

2008-04-10T08:41:00.001-05:00

I've started using an exec method as shown below to run sub processes from groovy. If you run the standard exec like:



   def torun = "ls -l "
   proc = torun.execute()
   proc.waitFor()

This will hang on UNIX systems if these programs generate more STDOUT data than the buffer can handle. Better to handle this yourself with the following:



#!/usr/bin/env groovy

void execute(def command )  {

 println "Running command : " + command
        ProcessBuilder  pb = new ProcessBuilder("/bin/sh", "-c", command) 
        pb.redirectErrorStream(true)
        def process = pb.start()
        def output = process.inputStream
        output.each() { // Throw away stdout in this example, could return it at the end. 
        }
        process.waitFor()
        println "Done executing command , return value : " + process.exitValue()
        process.destroy()
        process = null
        pb = null
        System.gc()

}

execute("ls -l /usr/bin ")

NOTE From: http://jira.codehaus.org/browse/GROOVY-2620

A Better solution is :



StringWriter stringWriterOutput = new StringWriter()
StringWriter stringWriterError = new StringWriter()
Process proc = command.execute()
stringWriterOutput << proc.in
stringWriterError << proc.err
proc.waitFor()

Multithreaded simple URL Crawler

2008-04-09T14:47:00.000-05:00

Here is a quick program to create X threads that crawl a given URL given a set of results for a database.


#!/usr/bin/env groovy
import groovy.sql.Sql
// 
// MultiThreaded query script. 
// Runs a query and then submits all the jobs as threads using the Executor.newFixedThreadPool
//  
// 3/2008 by George kowalski
// 

// How may Threads to allow to run at one time. 
def MAX_THREADS = 10

println "Processing started .. Quering Database for RGDIds ... "

def sql = Sql.newInstance("jdbc:oracle:thin:@site.edu:1521:SCHEMA", "USERID", "passwd", "oracle.jdbc.driver.OracleDriver");
def service = java.util.concurrent.Executors.newFixedThreadPool(MAX_THREADS)

def rgdIDList  = []

sql.eachRow("select * from genes, rgd_ids where genes.rgd_id = rgd_ids.rgd_id and rgd_ids.object_status = 'ACTIVE' and rgd_ids.species_type_key = 3", { rgdIDList << it.rgd_id })

println "Done with Query we will be processing ${rgdIDList.size} ids "

// Class that is run. 
class toRun implements  Runnable {
            String id
            toRun(String newid) {
                        this.id = newid
            }
            public void run() {
                        println "Calling URL with id: ${id}"
   def contents = new URL("http://rgddev.mcw.edu/tools/genes/genes_view.cgi?id=${id}").getText()
                        println "Return from: ${id}"
            }
}

for (id in rgdIDList) {
    println "Submitting Thread for id: ${id}"
    service.execute( new toRun(id.toString()) )
    // This is just to slow down display on console, not needed.
    Thread.sleep(500)
}

Running Oracle SQL / Call URL

2008-04-08T10:37:00.000-05:00

The following program was whipped together to pre-cache pages on our server by hitting them with an HTTP request based on a query.

Running a SQL query and calling that URL like Curl to precache results


#!/usr/bin/env groovy
//
// Groovy Script to call each gene page in turn
// Requirements : Drop the Oracle jdbc14.jar driver in your $HOME/.groovy/lib directory then run this script
//
import groovy.sql.Sql

println "Started"
sql = Sql.newInstance("jdbc:oracle:thin:@server.com:1521:SID", "USERID", "password", "oracle.jdbc.driver.OracleDriver");

sql.eachRow("select * from genes, rgd_ids where genes.rgd_id = rgd_ids.rgd_id and rgd_ids.object_status = 'ACTIVE' and rgd_ids.species_type_key = 3", {
      println "Calling:   http://rgd.mcw.edu/tools/genes/genes_view.cgi?id=${it.rgd_id}"
      // creates a new URL obect and downloads that HTML into the contents variable
      def contents = new URL("http://rgd.mcw.edu/tools/genes/genes_view.cgi?id=${it.rgd_id}").getText()
      // Uncomment next line to print contents of page
      // println contents
  }
)
println "Done"

New Groovy Code Site

2008-04-08T09:20:00.000-05:00

I intend to post the various pieces of groovy / grails code I am working on . Please feel to use this code and / or comment on this code.

- George K.