build your polyglot app with jruby and rake

outcastgeek

Use the power of and flexibility of JRuby and Rake, to create your ultimate build tool…

Custom software demands a custom build. This is a demonstration of how you could build your JVM based project with JRuby and Rake.


Earlier this month, I gave a presentation at clojure.mn on how you could build your next polyglot application, using JRuby and Rake.

I have migrated the most interesting code to the clojure.mn public Mercurial Repository.

To be able to run the polyglot app you will need JRuby 1.6.5.1 and the bundler gem. Run bundle install, and you should be good to go. Also, Ant is required on your classpath.

Gemfile:

1
2
3
4
5
6
7
8
9
source :rubygems

# Project requirements
gem 'rake', '0.8.7'
gem 'buildr', '1.4.6'

# JRuby deployment requirements
# please add these lines...
gem 'jruby-openssl'

The following shell script and batch file were included as an example of how to fetch the dependencies, compile, and run the app.

run.sh:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
export JAVA_OPTS="-Djava.awt.headless=true -server -XX:CompileThreshold=4 -XX:+AggressiveOpts -XX:+UseCompressedOops -XX:MaxHeapFreeRatio=70 -XX:MinHeapFreeRatio=40 -XX:HeapDumpPath=./logs -XX:+HeapDumpOnOutOfMemoryError -Xms512m -Xmx512M -XX:MaxPermSize=512m -XX:+UseParallelGC -XX:ParallelGCThreads=24 -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:./logs/gc.log -XX:+DisableExplicitGC"

case $1 in
  Server)
    ~/jruby -S rake runServer
    ;;
  Task)
    ~/jruby -S rake $2 $3 $4 $5
    ;;
  Repl)
    ~/jruby -S rake cljRepl
    ;;
  Idea)
    ~/jruby -S buildr idea
    ;;
  Eclipse)
    ~/jruby -S buildr eclipse
    ;;
  esac
exit 0

run.bat:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
set JAVA_OPTS="-Djava.awt.headless=true -XX:CompileThreshold=4 -XX:+AggressiveOpts -XX:MaxHeapFreeRatio=70 -XX:MinHeapFreeRatio=40 -XX:HeapDumpPath=./logs -XX:+HeapDumpOnOutOfMemoryError -Xms128m -Xmx128M -XX:MaxPermSize=128m -XX:+UseParallelGC -XX:ParallelGCThreads=8 -XX:+DisableExplicitGC"

set var=%1

if "%var%"=="One" goto :One
if "%var%"=="Task" goto :Task
if "%var%"=="Repl" goto :Repl
if "%var%"=="Idea" goto :Idea
if "%var%"=="Eclipse" goto :Eclipse

:Server
jruby -S rake runServer
goto :EOF

:Task
jruby -S rake %2 %3 %4 %5
goto :EOF

:Repl
jruby -S rake cljRepl
goto :EOF

:Repl
jruby -S rake cljRepl
goto :EOF

:Idea
jruby -S buildr idea
goto :EOF

Buildr file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
ENV['JAVA_OPTS'] = '-Xms1g -Xmx1g'

Buildr.settings.build['scala.version'] = "2.9.1"
require 'buildr/scala'

repositories.remote << 'http://repo1.maven.org/maven2'
repositories.remote << 'http://scala-tools.org/repo-releases'
repositories.remote << 'http://build.clojure.org/releases'
repositories.remote << 'http://build.clojure.org/snapshots'
repositories.remote << 'http://clojars.org/repo'
repositories.remote << 'http://repository.springsource.com/maven/bundles/release'
repositories.remote << 'http://repository.springsource.com/maven/bundles/milestone'
repositories.remote << 'http://repository.springsource.com/maven/bundles/snapshot'

define 'clojure.mn' do
  project.version = '0.1.0'
  compile.with 'org.clojure:clojure:jar:1.3.0',
               'org.scala-lang:scala-library:jar:2.9.1',
               'org.clojure.contrib:complete:jar:1.3.0-SNAPSHOT',
               'org.clojure:clojure-contrib:jar:1.2.0',
               'org.clojure:tools.macro:jar:0.1.1',
               'org.clojure:tools.logging:jar:0.2.3',
               'org.clojure:core.incubator:jar:0.1.0',
               'org.clojure:algo.generic:jar:0.1.0',
               'org.clojure:data.json:jar:0.1.2',
               'congomongo:congomongo:jar:0.1.8',
               'enlive:enlive:jar:1.0.0',
               'hiccup:hiccup:jar:0.3.8',
               transitive('clj-style:clj-style:jar:1.0.1'),
               'clout:clout:jar:1.0.0',
               transitive('ring💍jar:1.0.1'),
               'compojure:compojure:jar:1.0.1',
               'javax.ws.rs:jsr311-api:jar:1.1-ea',
               'com.google.code.gson:gson:jar:2.0',
               'commons-fileupload:commons-fileupload:jar:1.2.2',
               'org.jdom:jdom:jar:1.1.2',
               'rome:rome:jar:1.0',
               transitive('org.apache.tika:tika-core:jar:1.0'),
               transitive('org.apache.tika:tika-parsers:jar:1.0'),
               transitive('org.glassfish:javax.servlet:jar:3.1.1'),
               'org.slf4j:slf4j-log4j12:jar:1.6.4',
               'org.slf4j:slf4j-api:jar:1.6.4',
               'org.slf4j:jcl-over-slf4j:jar:1.6.4',
               'ch.qos.logback:logback-core:jar:0.9.30',
               'ch.qos.logback:logback-classic:jar:0.9.30',
               'ch.qos.logback:logback-access:jar:0.9.30',
               'ch.qos.logback:logback-site:jar:0.9.30',
               'junit:junit:jar:4.10',
               'org.jboss.netty:netty:jar:3.2.7.Final',
               'org.eclipse.jetty:jetty-server:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-security:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-servlet:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-webapp:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-servlets:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-xml:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-util:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-jmx:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-http:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-io:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-continuation:jar:8.1.0.RC5',
               'org.eclipse.jetty:jetty-websocket:jar:8.1.0.RC5',
               transitive('jline:jline:jar:0.9.94'),
               transitive('swank-clojure:swank-clojure:jar:1.4.0')

  shell.using :jirb

  iml.local_repository_env_override = nil

  iml.main_source_directories << _("src/main/clojure")
  iml.main_source_directories << _("src/main/java")
  iml.main_source_directories << _("src/main/scala")

  iml.add_facet("clojure", "Clojure")
  #iml.add_facet("Scala", "scala")

  ipr.vcs = "Hg"

  ipr.jdk_version = "1.7"

  test.using :java_args => [ '-Xmx1g' ]

  package :jar

  task :deps => :compile do
    mkdir "target/lib" unless File.exist?("target/lib")
    cp project.compile.dependencies.collect { |t| t.to_s }, project.path_to('target/lib')
  end

end

Rakefile:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
require 'rake'

begin
  require 'ant'
  ant_import
rescue LoadError
  puts 'This Rakefile requires JRuby.'
  exit 1
end

require 'rbconfig'

is_windows = (RbConfig::CONFIG['host_os'] =~ /mswin|mingw|cygwin/)

def delete_all(*wildcards)
  wildcards.each do |wildcard|
    Dir[wildcard].each do |fn|
      next if ! File.exist?(fn)
      if File.directory?(fn)
        Dir["#{fn}/*"].each do |subfn|
          next if subfn=='.' || subfn=='..'
          delete_all(subfn)
        end
        puts "Deleting directory #{fn}"
        Dir.delete(fn)
      else
        puts "Deleting file #{fn}"
        File.delete(fn)
      end
    end
  end
end

desc "Clean all but deps!!!!"
task "clean:butdeps" do
  ant do
    delete :dir => "log"
    mkdir :dir => "log"
    delete :dir => "logs"
    mkdir :dir => "logs"
    delete :dir => "target/out"
    delete :dir => "out"
    delete :dir => "target/node"
  end
  ["/tmp/uploads", "tmp", "work", "**/*.zip",  "**/*~", "**/*.hprof"].each do |pattern|
    delete_all(pattern)
  end
end

desc "Clean Workspace!!!!"
task "clean:workspace" do
  ant do
    delete :dir => "target"
    delete :dir => "log"
    delete :dir => "logs"
    delete :dir => "vendor"
    delete :dir => "target/out"
    delete :dir => "out"
    delete :dir => "target/node"
  end
  ["/tmp/uploads", "tmp", "bin", "work", "**/*.zip", "**/*.class", "**/*~", "**/*.hprof"].each do |pattern|
    delete_all(pattern)
  end
end

desc "Package and Add All Java dependencies"
task "deps:all" do
  unless is_windows
    sh "~/jruby -S buildr clojure.mn:deps"
  else
    sh "jruby -S buildr clojure.mn:deps"
  end
end

desc "Downloads all Sources for the Dependencies"
task "sources:all" do
  unless is_windows
    sh "~/jruby -S buildr artifacts:sources"
  else
    sh "jruby -S buildr artifacts:sources"
  end
end

task "clj_comp" do
  ["target/lib/clojure-1.2*", "target/lib/jetty-6.1*.jar",
   "target/lib/servlet-api*", "target/lib/slf4j-api-1.5*.jar"].each do |pattern| #Remove undesired jars here!!!!
    delete_all(pattern)
  end
  ant do
    @build_path = ["src/main/clojure"] unless @build_path
    source_files = @build_path.collect do |d|
      Dir.glob("#{d}/**/*.clj").select do |clj_file|
        classfile = 'target/classes/' + clj_file.sub(".clj", ".class")
        File.exist?(classfile) ? File.stat(clj_file).mtime > File.stat(classfile).mtime : true
      end
    end.flatten
    source_file_list = source_files.join ' '
    namespaces = source_files.map do |f|
      f.sub("src/main/clojure/", "").sub(".clj", "").gsub("/", ".")
    end.join ' '
    java  :classname => "clojure.lang.Compile", :fork => true, :failonerror => true do
      sysproperty :key => "clojure.compile.path", :value => "target/classes"
      classpath do
        pathelement :location => "src/main/clojure"
        pathelement :location => "src/main/resources"
        pathelement :location => "target/classes"
        fileset :dir => "target/lib" do
          include :name => "*.jar"
        end
      end
      arg :line => "#{namespaces}"
    end
  end
end

task "compile" => ["clean:butdeps", "deps:all", "clj_comp"] do
  puts "Done compiling!!!!"
end

task "runServer" do
  ant do
    java :classname => "com.clojure.mn.server.runner.Jetty", :fork => false, :failonerror => true do
      classpath do
        pathelement :location => "target/classes"
        pathelement :location => "target/lib"
        pathelement :location => "src/main/resources"
        pathelement :location => "src/main/webapp"
        fileset :dir => "target/lib" do
          include :name => "*.jar"
        end
      end
    end
  end
end

task "cljRepl" do
  ant do
    java :classname => "clojure.main", :fork => false, :failonerror => true do
      classpath do
        pathelement :location => "target/classes"
        pathelement :location => "target/lib"
        pathelement :location => "src/main/java"
        pathelement :location => "src/main/scala"
        pathelement :location => "src/main/resources"
        pathelement :location => "src/main/clojure"
        fileset :dir => "target/lib" do
          include :name => "*.jar"
        end
      end
    end
  end
end

Note that:
The dependencies are resolved through Apache Buildr
The current project setup supports building java and scala, by placing the source files under “main/java” and “main/scala” respectively.
The “runServer” Rake task inherits the JVM parameters from the shell or batch script files, as the process is not forked.
You can generate intellij or eclipse projects by running “run Idea” or “run Eclipse”.
You can start swank like by launching the REPL with “run Repl”, and then do the following inside:

1
2
(use 'swank.swank)
(swank.swank/start-repl)


I could enhance the example project, to add support for additional languages / tools if you’d like me to.
I hope you take this idea, and make something awesome.

Any questions? Comments? Suggestions?