Building and Maintaining a Ruby Infrastructure

Speakers: jeremey Hinegardner and Fernand Galiana

Work for:  Collect Intellect

copiousfreetime

gems:  amalgalite rabal htauth heel launchy …

mole - tells you about your rails application and it being used…

http://www.rubyinside.com/the-mole-spying-inside-your-rails-applications-457.html

http://liquidrail.com/2007/4/15/mole-plugin-inspect-the-rails/

ZiYa - charting the rails

World 1 - the grab bag world..

World 2 - the plugins world

World 3 - Gems - the one true path:

Primordial Gem - developer tools. - new project generation  -bones, consistent project layout, common tasks, base classes, etc..

Schema Management

Cruise Control - continuous integration’

~>  (twiddle wakka) 

~> 1.1 <=> 1.1.*

“~>” in dependency spec for Ruby Gems will ignore the “3rd” part of the version…

Should have a Internal Gem Server

Stickler - manage an internal gem server  (work in progress)

Grammar: Parsing DSL for Ruby

rubyforge.org/projects/grammar

Javascript and Ruby Framework

Speaker: Wynn Netherland  (wynn@squeejee.com  web: http://squeejee.com  blog: http://locomotivation.com)

Ruby: Rails, Merb, Camping, Sinatra

Javascript: Prototype, script.aculo.us, jWuery, MooTools, UI, ExtJS

Yehuda Katz

April 2009 - Katz’ new book:  Merb in action..

Abstract Browser quirks

Extend Language

Nifty Ajaxification

Prototype (script.aculo.us) - http://script.aculo.us/

- integrated with rails

-Effects: appear, fade, blinds, puff

- drag & drop

- builder

jQuery -

Simpler Ajax

designed for plugins

MooTools

–eye candy - lots of cool designs

ExtJS

good with data manipulation… grids..  themed like windows controls.  Good for apps trying to recreate a desktop application in the browser.

Widgets: tabs, grids, date pickers, dialogs, panels, sliders

jQuery UI project…   http://ui.jquery.com/demos  (http://ui.jquery.com/repository/real-world/layout/)

http://plugins.jquery.com

Prototype - LivePipe UI - http://scripteka.com

SproutCore - Javascript based; MVC in the browser

http://www.sproutcore.com/

http://github.com/sproutit/sproutcore/wikis/

supports testing…

HTML 5 datastore - SproutCore does this…  (HTML 5 datastore)

Dojo

Rails best practices

really funny talk that touched on how easy to maintained code goes bad…l

have to see the presentaiton…

webby - static site generation tool… — http://webby.rubyforge.org/

“unfactor your code and keep your job”

“small talk best practice patterns” book

rails

around_filter

Hidden Gems

Ruby in the Fast Lane

James Edward Gray II  (works on “Scout” at his job)

Created the Ruby Quiz at the beginning but don’t do it anymore

Evil rumor “Ruby is Slow”

James says it is “BS”..

Ruby goes as fast as we want it to.

Tools to speed up:

NArray, SQLite, RBTree, FSDB, Rinda, and most important tool: “Thinking out of the box”

NArray

Numeric Array … very fast..

Ruby’s numbers were built for “easy of use” but sometimes this makes it slower

C’s numbers were built for speed

Ruby can borrow C’s numbers in NArray…

Example Problem: Faster imaging

1.3 seconds to create 400 by 200 image using 2d array

changing to NArray made it less than a 1/100 of a second..

require “narray”

sqlite

sqlite has already solved many hard problems for data storage

It gives you an entire language to express your data needs…

IP to country

webnet77.com

code:  require: open-uri, zlib, rubygems, faster_csv, sqlite3

sqlite3 has a lot of cool features including unicode support

db = SQLite3::Database.new(<name of file)

db.exectute <<-END_Table.strip)

Create table…

END_TABLE

…. loading

puts db.get_first_value(…)

sqlite can use named parameter

SQlite is totally free…

Can receive results as a hash…

Can work with in memory databases ( this is crazy fast)

You can run queries across multiple database files

You can define SQL functions in pure RUBY code!

“Attach Database”

Rbtree

a binary tree

RBTree is super efficent…

Real binary search..

drops search below 1/1,000ths of a second

require: rbtree

Ruby SortedSet gets 15 times faster if RBTree is loaded…

FSDb - filesystem as a has

not a gem… but can be found on sourceforge

http://redshift.sourceforge.com/fsdb

Relational databases do not use times series data at all..

The has is a path in a file system…

db = FSDB::Database.new(”server_stats/”)

DB[time.strftime("%Y/%m/%d/%H/%M.obj")] = data…

Very simple to use… makes getting data very easily…

Can define your own format for files…

Rinda

dirt simple inter-process communication (IPC)

don’t have to install it…comes with Ruby

example problem: take a scramble word and find what it could be.

Create a “signature” by sorting the letters alphabetically…  just need to find matching signatures..  just takes time to find it then..

Rinda::TupleSapceProxy.new( DRbObject.new_with_uri(”druby://loclahost:61676″)

)

Creates tuple space:

results = Rinda:TupleSpace.new

DRb.start_service(”druby://localhost:61676″, results)

RingServer — zero config networking..

q&a

q: url for slides

a: not online yet, but will be put online… there will be a file server that is put up for the conference

The Next ruby 1.9 - Bruce Williams

Bruce (based in Austin) - Language Tourist

Blog: http://codefluency.com

work: http://fiveruns.com   (TuneUp, Manage, etc.)

Why Ruby 1.9?

(almost no one in the room had started using Ruby 1.9)

In a nutshell:

Many new syntax and language features…

Not strictly backwards compatible to 1.8..

Don’t get 1.9 just yet…. (http://www.ruby-lang.org/en/downloads/)

Standar library changes: rubygems, rake, json, ripper, probeprofiler, securerandom, HMAC digests… csv replaced by fasterCSV implementaiton

soap, sdle, base64 and some others were removed…

Migration Risk Factors

You need to have tests!  That is one of the quickest ways to find out what will break with the new version…

Risk: Text Processing - Anything relying on $KCODE, String#[], or String internals… Parsers especially at risk…different encodings for source files.. 

upack(’c*’)

Risk: Block Var Scope

“clever is a synonym for stupid”

Risk:  Hash#select (now returns hashes)

Risk: Hash Enumerations

Risk: Gems

The biggest obstacle to Ruby 1.9’s adoption is the sheer number of mostly working but essentially unmaintained gems that virtually everybody in the Ruy community depends on

–Sam Ruby

New features

multilingualization (m17n)

Enumerable - has index… can call next.. etc…  take, group_by, drop, min_by, max_by, count,and others…

Hash changes… (track insertion order)

Object: - tap…

New Proc Literal

(proc and lambda interchangeably)

Threads - native threading model from a user model…

Fibers - “semi-coroutines.”  Think of them as lightweight user threads with manual sceheduling…  some projects that use it Revactor  & NeverBlock…

question and answers

Why is Tap called tap?

(matz will probably talk more about this in his talk)

Are there ways to change the order?

Reverse, sort, index?  - not sure if there is a way…

Script Editor - Basic Version deployed to the server

I cleaned up the look at little.  Here are some screen shots to give you an idea of what it looks like:

Login Screen:

Welcome Screen:

New Script Page:

Script List page:

Script view page:

You can click the pencil icon to edit a line, click the up and down arrows to move the lines, or click the delete button to delete the line (no warning!! on the delete!)

At the bottom, there is a blank line to add a new line:

The record lines page allows for you to record lines that do not have a recording:

The final page shows the Spanish lines in the correct order:

The page uses SOX to combine the waves and LAME to convert them into MP3.  I am not able to get it to work on the server at dreamhost.  I downloaded SOX source, compiled, and it runs…  But when I pass in the wav files to combine them, I get an error: “can’t open file”… ” Could not find data chunk.”  Any ideas?

Script Editor - a look

Here is the look I was thinking about for the HabloHindu script editor (the graphic needs to have the last “i” changed to a “u”.

Today, I added the ability for people to log in, and for the individual lines to be edited.  I have all the pieces figured out.  Now I just need to bring them together, polish it, and test.

Script Editor - Combining the audio files

Combining the recordings has taken a little time to figure out.  But now I can merge them, and convert them to MP3.  Here is a screenshot of the page that produces the script in Spanish (just filter the Spanish lines) and puts links to the combined .Wav and .Mp3

‘

To get this to work, I had to call two command line functions “sox.exe” to combine the wavs  (http://sox.sourceforge.net/)   together.  and “Lame.exe” to convert it to an MP3 (http://lame.sourceforge.net/index.php).

I had some trouble with LAME.  It did not like the .Wav in the default format that the Java Sonics ListenUp Java Applet records in, i.e. ADPCM.  It kept saying “unsupported format”.  But it did work with “u8″.  There is some type of distortion in the background though..

I should also point out that I found a really neat looking PHP class to work with wav files called wav edit.  I could not get it to work with the wavs I recorded though.  Here are some screenshots of the sample page and a link to give you and idea of what I’m talking about.

http://www.pviet.com/wavedit/

Sample page using my recordings:

using the recordings provided in the .zip

Script Editor - Adding lines and recording audio

Making some good progress today!!!  I have added the ability to create lines for the script, delete lines, move lines up and down, record audio, and the like.  Here are some screenshots: 

The JavaSonic ListenUP applet records things as a .wav.  I will need to join these together.  I found a function to do this:

http://www.splitbrain.org/blog/2006-11/15-joining_wavs_with_php

function joinwavs($wavs){
    $fields = join(’/',array( ‘H8ChunkID’, ‘VChunkSize’, ‘H8Format’,
                              ‘H8Subchunk1ID’, ‘VSubchunk1Size’,
                              ‘vAudioFormat’, ‘vNumChannels’, ‘VSampleRate’,
                              ‘VByteRate’, ‘vBlockAlign’, ‘vBitsPerSample’ ));
    $data = ”;
    foreach($wavs as $wav){
        $fp     = fopen($wav,’rb’);
        $header = fread($fp,36);
        $info   = unpack($fields,$header);
        // read optional extra stuff
        if($info['Subchunk1Size'] > 16){
            $header .= fread($fp,($info['Subchunk1Size']-16));
        }
        // read SubChunk2ID
        $header .= fread($fp,4);
        // read Subchunk2Size
        $size  = unpack(’vsize’,fread($fp, 4));
        $size  = $size['size'];
        // read data
        $data .= fread($fp,$size);
    }
    return $header.pack(’V',strlen($data)).$data;
}

It streams it out to the client.. .I will want to save it to the filesystem and use LAME to convert it into MP3.

This is the next step…  After I am able to join them and convert them, then I will be at the point that I will have all the technical know how to do what I’m trying to do.  Then it will be a matter of cleaning up the design, (in fact redesign (ok.. ok… not much of  design now… just to design it)), make it look pretty, test test test… (I’m going to try to put the 15 scripts we have done so far for HabloHindu in there and see how that works), I need to add some authentication/security to it, then I will upload it and have Sara give it a try to see what she thinks…