MS SQL Server Replication with Ruby on Rails

May 25th, 2008

For my day job I have the following setup:

  • An intranet website with (of course) networking and password restrictions
  • A ‘public’ website with no restrictions on access

For reasons that aren’t really worth getting into, the ‘public’ website has no content management system (CMS) tied to it making it very difficult to update. I needed to be able to use the intranet as both an application for manipulating business data (which is not appropriate for public consumption), and as a CMS for the public website. Ruby on Rails works well for this since it allows for rapid creation of web forms for manipulating the data. At first I contemplated the idea of simply using the same database for both websites, with the intranet being the only one of the two having write access to the DB. But security concerns necessitate that the intranet DB never be accessed by the public at large since it may contain sensitive business data.

Hence SQL Server Replication with Ruby on Rails. The end result is two databases: whenever changes are made on the intranet website database they are automatically pushed (per transaction) to the public website database. This happens in real time (depending on the number of transactions occurring) - usually just a matter of seconds. And you can choose, with great granularity which tables/columns get replicated; so tables which contain data not appropriate for the public website simply don’t exist in the public website database. All in all very cool.

The details and a few gotchas:

  • There are three kinds of SQL Server Replication: Snapshot, Transactional, and Merge. I’m using Transactional replication - which pushes across each transaction in real near time - appropriate for a one way push.
  • This is a one-way push for security reasons. Two-way replication can apparently be set up - especially with Merge replication - but isn’t appropriate for my needs.
  • Every table that needs to have columns replicated from it MUST have an ID column. This is how SQL Server maps rows between the databases. Hence even the join tables for Has and Belongs To Many relationships must have an ID. And these columns must be named something other than ‘id’. I forget the reason for this - something to do with Rails pulling back the wrong ID as it does it’s nifty queries.
  • Any time you change what tables/columns will be replicated you must re-initialize the replication. This doesn’t result (to the best of my knowledge) in any down time for the public website but does add to the workload.
  • Most SQL Server objects can be replicated - including stored procedures. For my setup I’m using stored procedures for all queries on my public website and so what has worked out well is to create the stored procedures in Rails migrations. I know, I know - this is probably the greatest sin a Rails developer can commit. I apologize to you all for this heresy, yet it has proven useful.

I’ll update this post soon with more details and code examples.

Coldfusion testing with CFCUnit

March 23rd, 2008

So i’ve been working in Coldfusion for a few years as part of my day job - can’t complain too much, it does the job (though we’re planning on migrating to PHP). Recently, though, we’ve created an intranet site using Ruby on Rails and comparisons between the two are, well, interesting. Like a lot of people i’m impressed with RoR and all the agile goodness contained therein and the testing framework has been the thing that’s impressed me the most.

I’ve learned first hand the value of unit testing and the dangers of not having a thorough testing suite. But i’ve been dreading the task of trying to find a way of setting up a testing framework in CF. Seems like most of the things i’ve come across have involved needing the Mach II framework and perhaps some knowledge of JUnit (upon which some of the CF testing frameworks i’ve seen are based).

But today I bit the bullet and did some looking into it. Seems there are two primary testing frameworks: CFCUnit and CFUnit. Neither seems to have a large user base and the documentation seems to be almost noexistent, but, well, they exist (Wahoo!!). After a quick look at both I downloaded CFCUnit and gave it a whirl. Decided on CFCUnit because it appears to be the older and more fully featured (??) of the two. Install was extremely simple (copy two folders into your webroot) and didn’t involve needing to set up Mach II (CFCUnit version 1.1 includes it - no need to install/configure). Instant gratification.

Of course, the gratification wasn’t too instantaneous - once I had the thing set up I had to figure out how to write tests, in particular what assertions were available. Which is where Mark Drew’s post on CFCUnit assertions came in EXTREMELY handy.

All in all, extremely happy I got this set up - we shall see if it improves my code. I’ll post again after using for a while.

Markov Chains in Ruby

December 17th, 2007

So the plan for a piece i’m writing is this: a set of very simple melodies, each with some set of probabilities of progressing to one of the other melodies; in other words - markov chains. Each of the melodies will have a set of possible doublings/harmonizations, and as the piece progresses the chance of those doublings/harmonizations occurring will increase - the piece will ‘thicken’ over time. This gradual process will repeat a few times, each repetition forming a major unit of the piece, probably three in all.

One of my goals is to see if the piece (and the process of writing it) can ‘evolve’ over time; by generating many versions of each major unit of the piece and choosing the one I like best - perhaps one per day - I hope to gradually improve the melodies and the probabilities to make better versions; constrained randomness generating a kind of feed-back into the compositional process. In the end i’m hoping this will be something better (or a least different) from what I would have created just sitting down and deliberately selecting each note. At the same time, though, I don’t want to dive into something totally aleatoric - I still want some control. I’m not hoping for something radically different to come about because of all this - just for the way that I write music to be altered enough so that what I create is a variant of what I would have created.

At any rate, I needed some code to do this and this post shows the beginnings of that effort. So here’s the ruby code to generate a sequence of melodies - using a markov chain of course. It only spits out the array index number of each melody in the sequence (5 in all) - it’s designed to tell me what order the melodies should be played in, always starts on 0, and will append melody numbers to the sequence until it reaches the maximum value passed into the initialization method. A lot of the inspiration for this code plus some other (probably superior) ways of accomplishing this can be found at the rubyquiz.com

class RiffSelector
  def initialize(riffCount)
    # define the markov chain for the riffs...
    @chain = [[0, 1, 0, 0, 1], [0, 0, 1, 2, 0], [0, 1, 0, 1, 0], [0, 0, 2, 0, 1], [1, 0, 1, 0, 0]]
    # the starting state...
    @current = 0
    # the eventual list of riffs (with doublings)...
    @result = "0"
    # loop through, appending the next riff in the chain...
    (1..riffCount).each do
      nextState = getNext()
      @result += ', ' + nextState.to_s
    end
  end

  def getNext()
    probs = @chain[@current]# the array of probabilities of what state will be chosen next given the current state
    total_prob = 0# the sum of all the probabilites
    probs.each { |prob| total_prob += prob }
    random = rand(total_prob.to_i) + 1# pick a random number within the range of 1..[sum of probabilities]
    likelihood = 0
    for state in (0..(probs.length - 1))# loop through each of the states
      likelihood += probs.at(state)# sum probabilities as we go so that we'll eventually have a high enough number to equal the total_prob
      for iter in (0..likelihood)
        if random < = iter# iterate through until we reach or exceed the random number chosen earlier or until we run out of 'likelihood'
          @current = state
          return state
        end
      end
    end
  end

  def getChain()
    return @result
  end
end

foo = RiffSelector.new(19)# pass in the number of riffs to put into the chain
puts foo.getChain()# output the result

VOIP (Skype) Accessibility?

November 28th, 2007

So in an effort to prevent my wonderful wife from bankrupting us with cell phone bills, i’ve been trying to get her set up on Skype. The challenge being, of course, that she is blind and so must be able to control skype without the use of the mouse and with only those commands that her screen-reader, Jaws, can read to her. She seems to be able to get a fair amount of utility out of the program, though definitely not the full functionality a sighted person enjoys. There’s certainly some work the Skype folks need to do to make the program fully functional for visually-impaired users.

On the plus side Skype, like almost all windows software, has most of the programs commands available from the top menu, negating most of the limitations that lack of a mouse presents. The contact list can be traversed with the arrow keys, and I believe my wife has found a way to move through the tabbed interface with the some combination key. Dialing is fairly quick and easy, either via the contact list (enter key) or by dialing directly.

There are some problems, though. Dialing on the number pad once a call has begun, for example to navigate through an automated telephone system, seems impossible without the mouse. The form window to search for a contact seems fully accessible up until the time when you actually get the results of your search, at which point the results aren’t read - rendering the search functionality useless. And we still have yet to (though I hope we’ll soon discover the key-stroke to do this — it’s probably something simple) find a way to answer an incoming call.

All in all it’s still a useful program, even without that functionality; would be wonderful, though, if the Skype programmers would give some more thought to users with physical or cognitive disabilities.

Basic macro writing for Jedit.

October 13th, 2007

So i’ve been searching for a while for a good editor (mostly for XHTML/web development, though also for some work with music programming languages), and after trying out quite a few i’ve settled on Jedit. While it can be a little buggy at times i’m extremely impressed by it. My ideal editor is lightweight, without any extra bells and whistles or heavy-weight integration with the rest of the work environment (yeah, I know many folks love that stuff, but i’m perfectly happy to keep version-control and file navigator windows open as I work). One thing I do demand, though, is highly configurable keyboard shortcuts - and Jedit delivers in spades.

Within Jedit you have jedit command shortcuts (open, save, etc.), abbreviations and/or super-abbreviations, and macros. The basic Jedit command shortcuts give you rudimentary control over the program, the (super-)abbreviations give you the ability to very rapidly bang out common chunks of code (type a short command shortcut and then press space or tab and the abbreviation ‘expands’ to the code chunk you desire), and macros give you programmatic control over just about anything else you might want to do in a text editor.

I started off wanting to create basic ’snippet’ functionality and settled on macros as being the way to do it. Not being familiar with Java or Beanshell (in which Jedit macros are written), it took a little bit to figure out the best way of programming macros for my preferred way of editing, say, XHTML. The Jedit macro tutorial was, of course, extremely helpful and if you need details that’s definitely the place to start. But if you just want to bang out some quick macros, here are some tips for getting some simple functionality for text editing:

  • All macros have the file extension .bsh and are kept in the .jedit/macros folder within your home folder.
  • For a very simple macro to simply put a single chunk of text, for example the break tag for XHTML, try the following:
    textArea.setSelectedText("[your text here]");
  • To wrap a bit of text you’ve selected in tags , for example wrapping a few words with strong tags, try:
    text = textArea.getSelectedText();
    if(text == null) text = "";
    textArea.setSelectedText("[opening tag]" + text + "[closing tag]");
  • To wrap an entire line in tags (not selected text) use the following:
    textArea.goToStartOfWhiteSpace(false);
    textArea.setSelectedText("[opening tag]");
    textArea.goToEndOfWhiteSpace(false);
    textArea.setSelectedText("[closing tag]");
  • And to wrap some selected text in tags and use the selected text as an attribute within the tags, for example to wrap the text of a url (http://blah.blah) in an anchor tag which links to that url, use the following (note that double quotes within the tag must be escaped with backslash):
    text = textArea.getSelectedText();
    if(text == null) text = "";
    textArea.setSelectedText("[opening tag beginning, attribute beginning {ie: href=\"}]" + text + "[attribute end {ie: \"}, opening tag end]" + text + "[closing tag]");
  • All of your macros can be assigned shortcuts: Utilities-> Global Options->Shortcuts->Macros (drop-down menu)

Perceptions…

October 13th, 2007

Three short solo piano pieces for up and coming director Jesse Armstrong. Guess I’ve been in a minimalist mood lately…

Download Perception 1
Download Perception 2
Download Perception 3

Sound File Amplitude Modulation

September 9th, 2007

A Supercollider script I wrote to allow amplitude modulation of sound files, with the modulation depth being variable over time. Idea being, of course, to ‘morph’ the sound over time. Not the prettiest code, but effective (suggestions welcome, of course).

(
b = Buffer.read(s,"sounds/a11wlk01.wav");

	SynthDef
	(
		"playbuf",
		{ arg out=0, bufnum=0, modfreq=1000, startdepth=0.0, middepth=0.25, enddepth=0.0, attack=5.0, decay=5.0, rate=1, trigger=1, startPos=0, loop=1, multiplier=0.5;
			var modulator;
			modulator= SinOsc.ar
			(
				modfreq,
				0,
				EnvGen.kr
				(
					Env.new
					(
						[startdepth,middepth,enddepth],
						[attack, decay],
						'linear'
					)
				),
				multiplier
			);
			Out.ar
			(
				out,
				PlayBuf.ar
				(
					1,
					bufnum,
					BufRateScale.kr(bufnum)*rate,
					trigger,
					BufFrames.ir(bufnum)*startPos,
					loop
				)*modulator//...and amplitude modulate it
			)
		}
	).send(s);
)

Synth(playbuf, [out, 0, bufnum, b.bufnum, modfreq, 1568, attack, 2.0, decay, 2.0, startdepth, 0.0, middepth, 0.5, enddepth, 0.0, loop, 0]);

Tick-Tock

September 9th, 2007

Download Tick-Tock

Another piece written for director Jun Kang. Simple sequences from a sampler with varying amounts of delay to produce the textures.

Tone Poem in Shudders

August 26th, 2007

Download Tone Poem in Shudders

One of the many small pieces i’ve written for director Jun Kang. A simple tone poem performed by strings, run through granular synthesis in CSound, and sequenced in Cubase

Tenchi

August 26th, 2007

Download Tenchi

I’m told that translated from JapaneseTenchi means something like “Heaven and Earth”. So as I inaugurate this new website (or at least new version of it) I thought this piece, which means a lot to me, would be an appropriate choice for the first post.

Written as my master’s thesis while studying at the University of Texas at Austin, it was conducted by the phenomenal maestro Kevin Noe, and played wonderfully by the University of Texas Symphony Orchestra.

Creative Commons License

All music/content on this site is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 2.5 License.