Posted by Doug
Tue, 05 Feb 2008 21:35:00 GMT
I’ve just fixed a bug in production that took me more than eight hours to find. When I show you the code, you’ll wonder why it took me so long. I have lots of excuses, but it’s a fairly interesting bug to think about. It shows some of the weaknesses in my usual modus operandi. The code that looks something like this:
class Article
CURRENT="start_publish_on <= #{Date.today} AND stop_publish_on > #{Date.today}"
def self.find_latest
find(:all, :conditions => CURRENT)
end
end
In retrospect the bug is obvious and probably is obvious to you as well. The Article::CURRENT constant is dynamically generated using the date at the time the class is evaluated. With rails in production mode, that could be a long time; certainly more than a day. The conclusion to draw here is to be very, very careful about dynamically generating constant strings. As a rule, I might suggest not doing it.
The most interesting thing about this is you can’t write a test to catch this error. I think that’s the the biggest thing that took me so long to find the error. I tend to be over confident in our test suite. As the new guy to the project, I’m proud of them for how conscientious they are about testing their code. I’m trying to fix this bug by triggering it in a test. Well, it can’t be done. The difference is how the production environment cache classes versus how testing and development does it.
Here’s another tidbit that threw me off the trail for a long time. Our copy editor that is responsible for publishing articles says to fix it she simply back dates the articles by a day. So I spent a lot of time looking for off-by-one errors. I had recently fixed a problem with comparing times to dates and causing off-by-one, so I thought that might be it. As it turns out, this was a red herring. There’s such a tight loop for feature request, implement, deploy that the production environment gets restarted fairly regularly (like nearly every day).
I guess what prompted me to write about this particular bug was what it said about our testing. Clearly automated testing can’t find all the bugs. It also says something about our rapid development. As long as we’re really busy, this bug didn’t bite us. It’s not until our deployment slows down (like a weekend) that it showed up.
Posted in Programming, Ruby on Rails | 1 comment
Posted by Doug
Wed, 07 Nov 2007 16:04:00 GMT
One of the things that’s really nice about TextMate is the Cmd-T navigation of files in your project. It pops up this little dialog with fancy pattern matching input to select one of the files in your project and then jumps to that file. There’s a similar command once your in a file to jump to symbols in that file. I’ve tried a couple things to achieve that behavior in emacs that I’d like to talk about.
The first thing I tried was a method called find-file-in-project. It was originally implemented by Phil Hagelberg as part of his “Another Ruby on Rails Mode” (arorem) and then also ported over to the current Rinari Is Not a Rails IDE. Basically, it indexes all the files in a “project” and then provides a nice interactive completing read to switch between them. It basically works just like Cmd-T in TextMat. I did quite a bit of work optimizing the completing read so that it would behave nicely. The problem is that emacs is slow to index the files in your project.
What I’m using now is the tried and true find-tag method which is part of etags.el. ETags depends on an external TAGS file as an index of all the symbols in your project. When you invoke find-tag (by default bound to M-.) it prompts you with completing read for the symbol to find. It then jumps directly to the file and location where that symbol is defined. It’s basically combining the find-file-in-project with find-symbol-in-buffer. It’s also very, very fast.
As an added bonus, you can use tags-search to search through your project finding places that tag is used. This is similar to TextMate’s slowly grep all the files and render a buffer with those results in it, but much faster.
The bad news is you have to manually build the TAGS file periodically. Emacs is pretty good about continuing to work when you’ve made modifications to the files, but if you add new methods, new files, or refactor significantly where thing are located then it gets confused. When that happens, simply rebuild the TAGS file.
Jim Weirich gave me a nice little rake task to do the job:
module Tags
RUBY_FILES = FileList['**/*.rb'].exclude("pkg")
end
namespace "tags" do
task :emacs => Tags::RUBY_FILES do
puts "Making Emacs TAGS file"
sh "xctags -e #{Tags::RUBY_FILES}", :verbose => false
end
end
task :tags => ["tags:emacs"]
Just put that in lib/tasks/tags.rake and then run rake tags:emacs when you invoke find-tag and it can’t find it. That shouldn’t happen very often. I’ve found it’s also very fast to build the TAGS file. I might consider putting that tags task as part of the normal run tests task, but I’m not sure that’s necessary.
The other catch here is that the rake task above calls out the exuberant ctags rather than the ctags that comes with emacs. The exuberant ctags knows how to parse ruby files whereas the ctags that comes with emacs can’t.
I’ve installed exuberant ctags from MacPorts. It is xctags even though MacPorts doesn’t install it that way. So (for better or worse) I’ve renamed the ctags files installed with ports to xctags. This also gets around the conflict that MacPorts thinks there is between the ctags installed with emacs and the exuberant ctags.
Give this a spin and let me know what you think. I’ve found it to be very accurate and very fast.
Posted in Emacs, Programming, Lisp, Ruby on Rails, Mac OS X | Tags emacs, etags, rinari, Ruby, TextMate, xctags | 4 comments
Posted by Doug
Tue, 28 Aug 2007 09:38:00 GMT
blah blah blah (lawyers made me change this) has been going through a major re-branding effort for the last long time. This morning we’re launching a major re-write of our main application. There’s a pretty cool on-line demo that shows off our new sexiness.
Simultaneously we’re rebranding our website and also restructuring most of our internal item catalog. Pretty much everyone in the company has been involved in this effort in some way. After months of work, we’re live!
The new website is running Radiant from index to thank you page. We have a web service for handling all of the EC business logic and multiple user facing web sites that connect to the service for managing their cart. We’ve written a “store behavior” to handle this xml-rpc client interface with Radiant.
This actually isn’t our first production Radiant site. Our first was our Japanese RosettaWorld. That site isn’t getting near the traffic that our US site does though. The US site runs steady-state at about 5 user sessions per second. We had a bit of a problem serving our CSS and JavaScript out of Radiant. It was causing a session hit that forced an update to the DB. Turns out that was a pretty big bottleneck in our load testing. As a fix, we just moved those files out of Radiant for this rollout. I’m sure there’s some in-Radiant solution to tell it not to track sessions on those pages. We’ll figure that out for this next release.
Hopefully, after all the stress of getting this release done we can take a collective deep breath. However, I’m guessing it’s more likely we’ll be scrambling to make all kinds of tweaks and such.
Posted in Ruby on Rails | Tags radiant, rosettastone | 7 comments
