The Artima Developer Community
Sponsored Link

Ruby Buzz Forum
Finding out which Rails revision caused a test to fail

0 replies on 1 page.

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 0 replies on 1 page
Chris McGrath

Posts: 20
Nickname: octopod
Registered: Mar, 2006

Chris McGrath is a Ruby and Ruby On Rails developer
Finding out which Rails revision caused a test to fail Posted: Jan 24, 2008 1:32 PM
Reply to this message Reply

This post originated from an RSS feed registered with Ruby Buzz by Chris McGrath.
Original Post: Finding out which Rails revision caused a test to fail
Feed Title: Octoblog
Feed URL: http://feeds.feedburner.com/octoblog
Feed Description: Occasional blogging from octopod
Latest Ruby Buzz Posts
Latest Ruby Buzz Posts by Chris McGrath
Latest Posts From Octoblog

Advertisement

I’ve been upgrading an app from 1.1.6 to edge recently and ended up with failing tests. To get to the bottom of why a particular test was failing, and find the revision of Rails that caused it I wrote the following little script:

UPDATED: Yes, this should have been a binary search from the start :)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114

#!/usr/bin/env ruby
require 'optparse'

KNOWN_BAD_REVISIONS = [4634, 4635]

class Args < Hash
  def initialize(args) 
    super()
    self[:quiet] = false
     
    opts = OptionParser.new do |opts| 
      opts.banner = "Usage: #$0 [options]"
      opts.on('-s', '--start_revision NUMBER', Integer,
              'start from revision NUMBER') do |revision| 
        self[:start_revision] = revision 
      end 
      opts.on('-f', '--finish_revision [NUMBER]', Integer,
              'finish on revision [NUMBER]', 
              '(defaults to latest revision of',
              ' http://dev.rubyonrails.org/svn/rails/trunk)') do |revision| 
        self[:end_revision] = revision 
      end 
      opts.on('-t', '--test_file STRING', 
              'path to test file to run') do |test_file| 
        self[:test_file] = test_file 
      end 
      opts.on('-n', '--name STRING', 
              'name of test to run') do |test_name| 
        self[:test_name] = test_name 
      end 
      opts.on('-b', '--know_bad_revisions LIST',
              'Comma separated list if known bad revisions to skip',
              'These are added to the list', 
              "current list: #{KNOWN_BAD_REVISIONS.inspect}") do |extra_known_bad_revisions|
         self[:extra_known_bad_revisions] = extra_known_bad_revisions.split(",").map { |r| r.to_i }
      end
      opts.on('-q', '--quiet', 
              "don't output each revision as it is tested") do  
        self[:quiet] = true
      end 
      opts.on_tail('-h', '--help', 'display this help and exit') do 
        show_help(opts)
      end 
    end 
    opts.parse!(args)
    self[:extra_known_bad_revisions] ||= []
    show_help(opts) if self[:start_revision].nil? || self[:test_file].nil? || self[:test_name].nil?
  end
  
  def show_help(opts) 
    puts opts 
    puts "\n***Assumes a checkout of rails trunk in vendor/rails***"
    exit 
  end
end

class BadRevisionFinder

  def initialize(options)
    @options = options
    @end_revision = @options[:end_revision] ||= latest_rails_revision
    @start_revision = @options[:start_revision]
    @known_bad_revisions = KNOWN_BAD_REVISIONS + options[:extra_known_bad_revisions]
    puts "Known bad revision list: #{@known_bad_revisions.inspect}"
  end
  
  def find_bad_revision
    sanity_check
    while (@end_revision - @start_revision) > 1
      check_revision = ((@start_revision + @end_revision) / 2.0).floor
      while @known_bad_revisions.include?(check_revision)
        check_revision += 1
      end
      if revision_bad?(check_revision)
        puts "bad revision"
        @end_revision = check_revision
      else
        puts "good revision"
        @start_revision = check_revision
      end
    end
    bad_revision = revision_bad?(@start_revision) ? @start_revision : @end_revision
    puts "r#{bad_revision} caused #{@options[:test_name]} to fail! http://dev.rubyonrails.org/changeset/#{bad_revision}"
  end
  
  def sanity_check
    # make sure it's passing at the start revision
    if revision_bad?(@start_revision)
      puts "Hey, this test fails with the start revision #{@start_revision}!"
      exit
    end
    # make sure it's failing on end revision
    unless revision_bad?(@end_revision)
      puts "Hey, this test passed with the end revision #{@end_revision}!"
      exit
    end
  end
  
  def revision_bad?(revision)
    puts "checking r#{revision}" unless @options[:quiet]
    `svn up -r #{revision} vendor/rails`
    `ruby #{@options[:test_file]} -n #{@options[:test_name]}`
    $? != 0
  end
  
  def latest_rails_revision
    puts "Finding latest edge revision number"
    `svn info http://dev.rubyonrails.org/svn/rails/trunk`.match(/Revision: (\d*)/m).captures[0].to_i
  end
end

BadRevisionFinder.new(Args.new(ARGV)).find_bad_revision
 

Save the above as find_culprit.rb in root of your application, then run with something like:

1
2
3

      find_culprit.rb -r 4191 -t test/unit/user.rb -n test_something_that_should_still_be_working 
 

Start at whatever revision you know your app is good at. It requires a checkout of rails trunk to vendor/rails.

NB: This can be quite a slow process, as it’s doing an svn up each time. I’ve finished using it now, but next time I’m going to see if I can use svk to setup my own mirror of the rails repo locally to speed things up

Read: Finding out which Rails revision caused a test to fail

Topic: Visual Rails Workbench - Preview Previous Topic   Next Topic Topic: RubyForge - 5000 projects

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use