Ah, Ruby doesn't have a way of going from an exception to the frames it passed through (called bindings in Ruby).
Having that makes all this quite straightforward. Still quite cool stuff.
What ruby-breakpoint uses is a hackish way of getting the breakpoint() caller's binding which will then be used for the debugging session.
pdb seems similar to debug.rb that comes with Ruby. It's more of a traditional debugger, but I found an interactive environment very natural to do this kind of thing.
Currently ruby-breakpoint doesn't require a permanent global trace function which is a good thing because those are quite slow in Ruby.
WinPDB looks nice. Already has some of the features I would like my GUI client to have.
Thanks for all the information. This has been very insightful!