If it starts to bug you you can always copy bzr.bat to a different file name. :)
Seriously, I've also been trying out bzr and have had some of the same problems that you have. bzr is still in development phase so I feel somewhat forgiving that it is still rough and missing some features. It is a distributed system so it probably will never be a perfect replacement for svn. The model of bzr is based on the idea that every programmer has their own copy (branch) of the entire code history.