Bees nesting in wood pallet

I have a wood half pallet on my balcony that I've been drying out to take apart for use in a shelf, but it looks like some bees (probably mason or leafcutter) have taken up residence in a hole on one of the connecting 2x4 parts. (The top most horizontal section in the picture) I want to take the 2x4 part out and make it into a better home with ant and rain protection so I can use the rest of the wood, but I'm not sure how to go about doing that since they've already established themselves. I'd need to either saw the part off or take it apart with a crowbar, and I figure I'd have to get the bees to come out and maybe keep them somewhere safe, like in a cup or something, during the hour or so it'll take me to set it up. Any advice? This is my first experience doing anything other than netting and marking bumblebees for science.