You might gain a more comfortable ride using 650B wheels and tires, since the increased volume in the tires might allow you to use a lower pressure than with the 700X 28mm-30mm tires. Traction is another area where this swap might make sense for the rider, as a bigger, wider contact patch results when a lower pressure 650B tire is used over that of the 700c X 28-30mm tire.
I could always feel that I was pedaling more with the 650B wheels versus a set of 700c X 38-40mm wheels and tires. Momentum can be an important thing on looser gravel, or softer terrain, and in these cases, I feel a 700c wheel set has a bit of an upper hand. That said, it is hard to argue that a 650B and fat tire set up tubeless does not have a better ride feel than anything 700c X 40mm or so.
I feel a smaller diameter wheel is a bit less stable in terms of getting knocked off line, or in terms of lateral stability on loose gravel, than a 700c wheel and fat gravel tire.