Conditional ref qualifier

I have CRTP-like hierarchy where derived classes may be implemented a bit differently and for one class some methods are allowed to call on rvalue references but for another class this would be undesired:

template <class TDerived>
struct Base {
    TDerived& foo() {
        // Body
        return static_cast<TDerived&>(*this);
    }
};

struct Derived1: public Base<Derived1> {};
struct Derived2: public Base<Derived2> {};

int main() {
    // Should not compile:
    // Derived1().foo();

    // Should compile:
    Derived2().foo();
    return 0;
}

So to prohibit calling foo() on rvalue references one can add ref-qualifier like this:

TDerived& foo() & {...}

But I'm wondering is there any simple way to add such ref-qualifier conditionally without duplicating foo() method (and adding a bit of SFINAE-like code)?