I don’t see spam jet, but it will arrive eventually.
One simple way is mutual adding of the contacts; mutuals should bypass any other mechanism because allows full control for both contacts (removing a contact would be equivalent to blocking).
Another idea is to enable notifications for new contacts for a short configurable time (few hours).
Another idea is to generate a temporary id (could be the current id + a time based auth code) valid for a longer time (~1 week)
The last two options can be both in the QRcode sharing screen, and I’d leave “enable interactions with unknown contacts” disabled by default.
It doesn’t need to be One Right mechanism, different users may want more or less convenience, at the cost of risk of spam.