If you are logged in, it adds a link to the bottom of the list of interests on any user info page.
If you are looking at your own user info, the link reads "Remove some of these interests", otherwise "Add or remove some of these interests to/from your own".
When clicked on, this link leads to a new page (actually, a new mode of editinfo.bml, mode=interests) which displays all the interests of this user with a checkbox. The common interests have the checkbox checked already, and are bolded.
You can select interests to add them to your list, and deselect common interests to remove them from your list.
This doesn't clutter any page because it only adds a small link to an existing page. Everything else is on a new page.
This feature was suggested here and seems to be welcome.