If you don't want to come of as spammy, you can use the @ mention at the begining of any conversation you are having. How does it work?
Any time you click reply or type @ to begin a tweet, only the person you mention and yourself will see the tweet. However, if someone follows both of you [in this case you may have mutual interest] they would also see the conversation.
You can be sure your conversation with people on twitter are not spamming others, if you use this rule.
What actions by some twitter users annoys you?