Arguments that appeal to the character of the arguer -- ethotic arguments -- are generally consider poor. I say "generally" to mean the general public, including particularly those who have had some education in basic logic. I'm not sure where else people could get the idea that there's something illogical, or even irrational, about criticizing an argument by criticizing the person who has made it.
For example, the infamous ad hominem, which rejects a conclusion based on some unsavoury or unacceptable characteristic of the person defending it. (The contrast would be the pro homine, which endorses a conclusion based on some acceptable or desirable characteristic of the person defending it.) Or what we might call a "guilt by association" argument, which rejects a conclusion based on the unsavoury characteristics of people who tend to endorse it. (The contrast here we could call "honour by association".)
I've never quite grasped why people think these are necessarily bad arguments, though. They obviously have at least a use as shortcuts. When evidence is difficult to assess, either because of a lack of availability or the need for expertise in order to assess it, it makes sense to appeal to characteristics of the arguer, such as trustworthiness or sincerity, in order to assess the strength of the argument. After all, what else is there to go on?
Putting that aside, there is clearly a point to appealing to characteristics of the arguer when the argument being replied to does the same. It's one of the funny aspects of ethotic arguments. If someone makes an ethotic argument, it seems logically sensible to reply to it with another ethotic argument. If I argue that you should agree with a conclusion because I am trustworthy (pro homine), then it is reasonable for you to argue in reply that I am not trustworthy (ad hominem).
While it might be nice to pretend that arguments are devoid of arguers, coldly rational assemblies of facts that either succeed or fail, the reality is that argument is necessarily embedded in the activity of arguing. This implies that any logical system which tells us to ignore the ethos of the arguer is, to that extent, a bad system.