So lets say you have a character who is on the run from a particularly powerful organization. Over the course of the story he becomes more and more desperate, and begins committing more and more morally questionable actions. It's only his wellbeing that is at risk, and he's not out to help some greater cause or anything like that. Early on he would occasionally help people in need, but as he got more desperate he would help people less and less and even begin hurting, or taking advantage of, people weaker than him in order to escape. At what point, or what action, would be going to far for you, when you could no longer root for this character.
To me, it depends entirely on why this organization is after him. If I feel that the organization is morally in the wrong, then I will have a lot more sympathy for the character, if the character acknowledges the poor morality of these wrongful actions. The limit point to me is if the character's morally wrong actions are made out of convenience rather than necessity. In the end, I can't condemn someone who shoves, steals and lies because they have a gun to their head. I can certainly beguilt that person because it is wrong to do these things, but the eyes of judgement would ultimately fall on the one holding the gun.
I think there is always a "too far", but it totally depends on the character. For example, if the guy was an ex-criminal, surely if he was on the run and desperate to stay alive he would be quicker killing someone than a goody-goody boy (unless of course the criminal wanted to turn away from his past and the goody-goody wanted a thrill, but you get the idea)? Have a look at your character and decide how quickly he would do morally-questionable actions, then decide from his personality when he would be going "too far". Everyone has a survival instinct in them, so people would go to extraordinary lengths to survive. Even though it's an extreme example, look at the Saw franchise. Gruesome stuff, but I'm sure tere would be people selfish enough to do horrible things just so they themselves could live. As long as your character gets worse gradually and not too quickly, and as long as the readers are given a reasonable explanation for his actions (to a point), then you can write it to suit your story. You must look at your plot and characters to truly know the answer to your question.
Dexter Morgan's character by Jeff Lindsay is a serial killer, but we still root for him, and he's a psychopath. He doesn't kill bad guys out of vigilantism, but the fact that he kills them instead of "innocents" makes the reader root for him. No one I know ever rooted for Patrick Bateman. So basically if you make your guy hurt "the bad guys", we are inclined to take his side. But if you make him steal from children or kick puppies, we might just want to punch him. However, even then, if he kicks the puppy to save his lover, we might still understand his decision even though kicking a puppy in that case still feels wrong. What I'm trying to say, if the reader is inside your character's head and he justifies his actions somehow, the reader can root for him even if he's a puppy-kicker. However, if he's as cold as Patrick Bateman, the reader can be fascinated, aching to find out how the universe will punish this perp, but s/he wouldn't necessarily side with him.
Tee Hee, Ed. This idea reminds me more of Breaking Bad than Dexter. The writers actually wanted to see how long it would take for viewers to realize that the character was becoming bad, and whether they would root against him. It's an interesting issue -- the reader naturally wants to identify with the protagonist. We want to root for him, even if he's doing things that objectively are not "right." (See most of the mobster movies, or movies following some sort of big bank heist or something.) In Breaking Bad, we start out with a character who is extremely sympathetic. He comes across as a fairly timid guy, who's a high school teacher and has cancer. He wants only to make enough money so that when he dies, his family will still be able to afford their house, and live a decent enough life. He actually has it figured out to some exact figure he wants to make. But as the series progresses, he changes, and becomes a pretty terrible person. But viewers are still hanging on him, and it takes a long time for them to even question the fact that they are rooting for him. At what point does the viewer say, "you know what? I don't think I like this guy anymore." I think if you can do that in a book, that is very impressive. To start out with us liking the MC, and gradually questioning him is a complicated proposition. But if you're able to do it, you will have created a very complex, fascinating character. The trick is in being able to do it right -- it still has to be believable, and he can't suddenly do a 180. But if you can do it gradually and in ways that make at least some logical sense, (logical meaning that even if we can see that the choice is a bad choice, there is at least some underlying logic as to why he would make the decisions he makes -- i.e some formerly smaller motivation becomes larger), it could be fantastic.
The readers' reaction is not to the character, but rather to how that character is presented. In other words, there;s no limit to how far the character can go, provided the writer manages the reader well.
I've come across that problem before. But write the story first, then identify when the story goes too far. There isn't much point of discussing before the writings.
If the character does evil things without guilt then it's difficult to still root for them. For example, in Death Note, Light is killing criminals so while it's bad to kill people it's also a moral conflict because you feel a sense of injustice that criminals aren't punished in a way that makes them feel how their victims did but when Light kills innocent Ray Penber the guy investigating him the line is crossed. So as the main character you'll kind of go along with them as long as it's in a way justified, its only when it becomes unjustified that it crosses the line. To me it's when you say to yourself after reading it that they 'could've just walked away, they didn't have to'. But then again emotions will push people and they may act terribly, but if they don't feel guilt for what they do then in a way they're irredeemable. Sorry for the Death Note example, haha
