Suppose $G$ acts transitively on $A$, for a moment, and let $g\
eq e$ be in the stabilizer of $a\in A$.
For an arbitrary element $b\in A$, there exists $h\in G$ such that $b=h\cdot a$. Then $g\cdot b=g\cdot(h\cdot a)=h\cdot (g\cdot a)=h\cdot a=b$, so $g$ would also stabilize $b$. Thus $g$ stabilizes everything, and the action isn't faithful.
* * *
Now go back to the general situation and suppose that $G$ acts on $A$ such that $g\
eq e$ stablizes $a\in A$. Form the new set $C$ which is the disjoint union of $A$ and $G$, and let $G$ act on $C$ in the obvious way (by acting on elements according to $A$'s action if they are from $A$, and according to $G$'s action on itself if they are from $G$).
Now, $g$ still stabilizes $a\in A$, but it can't stabilize $e\in G$, so the action is not unfaithful on $C$. This counterexample leads me to believe your question was intended to be for transitive actions only.