It's not an apostrophe but an opening quotation mark, paired with a closing quotation mark at the end of the stanza. If I am understanding rightly, this stanza is spoken by a monk into whose monastery the Giaour has come, and he is describing the Giaour's behaviour. (Hence e.g. his invocation of St Francis later in the stanza.)
In at least one early edition the typographical convention is different and more explicit, with every single line of the stanza preceded by a quotation mark.
(This isn't something you asked about but may also be worth mentioning: Some lines in this poem do in fact begin with apostrophes -- there are several beginning "'Tis". This is just an abbreviated version of "It is".)