I recently murdered a guard outside Riften because he mentioned my furry ears. As a Khajit assassin, I only have a certain amount of tolerance to racist riff raff.
I slit his throat, drank his mead, then proceded to do the T-bag.
I was especially careful to make sure no one saw me, but alas, a bounty appeared on my head. I looked around and saw nothing but a horse standing there, all innocent like. I proceded to make dog food out of it, but at this point it was too late and the bounty remained.
Now this begs the question, did the horse act as a witness and somehow manage to alert Riften about my acts of justice, or did someone else see me?
This has happened to me on more than a couple of occasions.
For this answer I decided to test this. I rode up to Riften, hopped off my horse, crouched down saved and shot a guard. And I got a bounty. So I loaded, shot my horse and then shot the guard and did not get a bounty.