Well, this really depends on what you want to recover in case of an emergency.
For most websites, I'd recommend a backup of your files and database.
With a typical WordPress site, the WP Content folder would be a good place to start. This is what essentially makes your site your site. The plugins, themes and media folder are all stored in there, so keeping a backup of that will quickly enable you to get your site back online if the worst happens.
A copy of your database (I prefer in .sql format) is also a good idea, because this will allow you to quickly import all your site settings, posts, and everything else when your site goes down.
If you do use a repo of some kind then you practically should have a backup of your site's files and folders there; that's half the job done.
So, with that in mind, you have two things to think about with backups - files and database.
There are many different tools you can use to run backups and checks. Scheduled/automated backups will always be your friend, because when set up correctly, you will have a regular backup ready. I personally like the manual approach, such as using the repo and running a database export, but that does take time.
You can automate this manual process, which is what it should be like anyway, but it does take some time and know-how to understand what is going on; it shouldn't be too difficult for a seasoned developer.
There are also plugins you can use which will create a backup of your site, with varying results, as well as third-party sites which can create a backup and/or cache your site (think, short term memory).
However, without a doubt, I would recommend something that runs on the server-side of things. This can be done through your hosting (a weekly backup or snapshot) or indeed something you can set up yourself.
I personally offer free weekly backups with the hosting packages I provide; so if something goes wrong (which is very rare with sites that I've set up), we can quickly restore the site to the last backup. You may have to redo some work, but it should be a quick job for relatively small sites.
It also helps that all the projects I work on have a repository which has all the files used in development.
Of course, the bigger the project, the more redundancy you need (I would recommend daily backups to the database for data-heavy sites).
