I absolutely love getting a physical book, particularly ones from charity shops etc because they're so cheap!
However I totally get your points about reading on a kindle. I have one and when I read something on there I do think I find it easier and so read faster, and for longer. Turning pages is a little easier so you can read at any angle easily, it packs into any bag (and most coats), and a book is often way cheaper digitally. Plus, if you use Amazon's email-to-kindle service, you can actually send any pdf/epub to the kindle, meaning you have the option to read anything in public domain (e.g. Project Gutenberg) or...uh... anywhere else you magically find online, for free!
Granted, I'm talking about a physical kindle. I don't use the app on my phone much, but it is pretty good as well. I would suggest taking a look on eBay or the like for older Paperwhite models — mine is probably from 2014 and it's perfect. Not sure how much they go for second hand but weigh it against the price of another haul of books!