I think my Byzantine bug was set off by Guy Gavriel Kay's "Sarantine Mosaic", comprising two novels Sailing to Sarantium and Lord of Emperors. They're fantasy, set in a slightly disguised Byzantine Empire at the time of Emperor Justinian (about 535AD). The ending is very different from the real history, though.
I got more historical background from Tom Holland's In the Shadow of the Sword - it's a non-academic history looking at the area of the Byzantine and Persian Empires between roughly 400 and 750AD, and the development of Judaism, Christianity and Islam in that period.
I recommend all these books as well worth a read.