The easiest time to kick off this one is at the end of the Broken Flowers quest in Act One: Novigrad. Having completed this quest, you can challenge Zoltan Chivay to Gwent just outside Dandelion's ...