Note that the Agar agar should be added to warm liquid and be brought to a boil in order to properly "activate" its gelling qualities. Also note that you could probably get a very nice stabilising result even with less Agar agar than what is set out in the recipe - if you do not like ice cream bases that settles too firmly, I would suggest trying with only about ½ of the Agar (= slightly more than 1 teaspoon).
Xylitol is slightly sweeter and also more effective than "ordinary" sugar when it comes to keeping an ice cream soft: If you like, you could replace part of the sugar against an equal amount of xylitol. If you replace all sugar, however, you could well reduce the total overall amount of xylitol a bit too.