Affection can be checked by clicking on the 'MomoTalk' smartphone icon on your home screen. This stat can be raised by tapping on them on your dorms, giving them gifts while they are in your dorms or select them on the schedule. Check the table on the bottom of the page for a list of who likes what gifts.
1 and 2 star characters are only able to reach an Affection level of up to 10. Notably, 1 & 2 star characters can be ranked up to 3 stars via their character shard and can go past this 10 Affection level cap.
Raising the Affection stat will give the character small bonuses to their stats and unlock their Live2D background (Memorial Lobby) for your homescreen after reaching a certain milestone. However, raising Affection levels past 20 no longer gives these bonuses. Here is the table of when each Live2D CG is unlocked.