Doing my first play through and got up over 40 in 8 years and things looked solid, then the mother of all famines hit.
In one season I went to 18 residents, 3 of which were too old to reproduce, and a bunch of single residents. Luckily the town teacher is a bit loose and has had 2 kids out of wedlock back to back years and two of her students had a kid too
Currently at 22 in year 12, but it is a struggle. I am currently juggling several residents from farmers in the spring/summer to hunters/fishermen in the winter. Right now it is just a matter of eeking out an existence keeping everyone fed.
This game is a challenge.