Use your best judgement
When I sell filter-equipped fridges at THD I always ask if the customer has well or city water. Well water usually has more particulate matter, which will build up in the filter faster. Also it will filter out other stuff as advertised, such as benzine, lead, mercury etc. If you have city water you should not have this stuff ANYWAY. My town publishes and distributes a water test result yearly. So with city water you just have to be concerned with particulate matter and chlorine. I recommend yearly replacement, or as gauged by the flow of water from the dispenser. When you notice a susbstantial decrease in the flow, or notice off-taste, it's time. With well water, go by your indicator or every six months.
One other thing, concerning bacteria. Since the filter removes chlorine, if your filter is inside the fridge the cold itself will retard bacteria growth. If your filter is in the bottom near the warm motor, well.... Just take this into consideration.
Finally, when you do change the filter it may dislodge stuff in the water line. Run a pitcher of water and use it on the plants.